2024-09-15T23:53:19.762 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T23:53:19.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T23:53:19.876 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906370' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM2BRep29EiaXN8NG8mg9tTBo3/uQmwqqdS34cW3YwShNvDFgfExctzU2A1Nt5ut26Kt3mqs7hy60OX2TR0DyCA= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFcGZ+7AErqNs75UlAJPL+qi5VT3q141nouMEvw07JLBGVcP1GEqLPUxm8jJVBDjZRIZZlMSRKZZyk1AaEz5GwA= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD4wUyX9zrh/5WMnJTNmy3KeDDcMX/zlyVapYQTbp5vPvbN+R0i7d75FmWd6Kq6nJlEta88JhctyDg19b1bi5QY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T23:53:19.876 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T23:53:19.877 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T23:53:19.877 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T23:53:19.879 INFO:teuthology.task.internal:Checking packages... 2024-09-15T23:53:19.900 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T23:53:19.901 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T23:53:19.901 INFO:teuthology.packaging:ref: None 2024-09-15T23:53:19.901 INFO:teuthology.packaging:tag: None 2024-09-15T23:53:19.901 INFO:teuthology.packaging:branch: squid 2024-09-15T23:53:19.901 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:53:19.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T23:53:20.134 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-15T23:53:20.136 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T23:53:20.142 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T23:53:20.143 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T23:53:20.150 INFO:teuthology.task.internal:Saving configuration 2024-09-15T23:53:20.172 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T23:53:20.178 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T23:53:20.204 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:48:15.487275', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM2BRep29EiaXN8NG8mg9tTBo3/uQmwqqdS34cW3YwShNvDFgfExctzU2A1Nt5ut26Kt3mqs7hy60OX2TR0DyCA='} 2024-09-15T23:53:20.228 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:48:15.488303', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFcGZ+7AErqNs75UlAJPL+qi5VT3q141nouMEvw07JLBGVcP1GEqLPUxm8jJVBDjZRIZZlMSRKZZyk1AaEz5GwA='} 2024-09-15T23:53:20.249 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:48:15.486030', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD4wUyX9zrh/5WMnJTNmy3KeDDcMX/zlyVapYQTbp5vPvbN+R0i7d75FmWd6Kq6nJlEta88JhctyDg19b1bi5QY='} 2024-09-15T23:53:20.249 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T23:53:20.255 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-15T23:53:20.255 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-15T23:53:20.255 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-15T23:53:20.255 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T23:53:20.375 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f19ea59fbe0>, signals=[15]) 2024-09-15T23:53:20.375 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T23:53:20.381 INFO:teuthology.task.internal:Opening connections... 2024-09-15T23:53:20.381 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-09-15T23:53:20.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:53:20.462 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-09-15T23:53:20.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:53:20.540 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-09-15T23:53:20.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:53:20.623 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T23:53:20.629 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-09-15T23:53:20.648 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-09-15T23:53:20.648 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-09-15T23:53:20.707 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:53:20.708 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:53:20.709 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:53:20.709 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:53:20.709 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:53:20.709 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-09-15T23:53:20.744 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-09-15T23:53:20.770 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-09-15T23:53:20.770 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-09-15T23:53:20.840 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:53:20.841 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:53:20.842 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:53:20.842 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:53:20.842 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:53:20.842 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:53:20.842 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-09-15T23:53:20.868 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-09-15T23:53:20.887 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-09-15T23:53:20.888 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-09-15T23:53:20.947 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-09-15T23:53:20.947 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-09-15T23:53:20.947 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-09-15T23:53:20.947 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:53:20.947 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-09-15T23:53:20.947 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:53:20.947 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:53:20.948 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-09-15T23:53:20.948 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:53:20.948 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:53:20.948 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:53:20.948 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:53:20.948 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:53:20.948 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:53:20.948 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-09-15T23:53:20.968 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T23:53:21.238 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T23:53:21.261 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T23:53:21.261 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:53:21.264 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:53:21.267 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:53:21.284 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T23:53:21.293 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T23:53:21.293 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:53:21.321 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:53:21.324 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:53:21.419 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T23:53:21.498 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T23:53:21.513 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-15T23:53:21.513 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-15T23:53:21.513 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-15T23:53:21.513 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:53:21.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:53:21.530 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-09-15T23:53:21.587 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-503.el9.x86_64 2024-09-15T23:53:21.588 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-503.el9.x86_64 2024-09-15T23:53:21.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-09-15T23:53:23.848 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-15T23:53:25.658 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-15T23:53:29.975 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-15T23:53:31.442 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-15T23:53:32.539 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-15T23:53:38.728 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 2.1 kB/s | 1.7 kB 00:00 2024-09-15T23:53:40.750 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T23:53:40.751 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T23:53:40.845 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-09-15T23:53:40.855 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-09-15T23:53:40.855 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-09-15T23:53:40.855 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-09-15T23:53:40.855 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-09-15T23:53:40.855 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T23:53:40.855 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-09-15T23:53:40.855 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T23:53:40.856 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T23:53:40.856 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T23:53:40.856 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-15T23:53:40.856 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-09-15T23:53:40.856 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-09-15T23:53:40.856 INFO:teuthology.orchestra.run.smithi078.stdout:Install 4 Packages 2024-09-15T23:53:40.856 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-15T23:53:40.857 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 79 M 2024-09-15T23:53:40.857 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 126 M 2024-09-15T23:53:40.857 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-09-15T23:53:41.029 INFO:teuthology.orchestra.run.smithi078.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 310 kB/s | 23 kB 00:00 2024-09-15T23:53:43.438 INFO:teuthology.orchestra.run.smithi078.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 14 MB/s | 35 MB 00:02 2024-09-15T23:53:43.863 INFO:teuthology.orchestra.run.smithi078.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 10 MB/s | 29 MB 00:02 2024-09-15T23:53:45.463 INFO:teuthology.orchestra.run.smithi078.stdout:(4/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.5 MB/s | 16 MB 00:04 2024-09-15T23:53:45.463 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:53:45.463 INFO:teuthology.orchestra.run.smithi078.stdout:Total 17 MB/s | 79 MB 00:04 2024-09-15T23:53:46.143 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-09-15T23:53:46.500 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-09-15T23:53:46.500 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-09-15T23:53:47.471 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-09-15T23:53:47.472 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-09-15T23:53:49.910 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-09-15T23:53:50.555 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:53:50.631 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:53:53.430 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:53:53.559 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:53:58.407 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:53:58.451 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:54:03.288 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:54:41.547 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:54:41.585 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:54:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:54:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:54:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:54:41.960 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:54:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:54:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-15T23:54:42.807 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-09-15T23:54:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T23:54:42.807 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T23:54:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T23:54:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T23:54:42.808 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-15T23:54:42.808 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-09-15T23:54:43.162 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:54:43.242 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:54:43.243 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:54:43.243 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T23:54:43.243 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T23:54:43.243 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T23:54:43.243 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-15T23:54:43.243 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi078.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T23:54:43.243 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:54:43.302 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:54:43.302 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:54:43.302 DEBUG:teuthology.orchestra.run.smithi078:> sudo rpm -qi grub2-tools 2024-09-15T23:54:43.336 INFO:teuthology.orchestra.run.smithi078.stdout:Name : grub2-tools 2024-09-15T23:54:43.336 INFO:teuthology.orchestra.run.smithi078.stdout:Epoch : 1 2024-09-15T23:54:43.336 INFO:teuthology.orchestra.run.smithi078.stdout:Version : 2.06 2024-09-15T23:54:43.336 INFO:teuthology.orchestra.run.smithi078.stdout:Release : 82.el9 2024-09-15T23:54:43.336 INFO:teuthology.orchestra.run.smithi078.stdout:Architecture: x86_64 2024-09-15T23:54:43.336 INFO:teuthology.orchestra.run.smithi078.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T23:54:43.336 INFO:teuthology.orchestra.run.smithi078.stdout:Group : Unspecified 2024-09-15T23:54:43.336 INFO:teuthology.orchestra.run.smithi078.stdout:Size : 8274670 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:License : GPLv3+ 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:Packager : builder@centos.org 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:Vendor : CentOS 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:Summary : Support tools for GRUB. 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:Description : 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T23:54:43.337 INFO:teuthology.orchestra.run.smithi078.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T23:54:43.338 INFO:teuthology.orchestra.run.smithi078.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T23:54:43.338 INFO:teuthology.orchestra.run.smithi078.stdout:hardware devices. 2024-09-15T23:54:43.338 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-15T23:54:43.338 INFO:teuthology.orchestra.run.smithi078.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T23:54:43.339 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T23:54:43.339 INFO:teuthology.task.kernel:Updating grub on smithi078 to boot 5.14.0-505.el9.x86_64 2024-09-15T23:54:43.339 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T23:54:43.984 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2024-09-15T23:54:45.261 INFO:teuthology.orchestra.run.smithi078.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T23:54:45.302 INFO:teuthology.orchestra.run.smithi078.stderr:done 2024-09-15T23:54:45.305 DEBUG:teuthology.orchestra.run.smithi078:> mktemp 2024-09-15T23:54:45.320 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/tmp.5LT2N1lCee 2024-09-15T23:54:45.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5LT2N1lCee 2024-09-15T23:54:45.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0666 /tmp/tmp.5LT2N1lCee 2024-09-15T23:54:45.533 DEBUG:teuthology.orchestra.remote:smithi078:/tmp/tmp.5LT2N1lCee is 6KB 2024-09-15T23:54:45.581 DEBUG:teuthology.orchestra.run.smithi078:> rm -fr /tmp/tmp.5LT2N1lCee 2024-09-15T23:54:45.597 DEBUG:teuthology.orchestra.run.smithi078:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T23:54:45.659 INFO:teuthology.orchestra.run.smithi078.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T23:54:45.659 INFO:teuthology.orchestra.run.smithi078.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T23:54:45.659 INFO:teuthology.orchestra.run.smithi078.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T23:54:45.659 INFO:teuthology.orchestra.run.smithi078.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T23:54:45.661 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T23:54:45.793 DEBUG:teuthology.orchestra.run.smithi078:> sudo shutdown -r now 2024-09-15T23:55:15.796 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-15T23:55:15.796 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-15T23:55:15.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:56:15.801 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T23:56:24.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-15T23:56:24.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:56:25.186 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-09-15T23:56:25.597 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-15T23:56:25.597 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-15T23:56:25.598 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-09-15T23:56:25.612 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-505.el9.x86_64 2024-09-15T23:56:25.612 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T23:56:25.612 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T23:56:25.612 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T23:56:26.613 DEBUG:teuthology.parallel:result is None 2024-09-15T23:56:26.614 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T23:56:26.619 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T23:56:26.620 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T23:56:26.622 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T23:56:26.625 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T23:56:26.643 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T23:56:26.650 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T23:56:26.658 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T23:56:26.658 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T23:56:26.681 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T23:56:26.683 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T23:56:26.823 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T23:56:26.834 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T23:56:26.834 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:56:26.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:56:26.852 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:56:26.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:56:26.872 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T23:56:26.874 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T23:56:26.895 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T23:56:26.952 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:56:26.962 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:56:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:56:26.971 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:56:26.982 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:56:27.016 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T23:56:27.018 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T23:56:27.025 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T23:56:27.025 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T23:56:27.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T23:56:27.031 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T23:56:27.060 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T23:56:27.089 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T23:56:27.089 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T23:56:27.097 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T23:56:27.099 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T23:56:27.121 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:56:27.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:56:27.242 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:56:27.327 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:56:27.372 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-15T23:56:27.372 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T23:56:27.440 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:56:27.487 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:56:27.542 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:56:27.627 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:56:27.669 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-15T23:56:27.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T23:56:27.737 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:56:27.781 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T23:56:27.831 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:56:27.916 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T23:56:27.959 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-15T23:56:27.959 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T23:56:28.027 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-09-15T23:56:28.029 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-09-15T23:56:28.031 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-09-15T23:56:28.068 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:56:28.075 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:56:28.111 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:56:28.632 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T23:56:28.640 INFO:teuthology.task.internal:Starting timer... 2024-09-15T23:56:28.640 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T23:56:28.652 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T23:56:28.660 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T23:56:28.660 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-09-15T23:56:28.746 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-09-15T23:56:28.748 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-09-15T23:56:28.810 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-09-15T23:56:28.813 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-09-15T23:56:28.868 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-09-15T23:56:28.870 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T23:56:28.870 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-09-15T23:56:28.894 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-09-15T23:56:28.895 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-09-15T23:56:28.924 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-09-15T23:56:28.925 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-09-15T23:56:28.953 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-09-15T23:56:28.953 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-09-15T23:56:28.953 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:56:28.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:56:28.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:56:29.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:56:29.010 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:56:29.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:56:29.039 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T23:56:29.040 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-09-15T23:56:29.068 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-09-15T23:56:29.113 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-09-15T23:56:29.145 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T23:56:29.155 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T23:56:29.159 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T23:56:29.270 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T23:56:29.299 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T23:56:29.300 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi148.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T00:00:27.935 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-09-16T00:00:27.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-16T00:00:27.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:00:28.016 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-09-16T00:00:28.107 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-16T00:00:28.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-16T00:00:28.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:00:28.184 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-16T00:00:28.264 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-16T00:00:28.265 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-16T00:00:28.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:00:28.341 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-16T00:00:28.421 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-16T00:00:28.422 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T00:00:28.454 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T00:00:28.455 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:00:28.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:00:28.457 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:00:28.458 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:00:28.460 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:00:28.460 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:00:28.484 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:00:28.487 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:00:28.494 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:00:28.498 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:00:28.502 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:00:28.510 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:00:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-09-16T00:00:28.526 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-09-16T00:00:28.532 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-09-16T00:00:28.535 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-09-16T00:00:28.537 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-09-16T00:00:28.546 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:00:28.547 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-09-16T00:00:28.551 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:00:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:00:28.562 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:00:28.566 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:00:28.577 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:00:28.621 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:00:28.624 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:00:28.624 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-09-16T00:00:28.624 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.625 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.625 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.625 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.625 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:00:28.629 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:00:28.629 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-16T00:00:28.629 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.629 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.629 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.629 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.633 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:00:28.636 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:00:28.636 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-16T00:00:28.636 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.636 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.636 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.636 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:00:28.637 INFO:teuthology.run_tasks:Running task install... 2024-09-16T00:00:28.678 DEBUG:teuthology.task.install:project ceph 2024-09-16T00:00:28.678 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T00:00:28.678 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T00:00:28.679 INFO:teuthology.task.install:Using flavor: default 2024-09-16T00:00:28.684 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T00:00:28.684 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T00:00:28.685 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:00:28.686 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:00:28.688 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:00:28.688 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:00:28.689 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:00:28.689 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:00:28.828 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:00:28.828 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:00:28.849 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:00:28.849 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:00:28.851 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:00:28.852 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:00:28.943 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:00:28.943 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:00:28.943 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:00:28.954 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:00:28.954 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:00:28.954 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:00:28.962 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:00:28.963 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:00:28.963 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:00:28.969 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:00:28.969 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:00:28.979 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:00:28.980 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:00:28.989 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:00:28.989 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:00:29.034 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:00:29.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:00:29.054 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:00:29.109 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:00:29.120 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:00:29.128 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:00:29.174 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-09-16T00:00:29.175 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-09-16T00:00:29.185 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-09-16T00:00:29.186 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-09-16T00:00:29.192 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-09-16T00:00:29.194 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-09-16T00:00:29.396 INFO:teuthology.orchestra.run.smithi078.stdout:45 files removed 2024-09-16T00:00:29.411 INFO:teuthology.orchestra.run.smithi148.stdout:45 files removed 2024-09-16T00:00:29.419 INFO:teuthology.orchestra.run.smithi193.stdout:45 files removed 2024-09-16T00:00:29.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:00:29.439 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:00:29.445 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:00:29.972 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 314 kB/s | 83 kB 00:00 2024-09-16T00:00:29.997 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 301 kB/s | 83 kB 00:00 2024-09-16T00:00:30.128 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-09-16T00:00:30.256 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T00:00:30.281 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-16T00:00:30.395 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-16T00:00:30.532 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T00:00:30.549 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T00:00:30.671 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T00:00:31.866 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 6.4 MB/s | 8.2 MB 00:01 2024-09-16T00:00:32.265 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-16T00:00:32.371 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-16T00:00:33.859 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-16T00:00:36.841 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 20 MB 00:03 2024-09-16T00:00:38.114 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-16T00:00:38.435 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-16T00:00:39.844 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-16T00:00:40.648 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-16T00:00:42.373 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-16T00:00:42.425 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-16T00:00:43.990 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-16T00:00:44.115 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 19 kB 00:00 2024-09-16T00:00:45.163 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-16T00:00:45.236 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-16T00:00:46.179 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T00:00:48.439 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:00:48.439 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:00:48.450 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:00:48.451 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:00:48.452 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:00:48.453 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:00:48.454 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:00:48.457 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:00:48.458 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:00:48.521 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-16T00:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-09-16T00:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-16T00:00:48.535 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:00:48.536 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:00:48.537 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:00:48.538 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:00:48.539 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:00:48.540 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:00:48.540 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:00:48.540 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-09-16T00:00:48.540 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:00:48.540 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:00:48.540 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:00:48.540 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:00:48.540 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:00:48.541 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:00:48.541 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:00:48.541 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:00:48.541 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-16T00:00:48.541 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:00:48.541 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:00:48.541 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:00:48.541 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:00:48.542 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:00:48.542 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:00:48.542 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:00:48.542 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:00:48.542 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:00:48.542 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:00:48.542 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:00:48.542 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:00:48.543 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:00:48.544 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:00:48.544 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:00:48.544 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:00:48.544 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:00:48.544 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:00:48.544 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:00:48.544 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:00:48.545 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:00:48.546 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:00:48.546 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:00:48.546 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:00:48.546 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:00:48.546 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:00:48.546 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:00:48.546 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:00:48.546 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:00:48.547 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:00:48.547 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:00:48.547 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:00:48.547 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:00:48.547 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:00:48.547 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:00:48.547 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:00:48.547 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:00:48.548 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:00:48.548 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:00:48.548 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:00:48.548 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:00:48.548 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:00:48.548 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:00:48.548 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:00:48.548 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:00:48.549 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:00:48.549 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:00:48.549 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:00:48.549 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:00:48.549 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:00:48.549 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:00:48.549 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:00:48.549 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:00:48.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:00:48.551 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:00:48.551 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:00:48.551 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:00:48.551 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:00:48.551 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:00:48.551 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:00:48.551 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:00:48.551 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:00:48.552 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:00:48.552 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:00:48.552 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:00:48.552 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:00:48.552 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:00:48.552 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:00:48.552 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:00:48.552 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:00:48.553 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout:Install 148 Packages 2024-09-16T00:00:48.554 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 8 Packages 2024-09-16T00:00:48.555 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:00:48.555 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 226 M 2024-09-16T00:00:48.555 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-16T00:00:49.039 INFO:teuthology.orchestra.run.smithi193.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-16T00:00:49.281 INFO:teuthology.orchestra.run.smithi193.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.6 MB/s | 896 kB 00:00 2024-09-16T00:00:49.398 INFO:teuthology.orchestra.run.smithi193.stdout:(3/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-16T00:00:49.423 INFO:teuthology.orchestra.run.smithi193.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 1.0 MB/s | 145 kB 00:00 2024-09-16T00:00:49.516 INFO:teuthology.orchestra.run.smithi193.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 20 MB/s | 2.4 MB 00:00 2024-09-16T00:00:49.657 INFO:teuthology.orchestra.run.smithi193.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-16T00:00:49.766 INFO:teuthology.orchestra.run.smithi193.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 19 MB/s | 4.7 MB 00:00 2024-09-16T00:00:50.066 INFO:teuthology.orchestra.run.smithi193.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 18 MB/s | 21 MB 00:01 2024-09-16T00:00:50.117 INFO:teuthology.orchestra.run.smithi193.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 505 kB/s | 25 kB 00:00 2024-09-16T00:00:50.325 INFO:teuthology.orchestra.run.smithi193.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 19 MB/s | 11 MB 00:00 2024-09-16T00:00:50.392 INFO:teuthology.orchestra.run.smithi193.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 3.4 MB/s | 228 kB 00:00 2024-09-16T00:00:50.475 INFO:teuthology.orchestra.run.smithi193.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 325 kB/s | 27 kB 00:00 2024-09-16T00:00:50.576 INFO:teuthology.orchestra.run.smithi193.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 317 kB/s | 32 kB 00:00 2024-09-16T00:00:50.594 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-16T00:00:50.642 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T00:00:50.693 INFO:teuthology.orchestra.run.smithi193.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 6.3 MB/s | 751 kB 00:00 2024-09-16T00:00:50.901 INFO:teuthology.orchestra.run.smithi193.stdout:(15/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 14 MB/s | 17 MB 00:01 2024-09-16T00:00:50.927 INFO:teuthology.orchestra.run.smithi193.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 699 kB/s | 163 kB 00:00 2024-09-16T00:00:50.952 INFO:teuthology.orchestra.run.smithi193.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-16T00:00:50.994 INFO:teuthology.orchestra.run.smithi193.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 7.4 MB/s | 506 kB 00:00 2024-09-16T00:00:51.061 INFO:teuthology.orchestra.run.smithi193.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 676 kB/s | 45 kB 00:00 2024-09-16T00:00:51.161 INFO:teuthology.orchestra.run.smithi193.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.3 MB/s | 136 kB 00:00 2024-09-16T00:00:51.245 INFO:teuthology.orchestra.run.smithi193.stdout:(21/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-16T00:00:51.270 INFO:teuthology.orchestra.run.smithi193.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.4 MB/s | 161 kB 00:00 2024-09-16T00:00:51.304 INFO:teuthology.orchestra.run.smithi193.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 5.4 MB/s | 320 kB 00:00 2024-09-16T00:00:51.329 INFO:teuthology.orchestra.run.smithi193.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-16T00:00:51.372 INFO:teuthology.orchestra.run.smithi193.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.4 MB/s | 99 kB 00:00 2024-09-16T00:00:51.397 INFO:teuthology.orchestra.run.smithi193.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-16T00:00:51.456 INFO:teuthology.orchestra.run.smithi193.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.8 MB/s | 170 kB 00:00 2024-09-16T00:00:51.514 INFO:teuthology.orchestra.run.smithi193.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 22 MB/s | 3.1 MB 00:00 2024-09-16T00:00:51.531 INFO:teuthology.orchestra.run.smithi193.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 425 kB/s | 32 kB 00:00 2024-09-16T00:00:51.564 INFO:teuthology.orchestra.run.smithi193.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-16T00:00:51.907 INFO:teuthology.orchestra.run.smithi193.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 22 MB/s | 7.4 MB 00:00 2024-09-16T00:00:51.965 INFO:teuthology.orchestra.run.smithi193.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.2 MB/s | 249 kB 00:00 2024-09-16T00:00:52.024 INFO:teuthology.orchestra.run.smithi193.stdout:(33/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 7.2 MB/s | 3.6 MB 00:00 2024-09-16T00:00:52.053 INFO:teuthology.orchestra.run.smithi193.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 561 kB/s | 49 kB 00:00 2024-09-16T00:00:52.079 INFO:teuthology.orchestra.run.smithi193.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 302 kB/s | 16 kB 00:00 2024-09-16T00:00:52.112 INFO:teuthology.orchestra.run.smithi193.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.8 MB/s | 285 kB 00:00 2024-09-16T00:00:52.146 INFO:teuthology.orchestra.run.smithi193.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 11 MB/s | 765 kB 00:00 2024-09-16T00:00:52.188 INFO:teuthology.orchestra.run.smithi193.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 241 kB/s | 18 kB 00:00 2024-09-16T00:00:52.631 INFO:teuthology.orchestra.run.smithi193.stdout:(39/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 19 MB/s | 49 MB 00:02 2024-09-16T00:00:52.938 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:00:52.939 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:00:52.952 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:00:52.953 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:00:52.954 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:00:52.955 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:00:52.957 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:00:52.959 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:00:52.960 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:00:52.979 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:00:52.980 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:00:52.992 INFO:teuthology.orchestra.run.smithi078.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:00:52.992 INFO:teuthology.orchestra.run.smithi078.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:00:52.993 INFO:teuthology.orchestra.run.smithi078.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:00:52.994 INFO:teuthology.orchestra.run.smithi078.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:00:52.996 INFO:teuthology.orchestra.run.smithi078.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:00:52.998 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:00:52.999 INFO:teuthology.orchestra.run.smithi078.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:00:53.023 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-16T00:00:53.044 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-16T00:00:53.044 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-16T00:00:53.044 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-16T00:00:53.044 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-16T00:00:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:00:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:00:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:00:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:00:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:00:53.045 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:00:53.046 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:00:53.047 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:00:53.048 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:00:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:00:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:00:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:00:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:00:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:00:53.049 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-16T00:00:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:00:53.049 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:00:53.050 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:00:53.051 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:00:53.052 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:00:53.053 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:00:53.054 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:00:53.055 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:00:53.056 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:00:53.057 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-16T00:00:53.058 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-16T00:00:53.059 INFO:teuthology.orchestra.run.smithi148.stdout:Install 148 Packages 2024-09-16T00:00:53.059 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 8 Packages 2024-09-16T00:00:53.059 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:00:53.059 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 226 M 2024-09-16T00:00:53.059 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-16T00:00:53.061 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-09-16T00:00:53.076 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:00:53.077 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:00:53.078 INFO:teuthology.orchestra.run.smithi078.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:00:53.079 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:00:53.080 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:00:53.081 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:00:53.082 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:00:53.083 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:00:53.084 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:00:53.085 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:00:53.086 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:00:53.087 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:00:53.088 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout:Installing weak dependencies: 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================== 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout:Install 148 Packages 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 8 Packages 2024-09-16T00:00:53.089 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:00:53.090 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 226 M 2024-09-16T00:00:53.090 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-09-16T00:00:53.198 INFO:teuthology.orchestra.run.smithi193.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 76 kB/s | 80 kB 00:01 2024-09-16T00:00:53.373 INFO:teuthology.orchestra.run.smithi193.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 258 kB/s | 191 kB 00:00 2024-09-16T00:00:53.465 INFO:teuthology.orchestra.run.smithi193.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 628 kB/s | 801 kB 00:01 2024-09-16T00:00:53.877 INFO:teuthology.orchestra.run.smithi148.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 48 kB/s | 6.5 kB 00:00 2024-09-16T00:00:53.956 INFO:teuthology.orchestra.run.smithi193.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 334 kB/s | 253 kB 00:00 2024-09-16T00:00:54.029 INFO:teuthology.orchestra.run.smithi078.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-16T00:00:54.093 INFO:teuthology.orchestra.run.smithi148.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 4.0 MB/s | 896 kB 00:00 2024-09-16T00:00:54.148 INFO:teuthology.orchestra.run.smithi193.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 156 kB/s | 106 kB 00:00 2024-09-16T00:00:54.185 INFO:teuthology.orchestra.run.smithi148.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.5 MB/s | 145 kB 00:00 2024-09-16T00:00:54.260 INFO:teuthology.orchestra.run.smithi148.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-16T00:00:54.324 INFO:teuthology.orchestra.run.smithi193.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-16T00:00:54.353 INFO:teuthology.orchestra.run.smithi148.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-16T00:00:54.419 INFO:teuthology.orchestra.run.smithi148.stdout:(6/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 10 MB/s | 2.4 MB 00:00 2024-09-16T00:00:54.612 INFO:teuthology.orchestra.run.smithi078.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.5 MB/s | 896 kB 00:00 2024-09-16T00:00:54.674 INFO:teuthology.orchestra.run.smithi193.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 189 kB/s | 135 kB 00:00 2024-09-16T00:00:54.678 INFO:teuthology.orchestra.run.smithi148.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-16T00:00:54.713 INFO:teuthology.orchestra.run.smithi078.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.4 MB/s | 145 kB 00:00 2024-09-16T00:00:54.899 INFO:teuthology.orchestra.run.smithi193.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 166 kB/s | 125 kB 00:00 2024-09-16T00:00:54.937 INFO:teuthology.orchestra.run.smithi148.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 18 MB/s | 21 MB 00:01 2024-09-16T00:00:54.958 INFO:teuthology.orchestra.run.smithi193.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-16T00:00:54.978 INFO:teuthology.orchestra.run.smithi148.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 608 kB/s | 25 kB 00:00 2024-09-16T00:00:55.016 INFO:teuthology.orchestra.run.smithi193.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-16T00:00:55.067 INFO:teuthology.orchestra.run.smithi193.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-16T00:00:55.125 INFO:teuthology.orchestra.run.smithi193.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-09-16T00:00:55.151 INFO:teuthology.orchestra.run.smithi193.stdout:(52/156): python3-urllib3-1.26.5-6.el9.noarch.r 260 kB/s | 215 kB 00:00 2024-09-16T00:00:55.176 INFO:teuthology.orchestra.run.smithi193.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-16T00:00:55.210 INFO:teuthology.orchestra.run.smithi193.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-16T00:00:55.235 INFO:teuthology.orchestra.run.smithi193.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-16T00:00:55.260 INFO:teuthology.orchestra.run.smithi193.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-16T00:00:55.327 INFO:teuthology.orchestra.run.smithi193.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-16T00:00:55.346 INFO:teuthology.orchestra.run.smithi078.stdout:(4/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 3.8 MB/s | 2.4 MB 00:00 2024-09-16T00:00:55.352 INFO:teuthology.orchestra.run.smithi193.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 493 kB/s | 45 kB 00:00 2024-09-16T00:00:55.411 INFO:teuthology.orchestra.run.smithi193.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-16T00:00:55.444 INFO:teuthology.orchestra.run.smithi193.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-16T00:00:55.470 INFO:teuthology.orchestra.run.smithi193.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-16T00:00:55.495 INFO:teuthology.orchestra.run.smithi193.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-16T00:00:55.595 INFO:teuthology.orchestra.run.smithi148.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 14 MB/s | 17 MB 00:01 2024-09-16T00:00:55.596 INFO:teuthology.orchestra.run.smithi078.stdout:(5/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 3.2 MB/s | 5.4 MB 00:01 2024-09-16T00:00:55.612 INFO:teuthology.orchestra.run.smithi193.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-16T00:00:55.663 INFO:teuthology.orchestra.run.smithi193.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 771 kB/s | 38 kB 00:00 2024-09-16T00:00:55.697 INFO:teuthology.orchestra.run.smithi193.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.3 MB/s | 292 kB 00:00 2024-09-16T00:00:55.712 INFO:teuthology.orchestra.run.smithi148.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 10 MB/s | 11 MB 00:01 2024-09-16T00:00:55.730 INFO:teuthology.orchestra.run.smithi078.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 4.0 MB/s | 1.5 MB 00:00 2024-09-16T00:00:55.737 INFO:teuthology.orchestra.run.smithi148.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.6 MB/s | 228 kB 00:00 2024-09-16T00:00:55.763 INFO:teuthology.orchestra.run.smithi148.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 539 kB/s | 27 kB 00:00 2024-09-16T00:00:55.796 INFO:teuthology.orchestra.run.smithi148.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 542 kB/s | 32 kB 00:00 2024-09-16T00:00:55.846 INFO:teuthology.orchestra.run.smithi148.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-16T00:00:55.871 INFO:teuthology.orchestra.run.smithi148.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.1 MB/s | 163 kB 00:00 2024-09-16T00:00:55.914 INFO:teuthology.orchestra.run.smithi148.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.9 MB/s | 127 kB 00:00 2024-09-16T00:00:55.956 INFO:teuthology.orchestra.run.smithi148.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-16T00:00:56.039 INFO:teuthology.orchestra.run.smithi148.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 542 kB/s | 45 kB 00:00 2024-09-16T00:00:56.047 INFO:teuthology.orchestra.run.smithi078.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-16T00:00:56.106 INFO:teuthology.orchestra.run.smithi148.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.0 MB/s | 136 kB 00:00 2024-09-16T00:00:56.190 INFO:teuthology.orchestra.run.smithi148.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-16T00:00:56.282 INFO:teuthology.orchestra.run.smithi148.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.4 MB/s | 320 kB 00:00 2024-09-16T00:00:56.330 INFO:teuthology.orchestra.run.smithi193.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-16T00:00:56.373 INFO:teuthology.orchestra.run.smithi148.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.2 MB/s | 300 kB 00:00 2024-09-16T00:00:56.516 INFO:teuthology.orchestra.run.smithi148.stdout:(24/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 8.9 MB/s | 5.3 MB 00:00 2024-09-16T00:00:56.541 INFO:teuthology.orchestra.run.smithi148.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 595 kB/s | 99 kB 00:00 2024-09-16T00:00:56.599 INFO:teuthology.orchestra.run.smithi148.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-16T00:00:56.697 INFO:teuthology.orchestra.run.smithi193.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-09-16T00:00:56.833 INFO:teuthology.orchestra.run.smithi148.stdout:(27/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-16T00:00:56.892 INFO:teuthology.orchestra.run.smithi148.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 547 kB/s | 32 kB 00:00 2024-09-16T00:00:56.931 INFO:teuthology.orchestra.run.smithi078.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 7.0 MB/s | 21 MB 00:03 2024-09-16T00:00:56.950 INFO:teuthology.orchestra.run.smithi148.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.5 MB/s | 146 kB 00:00 2024-09-16T00:00:56.989 INFO:teuthology.orchestra.run.smithi078.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 432 kB/s | 25 kB 00:00 2024-09-16T00:00:56.992 INFO:teuthology.orchestra.run.smithi148.stdout:(30/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 434 kB/s | 170 kB 00:00 2024-09-16T00:00:57.131 INFO:teuthology.orchestra.run.smithi078.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 9.8 MB/s | 11 MB 00:01 2024-09-16T00:00:57.172 INFO:teuthology.orchestra.run.smithi193.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 1.9 MB/s | 910 kB 00:00 2024-09-16T00:00:57.240 INFO:teuthology.orchestra.run.smithi078.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 2.1 MB/s | 228 kB 00:00 2024-09-16T00:00:57.268 INFO:teuthology.orchestra.run.smithi148.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 11 MB/s | 3.6 MB 00:00 2024-09-16T00:00:57.281 INFO:teuthology.orchestra.run.smithi193.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-16T00:00:57.290 INFO:teuthology.orchestra.run.smithi078.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 542 kB/s | 27 kB 00:00 2024-09-16T00:00:57.352 INFO:teuthology.orchestra.run.smithi148.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.9 MB/s | 249 kB 00:00 2024-09-16T00:00:57.381 INFO:teuthology.orchestra.run.smithi078.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 346 kB/s | 32 kB 00:00 2024-09-16T00:00:57.507 INFO:teuthology.orchestra.run.smithi078.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 5.9 MB/s | 751 kB 00:00 2024-09-16T00:00:57.574 INFO:teuthology.orchestra.run.smithi078.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.4 MB/s | 163 kB 00:00 2024-09-16T00:00:57.641 INFO:teuthology.orchestra.run.smithi078.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.9 MB/s | 127 kB 00:00 2024-09-16T00:00:57.724 INFO:teuthology.orchestra.run.smithi078.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-16T00:00:57.843 INFO:teuthology.orchestra.run.smithi148.stdout:(33/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 17 MB/s | 49 MB 00:02 2024-09-16T00:00:57.869 INFO:teuthology.orchestra.run.smithi148.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 95 kB/s | 49 kB 00:00 2024-09-16T00:00:57.931 INFO:teuthology.orchestra.run.smithi193.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-09-16T00:00:57.935 INFO:teuthology.orchestra.run.smithi148.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 180 kB/s | 16 kB 00:00 2024-09-16T00:00:57.969 INFO:teuthology.orchestra.run.smithi148.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 2.8 MB/s | 285 kB 00:00 2024-09-16T00:00:58.077 INFO:teuthology.orchestra.run.smithi148.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 168 kB/s | 18 kB 00:00 2024-09-16T00:00:58.120 INFO:teuthology.orchestra.run.smithi148.stdout:(38/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 4.1 MB/s | 765 kB 00:00 2024-09-16T00:00:58.173 INFO:teuthology.orchestra.run.smithi193.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.0 MB/s | 4.9 MB 00:02 2024-09-16T00:00:58.245 INFO:teuthology.orchestra.run.smithi148.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 5.9 MB/s | 7.4 MB 00:01 2024-09-16T00:00:58.256 INFO:teuthology.orchestra.run.smithi193.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-16T00:00:58.270 INFO:teuthology.orchestra.run.smithi148.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 415 kB/s | 80 kB 00:00 2024-09-16T00:00:58.400 INFO:teuthology.orchestra.run.smithi078.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 7.9 MB/s | 5.3 MB 00:00 2024-09-16T00:00:58.420 INFO:teuthology.orchestra.run.smithi148.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-16T00:00:58.454 INFO:teuthology.orchestra.run.smithi148.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 917 kB/s | 191 kB 00:00 2024-09-16T00:00:58.479 INFO:teuthology.orchestra.run.smithi148.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-16T00:00:58.504 INFO:teuthology.orchestra.run.smithi148.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-16T00:00:58.506 INFO:teuthology.orchestra.run.smithi193.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-09-16T00:00:58.530 INFO:teuthology.orchestra.run.smithi148.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-16T00:00:58.555 INFO:teuthology.orchestra.run.smithi148.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-16T00:00:58.557 INFO:teuthology.orchestra.run.smithi193.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-16T00:00:58.589 INFO:teuthology.orchestra.run.smithi148.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-16T00:00:58.639 INFO:teuthology.orchestra.run.smithi148.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-16T00:00:58.707 INFO:teuthology.orchestra.run.smithi193.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-16T00:00:58.773 INFO:teuthology.orchestra.run.smithi148.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 803 kB/s | 107 kB 00:00 2024-09-16T00:00:58.814 INFO:teuthology.orchestra.run.smithi148.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-16T00:00:58.833 INFO:teuthology.orchestra.run.smithi078.stdout:(19/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 26 MB/s | 49 MB 00:01 2024-09-16T00:00:58.857 INFO:teuthology.orchestra.run.smithi193.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-16T00:00:58.858 INFO:teuthology.orchestra.run.smithi078.stdout:(20/156): python3-ceph-argparse-19.1.1-326.g892 98 kB/s | 45 kB 00:00 2024-09-16T00:00:58.864 INFO:teuthology.orchestra.run.smithi148.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-16T00:00:58.884 INFO:teuthology.orchestra.run.smithi078.stdout:(21/156): python3-ceph-common-19.1.1-326.g8921c 2.7 MB/s | 136 kB 00:00 2024-09-16T00:00:58.907 INFO:teuthology.orchestra.run.smithi193.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-16T00:00:58.909 INFO:teuthology.orchestra.run.smithi078.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 3.1 MB/s | 161 kB 00:00 2024-09-16T00:00:58.923 INFO:teuthology.orchestra.run.smithi148.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-16T00:00:58.934 INFO:teuthology.orchestra.run.smithi078.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 6.2 MB/s | 320 kB 00:00 2024-09-16T00:00:58.958 INFO:teuthology.orchestra.run.smithi193.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-16T00:00:58.993 INFO:teuthology.orchestra.run.smithi148.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 943 kB 00:00 2024-09-16T00:00:58.994 INFO:teuthology.orchestra.run.smithi078.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.9 MB/s | 300 kB 00:00 2024-09-16T00:00:58.994 INFO:teuthology.orchestra.run.smithi078.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T00:00:58.999 INFO:teuthology.orchestra.run.smithi148.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 244 kB/s | 33 kB 00:00 2024-09-16T00:00:59.010 INFO:teuthology.orchestra.run.smithi078.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T00:00:59.032 INFO:teuthology.orchestra.run.smithi148.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-16T00:00:59.058 INFO:teuthology.orchestra.run.smithi148.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-16T00:00:59.067 INFO:teuthology.orchestra.run.smithi193.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-16T00:00:59.077 INFO:teuthology.orchestra.run.smithi078.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-16T00:00:59.108 INFO:teuthology.orchestra.run.smithi148.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-16T00:00:59.117 INFO:teuthology.orchestra.run.smithi193.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-09-16T00:00:59.136 INFO:teuthology.orchestra.run.smithi078.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-16T00:00:59.150 INFO:teuthology.orchestra.run.smithi148.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-16T00:00:59.152 INFO:teuthology.orchestra.run.smithi078.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 425 kB/s | 32 kB 00:00 2024-09-16T00:00:59.186 INFO:teuthology.orchestra.run.smithi078.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-16T00:00:59.292 INFO:teuthology.orchestra.run.smithi148.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-09-16T00:00:59.342 INFO:teuthology.orchestra.run.smithi148.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 490 kB/s | 24 kB 00:00 2024-09-16T00:00:59.376 INFO:teuthology.orchestra.run.smithi148.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 2.5 MB/s | 581 kB 00:00 2024-09-16T00:00:59.401 INFO:teuthology.orchestra.run.smithi148.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-16T00:00:59.470 INFO:teuthology.orchestra.run.smithi078.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 26 MB/s | 7.4 MB 00:00 2024-09-16T00:00:59.476 INFO:teuthology.orchestra.run.smithi148.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-09-16T00:00:59.502 INFO:teuthology.orchestra.run.smithi148.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-16T00:00:59.520 INFO:teuthology.orchestra.run.smithi078.stdout:(32/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 9.7 MB/s | 3.6 MB 00:00 2024-09-16T00:00:59.527 INFO:teuthology.orchestra.run.smithi148.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 764 kB/s | 38 kB 00:00 2024-09-16T00:00:59.545 INFO:teuthology.orchestra.run.smithi078.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 3.3 MB/s | 249 kB 00:00 2024-09-16T00:00:59.562 INFO:teuthology.orchestra.run.smithi078.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 1.2 MB/s | 49 kB 00:00 2024-09-16T00:00:59.721 INFO:teuthology.orchestra.run.smithi078.stdout:(35/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 4.2 MB/s | 17 MB 00:03 2024-09-16T00:00:59.744 INFO:teuthology.orchestra.run.smithi148.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-09-16T00:00:59.746 INFO:teuthology.orchestra.run.smithi078.stdout:(36/156): ceph-prometheus-alerts-19.1.1-326.g89 82 kB/s | 16 kB 00:00 2024-09-16T00:00:59.772 INFO:teuthology.orchestra.run.smithi078.stdout:(37/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 1.3 MB/s | 285 kB 00:00 2024-09-16T00:00:59.797 INFO:teuthology.orchestra.run.smithi078.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 361 kB/s | 18 kB 00:00 2024-09-16T00:00:59.856 INFO:teuthology.orchestra.run.smithi078.stdout:(39/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 5.6 MB/s | 765 kB 00:00 2024-09-16T00:00:59.998 INFO:teuthology.orchestra.run.smithi078.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 354 kB/s | 80 kB 00:00 2024-09-16T00:01:00.056 INFO:teuthology.orchestra.run.smithi078.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 956 kB/s | 191 kB 00:00 2024-09-16T00:01:00.106 INFO:teuthology.orchestra.run.smithi078.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-16T00:01:00.140 INFO:teuthology.orchestra.run.smithi078.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-16T00:01:00.144 INFO:teuthology.orchestra.run.smithi148.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-09-16T00:01:00.165 INFO:teuthology.orchestra.run.smithi078.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-16T00:01:00.199 INFO:teuthology.orchestra.run.smithi078.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-16T00:01:00.224 INFO:teuthology.orchestra.run.smithi078.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-16T00:01:00.258 INFO:teuthology.orchestra.run.smithi078.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-16T00:01:00.259 INFO:teuthology.orchestra.run.smithi193.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 1.9 MB/s | 5.8 MB 00:02 2024-09-16T00:01:00.325 INFO:teuthology.orchestra.run.smithi078.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-09-16T00:01:00.386 INFO:teuthology.orchestra.run.smithi148.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.7 MB/s | 910 kB 00:00 2024-09-16T00:01:00.401 INFO:teuthology.orchestra.run.smithi193.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-16T00:01:00.445 INFO:teuthology.orchestra.run.smithi148.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-16T00:01:00.466 INFO:teuthology.orchestra.run.smithi078.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 756 kB/s | 107 kB 00:00 2024-09-16T00:01:00.503 INFO:teuthology.orchestra.run.smithi148.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-09-16T00:01:00.525 INFO:teuthology.orchestra.run.smithi078.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-16T00:01:00.542 INFO:teuthology.orchestra.run.smithi078.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 288 kB/s | 22 kB 00:00 2024-09-16T00:01:00.551 INFO:teuthology.orchestra.run.smithi193.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-16T00:01:00.584 INFO:teuthology.orchestra.run.smithi078.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 790 kB/s | 33 kB 00:00 2024-09-16T00:01:00.618 INFO:teuthology.orchestra.run.smithi078.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-09-16T00:01:00.635 INFO:teuthology.orchestra.run.smithi193.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 928 kB/s | 77 kB 00:00 2024-09-16T00:01:00.643 INFO:teuthology.orchestra.run.smithi078.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-16T00:01:00.660 INFO:teuthology.orchestra.run.smithi078.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 434 kB/s | 18 kB 00:00 2024-09-16T00:01:00.685 INFO:teuthology.orchestra.run.smithi193.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-16T00:01:00.687 INFO:teuthology.orchestra.run.smithi148.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.2 MB/s | 4.9 MB 00:01 2024-09-16T00:01:00.693 INFO:teuthology.orchestra.run.smithi078.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-16T00:01:00.744 INFO:teuthology.orchestra.run.smithi078.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-16T00:01:00.745 INFO:teuthology.orchestra.run.smithi148.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-16T00:01:00.769 INFO:teuthology.orchestra.run.smithi078.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-09-16T00:01:00.812 INFO:teuthology.orchestra.run.smithi148.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-09-16T00:01:00.860 INFO:teuthology.orchestra.run.smithi193.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-09-16T00:01:00.871 INFO:teuthology.orchestra.run.smithi148.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-16T00:01:00.911 INFO:teuthology.orchestra.run.smithi078.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-16T00:01:00.936 INFO:teuthology.orchestra.run.smithi078.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-16T00:01:00.954 INFO:teuthology.orchestra.run.smithi148.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-16T00:01:00.961 INFO:teuthology.orchestra.run.smithi078.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-16T00:01:00.977 INFO:teuthology.orchestra.run.smithi193.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-16T00:01:00.987 INFO:teuthology.orchestra.run.smithi078.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-16T00:01:01.045 INFO:teuthology.orchestra.run.smithi078.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-16T00:01:01.055 INFO:teuthology.orchestra.run.smithi148.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-09-16T00:01:01.070 INFO:teuthology.orchestra.run.smithi078.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-16T00:01:01.096 INFO:teuthology.orchestra.run.smithi078.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-16T00:01:01.106 INFO:teuthology.orchestra.run.smithi148.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-16T00:01:01.161 INFO:teuthology.orchestra.run.smithi193.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-09-16T00:01:01.164 INFO:teuthology.orchestra.run.smithi148.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-09-16T00:01:01.211 INFO:teuthology.orchestra.run.smithi193.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T00:01:01.247 INFO:teuthology.orchestra.run.smithi148.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-16T00:01:01.261 INFO:teuthology.orchestra.run.smithi193.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-09-16T00:01:01.290 INFO:teuthology.orchestra.run.smithi148.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 834 kB/s | 35 kB 00:00 2024-09-16T00:01:01.337 INFO:teuthology.orchestra.run.smithi193.stdout:(91/156): python3-babel-2.9.1-2.el9.noarch.rpm 1.9 MB/s | 6.0 MB 00:03 2024-09-16T00:01:01.388 INFO:teuthology.orchestra.run.smithi193.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-09-16T00:01:01.438 INFO:teuthology.orchestra.run.smithi193.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-16T00:01:01.589 INFO:teuthology.orchestra.run.smithi193.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-16T00:01:01.605 INFO:teuthology.orchestra.run.smithi078.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-16T00:01:01.647 INFO:teuthology.orchestra.run.smithi193.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 617 kB/s | 36 kB 00:00 2024-09-16T00:01:01.697 INFO:teuthology.orchestra.run.smithi193.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-16T00:01:01.805 INFO:teuthology.orchestra.run.smithi193.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.9 MB/s | 5.1 MB 00:02 2024-09-16T00:01:01.823 INFO:teuthology.orchestra.run.smithi148.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-09-16T00:01:01.856 INFO:teuthology.orchestra.run.smithi193.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T00:01:01.898 INFO:teuthology.orchestra.run.smithi148.stdout:(82/156): postgresql-server-13.16-1.el9.x86_64. 4.0 MB/s | 5.8 MB 00:01 2024-09-16T00:01:01.914 INFO:teuthology.orchestra.run.smithi193.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-16T00:01:01.948 INFO:teuthology.orchestra.run.smithi148.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-16T00:01:01.955 INFO:teuthology.orchestra.run.smithi078.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-09-16T00:01:01.965 INFO:teuthology.orchestra.run.smithi193.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 881 kB/s | 96 kB 00:00 2024-09-16T00:01:01.999 INFO:teuthology.orchestra.run.smithi148.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-16T00:01:02.024 INFO:teuthology.orchestra.run.smithi193.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 586 kB/s | 64 kB 00:00 2024-09-16T00:01:02.024 INFO:teuthology.orchestra.run.smithi148.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-09-16T00:01:02.047 INFO:teuthology.orchestra.run.smithi078.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 5.1 MB/s | 4.9 MB 00:00 2024-09-16T00:01:02.049 INFO:teuthology.orchestra.run.smithi148.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-09-16T00:01:02.057 INFO:teuthology.orchestra.run.smithi193.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 486 kB/s | 45 kB 00:00 2024-09-16T00:01:02.097 INFO:teuthology.orchestra.run.smithi078.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-09-16T00:01:02.099 INFO:teuthology.orchestra.run.smithi148.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T00:01:02.125 INFO:teuthology.orchestra.run.smithi148.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-16T00:01:02.164 INFO:teuthology.orchestra.run.smithi078.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-09-16T00:01:02.175 INFO:teuthology.orchestra.run.smithi148.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T00:01:02.209 INFO:teuthology.orchestra.run.smithi148.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-16T00:01:02.223 INFO:teuthology.orchestra.run.smithi078.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-09-16T00:01:02.234 INFO:teuthology.orchestra.run.smithi148.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-09-16T00:01:02.281 INFO:teuthology.orchestra.run.smithi078.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-16T00:01:02.284 INFO:teuthology.orchestra.run.smithi148.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-16T00:01:02.343 INFO:teuthology.orchestra.run.smithi148.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 760 kB/s | 44 kB 00:00 2024-09-16T00:01:02.410 INFO:teuthology.orchestra.run.smithi148.stdout:(94/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-09-16T00:01:02.443 INFO:teuthology.orchestra.run.smithi148.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-16T00:01:02.469 INFO:teuthology.orchestra.run.smithi148.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-16T00:01:02.473 INFO:teuthology.orchestra.run.smithi078.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-09-16T00:01:02.494 INFO:teuthology.orchestra.run.smithi148.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-16T00:01:02.523 INFO:teuthology.orchestra.run.smithi078.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-16T00:01:02.544 INFO:teuthology.orchestra.run.smithi148.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T00:01:02.569 INFO:teuthology.orchestra.run.smithi148.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-09-16T00:01:02.599 INFO:teuthology.orchestra.run.smithi193.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-09-16T00:01:02.605 INFO:teuthology.orchestra.run.smithi148.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-16T00:01:02.607 INFO:teuthology.orchestra.run.smithi078.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-16T00:01:02.628 INFO:teuthology.orchestra.run.smithi148.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-16T00:01:02.654 INFO:teuthology.orchestra.run.smithi148.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-16T00:01:02.666 INFO:teuthology.orchestra.run.smithi193.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-16T00:01:02.679 INFO:teuthology.orchestra.run.smithi148.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-09-16T00:01:02.700 INFO:teuthology.orchestra.run.smithi193.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-16T00:01:02.725 INFO:teuthology.orchestra.run.smithi193.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-16T00:01:02.754 INFO:teuthology.orchestra.run.smithi148.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-16T00:01:02.796 INFO:teuthology.orchestra.run.smithi148.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-16T00:01:02.830 INFO:teuthology.orchestra.run.smithi148.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-09-16T00:01:02.841 INFO:teuthology.orchestra.run.smithi078.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.0 MB/s | 248 kB 00:00 2024-09-16T00:01:02.855 INFO:teuthology.orchestra.run.smithi148.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-09-16T00:01:02.889 INFO:teuthology.orchestra.run.smithi148.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T00:01:02.891 INFO:teuthology.orchestra.run.smithi078.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-16T00:01:02.900 INFO:teuthology.orchestra.run.smithi193.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-16T00:01:02.922 INFO:teuthology.orchestra.run.smithi148.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-09-16T00:01:02.926 INFO:teuthology.orchestra.run.smithi193.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T00:01:02.941 INFO:teuthology.orchestra.run.smithi078.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-16T00:01:02.943 INFO:teuthology.orchestra.run.smithi193.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-16T00:01:02.947 INFO:teuthology.orchestra.run.smithi148.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T00:01:02.968 INFO:teuthology.orchestra.run.smithi193.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T00:01:02.973 INFO:teuthology.orchestra.run.smithi148.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-16T00:01:03.001 INFO:teuthology.orchestra.run.smithi193.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 914 kB/s | 863 kB 00:00 2024-09-16T00:01:03.015 INFO:teuthology.orchestra.run.smithi148.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-16T00:01:03.033 INFO:teuthology.orchestra.run.smithi078.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-16T00:01:03.043 INFO:teuthology.orchestra.run.smithi193.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-16T00:01:03.057 INFO:teuthology.orchestra.run.smithi148.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-16T00:01:03.077 INFO:teuthology.orchestra.run.smithi193.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 17 MB/s | 548 kB 00:00 2024-09-16T00:01:03.082 INFO:teuthology.orchestra.run.smithi148.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:01:03.092 INFO:teuthology.orchestra.run.smithi078.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-09-16T00:01:03.103 INFO:teuthology.orchestra.run.smithi193.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:01:03.107 INFO:teuthology.orchestra.run.smithi148.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:01:03.128 INFO:teuthology.orchestra.run.smithi193.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:01:03.133 INFO:teuthology.orchestra.run.smithi148.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-16T00:01:03.153 INFO:teuthology.orchestra.run.smithi193.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-16T00:01:03.158 INFO:teuthology.orchestra.run.smithi148.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-16T00:01:03.179 INFO:teuthology.orchestra.run.smithi193.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-16T00:01:03.200 INFO:teuthology.orchestra.run.smithi148.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-16T00:01:03.200 INFO:teuthology.orchestra.run.smithi078.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 6.5 MB/s | 6.0 MB 00:00 2024-09-16T00:01:03.221 INFO:teuthology.orchestra.run.smithi193.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.7 MB/s | 839 kB 00:00 2024-09-16T00:01:03.267 INFO:teuthology.orchestra.run.smithi078.stdout:(82/156): postgresql-server-13.16-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-09-16T00:01:03.279 INFO:teuthology.orchestra.run.smithi193.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-09-16T00:01:03.281 INFO:teuthology.orchestra.run.smithi148.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.7 MB/s | 303 kB 00:00 2024-09-16T00:01:03.292 INFO:teuthology.orchestra.run.smithi078.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-16T00:01:03.305 INFO:teuthology.orchestra.run.smithi193.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-16T00:01:03.309 INFO:teuthology.orchestra.run.smithi148.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 7.7 MB/s | 220 kB 00:00 2024-09-16T00:01:03.330 INFO:teuthology.orchestra.run.smithi193.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T00:01:03.343 INFO:teuthology.orchestra.run.smithi078.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T00:01:03.356 INFO:teuthology.orchestra.run.smithi193.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-16T00:01:03.368 INFO:teuthology.orchestra.run.smithi078.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-16T00:01:03.381 INFO:teuthology.orchestra.run.smithi193.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-16T00:01:03.384 INFO:teuthology.orchestra.run.smithi148.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-09-16T00:01:03.393 INFO:teuthology.orchestra.run.smithi078.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-16T00:01:03.406 INFO:teuthology.orchestra.run.smithi193.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-09-16T00:01:03.420 INFO:teuthology.orchestra.run.smithi148.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-16T00:01:03.431 INFO:teuthology.orchestra.run.smithi193.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 936 kB/s | 23 kB 00:00 2024-09-16T00:01:03.435 INFO:teuthology.orchestra.run.smithi148.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 642 kB/s | 11 kB 00:00 2024-09-16T00:01:03.444 INFO:teuthology.orchestra.run.smithi078.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T00:01:03.452 INFO:teuthology.orchestra.run.smithi148.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-16T00:01:03.457 INFO:teuthology.orchestra.run.smithi193.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-16T00:01:03.469 INFO:teuthology.orchestra.run.smithi148.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-16T00:01:03.469 INFO:teuthology.orchestra.run.smithi078.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-16T00:01:03.482 INFO:teuthology.orchestra.run.smithi193.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-16T00:01:03.486 INFO:teuthology.orchestra.run.smithi148.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-16T00:01:03.502 INFO:teuthology.orchestra.run.smithi078.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-16T00:01:03.511 INFO:teuthology.orchestra.run.smithi148.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-16T00:01:03.519 INFO:teuthology.orchestra.run.smithi078.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T00:01:03.541 INFO:teuthology.orchestra.run.smithi193.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-16T00:01:03.544 INFO:teuthology.orchestra.run.smithi078.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-09-16T00:01:03.566 INFO:teuthology.orchestra.run.smithi193.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:01:03.570 INFO:teuthology.orchestra.run.smithi148.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-09-16T00:01:03.587 INFO:teuthology.orchestra.run.smithi078.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 995 kB/s | 42 kB 00:00 2024-09-16T00:01:03.591 INFO:teuthology.orchestra.run.smithi193.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T00:01:03.595 INFO:teuthology.orchestra.run.smithi148.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:01:03.620 INFO:teuthology.orchestra.run.smithi148.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T00:01:03.625 INFO:teuthology.orchestra.run.smithi193.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-16T00:01:03.628 INFO:teuthology.orchestra.run.smithi078.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-16T00:01:03.650 INFO:teuthology.orchestra.run.smithi193.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-16T00:01:03.654 INFO:teuthology.orchestra.run.smithi148.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-16T00:01:03.676 INFO:teuthology.orchestra.run.smithi193.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-16T00:01:03.679 INFO:teuthology.orchestra.run.smithi148.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-16T00:01:03.687 INFO:teuthology.orchestra.run.smithi078.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-16T00:01:03.701 INFO:teuthology.orchestra.run.smithi193.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-16T00:01:03.704 INFO:teuthology.orchestra.run.smithi148.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-16T00:01:03.726 INFO:teuthology.orchestra.run.smithi193.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-16T00:01:03.729 INFO:teuthology.orchestra.run.smithi078.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-09-16T00:01:03.730 INFO:teuthology.orchestra.run.smithi148.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-16T00:01:03.751 INFO:teuthology.orchestra.run.smithi193.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-16T00:01:03.755 INFO:teuthology.orchestra.run.smithi148.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-16T00:01:03.771 INFO:teuthology.orchestra.run.smithi078.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-16T00:01:03.777 INFO:teuthology.orchestra.run.smithi193.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-16T00:01:03.780 INFO:teuthology.orchestra.run.smithi148.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-16T00:01:03.802 INFO:teuthology.orchestra.run.smithi193.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:01:03.806 INFO:teuthology.orchestra.run.smithi148.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-16T00:01:03.828 INFO:teuthology.orchestra.run.smithi193.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-16T00:01:03.831 INFO:teuthology.orchestra.run.smithi148.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:01:03.853 INFO:teuthology.orchestra.run.smithi193.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-16T00:01:03.855 INFO:teuthology.orchestra.run.smithi078.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.5 MB/s | 304 kB 00:00 2024-09-16T00:01:03.864 INFO:teuthology.orchestra.run.smithi148.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-16T00:01:03.878 INFO:teuthology.orchestra.run.smithi193.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-16T00:01:03.890 INFO:teuthology.orchestra.run.smithi148.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-16T00:01:03.903 INFO:teuthology.orchestra.run.smithi193.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T00:01:03.905 INFO:teuthology.orchestra.run.smithi078.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T00:01:03.915 INFO:teuthology.orchestra.run.smithi148.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-16T00:01:03.929 INFO:teuthology.orchestra.run.smithi193.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T00:01:03.940 INFO:teuthology.orchestra.run.smithi148.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-16T00:01:03.946 INFO:teuthology.orchestra.run.smithi193.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-16T00:01:03.955 INFO:teuthology.orchestra.run.smithi078.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-16T00:01:03.965 INFO:teuthology.orchestra.run.smithi148.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-16T00:01:03.979 INFO:teuthology.orchestra.run.smithi193.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-16T00:01:03.991 INFO:teuthology.orchestra.run.smithi148.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 798 kB/s | 20 kB 00:00 2024-09-16T00:01:04.005 INFO:teuthology.orchestra.run.smithi193.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T00:01:04.006 INFO:teuthology.orchestra.run.smithi078.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T00:01:04.016 INFO:teuthology.orchestra.run.smithi148.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-16T00:01:04.041 INFO:teuthology.orchestra.run.smithi148.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-16T00:01:04.056 INFO:teuthology.orchestra.run.smithi078.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-16T00:01:04.063 INFO:teuthology.orchestra.run.smithi193.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-16T00:01:04.109 INFO:teuthology.orchestra.run.smithi148.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-16T00:01:04.148 INFO:teuthology.orchestra.run.smithi078.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-09-16T00:01:04.323 INFO:teuthology.orchestra.run.smithi078.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-09-16T00:01:04.339 INFO:teuthology.orchestra.run.smithi193.stdout:(148/156): librados2-19.1.1-326.g8921c8ef.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-16T00:01:04.373 INFO:teuthology.orchestra.run.smithi078.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T00:01:04.392 INFO:teuthology.orchestra.run.smithi148.stdout:(148/156): librados2-19.1.1-326.g8921c8ef.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-16T00:01:04.397 INFO:teuthology.orchestra.run.smithi193.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 9.5 MB/s | 3.2 MB 00:00 2024-09-16T00:01:04.398 INFO:teuthology.orchestra.run.smithi078.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-16T00:01:04.424 INFO:teuthology.orchestra.run.smithi078.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-16T00:01:04.501 INFO:teuthology.orchestra.run.smithi148.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.1 MB/s | 3.2 MB 00:00 2024-09-16T00:01:04.541 INFO:teuthology.orchestra.run.smithi078.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-16T00:01:04.566 INFO:teuthology.orchestra.run.smithi078.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T00:01:04.608 INFO:teuthology.orchestra.run.smithi078.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-16T00:01:04.634 INFO:teuthology.orchestra.run.smithi078.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T00:01:04.659 INFO:teuthology.orchestra.run.smithi078.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-16T00:01:04.717 INFO:teuthology.orchestra.run.smithi078.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-16T00:01:04.759 INFO:teuthology.orchestra.run.smithi078.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-16T00:01:04.776 INFO:teuthology.orchestra.run.smithi148.stdout:(150/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-09-16T00:01:04.785 INFO:teuthology.orchestra.run.smithi078.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:01:04.810 INFO:teuthology.orchestra.run.smithi078.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:01:04.835 INFO:teuthology.orchestra.run.smithi078.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-16T00:01:04.869 INFO:teuthology.orchestra.run.smithi078.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-16T00:01:04.901 INFO:teuthology.orchestra.run.smithi148.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 327 kB/s | 41 kB 00:00 2024-09-16T00:01:04.902 INFO:teuthology.orchestra.run.smithi078.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-16T00:01:04.936 INFO:teuthology.orchestra.run.smithi078.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-16T00:01:04.943 INFO:teuthology.orchestra.run.smithi148.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 4.1 MB/s | 1.8 MB 00:00 2024-09-16T00:01:05.003 INFO:teuthology.orchestra.run.smithi078.stdout:(120/156): python3-numpy-1.20.1-5.el9.x86_64.rp 2.7 MB/s | 5.1 MB 00:01 2024-09-16T00:01:05.028 INFO:teuthology.orchestra.run.smithi078.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-16T00:01:05.052 INFO:teuthology.orchestra.run.smithi148.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 3.5 MB/s | 2.3 MB 00:00 2024-09-16T00:01:05.062 INFO:teuthology.orchestra.run.smithi078.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.4 MB/s | 144 kB 00:00 2024-09-16T00:01:05.077 INFO:teuthology.orchestra.run.smithi148.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.4 MB/s | 249 kB 00:00 2024-09-16T00:01:05.087 INFO:teuthology.orchestra.run.smithi078.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 458 kB/s | 11 kB 00:00 2024-09-16T00:01:05.112 INFO:teuthology.orchestra.run.smithi078.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 719 kB/s | 18 kB 00:00 2024-09-16T00:01:05.137 INFO:teuthology.orchestra.run.smithi078.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-16T00:01:05.163 INFO:teuthology.orchestra.run.smithi078.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-16T00:01:05.188 INFO:teuthology.orchestra.run.smithi078.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 779 kB/s | 20 kB 00:00 2024-09-16T00:01:05.247 INFO:teuthology.orchestra.run.smithi078.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-16T00:01:05.272 INFO:teuthology.orchestra.run.smithi078.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:01:05.306 INFO:teuthology.orchestra.run.smithi078.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-16T00:01:05.331 INFO:teuthology.orchestra.run.smithi078.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-16T00:01:05.357 INFO:teuthology.orchestra.run.smithi078.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-16T00:01:05.357 INFO:teuthology.orchestra.run.smithi193.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 2.3 MB/s | 2.3 MB 00:01 2024-09-16T00:01:05.382 INFO:teuthology.orchestra.run.smithi078.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-16T00:01:05.398 INFO:teuthology.orchestra.run.smithi193.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-16T00:01:05.399 INFO:teuthology.orchestra.run.smithi078.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 390 kB/s | 16 kB 00:00 2024-09-16T00:01:05.424 INFO:teuthology.orchestra.run.smithi078.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-16T00:01:05.441 INFO:teuthology.orchestra.run.smithi078.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-09-16T00:01:05.458 INFO:teuthology.orchestra.run.smithi078.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 916 kB/s | 31 kB 00:00 2024-09-16T00:01:05.475 INFO:teuthology.orchestra.run.smithi078.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.4 MB/s | 59 kB 00:00 2024-09-16T00:01:05.517 INFO:teuthology.orchestra.run.smithi078.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-16T00:01:05.534 INFO:teuthology.orchestra.run.smithi078.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-16T00:01:05.559 INFO:teuthology.orchestra.run.smithi078.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-16T00:01:05.584 INFO:teuthology.orchestra.run.smithi078.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T00:01:05.609 INFO:teuthology.orchestra.run.smithi078.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T00:01:05.635 INFO:teuthology.orchestra.run.smithi078.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 795 kB/s | 20 kB 00:00 2024-09-16T00:01:05.644 INFO:teuthology.orchestra.run.smithi148.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.9 MB/s | 1.4 MB 00:00 2024-09-16T00:01:05.661 INFO:teuthology.orchestra.run.smithi078.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-16T00:01:05.685 INFO:teuthology.orchestra.run.smithi078.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T00:01:05.736 INFO:teuthology.orchestra.run.smithi078.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-16T00:01:05.932 INFO:teuthology.orchestra.run.smithi193.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 71 kB/s | 41 kB 00:00 2024-09-16T00:01:05.978 INFO:teuthology.orchestra.run.smithi078.stdout:(148/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-09-16T00:01:06.103 INFO:teuthology.orchestra.run.smithi078.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.6 MB/s | 3.2 MB 00:00 2024-09-16T00:01:06.157 INFO:teuthology.orchestra.run.smithi193.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 328 kB/s | 249 kB 00:00 2024-09-16T00:01:06.227 INFO:teuthology.orchestra.run.smithi148.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-16T00:01:06.232 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:01:06.232 INFO:teuthology.orchestra.run.smithi148.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-16T00:01:06.236 INFO:teuthology.orchestra.run.smithi078.stdout:(150/156): librados2-19.1.1-326.g8921c8ef.el9.x 6.2 MB/s | 3.4 MB 00:00 2024-09-16T00:01:06.362 INFO:teuthology.orchestra.run.smithi078.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 326 kB/s | 41 kB 00:00 2024-09-16T00:01:06.462 INFO:teuthology.orchestra.run.smithi078.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 4.8 MB/s | 2.3 MB 00:00 2024-09-16T00:01:06.504 INFO:teuthology.orchestra.run.smithi078.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 1.7 MB/s | 249 kB 00:00 2024-09-16T00:01:06.546 INFO:teuthology.orchestra.run.smithi078.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.1 MB/s | 1.8 MB 00:00 2024-09-16T00:01:06.891 INFO:teuthology.orchestra.run.smithi193.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-16T00:01:07.113 INFO:teuthology.orchestra.run.smithi078.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 2.4 MB 00:00 2024-09-16T00:01:07.191 INFO:teuthology.orchestra.run.smithi193.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-16T00:01:07.205 INFO:teuthology.orchestra.run.smithi078.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 1.8 MB/s | 1.4 MB 00:00 2024-09-16T00:01:07.210 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:01:07.210 INFO:teuthology.orchestra.run.smithi078.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-16T00:01:07.440 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-16T00:01:07.592 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-16T00:01:07.592 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-16T00:01:08.401 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-09-16T00:01:08.551 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-09-16T00:01:08.551 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-09-16T00:01:09.389 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-16T00:01:09.389 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-16T00:01:09.390 INFO:teuthology.orchestra.run.smithi193.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.0 MB/s | 16 MB 00:08 2024-09-16T00:01:09.396 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:01:09.397 INFO:teuthology.orchestra.run.smithi193.stdout:Total 11 MB/s | 226 MB 00:20 2024-09-16T00:01:10.376 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-09-16T00:01:10.376 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-09-16T00:01:10.620 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-16T00:01:10.770 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-16T00:01:10.771 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-16T00:01:12.562 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-16T00:01:12.562 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-16T00:01:15.602 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-16T00:01:15.739 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:01:15.861 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:01:16.104 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:01:16.141 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:01:16.166 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:01:16.218 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:01:16.548 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:01:16.650 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:01:16.772 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:01:16.809 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:01:16.811 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-09-16T00:01:16.901 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:01:16.926 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:01:16.938 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:01:16.971 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:01:17.035 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:01:17.068 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:01:17.088 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:01:17.093 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:01:17.143 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:01:17.154 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:01:17.172 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:01:17.276 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:01:17.297 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:01:17.343 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:01:17.374 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:01:17.397 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:01:17.500 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:01:17.549 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:01:17.603 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:01:17.678 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:01:17.734 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:01:17.763 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:01:17.797 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:01:17.846 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:01:17.877 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:01:17.890 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:17.973 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:18.060 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:01:18.126 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:01:18.151 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:01:18.198 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:18.254 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:01:18.276 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:01:18.309 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:01:18.339 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:01:18.361 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:01:18.380 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:01:18.410 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:01:18.454 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:01:18.497 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:01:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:01:18.571 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:01:18.623 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:01:18.663 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:01:18.684 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:01:18.707 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:01:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:01:18.775 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:01:18.809 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:01:18.820 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-16T00:01:18.846 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:01:18.899 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:01:18.917 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:18.936 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:01:18.944 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:01:18.980 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:01:19.002 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:19.056 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:01:19.075 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:01:19.091 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:01:19.129 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:01:19.158 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:01:19.218 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:19.228 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:01:19.273 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:01:19.305 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:01:19.323 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:01:19.346 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:01:19.347 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:01:19.371 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:01:19.416 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:01:19.419 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:01:19.466 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:01:19.490 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:01:19.509 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:01:19.550 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:01:19.573 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:01:19.618 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:01:19.650 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:01:19.693 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:01:19.726 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:01:19.753 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:01:19.754 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:01:19.822 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:01:19.864 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:01:19.884 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:01:19.924 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:01:19.967 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:01:19.969 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:01:20.006 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:01:20.042 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:01:20.078 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:01:20.078 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:01:20.081 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:01:20.111 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:01:20.121 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:01:20.150 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:01:20.154 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:01:20.192 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:01:20.220 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:01:20.234 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:01:20.258 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:01:20.264 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:01:20.272 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:01:20.314 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:01:20.316 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:01:20.348 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:01:20.353 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:01:20.457 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:01:20.479 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:01:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:01:20.504 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:01:20.542 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:01:20.542 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:01:20.559 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:01:20.563 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:01:20.581 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:01:20.597 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:01:20.679 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:01:20.710 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:01:20.723 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:01:20.754 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:01:20.801 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:01:20.837 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:01:20.857 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:01:20.880 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:01:20.923 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:01:20.987 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:01:21.067 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:01:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:01:21.102 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:21.115 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:01:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:01:21.172 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:01:21.241 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:01:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:01:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:01:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:01:21.399 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:01:21.479 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:01:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:01:21.524 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:01:21.527 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:01:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:01:21.564 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:01:21.590 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:01:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:01:21.614 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:01:21.623 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:01:21.664 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:01:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:01:21.691 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:01:21.699 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:01:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:01:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:01:21.771 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:01:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:01:21.817 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:01:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:01:21.825 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:01:21.849 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:01:21.863 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:01:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:01:21.891 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:01:21.892 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:01:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:01:21.924 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:01:21.925 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:01:21.953 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:01:21.965 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:01:22.004 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:01:22.029 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:01:22.044 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:01:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:01:22.085 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:01:22.091 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:01:22.119 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:01:22.123 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:01:22.160 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:01:22.166 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:01:22.195 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:01:22.242 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:01:22.275 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:01:22.277 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:01:22.281 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:01:22.321 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:01:22.325 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:01:22.350 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:01:22.356 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:01:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:01:22.418 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:01:22.460 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:01:22.472 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:01:22.488 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:01:22.524 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:01:22.524 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:01:22.554 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:01:22.565 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:01:22.606 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:01:22.631 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:01:22.686 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:01:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:01:22.741 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:01:22.850 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:01:22.889 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:01:22.925 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:01:22.967 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:01:23.006 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:01:23.038 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:01:23.069 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:01:23.102 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:01:23.139 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:01:23.181 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:01:23.215 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:01:23.244 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:01:23.262 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:01:23.288 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:01:23.301 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:01:23.321 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:01:23.345 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:01:23.346 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:01:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:01:23.385 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:01:23.418 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:01:23.420 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:01:23.456 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:01:23.477 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:01:23.487 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:01:23.497 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:01:23.530 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:01:23.536 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:01:23.618 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:01:23.655 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:01:23.657 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:01:23.722 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:01:23.777 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:01:23.842 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:01:23.849 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:01:23.872 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:01:23.882 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:01:23.918 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:01:23.958 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:01:23.972 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:01:24.018 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:01:24.060 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:01:24.102 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:01:24.238 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:01:24.248 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:01:24.279 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:01:24.308 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:01:24.321 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:01:24.342 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:01:24.422 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:01:24.440 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:01:24.463 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:01:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:01:24.502 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:01:24.528 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:01:24.541 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:01:24.553 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:01:24.602 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:01:24.627 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:01:24.629 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:01:24.661 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:01:24.736 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:01:24.763 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:01:24.803 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:01:24.831 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:01:24.845 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:01:24.865 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:01:24.876 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:01:24.904 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:01:24.944 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:01:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:01:24.983 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:01:25.017 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:01:25.050 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:01:25.094 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:01:25.129 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:01:25.215 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:01:25.268 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:01:25.299 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:01:25.401 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:01:25.458 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:01:25.615 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:01:25.714 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:01:25.971 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:01:26.044 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:01:26.072 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:01:26.105 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:01:26.173 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:01:26.223 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:01:26.256 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:01:26.357 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:01:26.411 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:01:26.415 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:01:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:01:26.468 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:01:26.545 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:01:26.558 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:01:26.584 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:01:26.597 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:01:26.633 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:01:26.767 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:01:26.798 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:01:26.875 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:01:26.924 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:01:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:01:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:01:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:01:27.115 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:01:27.146 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:01:27.200 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:01:27.231 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:01:27.374 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:01:27.412 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:01:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:01:27.516 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:01:27.534 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:01:27.570 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:01:27.641 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:01:28.049 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:01:28.082 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:01:28.153 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:01:28.183 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:01:28.861 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:01:28.963 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:01:28.992 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:01:29.023 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:01:29.331 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:01:29.379 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:01:29.469 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:01:29.500 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:01:29.536 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:01:29.873 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:29.910 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:30.024 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:01:30.057 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:01:30.111 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:01:30.134 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:01:30.441 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:30.455 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:30.497 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:01:30.571 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:01:30.571 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:01:30.571 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:01:30.571 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:30.572 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:30.598 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:01:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:31.167 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:01:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:01:31.241 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:01:31.241 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:01:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:01:32.369 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:32.413 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:32.954 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:01:32.992 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:01:33.076 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:01:33.076 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:01:33.076 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:01:33.076 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:33.110 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-09-16T00:01:42.153 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:42.334 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:01:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:01:42.350 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:42.350 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:01:42.350 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:01:43.059 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-09-16T00:01:43.059 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-09-16T00:01:43.059 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-09-16T00:01:43.059 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-09-16T00:01:43.059 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-09-16T00:01:43.059 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /root 2024-09-16T00:01:43.059 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-09-16T00:01:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:43.232 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:01:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:01:43.248 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:43.248 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:01:43.248 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:43.829 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:01:43.855 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:01:43.855 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:43.855 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:01:43.855 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:01:43.855 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:01:43.855 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:44.088 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:01:44.111 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:01:44.111 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:44.111 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:01:44.111 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:01:44.111 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:01:44.111 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:44.549 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:01:44.570 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:01:44.571 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:44.571 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:01:44.571 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:01:44.571 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:01:44.571 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-09-16T00:01:44.585 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:44.677 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:01:44.734 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:01:44.735 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:01:44.762 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:01:44.762 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:44.762 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:01:44.762 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:01:44.762 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:01:44.762 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:44.766 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:01:44.783 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:01:44.783 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:44.784 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:01:44.784 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:44.797 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:01:44.986 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:01:44.991 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:01:45.009 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:01:45.009 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:45.009 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:01:45.009 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:01:45.009 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:01:45.010 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:45.039 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:01:45.117 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:01:45.238 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:01:45.379 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:01:45.431 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:01:45.445 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:01:45.467 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:01:45.468 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:45.468 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:01:45.468 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:01:45.468 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:01:45.468 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:45.471 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:01:45.499 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:01:45.592 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:01:45.648 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:01:45.711 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:01:45.902 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:01:45.919 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:01:45.947 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:01:45.953 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:01:45.979 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:01:46.022 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:01:46.041 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:01:46.079 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:01:46.111 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:01:46.249 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:01:46.253 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:01:46.280 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:01:46.280 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:46.280 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:01:46.280 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:01:46.280 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:01:46.280 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:46.314 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:01:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:01:46.380 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:01:46.503 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:01:46.527 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:01:46.527 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:46.527 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:01:46.527 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:01:46.527 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:01:46.527 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:46.742 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:01:46.770 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:01:46.801 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:01:46.863 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:01:46.893 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:01:46.962 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:01:46.986 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:01:46.986 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:46.986 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:01:46.986 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:01:46.987 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:01:46.987 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:47.113 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:01:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:01:47.167 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:01:47.173 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:01:47.223 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:01:47.404 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:01:47.464 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:01:47.550 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:01:47.645 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:01:47.787 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:01:47.831 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:01:47.863 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:01:47.891 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:01:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:01:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:01:48.140 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:01:48.164 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:01:48.266 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:01:48.286 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:01:48.314 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:01:48.346 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:01:48.408 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:01:48.444 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:01:48.446 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:01:48.475 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:01:48.499 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:01:48.499 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:48.499 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:01:48.499 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:01:48.499 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:01:48.499 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:48.550 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:01:48.562 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:01:48.660 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:01:48.707 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:01:48.927 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:01:48.954 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:01:49.055 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:01:49.233 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:01:49.264 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:01:49.288 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:01:49.288 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:49.288 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:01:49.288 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:01:49.288 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:01:49.288 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:49.340 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:01:49.351 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:01:49.448 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:01:49.483 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:01:49.504 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:01:49.523 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:01:49.644 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:01:49.678 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:01:49.702 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:01:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:01:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:01:49.702 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:01:49.702 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:49.745 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:01:49.768 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:01:49.768 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:49.768 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:01:49.768 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:49.813 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:01:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:01:49.836 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:49.836 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:01:49.836 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:01:49.836 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:01:49.836 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:50.138 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:01:50.161 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:01:50.161 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:50.161 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:01:50.161 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:01:50.162 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:01:50.162 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:01:50.423 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:01:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:01:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:01:50.474 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:50.474 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:01:50.474 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:01:50.474 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:01:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:50.483 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:01:50.515 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:01:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:01:50.541 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:01:50.541 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:50.541 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:01:50.541 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:50.585 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:01:50.608 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:01:50.608 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:50.608 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:01:50.608 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:01:50.608 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:01:50.608 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:50.616 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:01:50.776 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:01:50.809 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:01:50.833 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:01:50.833 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:50.833 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:01:50.834 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:01:50.834 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:01:50.834 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:50.885 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:01:50.896 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:01:50.919 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:01:50.942 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:01:50.942 INFO:teuthology.orchestra.run.smithi078.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:50.943 INFO:teuthology.orchestra.run.smithi078.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:01:50.943 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:01:50.943 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:01:50.943 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:01:50.994 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:01:51.041 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:01:51.980 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:01:52.013 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:01:52.036 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:01:52.036 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:52.037 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:01:52.037 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:01:52.037 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:01:52.037 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:52.080 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:01:52.102 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:01:52.103 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:52.103 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:01:52.103 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:52.147 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:01:52.170 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:01:52.171 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:52.171 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:01:52.171 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:01:52.171 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:01:52.171 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:52.473 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:01:52.496 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:01:52.496 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:01:52.497 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:01:52.497 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:01:52.497 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:01:52.497 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:01:54.716 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:01:54.766 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:01:54.816 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:01:54.848 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:01:54.875 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:01:55.065 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:01:55.105 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:01:55.135 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:01:55.165 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:01:55.211 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:01:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:01:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:01:55.571 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:01:55.603 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:01:55.630 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:01:55.819 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:01:55.822 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:55.858 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:55.860 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:01:55.879 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:55.891 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:01:55.921 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:01:55.967 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:01:56.549 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:56.589 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:56.609 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:56.717 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:01:56.813 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:01:56.854 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:01:56.935 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:01:56.966 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:01:56.991 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:01:57.018 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:01:57.018 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:01:57.018 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:01:57.059 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:01:57.093 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:01:57.094 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:01:57.100 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:01:57.132 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:01:57.160 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:01:57.169 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:01:57.194 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:01:57.199 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:57.199 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:57.338 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:01:57.379 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:57.379 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:01:57.382 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:01:57.421 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:01:57.430 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:01:57.450 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:01:57.463 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:01:57.505 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:01:57.519 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:01:57.527 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:01:57.572 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:01:57.680 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:01:57.714 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:01:57.747 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:01:57.782 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:01:57.782 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:01:57.891 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:01:57.891 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:01:57.975 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:01:58.000 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:01:58.005 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:58.005 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:58.068 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:58.112 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:58.132 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:01:58.177 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:58.177 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:01:58.261 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:01:58.325 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:01:58.970 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:01:59.059 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:01:59.100 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:01:59.181 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:01:59.221 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:01:59.246 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:01:59.272 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:01:59.272 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:01:59.348 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:01:59.348 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:01:59.432 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:01:59.457 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:01:59.463 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:59.463 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:59.634 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:01:59.635 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:01:59.736 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:01:59.799 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:02:03.179 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:02:03.179 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:02:03.179 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:02:03.179 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:02:03.179 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:02:03.179 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:02:03.179 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:02:03.179 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:02:03.180 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:02:03.181 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:02:03.182 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:02:03.183 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:02:03.184 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:02:03.185 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:02:03.186 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:02:03.187 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:02:03.188 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:02:03.189 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:02:03.190 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:02:03.191 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:02:03.882 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:02:03.883 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:02:03.884 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:02:03.885 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:02:03.886 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:02:03.887 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:02:03.888 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:02:03.889 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:02:03.890 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:02:03.891 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:02:03.892 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:02:03.893 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:02:03.894 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:02:03.895 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:02:03.895 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:02:05.446 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:02:05.446 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:02:05.446 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:02:05.446 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:02:05.446 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:02:05.447 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:02:05.448 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:02:05.460 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:02:05.460 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:02:05.460 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:02:05.460 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:02:05.460 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:02:05.460 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:02:05.460 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:02:05.461 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:02:05.462 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:02:05.463 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:02:05.464 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:02:05.465 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:02:05.466 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:02:05.467 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:02:05.468 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:02:05.468 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:02:05.468 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:02:05.468 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:02:05.469 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:02:05.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:02:05.471 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.735 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.736 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.737 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:02:05.738 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:02:05.740 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:02:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:02:05.742 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:02:05.743 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:02:05.744 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.745 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:02:05.746 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:02:05.747 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:02:05.748 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:02:05.748 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:02:05.748 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:02:05.748 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:02:05.748 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-16T00:02:06.161 DEBUG:teuthology.parallel:result is None 2024-09-16T00:02:06.681 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:02:06.681 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:02:06.681 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-09-16T00:02:06.681 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:02:06.682 INFO:teuthology.orchestra.run.smithi078.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.683 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:02:06.684 INFO:teuthology.orchestra.run.smithi078.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.685 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:02:06.686 INFO:teuthology.orchestra.run.smithi078.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:02:06.687 INFO:teuthology.orchestra.run.smithi078.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:02:06.688 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:02:06.689 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:02:06.690 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:02:06.691 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:02:06.692 INFO:teuthology.orchestra.run.smithi078.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:02:06.693 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:02:06.694 INFO:teuthology.orchestra.run.smithi078.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:02:06.694 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:02:06.694 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-09-16T00:02:07.099 DEBUG:teuthology.parallel:result is None 2024-09-16T00:02:08.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:02:08.293 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:02:08.293 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-09-16T00:02:08.293 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:08.293 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.294 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.295 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:02:08.296 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.297 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:02:08.298 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:02:08.299 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:02:08.300 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:02:08.301 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:02:08.302 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:02:08.304 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:02:08.305 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:02:08.306 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:02:08.306 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:02:08.306 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:02:08.306 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-16T00:02:08.714 DEBUG:teuthology.parallel:result is None 2024-09-16T00:02:08.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:02:08.857 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:02:08.884 INFO:teuthology.orchestra.run.smithi078.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:02:08.885 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:02:08.885 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:02:08.887 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:02:09.036 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:02:09.062 INFO:teuthology.orchestra.run.smithi148.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:02:09.062 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:02:09.063 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:02:09.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:02:09.204 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:02:09.227 INFO:teuthology.orchestra.run.smithi193.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:02:09.227 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:02:09.228 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:02:09.230 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T00:02:09.230 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:02:09.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:02:09.263 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:02:09.263 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:02:09.296 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:02:09.297 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:02:09.324 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T00:02:09.325 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:02:09.325 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:02:09.354 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:02:09.421 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:02:09.421 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:02:09.450 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:02:09.520 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:02:09.520 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:02:09.546 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:02:09.611 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T00:02:09.611 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:02:09.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:02:09.640 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:02:09.708 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:02:09.708 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:02:09.737 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:02:09.803 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:02:09.804 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:02:09.829 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:02:09.895 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T00:02:09.895 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:02:09.895 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:02:09.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:02:09.991 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:02:09.991 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:02:10.019 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:02:10.086 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:02:10.086 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:02:10.111 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:02:10.177 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T00:02:10.297 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-16T00:02:10.297 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:02:10.298 INFO:tasks.cephadm:Cluster fsid is eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:02:10.298 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T00:02:10.298 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.78', 'mon.b': '172.21.15.148', 'mon.c': '172.21.15.193'} 2024-09-16T00:02:10.299 INFO:tasks.cephadm:First mon is mon.a on smithi078 2024-09-16T00:02:10.299 INFO:tasks.cephadm:First mgr is x 2024-09-16T00:02:10.299 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T00:02:10.299 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-09-16T00:02:10.329 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-09-16T00:02:10.357 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-09-16T00:02:10.382 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T00:02:10.383 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:02:10.524 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T00:02:10.672 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:02:10.673 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T00:02:10.673 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T00:02:10.674 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.038 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:02 /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.039 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.407 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:02 /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.408 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.789 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:02 /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.789 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.809 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.828 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:02:11.864 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-16T00:02:11.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:02:11.867 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:02:11.871 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:02:13.486 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:02:13.519 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:02:13.535 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:02:43.477 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:02:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:02:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:02:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-09-16T00:02:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:02:43.478 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:02:43.478 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:03:05.887 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-16T00:03:05.888 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:03:05.888 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:03:05.888 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-09-16T00:03:05.888 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:03:05.888 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-16T00:03:05.888 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-16T00:03:05.944 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-16T00:03:05.944 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:03:05.945 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:03:05.945 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-09-16T00:03:05.945 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:03:05.945 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-09-16T00:03:05.945 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-16T00:03:05.972 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-09-16T00:03:06.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-09-16T00:03:06.035 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-09-16T00:03:06.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-09-16T00:03:06.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-09-16T00:03:06.117 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-09-16T00:03:06.144 INFO:tasks.cephadm:Writing seed config... 2024-09-16T00:03:06.145 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T00:03:06.145 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T00:03:06.145 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T00:03:06.145 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T00:03:06.145 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T00:03:06.145 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-16T00:03:06.145 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T00:03:06.145 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T00:03:06.146 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T00:03:06.147 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T00:03:06.148 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-16T00:03:06.149 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:03:06.149 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T00:03:06.167 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = eb73a744-73be-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T00:03:06.167 DEBUG:teuthology.orchestra.run.smithi078:mon.a> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T00:03:06.210 DEBUG:teuthology.orchestra.run.smithi078:mgr.x> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T00:03:06.253 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T00:03:06.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid eb73a744-73be-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.78 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:03:06.456 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:03:06.456 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', 'eb73a744-73be-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.78', '--skip-admin-label'] 2024-09-16T00:03:06.456 INFO:teuthology.orchestra.run.smithi078.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T00:03:06.457 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying podman|docker is present... 2024-09-16T00:03:06.482 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:03:06.482 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying lvm2 is present... 2024-09-16T00:03:06.482 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying time synchronization is in place... 2024-09-16T00:03:06.489 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:03:06.489 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:03:06.496 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:03:06.496 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-16T00:03:06.503 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-09-16T00:03:06.509 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-09-16T00:03:06.509 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:03:06.509 INFO:teuthology.orchestra.run.smithi078.stdout:Repeating the final host check... 2024-09-16T00:03:06.532 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:03:06.532 INFO:teuthology.orchestra.run.smithi078.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T00:03:06.533 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl is present 2024-09-16T00:03:06.533 INFO:teuthology.orchestra.run.smithi078.stdout:lvcreate is present 2024-09-16T00:03:06.539 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:03:06.539 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:03:06.545 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:03:06.545 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-16T00:03:06.552 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout enabled 2024-09-16T00:03:06.558 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout active 2024-09-16T00:03:06.558 INFO:teuthology.orchestra.run.smithi078.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:03:06.558 INFO:teuthology.orchestra.run.smithi078.stdout:Host looks OK 2024-09-16T00:03:06.559 INFO:teuthology.orchestra.run.smithi078.stdout:Cluster fsid: eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:06.559 INFO:teuthology.orchestra.run.smithi078.stdout:Acquiring lock 140534976508928 on /run/cephadm/eb73a744-73be-11ef-bceb-c7b262605968.lock 2024-09-16T00:03:06.559 INFO:teuthology.orchestra.run.smithi078.stdout:Lock 140534976508928 acquired on /run/cephadm/eb73a744-73be-11ef-bceb-c7b262605968.lock 2024-09-16T00:03:06.559 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 3300 ... 2024-09-16T00:03:06.559 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying IP 172.21.15.78 port 6789 ... 2024-09-16T00:03:06.559 INFO:teuthology.orchestra.run.smithi078.stdout:Base mon IP(s) is [172.21.15.78:3300, 172.21.15.78:6789], mon addrv is [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-09-16T00:03:06.562 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.78 metric 100 2024-09-16T00:03:06.563 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.78 metric 100 2024-09-16T00:03:06.565 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T00:03:06.565 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T00:03:06.565 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T00:03:06.567 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T00:03:06.567 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T00:03:06.568 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:03:06.568 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T00:03:06.568 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link noprefixroute 2024-09-16T00:03:06.568 INFO:teuthology.orchestra.run.smithi078.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:03:06.568 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-09-16T00:03:06.569 INFO:teuthology.orchestra.run.smithi078.stdout:Mon IP `172.21.15.78` is in CIDR network `172.21.0.0/20` 2024-09-16T00:03:06.569 INFO:teuthology.orchestra.run.smithi078.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T00:03:06.569 INFO:teuthology.orchestra.run.smithi078.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T00:03:06.569 INFO:teuthology.orchestra.run.smithi078.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:03:07.006 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T00:03:07.006 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:03:07.006 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T00:03:07.006 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-16T00:03:07.006 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T00:03:07.007 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T00:03:07.007 INFO:teuthology.orchestra.run.smithi078.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T00:03:07.919 INFO:teuthology.orchestra.run.smithi078.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T00:03:07.919 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T00:03:07.919 INFO:teuthology.orchestra.run.smithi078.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T00:03:08.941 INFO:teuthology.orchestra.run.smithi078.stdout:stat: stdout 167 167 2024-09-16T00:03:08.941 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial keys... 2024-09-16T00:03:09.954 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQC9dedmTv9eDxAAby3VsuCxcRUw/5DileCUbw== 2024-09-16T00:03:10.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQC+dedmvoKkExAA7guO/vDw6e9bbTAmc9d3kg== 2024-09-16T00:03:11.875 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph-authtool: stdout AQC/dedmwfp2DxAAn64qN9Pyb2UhPC1GlGOX7Q== 2024-09-16T00:03:11.876 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial monmap... 2024-09-16T00:03:13.856 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool for a [v2:172.21.15.78:3300,v1:172.21.15.78:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = quincy 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: set fsid to eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:03:13.857 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mon... 2024-09-16T00:03:14.870 INFO:teuthology.orchestra.run.smithi078.stdout:create mon.a on 2024-09-16T00:03:15.082 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T00:03:15.224 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T00:03:15.425 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-eb73a744-73be-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-eb73a744-73be-11ef-bceb-c7b262605968.target. 2024-09-16T00:03:15.425 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-eb73a744-73be-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-eb73a744-73be-11ef-bceb-c7b262605968.target. 2024-09-16T00:03:15.637 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:03:15.637 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T00:03:15.795 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eb73a744-73be-11ef-bceb-c7b262605968.target.wants/ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-eb73a744-73be-11ef-bceb-c7b262605968@.service. 2024-09-16T00:03:16.294 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:16 smithi078 ceph-mon[30797]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:03:16.354 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:03:16.354 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-09-16T00:03:16.361 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:03:16.361 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-16T00:03:16.361 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-09-16T00:03:16.361 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:03:16.361 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon to start... 2024-09-16T00:03:16.361 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mon... 2024-09-16T00:03:16.593 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:16 smithi078 bash[30778]: cca2d4d5c967f0a17b9dbc38fab3b781f5b7adf66463c84e9b7c62a8b5229f0e 2024-09-16T00:03:16.593 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:16 smithi078 systemd[1]: Started Ceph mon.a for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:03:17.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout id: eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout services: 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.556953s) 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout data: 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T00:03:17.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.776+0000 7f54f0103640 1 Processor -- start 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.776+0000 7f54f0103640 1 -- start start 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.776+0000 7f54f0103640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e81079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.776+0000 7f54f0103640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8107fb0 con 0x7f54e81075e0 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.776+0000 7f54ede9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e81079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.776+0000 7f54ede9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e81079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60170/0 (socket says 172.21.15.78:60170) 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.776+0000 7f54ede9f640 1 -- 172.21.15.78:0/1230444177 learned_addr learned my addr 172.21.15.78:0/1230444177 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.777+0000 7f54ede9f640 1 -- 172.21.15.78:0/1230444177 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e8108790 con 0x7f54e81075e0 2024-09-16T00:03:17.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54ede9f640 1 --2- 172.21.15.78:0/1230444177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e81079e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009820 tx=0x7f54e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a657bc877ad88d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54ece9d640 1 -- 172.21.15.78:0/1230444177 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e4037070 con 0x7f54e81075e0 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54ece9d640 1 -- 172.21.15.78:0/1230444177 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e4032440 con 0x7f54e81075e0 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54ece9d640 1 -- 172.21.15.78:0/1230444177 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e4032740 con 0x7f54e81075e0 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54f0103640 1 -- 172.21.15.78:0/1230444177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 msgr2=0x7f54e81079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54f0103640 1 --2- 172.21.15.78:0/1230444177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e81079e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009820 tx=0x7f54e402a270 comp rx=0 tx=0).stop 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54f0103640 1 -- 172.21.15.78:0/1230444177 shutdown_connections 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54f0103640 1 --2- 172.21.15.78:0/1230444177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e81079e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54f0103640 1 -- 172.21.15.78:0/1230444177 >> 172.21.15.78:0/1230444177 conn(0x7f54e8102dd0 msgr2=0x7f54e81051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.779+0000 7f54f0103640 1 -- 172.21.15.78:0/1230444177 shutdown_connections 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.780+0000 7f54f0103640 1 -- 172.21.15.78:0/1230444177 wait complete. 2024-09-16T00:03:17.340 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.780+0000 7f54f0103640 1 Processor -- start 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.780+0000 7f54f0103640 1 -- start start 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.780+0000 7f54f0103640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.780+0000 7f54f0103640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8108d30 con 0x7f54e81075e0 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.780+0000 7f54ede9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.780+0000 7f54ede9f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60186/0 (socket says 172.21.15.78:60186) 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.780+0000 7f54ede9f640 1 -- 172.21.15.78:0/1379774987 learned_addr learned my addr 172.21.15.78:0/1379774987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54ede9f640 1 -- 172.21.15.78:0/1379774987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e819b2a0 con 0x7f54e81075e0 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54ede9f640 1 --2- 172.21.15.78:0/1379774987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e819ad60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f54e40090b0 tx=0x7f54e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54deffd640 1 -- 172.21.15.78:0/1379774987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e4041070 con 0x7f54e81075e0 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54deffd640 1 -- 172.21.15.78:0/1379774987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e403c7a0 con 0x7f54e81075e0 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54deffd640 1 -- 172.21.15.78:0/1379774987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54e4037040 con 0x7f54e81075e0 2024-09-16T00:03:17.341 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e819b530 con 0x7f54e81075e0 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e819e220 con 0x7f54e81075e0 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54deffd640 1 -- 172.21.15.78:0/1379774987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54e4044070 con 0x7f54e81075e0 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.781+0000 7f54deffd640 1 -- 172.21.15.78:0/1379774987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54e403d3f0 con 0x7f54e81075e0 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.782+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e8109380 con 0x7f54e81075e0 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.784+0000 7f54deffd640 1 -- 172.21.15.78:0/1379774987 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f54e4046450 con 0x7f54e81075e0 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.842+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f54e819e790 con 0x7f54e81075e0 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.842+0000 7f54deffd640 1 -- 172.21.15.78:0/1379774987 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f54e4046630 con 0x7f54e81075e0 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.843+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 msgr2=0x7f54e819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.843+0000 7f54f0103640 1 --2- 172.21.15.78:0/1379774987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e819ad60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f54e40090b0 tx=0x7f54e40046c0 comp rx=0 tx=0).stop 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.843+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 shutdown_connections 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.843+0000 7f54f0103640 1 --2- 172.21.15.78:0/1379774987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54e81075e0 0x7f54e819ad60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.843+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 >> 172.21.15.78:0/1379774987 conn(0x7f54e8102dd0 msgr2=0x7f54e818ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.843+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 shutdown_connections 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:16.843+0000 7f54f0103640 1 -- 172.21.15.78:0/1379774987 wait complete. 2024-09-16T00:03:17.342 INFO:teuthology.orchestra.run.smithi078.stdout:mon is available 2024-09-16T00:03:17.343 INFO:teuthology.orchestra.run.smithi078.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T00:03:17.447 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:03:17.542 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: monmap epoch 1 2024-09-16T00:03:17.542 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:17.542 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: last_changed 2024-09-16T00:03:12.254418+0000 2024-09-16T00:03:17.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:03:17.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: min_mon_release 19 (squid) 2024-09-16T00:03:17.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: election_strategy: 1 2024-09-16T00:03:17.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:03:17.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: fsmap 2024-09-16T00:03:17.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:03:17.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: mgrmap e1: no daemons active 2024-09-16T00:03:17.543 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:17 smithi078 ceph-mon[30797]: from='client.? 172.21.15.78:0/1379774987' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T00:03:18.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:18 smithi078 ceph-mon[30797]: from='client.? 172.21.15.78:0/3015098432' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:03:18.411 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:18 smithi078 ceph-mon[30797]: from='client.? 172.21.15.78:0/3015098432' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T00:03:18.413 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:18.413 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:03:18.419 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:18.419 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:03:18.420 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.749+0000 7f991109c640 1 Processor -- start 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.749+0000 7f991109c640 1 -- start start 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.749+0000 7f991109c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c10a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.749+0000 7f991109c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f990c07bd60 con 0x7f990c1080e0 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.749+0000 7f990ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c10a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.749+0000 7f990ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c10a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60196/0 (socket says 172.21.15.78:60196) 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.749+0000 7f990ad9d640 1 -- 172.21.15.78:0/421567985 learned_addr learned my addr 172.21.15.78:0/421567985 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.750+0000 7f990ad9d640 1 -- 172.21.15.78:0/421567985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c10aa10 con 0x7f990c1080e0 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.750+0000 7f990ad9d640 1 --2- 172.21.15.78:0/421567985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c10a4d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9904009820 tx=0x7f990402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97af6f629eea444 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.750+0000 7f9909d9b640 1 -- 172.21.15.78:0/421567985 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9904037070 con 0x7f990c1080e0 2024-09-16T00:03:18.421 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f9909d9b640 1 -- 172.21.15.78:0/421567985 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904032440 con 0x7f990c1080e0 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f9909d9b640 1 -- 172.21.15.78:0/421567985 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9904032740 con 0x7f990c1080e0 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f991109c640 1 -- 172.21.15.78:0/421567985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 msgr2=0x7f990c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f991109c640 1 --2- 172.21.15.78:0/421567985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c10a4d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9904009820 tx=0x7f990402a270 comp rx=0 tx=0).stop 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f991109c640 1 -- 172.21.15.78:0/421567985 shutdown_connections 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f991109c640 1 --2- 172.21.15.78:0/421567985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c10a4d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f991109c640 1 -- 172.21.15.78:0/421567985 >> 172.21.15.78:0/421567985 conn(0x7f990c101c80 msgr2=0x7f990c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f991109c640 1 -- 172.21.15.78:0/421567985 shutdown_connections 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.751+0000 7f991109c640 1 -- 172.21.15.78:0/421567985 wait complete. 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.752+0000 7f991109c640 1 Processor -- start 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.752+0000 7f991109c640 1 -- start start 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.752+0000 7f991109c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.752+0000 7f991109c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f990c10ae20 con 0x7f990c1080e0 2024-09-16T00:03:18.422 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.752+0000 7f990ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.752+0000 7f990ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60212/0 (socket says 172.21.15.78:60212) 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.752+0000 7f990ad9d640 1 -- 172.21.15.78:0/3015098432 learned_addr learned my addr 172.21.15.78:0/3015098432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.752+0000 7f990ad9d640 1 -- 172.21.15.78:0/3015098432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c19d4c0 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.753+0000 7f990ad9d640 1 --2- 172.21.15.78:0/3015098432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c19cf80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f99040090b0 tx=0x7f99040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.753+0000 7f98f3fff640 1 -- 172.21.15.78:0/3015098432 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9904041070 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.753+0000 7f98f3fff640 1 -- 172.21.15.78:0/3015098432 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990403c7a0 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.753+0000 7f98f3fff640 1 -- 172.21.15.78:0/3015098432 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9904037040 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.753+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c19d750 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.753+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c1a0440 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.753+0000 7f98f3fff640 1 -- 172.21.15.78:0/3015098432 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9904044070 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.753+0000 7f98f3fff640 1 -- 172.21.15.78:0/3015098432 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f990403d3f0 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.754+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990c1084c0 con 0x7f990c1080e0 2024-09-16T00:03:18.423 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.756+0000 7f98f3fff640 1 -- 172.21.15.78:0/3015098432 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9904046450 con 0x7f990c1080e0 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.812+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f990c02d910 con 0x7f990c1080e0 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.815+0000 7f98f3fff640 1 -- 172.21.15.78:0/3015098432 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f9904046630 con 0x7f990c1080e0 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.816+0000 7f98f3fff640 1 -- 172.21.15.78:0/3015098432 <== mon.0 v2:172.21.15.78:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9904056070 con 0x7f990c1080e0 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.817+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 msgr2=0x7f990c19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.817+0000 7f991109c640 1 --2- 172.21.15.78:0/3015098432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c19cf80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f99040090b0 tx=0x7f99040046c0 comp rx=0 tx=0).stop 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.817+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 shutdown_connections 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.817+0000 7f991109c640 1 --2- 172.21.15.78:0/3015098432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990c1080e0 0x7f990c19cf80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.817+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 >> 172.21.15.78:0/3015098432 conn(0x7f990c101c80 msgr2=0x7f990c103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.817+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 shutdown_connections 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:17.817+0000 7f991109c640 1 -- 172.21.15.78:0/3015098432 wait complete. 2024-09-16T00:03:18.424 INFO:teuthology.orchestra.run.smithi078.stdout:Generating new minimal ceph.conf... 2024-09-16T00:03:19.492 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:19 smithi078 ceph-mon[30797]: from='client.? 172.21.15.78:0/3115530300' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:03:19.498 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.788+0000 7f1540554640 1 Processor -- start 2024-09-16T00:03:19.498 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.788+0000 7f1540554640 1 -- start start 2024-09-16T00:03:19.498 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.789+0000 7f1540554640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f1538107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:19.498 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.789+0000 7f1540554640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1538107f40 con 0x7f1538107600 2024-09-16T00:03:19.498 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.789+0000 7f153e2f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f1538107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.789+0000 7f153e2f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f1538107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60214/0 (socket says 172.21.15.78:60214) 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.789+0000 7f153e2f0640 1 -- 172.21.15.78:0/3716926929 learned_addr learned my addr 172.21.15.78:0/3716926929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.789+0000 7f153e2f0640 1 -- 172.21.15.78:0/3716926929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15381080c0 con 0x7f1538107600 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f153e2f0640 1 --2- 172.21.15.78:0/3716926929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f1538107a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1528009820 tx=0x7f152802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9f7871cabeecd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f153d2ee640 1 -- 172.21.15.78:0/3716926929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1528037070 con 0x7f1538107600 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f153d2ee640 1 -- 172.21.15.78:0/3716926929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1528032900 con 0x7f1538107600 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f153d2ee640 1 -- 172.21.15.78:0/3716926929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1528030850 con 0x7f1538107600 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f1540554640 1 -- 172.21.15.78:0/3716926929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 msgr2=0x7f1538107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f1540554640 1 --2- 172.21.15.78:0/3716926929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f1538107a00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1528009820 tx=0x7f152802a270 comp rx=0 tx=0).stop 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f1540554640 1 -- 172.21.15.78:0/3716926929 shutdown_connections 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f1540554640 1 --2- 172.21.15.78:0/3716926929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f1538107a00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:19.499 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.790+0000 7f1540554640 1 -- 172.21.15.78:0/3716926929 >> 172.21.15.78:0/3716926929 conn(0x7f1538102e50 msgr2=0x7f1538105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f1540554640 1 -- 172.21.15.78:0/3716926929 shutdown_connections 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f1540554640 1 -- 172.21.15.78:0/3716926929 wait complete. 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f1540554640 1 Processor -- start 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f1540554640 1 -- start start 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f1540554640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f153819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f1540554640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f153810ba70 con 0x7f1538107600 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f153e2f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f153819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f153e2f0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f153819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60218/0 (socket says 172.21.15.78:60218) 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.791+0000 7f153e2f0640 1 -- 172.21.15.78:0/3115530300 learned_addr learned my addr 172.21.15.78:0/3115530300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f153e2f0640 1 -- 172.21.15.78:0/3115530300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153819bad0 con 0x7f1538107600 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f153e2f0640 1 --2- 172.21.15.78:0/3115530300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f153819b590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f15280097f0 tx=0x7f1528030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f152f7fe640 1 -- 172.21.15.78:0/3115530300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1528037040 con 0x7f1538107600 2024-09-16T00:03:19.500 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f152f7fe640 1 -- 172.21.15.78:0/3115530300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1528040070 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153819bd60 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f152f7fe640 1 -- 172.21.15.78:0/3115530300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152803a390 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153819ea50 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f152f7fe640 1 -- 172.21.15.78:0/3115530300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f152803a630 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.792+0000 7f152f7fe640 1 -- 172.21.15.78:0/3115530300 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f15280459a0 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.793+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1538107a80 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.795+0000 7f152f7fe640 1 -- 172.21.15.78:0/3115530300 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1528045460 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.852+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f153819ec20 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.852+0000 7f152f7fe640 1 -- 172.21.15.78:0/3115530300 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f152803b430 con 0x7f1538107600 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.853+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 msgr2=0x7f153819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.853+0000 7f1540554640 1 --2- 172.21.15.78:0/3115530300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f153819b590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f15280097f0 tx=0x7f1528030d80 comp rx=0 tx=0).stop 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.853+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 shutdown_connections 2024-09-16T00:03:19.501 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.853+0000 7f1540554640 1 --2- 172.21.15.78:0/3115530300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1538107600 0x7f153819b590 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:19.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.853+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 >> 172.21.15.78:0/3115530300 conn(0x7f1538102e50 msgr2=0x7f1538103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:19.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.853+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 shutdown_connections 2024-09-16T00:03:19.502 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:18.853+0000 7f1540554640 1 -- 172.21.15.78:0/3115530300 wait complete. 2024-09-16T00:03:19.502 INFO:teuthology.orchestra.run.smithi078.stdout:Restarting the monitor... 2024-09-16T00:03:19.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:19 smithi078 systemd[1]: Stopping Ceph mon.a for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:03:20.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a[30793]: 2024-09-16T00:03:19.778+0000 7efc740bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:03:20.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a[30793]: 2024-09-16T00:03:19.778+0000 7efc740bc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T00:03:20.254 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:19 smithi078 podman[31037]: 2024-09-16 00:03:19.983475428 +0000 UTC m=+0.319582532 container died cca2d4d5c967f0a17b9dbc38fab3b781f5b7adf66463c84e9b7c62a8b5229f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T00:03:20.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:20 smithi078 podman[31037]: 2024-09-16 00:03:20.353176686 +0000 UTC m=+0.689283802 container cleanup cca2d4d5c967f0a17b9dbc38fab3b781f5b7adf66463c84e9b7c62a8b5229f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:03:20.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:20 smithi078 bash[31037]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:03:20.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:20 smithi078 podman[31049]: 2024-09-16 00:03:20.620724387 +0000 UTC m=+0.638975452 container remove cca2d4d5c967f0a17b9dbc38fab3b781f5b7adf66463c84e9b7c62a8b5229f0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T00:03:20.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:20 smithi078 systemd[1]: ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T00:03:20.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:20 smithi078 systemd[1]: Stopped Ceph mon.a for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:03:20.976 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:20 smithi078 systemd[1]: Starting Ceph mon.a for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:03:21.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:20 smithi078 podman[31138]: 2024-09-16 00:03:20.877601635 +0000 UTC m=+0.015660762 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:03:21.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:20 smithi078 podman[31138]: 2024-09-16 00:03:20.99664289 +0000 UTC m=+0.134702057 container create 16361528f3de950291676d5378a55966c5370a41afc2a6b0f56bd00c6fa84411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:03:21.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 podman[31138]: 2024-09-16 00:03:21.198739466 +0000 UTC m=+0.336798587 container init 16361528f3de950291676d5378a55966c5370a41afc2a6b0f56bd00c6fa84411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T00:03:21.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 podman[31138]: 2024-09-16 00:03:21.201894039 +0000 UTC m=+0.339953161 container start 16361528f3de950291676d5378a55966c5370a41afc2a6b0f56bd00c6fa84411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:03:21.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:03:21.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T00:03:21.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: pidfile_write: ignore empty --pid-file 2024-09-16T00:03:21.227 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: load: jerasure load: lrc 2024-09-16T00:03:21.303 INFO:teuthology.orchestra.run.smithi078.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T00:03:21.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: RocksDB version: 7.9.2 2024-09-16T00:03:21.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Git sha 0 2024-09-16T00:03:21.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T00:03:21.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: DB SUMMARY 2024-09-16T00:03:21.503 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: DB Session ID: WP50DJJ69WVO1HENU9GM 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104824 ; 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.env: 0x5586f722d940 2024-09-16T00:03:21.504 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.info_log: 0x5586f955d6e0 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.statistics: (nil) 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.use_fsync: 0 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:03:21.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:03:21.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:03:21.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.db_log_dir: 2024-09-16T00:03:21.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.wal_dir: 2024-09-16T00:03:21.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:03:21.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:03:21.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:03:21.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:03:21.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:03:21.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:03:21.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:03:21.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:03:21.507 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.write_buffer_manager: 0x5586f956cdc0 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.unordered_write: 0 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:03:21.508 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.row_cache: None 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.wal_filter: None 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.wal_compression: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:03:21.509 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:03:21.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_open_files: -1 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Compression algorithms supported: 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: kZSTD supported: 0 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:03:21.511 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.merge_operator: 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_filter: None 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:03:21.512 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5586f955d320) 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_type: 0 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_index_type: 0 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_shortening: 1 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: checksum: 4 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: no_block_cache: 0 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache: 0x5586f95511f0 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:03:21.513 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_options: 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: capacity : 536870912 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_shard_bits : 4 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: strict_capacity_limit : 0 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_cache_compressed: (nil) 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: persistent_cache: (nil) 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size: 4096 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_size_deviation: 10 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_restart_interval: 16 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: index_block_restart_interval: 1 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: metadata_block_size: 4096 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: partition_filters: 0 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: use_delta_encoding: 1 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: filter_policy: bloomfilter 2024-09-16T00:03:21.514 INFO:journalctl@ceph.mon.a.smithi078.stdout: whole_key_filtering: 1 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: verify_compression: 0 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: format_version: 5 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_index_compression: 1 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: block_align: 0 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: max_auto_readahead_size: 262144 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: prepopulate_block_cache: 0 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: initial_auto_readahead_size: 8192 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression: NoCompression 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:03:21.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.num_levels: 7 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:03:21.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:03:21.517 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:03:21.518 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:03:21.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.ttl: 2592000 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:03:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T00:03:21.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d064cf68-ac23-4567-bf84-6c5bdcd49603 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445001228211, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445001229614, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99960, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97844, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14189, "raw_average_key_size": 49, "raw_value_size": 90140, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726445001, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d064cf68-ac23-4567-bf84-6c5bdcd49603", "db_session_id": "WP50DJJ69WVO1HENU9GM", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445001229748, "job": 1, "event": "recovery_finished"} 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5586f957ee00 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: DB pointer 0x5586f967a000 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** DB Stats ** 2024-09-16T00:03:21.522 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: L0 2/0 99.48 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Sum 2/0 99.48 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:03:21.523 INFO:journalctl@ceph.mon.a.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 116.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** Compaction Stats [default] ** 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 116.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:03:21.524 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: Cumulative compaction: 0.00 GB write, 21.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: Interval compaction: 0.00 GB write, 21.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache BinnedLRUCache@0x5586f95511f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: starting mon.a rank 0 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a@-1(???) e1 preinit fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a@-1(???).mds e1 new map 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a@-1(???).mds e1 print_map 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: e1 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: btime 2024-09-16T00:03:16:286868+0000 2024-09-16T00:03:21.525 INFO:journalctl@ceph.mon.a.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:03:21.526 INFO:journalctl@ceph.mon.a.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:03:21.526 INFO:journalctl@ceph.mon.a.smithi078.stdout: legacy client fscid: -1 2024-09-16T00:03:21.526 INFO:journalctl@ceph.mon.a.smithi078.stdout: 2024-09-16T00:03:21.526 INFO:journalctl@ceph.mon.a.smithi078.stdout: No filesystems configured 2024-09-16T00:03:21.526 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:03:21.526 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:03:21.526 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:03:21.526 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: monmap epoch 1 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: last_changed 2024-09-16T00:03:12.254418+0000 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: min_mon_release 19 (squid) 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: election_strategy: 1 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: fsmap 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 ceph-mon[31157]: mgrmap e1: no daemons active 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 bash[31138]: 16361528f3de950291676d5378a55966c5370a41afc2a6b0f56bd00c6fa84411 2024-09-16T00:03:21.527 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:21 smithi078 systemd[1]: Started Ceph mon.a for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:03:22.349 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.696+0000 7efdaf6b6640 1 Processor -- start 2024-09-16T00:03:22.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.696+0000 7efdaf6b6640 1 -- start start 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.697+0000 7efdaf6b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda8106890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.697+0000 7efdaf6b6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efda8106e60 con 0x7efda8106490 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.697+0000 7efdad452640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda8106890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.697+0000 7efdad452640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda8106890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60228/0 (socket says 172.21.15.78:60228) 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.697+0000 7efdad452640 1 -- 172.21.15.78:0/2407287300 learned_addr learned my addr 172.21.15.78:0/2407287300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.697+0000 7efdad452640 1 -- 172.21.15.78:0/2407287300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda8107690 con 0x7efda8106490 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.699+0000 7efdad452640 1 --2- 172.21.15.78:0/2407287300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda8106890 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e80 tx=0x7efd9802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0cdefa5dd67689 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.699+0000 7efda7fff640 1 -- 172.21.15.78:0/2407287300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98036070 con 0x7efda8106490 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.699+0000 7efda7fff640 1 -- 172.21.15.78:0/2407287300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd980318c0 con 0x7efda8106490 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.699+0000 7efda7fff640 1 -- 172.21.15.78:0/2407287300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98007ab0 con 0x7efda8106490 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.700+0000 7efdaf6b6640 1 -- 172.21.15.78:0/2407287300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 msgr2=0x7efda8106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.700+0000 7efdaf6b6640 1 --2- 172.21.15.78:0/2407287300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda8106890 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd98009e80 tx=0x7efd9802a7a0 comp rx=0 tx=0).stop 2024-09-16T00:03:22.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.700+0000 7efdaf6b6640 1 -- 172.21.15.78:0/2407287300 shutdown_connections 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.700+0000 7efdaf6b6640 1 --2- 172.21.15.78:0/2407287300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda8106890 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.700+0000 7efdaf6b6640 1 -- 172.21.15.78:0/2407287300 >> 172.21.15.78:0/2407287300 conn(0x7efda8101ca0 msgr2=0x7efda81040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.700+0000 7efdaf6b6640 1 -- 172.21.15.78:0/2407287300 shutdown_connections 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.700+0000 7efdaf6b6640 1 -- 172.21.15.78:0/2407287300 wait complete. 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.700+0000 7efdaf6b6640 1 Processor -- start 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdaf6b6640 1 -- start start 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdaf6b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdaf6b6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efda810aa40 con 0x7efda8106490 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdad452640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdad452640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60244/0 (socket says 172.21.15.78:60244) 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdad452640 1 -- 172.21.15.78:0/3480755463 learned_addr learned my addr 172.21.15.78:0/3480755463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdad452640 1 -- 172.21.15.78:0/3480755463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda819bad0 con 0x7efda8106490 2024-09-16T00:03:22.355 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdad452640 1 --2- 172.21.15.78:0/3480755463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda819b590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efd9802acc0 tx=0x7efd98031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efda67fc640 1 -- 172.21.15.78:0/3480755463 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98036050 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efda67fc640 1 -- 172.21.15.78:0/3480755463 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd98037040 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.701+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda819bd60 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.702+0000 7efda67fc640 1 -- 172.21.15.78:0/3480755463 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd9803d8c0 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.702+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda819ea50 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.702+0000 7efda67fc640 1 -- 172.21.15.78:0/3480755463 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd98044020 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.702+0000 7efda67fc640 1 -- 172.21.15.78:0/3480755463 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd98042020 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.703+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda810abc0 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.705+0000 7efda67fc640 1 -- 172.21.15.78:0/3480755463 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7efd9804b020 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.762+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7efda819f090 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.766+0000 7efda67fc640 1 -- 172.21.15.78:0/3480755463 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7efd980073c0 con 0x7efda8106490 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.767+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 msgr2=0x7efda819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:22.356 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.767+0000 7efdaf6b6640 1 --2- 172.21.15.78:0/3480755463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda819b590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efd9802acc0 tx=0x7efd98031ca0 comp rx=0 tx=0).stop 2024-09-16T00:03:22.357 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.767+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 shutdown_connections 2024-09-16T00:03:22.357 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.767+0000 7efdaf6b6640 1 --2- 172.21.15.78:0/3480755463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efda8106490 0x7efda819b590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:22.357 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.767+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 >> 172.21.15.78:0/3480755463 conn(0x7efda8101ca0 msgr2=0x7efda8102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:22.357 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.767+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 shutdown_connections 2024-09-16T00:03:22.357 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:21.767+0000 7efdaf6b6640 1 -- 172.21.15.78:0/3480755463 wait complete. 2024-09-16T00:03:22.357 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T00:03:22.372 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:03:22.372 INFO:teuthology.orchestra.run.smithi078.stdout:Creating mgr... 2024-09-16T00:03:22.372 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T00:03:22.373 INFO:teuthology.orchestra.run.smithi078.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T00:03:22.623 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:03:22.623 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Failed to reset failed state of unit ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-16T00:03:22.776 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-eb73a744-73be-11ef-bceb-c7b262605968.target.wants/ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-eb73a744-73be-11ef-bceb-c7b262605968@.service. 2024-09-16T00:03:23.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:22 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3480755463' entity='client.admin' 2024-09-16T00:03:23.020 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 podman[31370]: 2024-09-16 00:03:22.915545394 +0000 UTC m=+0.024639623 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:03:23.316 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:03:23.316 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-09-16T00:03:23.323 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:03:23.323 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-16T00:03:23.323 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-09-16T00:03:23.323 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:03:23.332 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:03:23.332 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-09-16T00:03:23.339 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:03:23.339 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-16T00:03:23.339 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-09-16T00:03:23.339 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T00:03:23.339 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr to start... 2024-09-16T00:03:23.340 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr... 2024-09-16T00:03:23.428 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 podman[31370]: 2024-09-16 00:03:23.243043218 +0000 UTC m=+0.352137450 container init f1c3878bc64b3ca51ff83b23473e37c1c9762987ba7a0ce4fcbf4ea76007a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:03:23.428 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 podman[31370]: 2024-09-16 00:03:23.24583288 +0000 UTC m=+0.354927095 container start f1c3878bc64b3ca51ff83b23473e37c1c9762987ba7a0ce4fcbf4ea76007a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T00:03:23.428 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 bash[31370]: f1c3878bc64b3ca51ff83b23473e37c1c9762987ba7a0ce4fcbf4ea76007a94c 2024-09-16T00:03:23.428 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 systemd[1]: Started Ceph mgr.x for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:03:23.429 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:23.368+0000 7f7d8f7fa040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:03:23.429 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:23.425+0000 7f7d8f7fa040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:03:23.705 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:23.578+0000 7f7d8f7fa040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:03:23.705 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:23.648+0000 7f7d8f7fa040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:03:23.962 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:23 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4053015655' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:03:23.962 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:23.813+0000 7f7d8f7fa040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:03:24.269 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:23.960+0000 7f7d8f7fa040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:03:24.586 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:24.281+0000 7f7d8f7fa040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:03:24.586 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:24.343+0000 7f7d8f7fa040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:03:24.586 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:24.530+0000 7f7d8f7fa040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:03:24.588 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:03:24.591 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:03:24.592 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:24.593 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:03:24.593 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:03:24.593 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:03:24.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:03:16:286868+0000", 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:03:24.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:03:16.287504+0000", 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:03:24.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.782+0000 7f7021e25640 1 Processor -- start 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.783+0000 7f7021e25640 1 -- start start 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.783+0000 7f7021e25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c106890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.783+0000 7f7021e25640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f701c106e60 con 0x7f701c106490 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.783+0000 7f701b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c106890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.783+0000 7f701b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c106890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60286/0 (socket says 172.21.15.78:60286) 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.783+0000 7f701b7fe640 1 -- 172.21.15.78:0/998696321 learned_addr learned my addr 172.21.15.78:0/998696321 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.783+0000 7f701b7fe640 1 -- 172.21.15.78:0/998696321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c107690 con 0x7f701c106490 2024-09-16T00:03:24.604 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.784+0000 7f701b7fe640 1 --2- 172.21.15.78:0/998696321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c106890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f700c009820 tx=0x7f700c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de708c9fe49a15e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.784+0000 7f701affd640 1 -- 172.21.15.78:0/998696321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f700c037070 con 0x7f701c106490 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.784+0000 7f701affd640 1 -- 172.21.15.78:0/998696321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f700c032900 con 0x7f701c106490 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.784+0000 7f701affd640 1 -- 172.21.15.78:0/998696321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f700c030850 con 0x7f701c106490 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.785+0000 7f7021e25640 1 -- 172.21.15.78:0/998696321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 msgr2=0x7f701c106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.785+0000 7f7021e25640 1 --2- 172.21.15.78:0/998696321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c106890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f700c009820 tx=0x7f700c02a270 comp rx=0 tx=0).stop 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.785+0000 7f7021e25640 1 -- 172.21.15.78:0/998696321 shutdown_connections 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.785+0000 7f7021e25640 1 --2- 172.21.15.78:0/998696321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c106890 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.785+0000 7f7021e25640 1 -- 172.21.15.78:0/998696321 >> 172.21.15.78:0/998696321 conn(0x7f701c101ca0 msgr2=0x7f701c1040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.785+0000 7f7021e25640 1 -- 172.21.15.78:0/998696321 shutdown_connections 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.785+0000 7f7021e25640 1 -- 172.21.15.78:0/998696321 wait complete. 2024-09-16T00:03:24.605 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f7021e25640 1 Processor -- start 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f7021e25640 1 -- start start 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f7021e25640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c075460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f7021e25640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10aa40 con 0x7f701c106490 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f701b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c075460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f701b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c075460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60296/0 (socket says 172.21.15.78:60296) 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f701b7fe640 1 -- 172.21.15.78:0/4053015655 learned_addr learned my addr 172.21.15.78:0/4053015655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f701b7fe640 1 -- 172.21.15.78:0/4053015655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c19e2c0 con 0x7f701c106490 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f701b7fe640 1 --2- 172.21.15.78:0/4053015655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c075460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f700c0097f0 tx=0x7f700c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.786+0000 7f70197fa640 1 -- 172.21.15.78:0/4053015655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f700c037040 con 0x7f701c106490 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.787+0000 7f70197fa640 1 -- 172.21.15.78:0/4053015655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f700c040070 con 0x7f701c106490 2024-09-16T00:03:24.606 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.787+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c19e490 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.787+0000 7f70197fa640 1 -- 172.21.15.78:0/4053015655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f700c03a390 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.787+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c19e830 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.787+0000 7f70197fa640 1 -- 172.21.15.78:0/4053015655 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f700c03a5e0 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.787+0000 7f70197fa640 1 -- 172.21.15.78:0/4053015655 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f700c045950 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.788+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c106910 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.790+0000 7f70197fa640 1 -- 172.21.15.78:0/4053015655 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f700c045460 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.862+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f701c19f0b0 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.862+0000 7f70197fa640 1 -- 172.21.15.78:0/4053015655 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f700c03b430 con 0x7f701c106490 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.863+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 msgr2=0x7f701c075460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.863+0000 7f7021e25640 1 --2- 172.21.15.78:0/4053015655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c075460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f700c0097f0 tx=0x7f700c030d80 comp rx=0 tx=0).stop 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.863+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 shutdown_connections 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.863+0000 7f7021e25640 1 --2- 172.21.15.78:0/4053015655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f701c106490 0x7f701c075460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:24.607 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.863+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 >> 172.21.15.78:0/4053015655 conn(0x7f701c101ca0 msgr2=0x7f701c102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:24.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.863+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 shutdown_connections 2024-09-16T00:03:24.608 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:23.863+0000 7f7021e25640 1 -- 172.21.15.78:0/4053015655 wait complete. 2024-09-16T00:03:24.608 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (1/15)... 2024-09-16T00:03:25.770 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:25.413+0000 7f7d8f7fa040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:03:25.770 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:25.479+0000 7f7d8f7fa040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:03:26.443 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:26.185+0000 7f7d8f7fa040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:03:26.443 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:26.301+0000 7f7d8f7fa040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:03:26.443 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:26.440+0000 7f7d8f7fa040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:03:26.717 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:26.501+0000 7f7d8f7fa040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:03:26.717 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:26.653+0000 7f7d8f7fa040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:03:27.014 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:26.720+0000 7f7d8f7fa040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:03:27.014 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:26.781+0000 7f7d8f7fa040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:03:27.014 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:26.944+0000 7f7d8f7fa040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:03:27.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:27 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3402941606' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:03:27.270 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:27.013+0000 7f7d8f7fa040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:03:27.270 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:27.084+0000 7f7d8f7fa040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:03:27.270 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:27.171+0000 7f7d8f7fa040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:03:27.270 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:27.232+0000 7f7d8f7fa040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:03:27.648 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:27.294+0000 7f7d8f7fa040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:03:27.653 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:03:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:03:27.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:03:27.655 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:03:27.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:03:27.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:03:27.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:27.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:03:27.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:03:27.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:03:27.662 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:03:16:286868+0000", 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:27.663 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:03:16.287504+0000", 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:03:27.664 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.011+0000 7f77e0f19640 1 Processor -- start 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.011+0000 7f77e0f19640 1 -- start start 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.012+0000 7f77e0f19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc106890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.012+0000 7f77e0f19640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc106e60 con 0x7f77dc106490 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.012+0000 7f77da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc106890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.012+0000 7f77da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc106890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41418/0 (socket says 172.21.15.78:41418) 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.012+0000 7f77da59c640 1 -- 172.21.15.78:0/3747348710 learned_addr learned my addr 172.21.15.78:0/3747348710 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.012+0000 7f77da59c640 1 -- 172.21.15.78:0/3747348710 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc107690 con 0x7f77dc106490 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.013+0000 7f77da59c640 1 --2- 172.21.15.78:0/3747348710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc106890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009820 tx=0x7f77cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc638e036675a5f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.013+0000 7f77d9d9b640 1 -- 172.21.15.78:0/3747348710 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77cc037070 con 0x7f77dc106490 2024-09-16T00:03:27.665 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.013+0000 7f77d9d9b640 1 -- 172.21.15.78:0/3747348710 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77cc032900 con 0x7f77dc106490 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.013+0000 7f77d9d9b640 1 -- 172.21.15.78:0/3747348710 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77cc030850 con 0x7f77dc106490 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.014+0000 7f77e0f19640 1 -- 172.21.15.78:0/3747348710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 msgr2=0x7f77dc106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.014+0000 7f77e0f19640 1 --2- 172.21.15.78:0/3747348710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc106890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009820 tx=0x7f77cc02a270 comp rx=0 tx=0).stop 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.014+0000 7f77e0f19640 1 -- 172.21.15.78:0/3747348710 shutdown_connections 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.014+0000 7f77e0f19640 1 --2- 172.21.15.78:0/3747348710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc106890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.014+0000 7f77e0f19640 1 -- 172.21.15.78:0/3747348710 >> 172.21.15.78:0/3747348710 conn(0x7f77dc101ca0 msgr2=0x7f77dc1040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.014+0000 7f77e0f19640 1 -- 172.21.15.78:0/3747348710 shutdown_connections 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.014+0000 7f77e0f19640 1 -- 172.21.15.78:0/3747348710 wait complete. 2024-09-16T00:03:27.666 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.014+0000 7f77e0f19640 1 Processor -- start 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77e0f19640 1 -- start start 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77e0f19640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77e0f19640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc10aa40 con 0x7f77dc106490 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77da59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc19b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41420/0 (socket says 172.21.15.78:41420) 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77da59c640 1 -- 172.21.15.78:0/3402941606 learned_addr learned my addr 172.21.15.78:0/3402941606 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77da59c640 1 -- 172.21.15.78:0/3402941606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc19baf0 con 0x7f77dc106490 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77da59c640 1 --2- 172.21.15.78:0/3402941606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc19b5b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009940 tx=0x7f77cc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77d37fe640 1 -- 172.21.15.78:0/3402941606 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77cc042070 con 0x7f77dc106490 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.015+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc19bd80 con 0x7f77dc106490 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.016+0000 7f77d37fe640 1 -- 172.21.15.78:0/3402941606 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77cc0308c0 con 0x7f77dc106490 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.016+0000 7f77d37fe640 1 -- 172.21.15.78:0/3402941606 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77cc037040 con 0x7f77dc106490 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.016+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc19ea70 con 0x7f77dc106490 2024-09-16T00:03:27.667 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.016+0000 7f77d37fe640 1 -- 172.21.15.78:0/3402941606 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f77cc045070 con 0x7f77dc106490 2024-09-16T00:03:27.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.016+0000 7f77d37fe640 1 -- 172.21.15.78:0/3402941606 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f77cc047bc0 con 0x7f77dc106490 2024-09-16T00:03:27.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.016+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc10abc0 con 0x7f77dc106490 2024-09-16T00:03:27.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.018+0000 7f77d37fe640 1 -- 172.21.15.78:0/3402941606 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f77cc0327b0 con 0x7f77dc106490 2024-09-16T00:03:27.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.080+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f77dc02d950 con 0x7f77dc106490 2024-09-16T00:03:27.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.081+0000 7f77d37fe640 1 -- 172.21.15.78:0/3402941606 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f77cc03c800 con 0x7f77dc106490 2024-09-16T00:03:27.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.082+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 msgr2=0x7f77dc19b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:27.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.082+0000 7f77e0f19640 1 --2- 172.21.15.78:0/3402941606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc19b5b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009940 tx=0x7f77cc032ce0 comp rx=0 tx=0).stop 2024-09-16T00:03:27.668 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.082+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 shutdown_connections 2024-09-16T00:03:27.669 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.082+0000 7f77e0f19640 1 --2- 172.21.15.78:0/3402941606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77dc106490 0x7f77dc19b5b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:27.669 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.082+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 >> 172.21.15.78:0/3402941606 conn(0x7f77dc101ca0 msgr2=0x7f77dc102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:27.669 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.082+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 shutdown_connections 2024-09-16T00:03:27.669 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:27.082+0000 7f77e0f19640 1 -- 172.21.15.78:0/3402941606 wait complete. 2024-09-16T00:03:27.669 INFO:teuthology.orchestra.run.smithi078.stdout:mgr not available, waiting (2/15)... 2024-09-16T00:03:28.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:28.085+0000 7f7d8f7fa040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:03:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: Activating manager daemon x 2024-09-16T00:03:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: mgrmap e2: x(active, starting, since 0.00312846s) 2024-09-16T00:03:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:03:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:03:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:03:28.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:03:28.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:03:28.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: Manager daemon x is now available 2024-09-16T00:03:28.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:03:28.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:03:28.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' 2024-09-16T00:03:28.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' 2024-09-16T00:03:28.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:28 smithi078 ceph-mon[31157]: from='mgr.14100 172.21.15.78:0/27146919' entity='mgr.x' 2024-09-16T00:03:30.336 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:30 smithi078 ceph-mon[31157]: mgrmap e3: x(active, since 1.00693s) 2024-09-16T00:03:31.019 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:31.019 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:03:31.019 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:03:31.019 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:03:31.020 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:03:31.020 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:03:31.020 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:03:31.020 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:31.020 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:03:31.020 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:03:31.020 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:03:31.020 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:31.021 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:03:31.021 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:03:31.021 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:31.021 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T00:03:31.021 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:03:31.021 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:31.021 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:03:31.021 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:03:31.022 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:31.022 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:03:31.022 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:31.022 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:03:31.022 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:03:31.023 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:03:31.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:03:31.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:03:31.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:03:31.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:31.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:03:31.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:03:31.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:03:31.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:03:31.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:03:31.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:03:31.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:03:31.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:03:31.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:03:31.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:31.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:03:31.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:31.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:03:16:286868+0000", 2024-09-16T00:03:31.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:03:31.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:03:31.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:31.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:03:31.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:03:31.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:03:31.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:03:31.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:03:31.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:03:31.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:03:31.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:03:31.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:03:31.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:31.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:03:31.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:03:31.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:03:16.287504+0000", 2024-09-16T00:03:31.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:03:31.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:03:31.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:03:31.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:03:31.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.079+0000 7fb2597a2640 1 Processor -- start 2024-09-16T00:03:31.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.079+0000 7fb2597a2640 1 -- start start 2024-09-16T00:03:31.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.079+0000 7fb2597a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb2541066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:31.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.079+0000 7fb2597a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb254106cc0 con 0x7fb2541062f0 2024-09-16T00:03:31.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.080+0000 7fb253fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb2541066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:31.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.080+0000 7fb253fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb2541066f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41482/0 (socket says 172.21.15.78:41482) 2024-09-16T00:03:31.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.080+0000 7fb253fff640 1 -- 172.21.15.78:0/330133489 learned_addr learned my addr 172.21.15.78:0/330133489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:31.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.080+0000 7fb253fff640 1 -- 172.21.15.78:0/330133489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2541074a0 con 0x7fb2541062f0 2024-09-16T00:03:31.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb253fff640 1 --2- 172.21.15.78:0/330133489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb2541066f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb24400bac0 tx=0x7fb24402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188bb8b697e9a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:31.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb252ffd640 1 -- 172.21.15.78:0/330133489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2440326d0 con 0x7fb2541062f0 2024-09-16T00:03:31.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb252ffd640 1 -- 172.21.15.78:0/330133489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb244038070 con 0x7fb2541062f0 2024-09-16T00:03:31.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb252ffd640 1 -- 172.21.15.78:0/330133489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb244033d70 con 0x7fb2541062f0 2024-09-16T00:03:31.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb2597a2640 1 -- 172.21.15.78:0/330133489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 msgr2=0x7fb2541066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:31.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb2597a2640 1 --2- 172.21.15.78:0/330133489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb2541066f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb24400bac0 tx=0x7fb24402c920 comp rx=0 tx=0).stop 2024-09-16T00:03:31.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb2597a2640 1 -- 172.21.15.78:0/330133489 shutdown_connections 2024-09-16T00:03:31.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb2597a2640 1 --2- 172.21.15.78:0/330133489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb2541066f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:31.044 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb2597a2640 1 -- 172.21.15.78:0/330133489 >> 172.21.15.78:0/330133489 conn(0x7fb254101ae0 msgr2=0x7fb254103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:31.044 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.081+0000 7fb2597a2640 1 -- 172.21.15.78:0/330133489 shutdown_connections 2024-09-16T00:03:31.044 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.082+0000 7fb2597a2640 1 -- 172.21.15.78:0/330133489 wait complete. 2024-09-16T00:03:31.044 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.082+0000 7fb2597a2640 1 Processor -- start 2024-09-16T00:03:31.044 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.082+0000 7fb2597a2640 1 -- start start 2024-09-16T00:03:31.044 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.082+0000 7fb2597a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb25419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:31.044 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.082+0000 7fb2597a2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb25407b0f0 con 0x7fb2541062f0 2024-09-16T00:03:31.044 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb253fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb25419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:31.045 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb253fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb25419d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41486/0 (socket says 172.21.15.78:41486) 2024-09-16T00:03:31.045 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb253fff640 1 -- 172.21.15.78:0/3318267788 learned_addr learned my addr 172.21.15.78:0/3318267788 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:31.045 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb253fff640 1 -- 172.21.15.78:0/3318267788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25419dab0 con 0x7fb2541062f0 2024-09-16T00:03:31.045 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb253fff640 1 --2- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb25419d570 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb24400b2c0 tx=0x7fb2440328b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:31.045 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb2517fa640 1 -- 172.21.15.78:0/3318267788 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb244042070 con 0x7fb2541062f0 2024-09-16T00:03:31.046 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb2517fa640 1 -- 172.21.15.78:0/3318267788 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb24403e900 con 0x7fb2541062f0 2024-09-16T00:03:31.046 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25419dd40 con 0x7fb2541062f0 2024-09-16T00:03:31.046 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb2517fa640 1 -- 172.21.15.78:0/3318267788 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb24403c9b0 con 0x7fb2541062f0 2024-09-16T00:03:31.046 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.083+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2541a0a30 con 0x7fb2541062f0 2024-09-16T00:03:31.046 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.084+0000 7fb2517fa640 1 -- 172.21.15.78:0/3318267788 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7fb244045070 con 0x7fb2541062f0 2024-09-16T00:03:31.046 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.084+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2541066f0 con 0x7fb2541062f0 2024-09-16T00:03:31.047 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.084+0000 7fb2517fa640 1 --2- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb23c039050 0x7fb23c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:31.047 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.084+0000 7fb2517fa640 1 -- 172.21.15.78:0/3318267788 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb244030050 con 0x7fb2541062f0 2024-09-16T00:03:31.047 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.085+0000 7fb2537fe640 1 --2- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb23c039050 0x7fb23c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:31.047 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.085+0000 7fb2537fe640 1 --2- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb23c039050 0x7fb23c03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2480098c0 tx=0x7fb248006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:31.047 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.089+0000 7fb2517fa640 1 -- 172.21.15.78:0/3318267788 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb244030250 con 0x7fb2541062f0 2024-09-16T00:03:31.047 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.144+0000 7fb2517fa640 1 -- 172.21.15.78:0/3318267788 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fb24404ccf0 con 0x7fb2541062f0 2024-09-16T00:03:31.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.323+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb2541925d0 con 0x7fb2541062f0 2024-09-16T00:03:31.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.323+0000 7fb2517fa640 1 -- 172.21.15.78:0/3318267788 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fb2440442c0 con 0x7fb2541062f0 2024-09-16T00:03:31.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.325+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb23c039050 msgr2=0x7fb23c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:31.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.325+0000 7fb2597a2640 1 --2- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb23c039050 0x7fb23c03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb2480098c0 tx=0x7fb248006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:31.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.325+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 msgr2=0x7fb25419d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:31.048 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.325+0000 7fb2597a2640 1 --2- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb25419d570 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb24400b2c0 tx=0x7fb2440328b0 comp rx=0 tx=0).stop 2024-09-16T00:03:31.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.326+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 shutdown_connections 2024-09-16T00:03:31.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.326+0000 7fb2597a2640 1 --2- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb23c039050 0x7fb23c03b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:31.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.326+0000 7fb2597a2640 1 --2- 172.21.15.78:0/3318267788 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2541062f0 0x7fb25419d570 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:31.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.326+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 >> 172.21.15.78:0/3318267788 conn(0x7fb254101ae0 msgr2=0x7fb25418f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:31.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.326+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 shutdown_connections 2024-09-16T00:03:31.049 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:30.326+0000 7fb2597a2640 1 -- 172.21.15.78:0/3318267788 wait complete. 2024-09-16T00:03:31.050 INFO:teuthology.orchestra.run.smithi078.stdout:mgr is available 2024-09-16T00:03:31.437 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:31 smithi078 ceph-mon[31157]: mgrmap e4: x(active, since 2s) 2024-09-16T00:03:31.437 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:31 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3318267788' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:03:32.169 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:32 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/214388097' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout fsid = eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.78:3300,v1:172.21.15.78:6789] 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:03:32.182 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.435+0000 7f0ccd7a7640 1 Processor -- start 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.435+0000 7f0ccd7a7640 1 -- start start 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.435+0000 7f0ccd7a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc810a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:32.183 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.435+0000 7f0ccd7a7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc8075820 con 0x7f0cc8108060 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.435+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc810a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.435+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc810a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41496/0 (socket says 172.21.15.78:41496) 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.435+0000 7f0cc6ffd640 1 -- 172.21.15.78:0/3827779051 learned_addr learned my addr 172.21.15.78:0/3827779051 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.436+0000 7f0cc6ffd640 1 -- 172.21.15.78:0/3827779051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc810a990 con 0x7f0cc8108060 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0cc6ffd640 1 --2- 172.21.15.78:0/3827779051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc810a450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009820 tx=0x7f0cb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8dd1c2675797e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0cc67fc640 1 -- 172.21.15.78:0/3827779051 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb8037070 con 0x7f0cc8108060 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0cc67fc640 1 -- 172.21.15.78:0/3827779051 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0cb8032900 con 0x7f0cc8108060 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0cc67fc640 1 -- 172.21.15.78:0/3827779051 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb8030850 con 0x7f0cc8108060 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/3827779051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 msgr2=0x7f0cc810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0ccd7a7640 1 --2- 172.21.15.78:0/3827779051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc810a450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009820 tx=0x7f0cb802a270 comp rx=0 tx=0).stop 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/3827779051 shutdown_connections 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0ccd7a7640 1 --2- 172.21.15.78:0/3827779051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc810a450 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:32.184 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/3827779051 >> 172.21.15.78:0/3827779051 conn(0x7f0cc8101ca0 msgr2=0x7f0cc81040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.437+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/3827779051 shutdown_connections 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.438+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/3827779051 wait complete. 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.438+0000 7f0ccd7a7640 1 Processor -- start 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.438+0000 7f0ccd7a7640 1 -- start start 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.438+0000 7f0ccd7a7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.438+0000 7f0ccd7a7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc810b9d0 con 0x7f0cc8108060 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.438+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.438+0000 7f0cc6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41498/0 (socket says 172.21.15.78:41498) 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.438+0000 7f0cc6ffd640 1 -- 172.21.15.78:0/214388097 learned_addr learned my addr 172.21.15.78:0/214388097 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.439+0000 7f0cc6ffd640 1 -- 172.21.15.78:0/214388097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc819dca0 con 0x7f0cc8108060 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.439+0000 7f0cc6ffd640 1 --2- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc819d760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8032ce0 tx=0x7f0cb8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.439+0000 7f0cc4ff9640 1 -- 172.21.15.78:0/214388097 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb8042070 con 0x7f0cc8108060 2024-09-16T00:03:32.185 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.439+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc819df30 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.439+0000 7f0cc4ff9640 1 -- 172.21.15.78:0/214388097 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0cb80308c0 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.439+0000 7f0cc4ff9640 1 -- 172.21.15.78:0/214388097 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb8037040 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.439+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc81a0c20 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.440+0000 7f0cc4ff9640 1 -- 172.21.15.78:0/214388097 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f0cb8045070 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.440+0000 7f0cc4ff9640 1 --2- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f0cb40391b0 0x7f0cb403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.440+0000 7f0cc4ff9640 1 -- 172.21.15.78:0/214388097 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0cb806c4e0 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.440+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc81084c0 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.441+0000 7f0cbffff640 1 --2- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f0cb40391b0 0x7f0cb403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.441+0000 7f0cbffff640 1 --2- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f0cb40391b0 0x7f0cb403b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00098c0 tx=0x7f0cb0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.445+0000 7f0cc4ff9640 1 -- 172.21.15.78:0/214388097 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cb8045520 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.610+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0cc81a2370 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.611+0000 7f0cc4ff9640 1 -- 172.21.15.78:0/214388097 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f0cb806bf00 con 0x7f0cc8108060 2024-09-16T00:03:32.186 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f0cb40391b0 msgr2=0x7f0cb403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 --2- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f0cb40391b0 0x7f0cb403b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00098c0 tx=0x7f0cb0006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 msgr2=0x7f0cc819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 --2- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc819d760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8032ce0 tx=0x7f0cb8032d10 comp rx=0 tx=0).stop 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 shutdown_connections 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 --2- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f0cb40391b0 0x7f0cb403b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 --2- 172.21.15.78:0/214388097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0cc8108060 0x7f0cc819d760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 >> 172.21.15.78:0/214388097 conn(0x7f0cc8101ca0 msgr2=0x7f0cc81027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 shutdown_connections 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:31.613+0000 7f0ccd7a7640 1 -- 172.21.15.78:0/214388097 wait complete. 2024-09-16T00:03:32.187 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling cephadm module... 2024-09-16T00:03:33.449 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/966196069' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T00:03:33.450 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: ignoring --setuser ceph since I am not root 2024-09-16T00:03:33.450 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: ignoring --setgroup ceph since I am not root 2024-09-16T00:03:33.450 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:33.285+0000 7f69fcdc9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:03:33.450 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:33.342+0000 7f69fcdc9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:03:33.709 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:33.492+0000 7f69fcdc9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:03:33.709 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:33.558+0000 7f69fcdc9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:03:33.709 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:33.707+0000 7f69fcdc9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:03:33.741 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.564+0000 7f3b65596640 1 Processor -- start 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.564+0000 7f3b65596640 1 -- start start 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.564+0000 7f3b65596640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b60105cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.564+0000 7f3b65596640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b60106280 con 0x7f3b601058b0 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.564+0000 7f3b5effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b60105cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.564+0000 7f3b5effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b60105cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41510/0 (socket says 172.21.15.78:41510) 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.564+0000 7f3b5effd640 1 -- 172.21.15.78:0/392415977 learned_addr learned my addr 172.21.15.78:0/392415977 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.564+0000 7f3b5effd640 1 -- 172.21.15.78:0/392415977 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b60108890 con 0x7f3b601058b0 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.565+0000 7f3b5effd640 1 --2- 172.21.15.78:0/392415977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b60105cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009820 tx=0x7f3b5002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4958b8239a0e1a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.565+0000 7f3b5dffb640 1 -- 172.21.15.78:0/392415977 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b50037070 con 0x7f3b601058b0 2024-09-16T00:03:33.742 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.565+0000 7f3b5dffb640 1 -- 172.21.15.78:0/392415977 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b50032900 con 0x7f3b601058b0 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.565+0000 7f3b5dffb640 1 -- 172.21.15.78:0/392415977 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b50030850 con 0x7f3b601058b0 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.566+0000 7f3b65596640 1 -- 172.21.15.78:0/392415977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 msgr2=0x7f3b60105cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.566+0000 7f3b65596640 1 --2- 172.21.15.78:0/392415977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b60105cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009820 tx=0x7f3b5002a270 comp rx=0 tx=0).stop 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.566+0000 7f3b65596640 1 -- 172.21.15.78:0/392415977 shutdown_connections 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.566+0000 7f3b65596640 1 --2- 172.21.15.78:0/392415977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b60105cb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.566+0000 7f3b65596640 1 -- 172.21.15.78:0/392415977 >> 172.21.15.78:0/392415977 conn(0x7f3b60101420 msgr2=0x7f3b60103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.566+0000 7f3b65596640 1 -- 172.21.15.78:0/392415977 shutdown_connections 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.566+0000 7f3b65596640 1 -- 172.21.15.78:0/392415977 wait complete. 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.566+0000 7f3b65596640 1 Processor -- start 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b65596640 1 -- start start 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b65596640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b6019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b65596640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6010bac0 con 0x7f3b601058b0 2024-09-16T00:03:33.743 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b5effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b6019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b5effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b6019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41516/0 (socket says 172.21.15.78:41516) 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b5effd640 1 -- 172.21.15.78:0/966196069 learned_addr learned my addr 172.21.15.78:0/966196069 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b5effd640 1 -- 172.21.15.78:0/966196069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6019bf30 con 0x7f3b601058b0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b5effd640 1 --2- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b6019b9f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009940 tx=0x7f3b500326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b47fff640 1 -- 172.21.15.78:0/966196069 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b50042070 con 0x7f3b601058b0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6019c1c0 con 0x7f3b601058b0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b47fff640 1 -- 172.21.15.78:0/966196069 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3b50032be0 con 0x7f3b601058b0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.567+0000 7f3b47fff640 1 -- 172.21.15.78:0/966196069 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b50037040 con 0x7f3b601058b0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.568+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6019eeb0 con 0x7f3b601058b0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.568+0000 7f3b47fff640 1 -- 172.21.15.78:0/966196069 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f3b50045070 con 0x7f3b601058b0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.568+0000 7f3b47fff640 1 --2- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f3b48039160 0x7f3b4803b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.568+0000 7f3b47fff640 1 -- 172.21.15.78:0/966196069 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b5006c100 con 0x7f3b601058b0 2024-09-16T00:03:33.744 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.568+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b60105d30 con 0x7f3b601058b0 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.569+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f3b48039160 0x7f3b4803b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.569+0000 7f3b5e7fc640 1 --2- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f3b48039160 0x7f3b4803b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b540098c0 tx=0x7f3b54006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.573+0000 7f3b47fff640 1 -- 172.21.15.78:0/966196069 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b50032d90 con 0x7f3b601058b0 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:32.781+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f3b6002d7c0 con 0x7f3b601058b0 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.147+0000 7f3b47fff640 1 -- 172.21.15.78:0/966196069 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f3b5002ed90 con 0x7f3b601058b0 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.149+0000 7f3b47fff640 1 -- 172.21.15.78:0/966196069 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f3b5003c720 con 0x7f3b601058b0 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.150+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f3b48039160 msgr2=0x7f3b4803b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.150+0000 7f3b65596640 1 --2- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f3b48039160 0x7f3b4803b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b540098c0 tx=0x7f3b54006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.150+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 msgr2=0x7f3b6019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.150+0000 7f3b65596640 1 --2- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b6019b9f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009940 tx=0x7f3b500326e0 comp rx=0 tx=0).stop 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.151+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 shutdown_connections 2024-09-16T00:03:33.745 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.151+0000 7f3b65596640 1 --2- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f3b48039160 0x7f3b4803b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:33.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.151+0000 7f3b65596640 1 --2- 172.21.15.78:0/966196069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b601058b0 0x7f3b6019b9f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:33.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.151+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 >> 172.21.15.78:0/966196069 conn(0x7f3b60101420 msgr2=0x7f3b601022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:33.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.151+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 shutdown_connections 2024-09-16T00:03:33.746 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:33.151+0000 7f3b65596640 1 -- 172.21.15.78:0/966196069 wait complete. 2024-09-16T00:03:33.984 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:33 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:33.853+0000 7f69fcdc9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:03:34.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:34 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/966196069' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T00:03:34.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:34 smithi078 ceph-mon[31157]: mgrmap e5: x(active, since 5s) 2024-09-16T00:03:34.237 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:34 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:34.173+0000 7f69fcdc9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:03:34.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:34 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:34.234+0000 7f69fcdc9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:03:34.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:34 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:34.423+0000 7f69fcdc9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.144+0000 7fb13547e640 1 Processor -- start 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.144+0000 7fb13547e640 1 -- start start 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.144+0000 7fb13547e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb1301068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:34.965 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.144+0000 7fb13547e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb130106eb0 con 0x7fb1301064e0 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.145+0000 7fb12effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb1301068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.145+0000 7fb12effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb1301068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41542/0 (socket says 172.21.15.78:41542) 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.145+0000 7fb12effd640 1 -- 172.21.15.78:0/1771688557 learned_addr learned my addr 172.21.15.78:0/1771688557 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.145+0000 7fb12effd640 1 -- 172.21.15.78:0/1771688557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb130107690 con 0x7fb1301064e0 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.146+0000 7fb12effd640 1 --2- 172.21.15.78:0/1771688557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb1301068e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb120009820 tx=0x7fb12002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5352132716775e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.146+0000 7fb12e7fc640 1 -- 172.21.15.78:0/1771688557 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120037070 con 0x7fb1301064e0 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.146+0000 7fb12e7fc640 1 -- 172.21.15.78:0/1771688557 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb120032900 con 0x7fb1301064e0 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.146+0000 7fb12e7fc640 1 -- 172.21.15.78:0/1771688557 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120030850 con 0x7fb1301064e0 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.147+0000 7fb13547e640 1 -- 172.21.15.78:0/1771688557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 msgr2=0x7fb1301068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.147+0000 7fb13547e640 1 --2- 172.21.15.78:0/1771688557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb1301068e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb120009820 tx=0x7fb12002a270 comp rx=0 tx=0).stop 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.147+0000 7fb13547e640 1 -- 172.21.15.78:0/1771688557 shutdown_connections 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.147+0000 7fb13547e640 1 --2- 172.21.15.78:0/1771688557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb1301068e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:34.966 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.147+0000 7fb13547e640 1 -- 172.21.15.78:0/1771688557 >> 172.21.15.78:0/1771688557 conn(0x7fb130101c70 msgr2=0x7fb1301040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.147+0000 7fb13547e640 1 -- 172.21.15.78:0/1771688557 shutdown_connections 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.147+0000 7fb13547e640 1 -- 172.21.15.78:0/1771688557 wait complete. 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.147+0000 7fb13547e640 1 Processor -- start 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.148+0000 7fb13547e640 1 -- start start 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.148+0000 7fb13547e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb13019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.148+0000 7fb13547e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb13010aa40 con 0x7fb1301064e0 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.148+0000 7fb12effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb13019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.148+0000 7fb12effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb13019daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41550/0 (socket says 172.21.15.78:41550) 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.148+0000 7fb12effd640 1 -- 172.21.15.78:0/3070961422 learned_addr learned my addr 172.21.15.78:0/3070961422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.148+0000 7fb12effd640 1 -- 172.21.15.78:0/3070961422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13019e030 con 0x7fb1301064e0 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.148+0000 7fb12effd640 1 --2- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb13019daf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb1200097f0 tx=0x7fb120030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.149+0000 7fb12cff9640 1 -- 172.21.15.78:0/3070961422 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120042070 con 0x7fb1301064e0 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.149+0000 7fb12cff9640 1 -- 172.21.15.78:0/3070961422 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb120032ca0 con 0x7fb1301064e0 2024-09-16T00:03:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.149+0000 7fb12cff9640 1 -- 172.21.15.78:0/3070961422 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb120037040 con 0x7fb1301064e0 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.149+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13019e2c0 con 0x7fb1301064e0 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.149+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1301a0fb0 con 0x7fb1301064e0 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.150+0000 7fb12cff9640 1 -- 172.21.15.78:0/3070961422 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fb120045070 con 0x7fb1301064e0 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.150+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13010abc0 con 0x7fb1301064e0 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.151+0000 7fb12cff9640 1 --2- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb11c039200 0x7fb11c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.151+0000 7fb12cff9640 1 -- 172.21.15.78:0/3070961422 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb12006c150 con 0x7fb1301064e0 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.151+0000 7fb127fff640 1 -- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb11c039200 msgr2=0x7fb11c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1515675480 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.151+0000 7fb127fff640 1 --2- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb11c039200 0x7fb11c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.155+0000 7fb12cff9640 1 -- 172.21.15.78:0/3070961422 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb120045360 con 0x7fb1301064e0 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.352+0000 7fb127fff640 1 -- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb11c039200 msgr2=0x7fb11c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1515675480 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.352+0000 7fb127fff640 1 --2- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb11c039200 0x7fb11c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.369+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb13002d7c0 con 0x7fb1301064e0 2024-09-16T00:03:34.968 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.369+0000 7fb12cff9640 1 -- 172.21.15.78:0/3070961422 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb12002ed90 con 0x7fb1301064e0 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.371+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb11c039200 msgr2=0x7fb11c03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.371+0000 7fb13547e640 1 --2- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb11c039200 0x7fb11c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.371+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 msgr2=0x7fb13019daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.371+0000 7fb13547e640 1 --2- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb13019daf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb1200097f0 tx=0x7fb120030ec0 comp rx=0 tx=0).stop 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.371+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 shutdown_connections 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.371+0000 7fb13547e640 1 --2- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7fb11c039200 0x7fb11c03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.371+0000 7fb13547e640 1 --2- 172.21.15.78:0/3070961422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb1301064e0 0x7fb13019daf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.371+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 >> 172.21.15.78:0/3070961422 conn(0x7fb130101c70 msgr2=0x7fb130102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.372+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 shutdown_connections 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:34.372+0000 7fb13547e640 1 -- 172.21.15.78:0/3070961422 wait complete. 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-09-16T00:03:34.969 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 5... 2024-09-16T00:03:35.413 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:35 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3070961422' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:03:35.413 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:35 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:35.319+0000 7f69fcdc9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:03:35.413 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:35 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:35.389+0000 7f69fcdc9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:03:36.353 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.097+0000 7f69fcdc9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:03:36.353 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.211+0000 7f69fcdc9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:03:36.627 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.352+0000 7f69fcdc9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:03:36.627 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.412+0000 7f69fcdc9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:03:36.628 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.559+0000 7f69fcdc9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:03:36.628 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.626+0000 7f69fcdc9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:03:36.908 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.686+0000 7f69fcdc9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:03:36.909 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.843+0000 7f69fcdc9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:03:36.909 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.907+0000 7f69fcdc9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:03:37.173 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:36 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:36.973+0000 7f69fcdc9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:03:37.173 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:37 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:37.055+0000 7f69fcdc9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:03:37.173 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:37 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:37.114+0000 7f69fcdc9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:03:37.173 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:37 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:37.171+0000 7f69fcdc9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:03:38.270 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:37 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:37.952+0000 7f69fcdc9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:03:38.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: Active manager daemon x restarted 2024-09-16T00:03:38.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: Activating manager daemon x 2024-09-16T00:03:38.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T00:03:38.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: mgrmap e6: x(active, starting, since 0.00391648s) 2024-09-16T00:03:38.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: Manager daemon x is now available 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:03:38.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:38 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:03:39.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-mon[31157]: Found migration_current of "None". Setting to last migration. 2024-09-16T00:03:39.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:39.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:39.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-mon[31157]: mgrmap e7: x(active, since 1.0056s) 2024-09-16T00:03:39.270 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:39.174+0000 7f696992e640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:03:39] ENGINE Error in HTTPServer.serve 2024-09-16T00:03:39.270 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: Traceback (most recent call last): 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: self._connections.run(self.expiration_interval) 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: self._run(expiration_interval) 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: s = self.context.wrap_socket( 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:03:39.271 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: return self.sslsocket_class._create( 2024-09-16T00:03:39.272 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:03:39.272 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: self.do_handshake() 2024-09-16T00:03:39.272 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:03:39.272 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: self._sslobj.do_handshake() 2024-09-16T00:03:39.272 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:03:39.272 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:39.597 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.411+0000 7f2641752640 1 Processor -- start 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.412+0000 7f2641752640 1 -- start start 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.412+0000 7f2641752640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c10a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.412+0000 7f2641752640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f263c07bd90 con 0x7f263c108100 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.412+0000 7f263affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c10a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.412+0000 7f263affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c10a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41554/0 (socket says 172.21.15.78:41554) 2024-09-16T00:03:39.598 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.412+0000 7f263affd640 1 -- 172.21.15.78:0/2142541672 learned_addr learned my addr 172.21.15.78:0/2142541672 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.412+0000 7f263affd640 1 -- 172.21.15.78:0/2142541672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c10aa30 con 0x7f263c108100 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.413+0000 7f263affd640 1 --2- 172.21.15.78:0/2142541672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c10a4f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f262c009820 tx=0x7f262c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bebce2d7cbd129 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.413+0000 7f263a7fc640 1 -- 172.21.15.78:0/2142541672 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f262c037070 con 0x7f263c108100 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.413+0000 7f263a7fc640 1 -- 172.21.15.78:0/2142541672 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f262c032900 con 0x7f263c108100 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.413+0000 7f263a7fc640 1 -- 172.21.15.78:0/2142541672 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f262c030850 con 0x7f263c108100 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.414+0000 7f2641752640 1 -- 172.21.15.78:0/2142541672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 msgr2=0x7f263c10a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.414+0000 7f2641752640 1 --2- 172.21.15.78:0/2142541672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c10a4f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f262c009820 tx=0x7f262c02a270 comp rx=0 tx=0).stop 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.414+0000 7f2641752640 1 -- 172.21.15.78:0/2142541672 shutdown_connections 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.414+0000 7f2641752640 1 --2- 172.21.15.78:0/2142541672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c10a4f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.414+0000 7f2641752640 1 -- 172.21.15.78:0/2142541672 >> 172.21.15.78:0/2142541672 conn(0x7f263c101d00 msgr2=0x7f263c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.414+0000 7f2641752640 1 -- 172.21.15.78:0/2142541672 shutdown_connections 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.414+0000 7f2641752640 1 -- 172.21.15.78:0/2142541672 wait complete. 2024-09-16T00:03:39.599 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.414+0000 7f2641752640 1 Processor -- start 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f2641752640 1 -- start start 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f2641752640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f2641752640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f263c10ba10 con 0x7f263c108100 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f263affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f263affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c19b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41556/0 (socket says 172.21.15.78:41556) 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f263affd640 1 -- 172.21.15.78:0/1786259686 learned_addr learned my addr 172.21.15.78:0/1786259686 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f263affd640 1 -- 172.21.15.78:0/1786259686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c19bad0 con 0x7f263c108100 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f263affd640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c19b590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f262c032ce0 tx=0x7f262c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f262c042070 con 0x7f263c108100 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f262c0308c0 con 0x7f263c108100 2024-09-16T00:03:39.600 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f262c037040 con 0x7f263c108100 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.415+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c19bd60 con 0x7f263c108100 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.416+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c19ea50 con 0x7f263c108100 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.416+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f262c045070 con 0x7f263c108100 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.417+0000 7f2638ff9640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 0x7f262803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.417+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 --> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f262803bd80 con 0x7f26280391b0 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.417+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f262c06c4d0 con 0x7f263c108100 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.417+0000 7f2633fff640 1 -- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 msgr2=0x7f262803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1515675480 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.417+0000 7f2633fff640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 0x7f262803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.618+0000 7f2633fff640 1 -- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 msgr2=0x7f262803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1515675480 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:35.618+0000 7f2633fff640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 0x7f262803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:36.019+0000 7f2633fff640 1 -- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 msgr2=0x7f262803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1515675480 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:36.019+0000 7f2633fff640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 0x7f262803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:03:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:36.820+0000 7f2633fff640 1 -- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 msgr2=0x7f262803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1515675480 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:36.820+0000 7f2633fff640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 0x7f262803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:37.957+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f262c06af90 con 0x7f263c108100 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:37.957+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 msgr2=0x7f262803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:37.957+0000 7f2638ff9640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 0x7f262803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.957+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f262c03cd00 con 0x7f263c108100 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.957+0000 7f2638ff9640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f262803cae0 0x7f262803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.957+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f262803bd80 con 0x7f262803cae0 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.958+0000 7f2633fff640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f262803cae0 0x7f262803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.958+0000 7f2633fff640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f262803cae0 0x7f262803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2624003980 tx=0x7f2624006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.959+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mgr.14118 v2:172.21.15.78:6800/1052494441 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f262803bd80 con 0x7f262803cae0 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f263c108560 con 0x7f262803cae0 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2638ff9640 1 -- 172.21.15.78:0/1786259686 <== mgr.14118 v2:172.21.15.78:6800/1052494441 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f263c108560 con 0x7f262803cae0 2024-09-16T00:03:39.602 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f262803cae0 msgr2=0x7f262803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f262803cae0 0x7f262803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2624003980 tx=0x7f2624006e10 comp rx=0 tx=0).stop 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 msgr2=0x7f263c19b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c19b590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f262c032ce0 tx=0x7f262c032d10 comp rx=0 tx=0).stop 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 shutdown_connections 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f262803cae0 0x7f262803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:6800/1515675480,v1:172.21.15.78:6801/1515675480] conn(0x7f26280391b0 0x7f262803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 --2- 172.21.15.78:0/1786259686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f263c108100 0x7f263c19b590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 >> 172.21.15.78:0/1786259686 conn(0x7f263c101d00 msgr2=0x7f263c102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.966+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 shutdown_connections 2024-09-16T00:03:39.603 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:38.967+0000 7f2641752640 1 -- 172.21.15.78:0/1786259686 wait complete. 2024-09-16T00:03:39.604 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 5 is available 2024-09-16T00:03:39.604 INFO:teuthology.orchestra.run.smithi078.stdout:Setting orchestrator backend to cephadm... 2024-09-16T00:03:40.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:40 smithi078 ceph-mon[31157]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T00:03:40.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:40 smithi078 ceph-mon[31157]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T00:03:40.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:40 smithi078 ceph-mon[31157]: [16/Sep/2024:00:03:39] ENGINE Bus STARTING 2024-09-16T00:03:40.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:40 smithi078 ceph-mon[31157]: [16/Sep/2024:00:03:39] ENGINE Serving on https://172.21.15.78:7150 2024-09-16T00:03:40.178 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:40 smithi078 ceph-mon[31157]: [16/Sep/2024:00:03:39] ENGINE Error in HTTPServer.serve 2024-09-16T00:03:40.178 INFO:journalctl@ceph.mon.a.smithi078.stdout: Traceback (most recent call last): 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._run(expiration_interval) 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: s = self.context.wrap_socket( 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:03:40.179 INFO:journalctl@ceph.mon.a.smithi078.stdout: return self.sslsocket_class._create( 2024-09-16T00:03:40.180 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:03:40.180 INFO:journalctl@ceph.mon.a.smithi078.stdout: self.do_handshake() 2024-09-16T00:03:40.180 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:03:40.180 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._sslobj.do_handshake() 2024-09-16T00:03:40.180 INFO:journalctl@ceph.mon.a.smithi078.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:03:40.180 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:40 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:03:40.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.985+0000 7f233b3bf640 1 Processor -- start 2024-09-16T00:03:40.779 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.985+0000 7f233b3bf640 1 -- start start 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.985+0000 7f233b3bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233410a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.985+0000 7f233b3bf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2334075800 con 0x7f2334108080 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.985+0000 7f233915b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233410a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.985+0000 7f233915b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233410a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44392/0 (socket says 172.21.15.78:44392) 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.985+0000 7f233915b640 1 -- 172.21.15.78:0/2902069251 learned_addr learned my addr 172.21.15.78:0/2902069251 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.986+0000 7f233915b640 1 -- 172.21.15.78:0/2902069251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233410a9b0 con 0x7f2334108080 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.986+0000 7f233915b640 1 --2- 172.21.15.78:0/2902069251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233410a470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f232c009820 tx=0x7f232c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb72f7a723f96c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.986+0000 7f2333fff640 1 -- 172.21.15.78:0/2902069251 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232c037070 con 0x7f2334108080 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.986+0000 7f2333fff640 1 -- 172.21.15.78:0/2902069251 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f232c032900 con 0x7f2334108080 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.986+0000 7f2333fff640 1 -- 172.21.15.78:0/2902069251 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232c030850 con 0x7f2334108080 2024-09-16T00:03:40.780 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.987+0000 7f233b3bf640 1 -- 172.21.15.78:0/2902069251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 msgr2=0x7f233410a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.987+0000 7f233b3bf640 1 --2- 172.21.15.78:0/2902069251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233410a470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f232c009820 tx=0x7f232c02a270 comp rx=0 tx=0).stop 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.987+0000 7f233b3bf640 1 -- 172.21.15.78:0/2902069251 shutdown_connections 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.987+0000 7f233b3bf640 1 --2- 172.21.15.78:0/2902069251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233410a470 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.987+0000 7f233b3bf640 1 -- 172.21.15.78:0/2902069251 >> 172.21.15.78:0/2902069251 conn(0x7f2334101ca0 msgr2=0x7f23341040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.987+0000 7f233b3bf640 1 -- 172.21.15.78:0/2902069251 shutdown_connections 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.987+0000 7f233b3bf640 1 -- 172.21.15.78:0/2902069251 wait complete. 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233b3bf640 1 Processor -- start 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233b3bf640 1 -- start start 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233b3bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233419b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233b3bf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f233410b990 con 0x7f2334108080 2024-09-16T00:03:40.781 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233915b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233419b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:40.782 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233915b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233419b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44396/0 (socket says 172.21.15.78:44396) 2024-09-16T00:03:40.782 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233915b640 1 -- 172.21.15.78:0/214907110 learned_addr learned my addr 172.21.15.78:0/214907110 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:40.782 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233915b640 1 -- 172.21.15.78:0/214907110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233419bad0 con 0x7f2334108080 2024-09-16T00:03:40.782 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233915b640 1 --2- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233419b590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f232c0097f0 tx=0x7f232c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:40.782 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f23327fc640 1 -- 172.21.15.78:0/214907110 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232c037040 con 0x7f2334108080 2024-09-16T00:03:40.782 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233419bd60 con 0x7f2334108080 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f23327fc640 1 -- 172.21.15.78:0/214907110 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f232c040070 con 0x7f2334108080 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f23327fc640 1 -- 172.21.15.78:0/214907110 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232c03a390 con 0x7f2334108080 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.988+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233419ea50 con 0x7f2334108080 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.989+0000 7f23327fc640 1 -- 172.21.15.78:0/214907110 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f232c03a650 con 0x7f2334108080 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.989+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23341084e0 con 0x7f2334108080 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.989+0000 7f23327fc640 1 --2- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f231c0390a0 0x7f231c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.989+0000 7f23327fc640 1 -- 172.21.15.78:0/214907110 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f232c06c770 con 0x7f2334108080 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.989+0000 7f233895a640 1 --2- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f231c0390a0 0x7f231c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:40.784 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.990+0000 7f233895a640 1 --2- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f231c0390a0 0x7f231c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f23280098c0 tx=0x7f2328006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:39.994+0000 7f23327fc640 1 -- 172.21.15.78:0/214907110 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f232c03b430 con 0x7f2334108080 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.009+0000 7f23327fc640 1 -- 172.21.15.78:0/214907110 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f232c0457e0 con 0x7f2334108080 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.177+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2334102f60 con 0x7f231c0390a0 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.182+0000 7f23327fc640 1 -- 172.21.15.78:0/214907110 <== mgr.14118 v2:172.21.15.78:6800/1052494441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2334102f60 con 0x7f231c0390a0 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f231c0390a0 msgr2=0x7f231c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 --2- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f231c0390a0 0x7f231c03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f23280098c0 tx=0x7f2328006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 msgr2=0x7f233419b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 --2- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233419b590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f232c0097f0 tx=0x7f232c030d80 comp rx=0 tx=0).stop 2024-09-16T00:03:40.785 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 shutdown_connections 2024-09-16T00:03:40.786 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 --2- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f231c0390a0 0x7f231c03b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:40.786 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 --2- 172.21.15.78:0/214907110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2334108080 0x7f233419b590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:40.786 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 >> 172.21.15.78:0/214907110 conn(0x7f2334101ca0 msgr2=0x7f2334102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:40.786 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 shutdown_connections 2024-09-16T00:03:40.786 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:40.184+0000 7f233b3bf640 1 -- 172.21.15.78:0/214907110 wait complete. 2024-09-16T00:03:41.113 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:41 smithi078 ceph-mon[31157]: [16/Sep/2024:00:03:39] ENGINE Serving on http://172.21.15.78:8765 2024-09-16T00:03:41.114 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:41 smithi078 ceph-mon[31157]: [16/Sep/2024:00:03:39] ENGINE Bus STARTED 2024-09-16T00:03:41.114 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:41 smithi078 ceph-mon[31157]: mgrmap e8: x(active, since 2s) 2024-09-16T00:03:41.114 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:41 smithi078 ceph-mon[31157]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:03:41.114 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:41 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:41.114 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:41 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:03:41.991 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.194+0000 7f372c9de640 1 Processor -- start 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.194+0000 7f372c9de640 1 -- start start 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.195+0000 7f372c9de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f3728107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.195+0000 7f372c9de640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3728107f40 con 0x7f3728107600 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.195+0000 7f372659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f3728107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.195+0000 7f372659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f3728107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44402/0 (socket says 172.21.15.78:44402) 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.195+0000 7f372659c640 1 -- 172.21.15.78:0/684105156 learned_addr learned my addr 172.21.15.78:0/684105156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.195+0000 7f372659c640 1 -- 172.21.15.78:0/684105156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37281080c0 con 0x7f3728107600 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.196+0000 7f372659c640 1 --2- 172.21.15.78:0/684105156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f3728107a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3718009820 tx=0x7f371802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c652ef07c81453 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.196+0000 7f3725d9b640 1 -- 172.21.15.78:0/684105156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3718037070 con 0x7f3728107600 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.196+0000 7f3725d9b640 1 -- 172.21.15.78:0/684105156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3718032900 con 0x7f3728107600 2024-09-16T00:03:41.992 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.196+0000 7f3725d9b640 1 -- 172.21.15.78:0/684105156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3718030850 con 0x7f3728107600 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.196+0000 7f372c9de640 1 -- 172.21.15.78:0/684105156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 msgr2=0x7f3728107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.196+0000 7f372c9de640 1 --2- 172.21.15.78:0/684105156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f3728107a00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3718009820 tx=0x7f371802a270 comp rx=0 tx=0).stop 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.197+0000 7f372c9de640 1 -- 172.21.15.78:0/684105156 shutdown_connections 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.197+0000 7f372c9de640 1 --2- 172.21.15.78:0/684105156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f3728107a00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.197+0000 7f372c9de640 1 -- 172.21.15.78:0/684105156 >> 172.21.15.78:0/684105156 conn(0x7f3728102e50 msgr2=0x7f3728105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.197+0000 7f372c9de640 1 -- 172.21.15.78:0/684105156 shutdown_connections 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.197+0000 7f372c9de640 1 -- 172.21.15.78:0/684105156 wait complete. 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.197+0000 7f372c9de640 1 Processor -- start 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.197+0000 7f372c9de640 1 -- start start 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372c9de640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f372819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372c9de640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f372810ba70 con 0x7f3728107600 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f372819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:41.993 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f372819b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44414/0 (socket says 172.21.15.78:44414) 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372659c640 1 -- 172.21.15.78:0/1857439440 learned_addr learned my addr 172.21.15.78:0/1857439440 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372659c640 1 -- 172.21.15.78:0/1857439440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372819baa0 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372659c640 1 --2- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f372819b560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3718032ce0 tx=0x7f3718032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f371f7fe640 1 -- 172.21.15.78:0/1857439440 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3718042070 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f371f7fe640 1 -- 172.21.15.78:0/1857439440 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f37180308c0 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372819bd30 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f371f7fe640 1 -- 172.21.15.78:0/1857439440 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3718037040 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.198+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372819ea20 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.199+0000 7f371f7fe640 1 -- 172.21.15.78:0/1857439440 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f3718045070 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.199+0000 7f371f7fe640 1 --2- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f3714039200 0x7f371403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.199+0000 7f371f7fe640 1 -- 172.21.15.78:0/1857439440 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f371806c330 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.199+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3728107a80 con 0x7f3728107600 2024-09-16T00:03:41.994 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.199+0000 7f371ffff640 1 --2- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f3714039200 0x7f371403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.200+0000 7f371ffff640 1 --2- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f3714039200 0x7f371403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37100098c0 tx=0x7f3710006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.204+0000 7f371f7fe640 1 -- 172.21.15.78:0/1857439440 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3718045360 con 0x7f3728107600 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.377+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3728104090 con 0x7f3714039200 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.378+0000 7f371f7fe640 1 -- 172.21.15.78:0/1857439440 <== mgr.14118 v2:172.21.15.78:6800/1052494441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3728104090 con 0x7f3714039200 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f3714039200 msgr2=0x7f371403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 --2- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f3714039200 0x7f371403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37100098c0 tx=0x7f3710006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 msgr2=0x7f372819b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 --2- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f372819b560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3718032ce0 tx=0x7f3718032d10 comp rx=0 tx=0).stop 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 shutdown_connections 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 --2- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f3714039200 0x7f371403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:41.995 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 --2- 172.21.15.78:0/1857439440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3728107600 0x7f372819b560 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:41.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 >> 172.21.15.78:0/1857439440 conn(0x7f3728102e50 msgr2=0x7f3728103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:41.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 shutdown_connections 2024-09-16T00:03:41.996 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:41.380+0000 7f372c9de640 1 -- 172.21.15.78:0/1857439440 wait complete. 2024-09-16T00:03:41.996 INFO:teuthology.orchestra.run.smithi078.stdout:Generating ssh key... 2024-09-16T00:03:42.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-mon[31157]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:03:42.956 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: Generating public/private rsa key pair. 2024-09-16T00:03:42.956 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: Your identification has been saved in /tmp/tmpnlne2yyl/key 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: Your public key has been saved in /tmp/tmpnlne2yyl/key.pub 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: The key fingerprint is: 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: SHA256:F0noOjCYI+Z6bfqfvOaPnUNNNSGUphTYoiowz6q3DTg ceph-eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: The key's randomart image is: 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: +---[RSA 3072]----+ 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: | o.++... | 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: | o +.ooo | 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: | o . + oo. . | 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: |+.+ + o .. | 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: |+= o o .So. | 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: |.o+ o ... | 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: |Eo.. o | 2024-09-16T00:03:42.957 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: |o.ooo..+.. | 2024-09-16T00:03:42.958 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: |ooo=o+Bo+. | 2024-09-16T00:03:42.958 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: +----[SHA256]-----+ 2024-09-16T00:03:43.255 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.358+0000 7f270d638640 1 Processor -- start 2024-09-16T00:03:43.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.358+0000 7f270d638640 1 -- start start 2024-09-16T00:03:43.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.359+0000 7f270d638640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f2708106250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:43.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.359+0000 7f270d638640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2708075820 con 0x7f2708105e50 2024-09-16T00:03:43.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.359+0000 7f2706ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f2708106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:43.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.359+0000 7f2706ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f2708106250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44420/0 (socket says 172.21.15.78:44420) 2024-09-16T00:03:43.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.359+0000 7f2706ffd640 1 -- 172.21.15.78:0/1900265162 learned_addr learned my addr 172.21.15.78:0/1900265162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:43.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.359+0000 7f2706ffd640 1 -- 172.21.15.78:0/1900265162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2708106790 con 0x7f2708105e50 2024-09-16T00:03:43.256 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f2706ffd640 1 --2- 172.21.15.78:0/1900265162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f2708106250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009e80 tx=0x7f26f802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397e42d8bab51c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f2705ffb640 1 -- 172.21.15.78:0/1900265162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f8036070 con 0x7f2708105e50 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f2705ffb640 1 -- 172.21.15.78:0/1900265162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26f80318c0 con 0x7f2708105e50 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f2705ffb640 1 -- 172.21.15.78:0/1900265162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f8007ab0 con 0x7f2708105e50 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f270d638640 1 -- 172.21.15.78:0/1900265162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 msgr2=0x7f2708106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f270d638640 1 --2- 172.21.15.78:0/1900265162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f2708106250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009e80 tx=0x7f26f802a7a0 comp rx=0 tx=0).stop 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f270d638640 1 -- 172.21.15.78:0/1900265162 shutdown_connections 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f270d638640 1 --2- 172.21.15.78:0/1900265162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f2708106250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.360+0000 7f270d638640 1 -- 172.21.15.78:0/1900265162 >> 172.21.15.78:0/1900265162 conn(0x7f2708101ca0 msgr2=0x7f27081040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.361+0000 7f270d638640 1 -- 172.21.15.78:0/1900265162 shutdown_connections 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.361+0000 7f270d638640 1 -- 172.21.15.78:0/1900265162 wait complete. 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.361+0000 7f270d638640 1 Processor -- start 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.361+0000 7f270d638640 1 -- start start 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f270d638640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f270819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:43.257 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f270d638640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2708107890 con 0x7f2708105e50 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f2706ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f270819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f2706ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f270819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44430/0 (socket says 172.21.15.78:44430) 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f2706ffd640 1 -- 172.21.15.78:0/3602078415 learned_addr learned my addr 172.21.15.78:0/3602078415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f2706ffd640 1 -- 172.21.15.78:0/3602078415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270819dca0 con 0x7f2708105e50 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f2706ffd640 1 --2- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f270819d760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26f8031ca0 tx=0x7f26f8031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f26effff640 1 -- 172.21.15.78:0/3602078415 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f8036050 con 0x7f2708105e50 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f26effff640 1 -- 172.21.15.78:0/3602078415 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26f8042070 con 0x7f2708105e50 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270819df30 con 0x7f2708105e50 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f26effff640 1 -- 172.21.15.78:0/3602078415 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26f803d820 con 0x7f2708105e50 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.362+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27081a0c20 con 0x7f2708105e50 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.363+0000 7f26effff640 1 -- 172.21.15.78:0/3602078415 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f26f803b610 con 0x7f2708105e50 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.363+0000 7f26effff640 1 --2- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f26f0039200 0x7f26f003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:43.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.363+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2708106250 con 0x7f2708105e50 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.364+0000 7f26effff640 1 -- 172.21.15.78:0/3602078415 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f26f8007e00 con 0x7f2708105e50 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.364+0000 7f27067fc640 1 --2- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f26f0039200 0x7f26f003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.364+0000 7f27067fc640 1 --2- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f26f0039200 0x7f26f003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0098c0 tx=0x7f26fc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.368+0000 7f26effff640 1 -- 172.21.15.78:0/3602078415 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26f804c200 con 0x7f2708105e50 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.533+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f2708062630 con 0x7f26f0039200 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.602+0000 7f26effff640 1 -- 172.21.15.78:0/3602078415 <== mgr.14118 v2:172.21.15.78:6800/1052494441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2708062630 con 0x7f26f0039200 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f26f0039200 msgr2=0x7f26f003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 --2- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f26f0039200 0x7f26f003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0098c0 tx=0x7f26fc006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 msgr2=0x7f270819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 --2- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f270819d760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26f8031ca0 tx=0x7f26f8031cd0 comp rx=0 tx=0).stop 2024-09-16T00:03:43.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 shutdown_connections 2024-09-16T00:03:43.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 --2- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f26f0039200 0x7f26f003b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:43.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 --2- 172.21.15.78:0/3602078415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2708105e50 0x7f270819d760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:43.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 >> 172.21.15.78:0/3602078415 conn(0x7f2708101ca0 msgr2=0x7f2708191a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:43.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 shutdown_connections 2024-09-16T00:03:43.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:42.604+0000 7f270d638640 1 -- 172.21.15.78:0/3602078415 wait complete. 2024-09-16T00:03:43.814 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:43 smithi078 ceph-mon[31157]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:03:43.814 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:43 smithi078 ceph-mon[31157]: Generating ssh key... 2024-09-16T00:03:43.814 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:43 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:43.814 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:43 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:44.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnv2jWhnxu1NZjNQaifTtjiFZbokoouQu+KvWy6CNYRIZ5eTLAzKCh3mqihFDIpklzON0W9zyvoo4mLj5G32OYkTpsvpjHfYBqA0LnQFGjNfkaHQ3+Pn1SZUImR06qhBGQURdu1uD4AXvLvv+v5pls6/6tjWwWseGiNvYaAR/nGfbgjb5KiXtc5kmxu2vVKsZRpVYqZmctptqhUfN5RMTDsKATfg0C4JIVI/faFGDyFN8qqkkLQfksOXPWWNuPleYpnkC2uLimu1bGNPrnd0mgHiod57K0XWbKN9mLbgng/jGS+im1jzFA5xLwFCXfUUTHQiTAWA8u2YWfapiNm246N20zL0jamGb1thfqDflKzKkHVgcE6E/SFF8dbFhP2NtCs0a33XIh8SsxqZ/K7n1GC2ib8AOVk9/DG9S3BuNzO1y7ZVgD1jRhBgj5yJBm2nKH+z/xr504M52I6iU/L4UdUUw5WKiKRV5n5HF2Y+X0q6L7CKZDKiv83Y45/IbeSD0= ceph-eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:03:44.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.627+0000 7f31aeea8640 1 Processor -- start 2024-09-16T00:03:44.471 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.628+0000 7f31aeea8640 1 -- start start 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.628+0000 7f31aeea8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a810a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.628+0000 7f31aeea8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a80757a0 con 0x7f31a81080e0 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.628+0000 7f31acc44640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a810a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.628+0000 7f31acc44640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a810a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44434/0 (socket says 172.21.15.78:44434) 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.628+0000 7f31acc44640 1 -- 172.21.15.78:0/3933919753 learned_addr learned my addr 172.21.15.78:0/3933919753 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.628+0000 7f31acc44640 1 -- 172.21.15.78:0/3933919753 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a810aa10 con 0x7f31a81080e0 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.629+0000 7f31acc44640 1 --2- 172.21.15.78:0/3933919753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a810a4d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f319c009820 tx=0x7f319c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cfa666fc2f31ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.629+0000 7f31a77fe640 1 -- 172.21.15.78:0/3933919753 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c037070 con 0x7f31a81080e0 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.629+0000 7f31a77fe640 1 -- 172.21.15.78:0/3933919753 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f319c032900 con 0x7f31a81080e0 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.629+0000 7f31a77fe640 1 -- 172.21.15.78:0/3933919753 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c030850 con 0x7f31a81080e0 2024-09-16T00:03:44.472 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 -- 172.21.15.78:0/3933919753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 msgr2=0x7f31a810a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 --2- 172.21.15.78:0/3933919753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a810a4d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f319c009820 tx=0x7f319c02a270 comp rx=0 tx=0).stop 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 -- 172.21.15.78:0/3933919753 shutdown_connections 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 --2- 172.21.15.78:0/3933919753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a810a4d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 -- 172.21.15.78:0/3933919753 >> 172.21.15.78:0/3933919753 conn(0x7f31a8101ca0 msgr2=0x7f31a81040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 -- 172.21.15.78:0/3933919753 shutdown_connections 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 -- 172.21.15.78:0/3933919753 wait complete. 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 Processor -- start 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.630+0000 7f31aeea8640 1 -- start start 2024-09-16T00:03:44.473 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31aeea8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31aeea8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31a810b990 con 0x7f31a81080e0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31acc44640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31acc44640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a819b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44448/0 (socket says 172.21.15.78:44448) 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31acc44640 1 -- 172.21.15.78:0/2462161930 learned_addr learned my addr 172.21.15.78:0/2462161930 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31acc44640 1 -- 172.21.15.78:0/2462161930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a819bad0 con 0x7f31a81080e0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31acc44640 1 --2- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a819b590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f319c032ce0 tx=0x7f319c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31a5ffb640 1 -- 172.21.15.78:0/2462161930 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c042070 con 0x7f31a81080e0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31a5ffb640 1 -- 172.21.15.78:0/2462161930 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f319c0308c0 con 0x7f31a81080e0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31a5ffb640 1 -- 172.21.15.78:0/2462161930 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c037040 con 0x7f31a81080e0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.631+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a819bd60 con 0x7f31a81080e0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.632+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a819ea50 con 0x7f31a81080e0 2024-09-16T00:03:44.474 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.632+0000 7f31a5ffb640 1 -- 172.21.15.78:0/2462161930 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f319c045070 con 0x7f31a81080e0 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.632+0000 7f31a5ffb640 1 --2- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f31980391b0 0x7f319803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.633+0000 7f31a5ffb640 1 -- 172.21.15.78:0/2462161930 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f319c06c3b0 con 0x7f31a81080e0 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.633+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a8108540 con 0x7f31a81080e0 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.633+0000 7f31a7fff640 1 --2- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f31980391b0 0x7f319803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.633+0000 7f31a7fff640 1 --2- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f31980391b0 0x7f319803b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f31940098c0 tx=0x7f3194006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.637+0000 7f31a5ffb640 1 -- 172.21.15.78:0/2462161930 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f319c04b170 con 0x7f31a81080e0 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.812+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f31a8102f30 con 0x7f31980391b0 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.813+0000 7f31a5ffb640 1 -- 172.21.15.78:0/2462161930 <== mgr.14118 v2:172.21.15.78:6800/1052494441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f31a8102f30 con 0x7f31980391b0 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.815+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f31980391b0 msgr2=0x7f319803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:44.475 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.815+0000 7f31aeea8640 1 --2- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f31980391b0 0x7f319803b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f31940098c0 tx=0x7f3194006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.815+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 msgr2=0x7f31a819b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.815+0000 7f31aeea8640 1 --2- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a819b590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f319c032ce0 tx=0x7f319c032d10 comp rx=0 tx=0).stop 2024-09-16T00:03:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.815+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 shutdown_connections 2024-09-16T00:03:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.815+0000 7f31aeea8640 1 --2- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f31980391b0 0x7f319803b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.815+0000 7f31aeea8640 1 --2- 172.21.15.78:0/2462161930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31a81080e0 0x7f31a819b590 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.815+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 >> 172.21.15.78:0/2462161930 conn(0x7f31a8101ca0 msgr2=0x7f31a8102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.816+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 shutdown_connections 2024-09-16T00:03:44.476 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:43.816+0000 7f31aeea8640 1 -- 172.21.15.78:0/2462161930 wait complete. 2024-09-16T00:03:44.477 INFO:teuthology.orchestra.run.smithi078.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:03:44.477 INFO:teuthology.orchestra.run.smithi078.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T00:03:44.478 INFO:teuthology.orchestra.run.smithi078.stdout:Adding host smithi078... 2024-09-16T00:03:44.710 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:44 smithi078 ceph-mon[31157]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:03:45.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:45 smithi078 ceph-mon[31157]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:03:46.781 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:46 smithi078 ceph-mon[31157]: Deploying cephadm binary to smithi078 2024-09-16T00:03:48.829 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Added host 'smithi078' with addr '172.21.15.78' 2024-09-16T00:03:48.829 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.886+0000 7f2613c2d640 1 Processor -- start 2024-09-16T00:03:48.829 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.887+0000 7f2613c2d640 1 -- start start 2024-09-16T00:03:48.829 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.887+0000 7f2613c2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.887+0000 7f2613c2d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f260c107f40 con 0x7f260c107600 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.887+0000 7f26119c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.887+0000 7f26119c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44460/0 (socket says 172.21.15.78:44460) 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.887+0000 7f26119c9640 1 -- 172.21.15.78:0/2191828675 learned_addr learned my addr 172.21.15.78:0/2191828675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.888+0000 7f26119c9640 1 -- 172.21.15.78:0/2191828675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c1080c0 con 0x7f260c107600 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.888+0000 7f26119c9640 1 --2- 172.21.15.78:0/2191828675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c107a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2600009820 tx=0x7f260002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a0abf062703b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.888+0000 7f26109c7640 1 -- 172.21.15.78:0/2191828675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600037070 con 0x7f260c107600 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.888+0000 7f26109c7640 1 -- 172.21.15.78:0/2191828675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2600032900 con 0x7f260c107600 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.888+0000 7f26109c7640 1 -- 172.21.15.78:0/2191828675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600030850 con 0x7f260c107600 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.889+0000 7f2613c2d640 1 -- 172.21.15.78:0/2191828675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 msgr2=0x7f260c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.889+0000 7f2613c2d640 1 --2- 172.21.15.78:0/2191828675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c107a00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2600009820 tx=0x7f260002a270 comp rx=0 tx=0).stop 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.889+0000 7f2613c2d640 1 -- 172.21.15.78:0/2191828675 shutdown_connections 2024-09-16T00:03:48.830 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.889+0000 7f2613c2d640 1 --2- 172.21.15.78:0/2191828675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c107a00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.889+0000 7f2613c2d640 1 -- 172.21.15.78:0/2191828675 >> 172.21.15.78:0/2191828675 conn(0x7f260c102e50 msgr2=0x7f260c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.889+0000 7f2613c2d640 1 -- 172.21.15.78:0/2191828675 shutdown_connections 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.889+0000 7f2613c2d640 1 -- 172.21.15.78:0/2191828675 wait complete. 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f2613c2d640 1 Processor -- start 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f2613c2d640 1 -- start start 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f2613c2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f2613c2d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f260c10ba70 con 0x7f260c107600 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f26119c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f26119c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c19b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44462/0 (socket says 172.21.15.78:44462) 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f26119c9640 1 -- 172.21.15.78:0/1269620820 learned_addr learned my addr 172.21.15.78:0/1269620820 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f26119c9640 1 -- 172.21.15.78:0/1269620820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c19bad0 con 0x7f260c107600 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f26119c9640 1 --2- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c19b590 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2600032ce0 tx=0x7f2600032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:48.831 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f25feffd640 1 -- 172.21.15.78:0/1269620820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600042070 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f25feffd640 1 -- 172.21.15.78:0/1269620820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26000308c0 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c19bd60 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.890+0000 7f25feffd640 1 -- 172.21.15.78:0/1269620820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2600037040 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.891+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c19ea50 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.891+0000 7f25feffd640 1 -- 172.21.15.78:0/1269620820 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2600045070 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.891+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260c107a80 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.892+0000 7f25feffd640 1 --2- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f25f80391b0 0x7f25f803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.892+0000 7f25feffd640 1 -- 172.21.15.78:0/1269620820 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f260006c470 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.892+0000 7f26111c8640 1 --2- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f25f80391b0 0x7f25f803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.892+0000 7f26111c8640 1 --2- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f25f80391b0 0x7f25f803b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26080098c0 tx=0x7f2608006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:44.896+0000 7f25feffd640 1 -- 172.21.15.78:0/1269620820 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2600045520 con 0x7f260c107600 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:45.068+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi078", "addr": "172.21.15.78", "target": ["mon-mgr", ""]}) -- 0x7f260c104070 con 0x7f25f80391b0 2024-09-16T00:03:48.832 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.207+0000 7f25feffd640 1 -- 172.21.15.78:0/1269620820 <== mgr.14118 v2:172.21.15.78:6800/1052494441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f260c104070 con 0x7f25f80391b0 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.209+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f25f80391b0 msgr2=0x7f25f803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.209+0000 7f2613c2d640 1 --2- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f25f80391b0 0x7f25f803b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26080098c0 tx=0x7f2608006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.210+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 msgr2=0x7f260c19b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.210+0000 7f2613c2d640 1 --2- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c19b590 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2600032ce0 tx=0x7f2600032d10 comp rx=0 tx=0).stop 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.210+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 shutdown_connections 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.210+0000 7f2613c2d640 1 --2- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f25f80391b0 0x7f25f803b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.210+0000 7f2613c2d640 1 --2- 172.21.15.78:0/1269620820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f260c107600 0x7f260c19b590 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.210+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 >> 172.21.15.78:0/1269620820 conn(0x7f260c102e50 msgr2=0x7f260c103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.210+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 shutdown_connections 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:48.210+0000 7f2613c2d640 1 -- 172.21.15.78:0/1269620820 wait complete. 2024-09-16T00:03:48.833 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mon service... 2024-09-16T00:03:49.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:49 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:49.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:49 smithi078 ceph-mon[31157]: Added host smithi078 2024-09-16T00:03:49.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:49 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:03:50.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:50 smithi078 ceph-mon[31157]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:03:50.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:50 smithi078 ceph-mon[31157]: Saving service mon spec with placement count:5 2024-09-16T00:03:50.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:50 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:51.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T00:03:51.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.278+0000 7f6a1cb92640 1 Processor -- start 2024-09-16T00:03:51.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.278+0000 7f6a1cb92640 1 -- start start 2024-09-16T00:03:51.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.278+0000 7f6a1cb92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a181061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:51.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.278+0000 7f6a1cb92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18075880 con 0x7f6a18105df0 2024-09-16T00:03:51.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.279+0000 7f6a1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a181061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.279+0000 7f6a1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a181061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35312/0 (socket says 172.21.15.78:35312) 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.279+0000 7f6a1659c640 1 -- 172.21.15.78:0/3706371176 learned_addr learned my addr 172.21.15.78:0/3706371176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.279+0000 7f6a1659c640 1 -- 172.21.15.78:0/3706371176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a18106730 con 0x7f6a18105df0 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.279+0000 7f6a1659c640 1 --2- 172.21.15.78:0/3706371176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a181061f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009e80 tx=0x7f6a0802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a5a277fa0517fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1559a640 1 -- 172.21.15.78:0/3706371176 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a08036070 con 0x7f6a18105df0 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1559a640 1 -- 172.21.15.78:0/3706371176 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a080318c0 con 0x7f6a18105df0 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1559a640 1 -- 172.21.15.78:0/3706371176 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a08007ab0 con 0x7f6a18105df0 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3706371176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 msgr2=0x7f6a181061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1cb92640 1 --2- 172.21.15.78:0/3706371176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a181061f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6a08009e80 tx=0x7f6a0802a7a0 comp rx=0 tx=0).stop 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3706371176 shutdown_connections 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1cb92640 1 --2- 172.21.15.78:0/3706371176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a181061f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3706371176 >> 172.21.15.78:0/3706371176 conn(0x7f6a18101ca0 msgr2=0x7f6a181040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:51.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3706371176 shutdown_connections 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.280+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3706371176 wait complete. 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.281+0000 7f6a1cb92640 1 Processor -- start 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.281+0000 7f6a1cb92640 1 -- start start 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.281+0000 7f6a1cb92640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a1819d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.281+0000 7f6a1cb92640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a18107830 con 0x7f6a18105df0 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a1819d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a1659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a1819d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35314/0 (socket says 172.21.15.78:35314) 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a1659c640 1 -- 172.21.15.78:0/3730902882 learned_addr learned my addr 172.21.15.78:0/3730902882 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a1659c640 1 -- 172.21.15.78:0/3730902882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1819dca0 con 0x7f6a18105df0 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a1659c640 1 --2- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a1819d760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6a08031ca0 tx=0x7f6a08031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a0f7fe640 1 -- 172.21.15.78:0/3730902882 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a08036050 con 0x7f6a18105df0 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a0f7fe640 1 -- 172.21.15.78:0/3730902882 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6a08042070 con 0x7f6a18105df0 2024-09-16T00:03:51.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a0f7fe640 1 -- 172.21.15.78:0/3730902882 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a0803d820 con 0x7f6a18105df0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1819df30 con 0x7f6a18105df0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.282+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a181a0c20 con 0x7f6a18105df0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.283+0000 7f6a0f7fe640 1 -- 172.21.15.78:0/3730902882 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6a0803db20 con 0x7f6a18105df0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.283+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a181061f0 con 0x7f6a18105df0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.283+0000 7f6a0f7fe640 1 --2- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f6a040391b0 0x7f6a0403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.284+0000 7f6a0f7fe640 1 -- 172.21.15.78:0/3730902882 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6a08007e00 con 0x7f6a18105df0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.284+0000 7f6a15d9b640 1 --2- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f6a040391b0 0x7f6a0403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.284+0000 7f6a15d9b640 1 --2- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f6a040391b0 0x7f6a0403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6a000098c0 tx=0x7f6a00006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.289+0000 7f6a0f7fe640 1 -- 172.21.15.78:0/3730902882 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a080371f0 con 0x7f6a18105df0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.470+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6a18062630 con 0x7f6a040391b0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.474+0000 7f6a0f7fe640 1 -- 172.21.15.78:0/3730902882 <== mgr.14118 v2:172.21.15.78:6800/1052494441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6a18062630 con 0x7f6a040391b0 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f6a040391b0 msgr2=0x7f6a0403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:51.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 --2- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f6a040391b0 0x7f6a0403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6a000098c0 tx=0x7f6a00006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 msgr2=0x7f6a1819d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 --2- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a1819d760 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6a08031ca0 tx=0x7f6a08031cd0 comp rx=0 tx=0).stop 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 shutdown_connections 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 --2- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f6a040391b0 0x7f6a0403b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 --2- 172.21.15.78:0/3730902882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a18105df0 0x7f6a1819d760 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 >> 172.21.15.78:0/3730902882 conn(0x7f6a18101ca0 msgr2=0x7f6a18191a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 shutdown_connections 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:49.476+0000 7f6a1cb92640 1 -- 172.21.15.78:0/3730902882 wait complete. 2024-09-16T00:03:51.264 INFO:teuthology.orchestra.run.smithi078.stdout:Deploying unmanaged mgr service... 2024-09-16T00:03:52.117 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:51 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:52.117 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:51 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:53.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:52 smithi078 ceph-mon[31157]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:03:53.189 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:52 smithi078 ceph-mon[31157]: Saving service mgr spec with placement count:2 2024-09-16T00:03:53.189 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:52 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:53.189 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:52 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:53.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T00:03:53.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.724+0000 7f1543ff8640 1 Processor -- start 2024-09-16T00:03:53.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.724+0000 7f1543ff8640 1 -- start start 2024-09-16T00:03:53.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.724+0000 7f1543ff8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c10a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:53.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.724+0000 7f1543ff8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f153c07bdc0 con 0x7f153c1080b0 2024-09-16T00:03:53.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.724+0000 7f1541d94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c10a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:53.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.724+0000 7f1541d94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c10a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35322/0 (socket says 172.21.15.78:35322) 2024-09-16T00:03:53.196 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.724+0000 7f1541d94640 1 -- 172.21.15.78:0/4247937658 learned_addr learned my addr 172.21.15.78:0/4247937658 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.725+0000 7f1541d94640 1 -- 172.21.15.78:0/4247937658 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c10a9e0 con 0x7f153c1080b0 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.725+0000 7f1541d94640 1 --2- 172.21.15.78:0/4247937658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c10a4a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f153800f960 tx=0x7f153802e550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc248f65983133c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.725+0000 7f1540d92640 1 -- 172.21.15.78:0/4247937658 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f153803a070 con 0x7f153c1080b0 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1540d92640 1 -- 172.21.15.78:0/4247937658 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1538033b20 con 0x7f153c1080b0 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1540d92640 1 -- 172.21.15.78:0/4247937658 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538035cb0 con 0x7f153c1080b0 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1543ff8640 1 -- 172.21.15.78:0/4247937658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 msgr2=0x7f153c10a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1543ff8640 1 --2- 172.21.15.78:0/4247937658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c10a4a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f153800f960 tx=0x7f153802e550 comp rx=0 tx=0).stop 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1543ff8640 1 -- 172.21.15.78:0/4247937658 shutdown_connections 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1543ff8640 1 --2- 172.21.15.78:0/4247937658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c10a4a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:53.197 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1543ff8640 1 -- 172.21.15.78:0/4247937658 >> 172.21.15.78:0/4247937658 conn(0x7f153c101cb0 msgr2=0x7f153c1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1543ff8640 1 -- 172.21.15.78:0/4247937658 shutdown_connections 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.726+0000 7f1543ff8640 1 -- 172.21.15.78:0/4247937658 wait complete. 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1543ff8640 1 Processor -- start 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1543ff8640 1 -- start start 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1543ff8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1543ff8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f153c106cf0 con 0x7f153c1080b0 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1541d94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1541d94640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35326/0 (socket says 172.21.15.78:35326) 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1541d94640 1 -- 172.21.15.78:0/1439331593 learned_addr learned my addr 172.21.15.78:0/1439331593 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1541d94640 1 -- 172.21.15.78:0/1439331593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c199c20 con 0x7f153c1080b0 2024-09-16T00:03:53.198 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.727+0000 7f1541d94640 1 --2- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c1996e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f15380357e0 tx=0x7f1538033740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.728+0000 7f1532ffd640 1 -- 172.21.15.78:0/1439331593 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538044020 con 0x7f153c1080b0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.728+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c199eb0 con 0x7f153c1080b0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.728+0000 7f1532ffd640 1 -- 172.21.15.78:0/1439331593 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1538040900 con 0x7f153c1080b0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.728+0000 7f1532ffd640 1 -- 172.21.15.78:0/1439331593 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f153803a040 con 0x7f153c1080b0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.728+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c19cba0 con 0x7f153c1080b0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.729+0000 7f1532ffd640 1 -- 172.21.15.78:0/1439331593 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f1538047020 con 0x7f153c1080b0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.729+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c1073a0 con 0x7f153c1080b0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.729+0000 7f1532ffd640 1 --2- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f15280391b0 0x7f152803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.729+0000 7f1532ffd640 1 -- 172.21.15.78:0/1439331593 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1538071300 con 0x7f153c1080b0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.729+0000 7f1541593640 1 --2- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f15280391b0 0x7f152803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:53.199 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.729+0000 7f1541593640 1 --2- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f15280391b0 0x7f152803b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f152c0098c0 tx=0x7f152c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.735+0000 7f1532ffd640 1 -- 172.21.15.78:0/1439331593 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f153804e020 con 0x7f153c1080b0 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.929+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f153c062630 con 0x7f15280391b0 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.936+0000 7f1532ffd640 1 -- 172.21.15.78:0/1439331593 <== mgr.14118 v2:172.21.15.78:6800/1052494441 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f153c062630 con 0x7f15280391b0 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f15280391b0 msgr2=0x7f152803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 --2- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f15280391b0 0x7f152803b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f152c0098c0 tx=0x7f152c006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 msgr2=0x7f153c1996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 --2- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c1996e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f15380357e0 tx=0x7f1538033740 comp rx=0 tx=0).stop 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 shutdown_connections 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 --2- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f15280391b0 0x7f152803b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 --2- 172.21.15.78:0/1439331593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f153c1080b0 0x7f153c1996e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:53.200 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 >> 172.21.15.78:0/1439331593 conn(0x7f153c101cb0 msgr2=0x7f153c102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:53.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 shutdown_connections 2024-09-16T00:03:53.201 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:51.938+0000 7f1543ff8640 1 -- 172.21.15.78:0/1439331593 wait complete. 2024-09-16T00:03:54.457 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:54 smithi078 ceph-mon[31157]: from='mgr.14118 172.21.15.78:0/673036905' entity='mgr.x' 2024-09-16T00:03:54.457 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:54 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1962313244' entity='client.admin' 2024-09-16T00:03:54.464 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.606+0000 7fa913a65640 1 Processor -- start 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.606+0000 7fa913a65640 1 -- start start 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.606+0000 7fa913a65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c07c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.606+0000 7fa913a65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c07e110 con 0x7fa90c07dd10 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.606+0000 7fa911801640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c07c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.606+0000 7fa911801640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c07c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35334/0 (socket says 172.21.15.78:35334) 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.606+0000 7fa911801640 1 -- 172.21.15.78:0/3766982252 learned_addr learned my addr 172.21.15.78:0/3766982252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.607+0000 7fa911801640 1 -- 172.21.15.78:0/3766982252 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c07c7c0 con 0x7fa90c07dd10 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.607+0000 7fa911801640 1 --2- 172.21.15.78:0/3766982252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c07c280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa908009820 tx=0x7fa90802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d15cdaef0ca941 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.607+0000 7fa903fff640 1 -- 172.21.15.78:0/3766982252 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa908037070 con 0x7fa90c07dd10 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.607+0000 7fa903fff640 1 -- 172.21.15.78:0/3766982252 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa908032920 con 0x7fa90c07dd10 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.607+0000 7fa903fff640 1 -- 172.21.15.78:0/3766982252 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa908030810 con 0x7fa90c07dd10 2024-09-16T00:03:54.465 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.608+0000 7fa913a65640 1 -- 172.21.15.78:0/3766982252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 msgr2=0x7fa90c07c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.608+0000 7fa913a65640 1 --2- 172.21.15.78:0/3766982252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c07c280 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa908009820 tx=0x7fa90802a270 comp rx=0 tx=0).stop 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.608+0000 7fa913a65640 1 -- 172.21.15.78:0/3766982252 shutdown_connections 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.608+0000 7fa913a65640 1 --2- 172.21.15.78:0/3766982252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c07c280 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.608+0000 7fa913a65640 1 -- 172.21.15.78:0/3766982252 >> 172.21.15.78:0/3766982252 conn(0x7fa90c102e50 msgr2=0x7fa90c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.608+0000 7fa913a65640 1 -- 172.21.15.78:0/3766982252 shutdown_connections 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.608+0000 7fa913a65640 1 -- 172.21.15.78:0/3766982252 wait complete. 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.609+0000 7fa913a65640 1 Processor -- start 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.609+0000 7fa913a65640 1 -- start start 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.609+0000 7fa913a65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.609+0000 7fa913a65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c10db60 con 0x7fa90c07dd10 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.609+0000 7fa911801640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.609+0000 7fa911801640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c19dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35342/0 (socket says 172.21.15.78:35342) 2024-09-16T00:03:54.466 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.609+0000 7fa911801640 1 -- 172.21.15.78:0/1962313244 learned_addr learned my addr 172.21.15.78:0/1962313244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.609+0000 7fa911801640 1 -- 172.21.15.78:0/1962313244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c19dff0 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.610+0000 7fa911801640 1 --2- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c19dab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa908030380 tx=0x7fa908032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.610+0000 7fa9027fc640 1 -- 172.21.15.78:0/1962313244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa908037040 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.610+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c19e280 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.610+0000 7fa9027fc640 1 -- 172.21.15.78:0/1962313244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa908040070 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.610+0000 7fa9027fc640 1 -- 172.21.15.78:0/1962313244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa90803a7b0 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.610+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c1a0f70 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.611+0000 7fa9027fc640 1 -- 172.21.15.78:0/1962313244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa90803aa30 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.611+0000 7fa9027fc640 1 --2- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fa8f80391b0 0x7fa8f803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.611+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90c07c300 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.611+0000 7fa9027fc640 1 -- 172.21.15.78:0/1962313244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa90806c8c0 con 0x7fa90c07dd10 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.611+0000 7fa911000640 1 --2- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fa8f80391b0 0x7fa8f803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:54.467 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.611+0000 7fa911000640 1 --2- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fa8f80391b0 0x7fa8f803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0098c0 tx=0x7fa8fc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:54.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.616+0000 7fa9027fc640 1 -- 172.21.15.78:0/1962313244 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa908044c70 con 0x7fa90c07dd10 2024-09-16T00:03:54.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.798+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fa90c02d7c0 con 0x7fa90c07dd10 2024-09-16T00:03:54.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.801+0000 7fa9027fc640 1 -- 172.21.15.78:0/1962313244 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fa908041270 con 0x7fa90c07dd10 2024-09-16T00:03:54.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.803+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fa8f80391b0 msgr2=0x7fa8f803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:54.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.803+0000 7fa913a65640 1 --2- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fa8f80391b0 0x7fa8f803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0098c0 tx=0x7fa8fc006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:54.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.803+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 msgr2=0x7fa90c19dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:54.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.803+0000 7fa913a65640 1 --2- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c19dab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa908030380 tx=0x7fa908032670 comp rx=0 tx=0).stop 2024-09-16T00:03:54.468 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.804+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 shutdown_connections 2024-09-16T00:03:54.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.804+0000 7fa913a65640 1 --2- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fa8f80391b0 0x7fa8f803b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:54.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.804+0000 7fa913a65640 1 --2- 172.21.15.78:0/1962313244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa90c07dd10 0x7fa90c19dab0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:54.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.804+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 >> 172.21.15.78:0/1962313244 conn(0x7fa90c102e50 msgr2=0x7fa90c103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:54.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.804+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 shutdown_connections 2024-09-16T00:03:54.469 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:53.804+0000 7fa913a65640 1 -- 172.21.15.78:0/1962313244 wait complete. 2024-09-16T00:03:56.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.306+0000 7fb4bbae6640 1 Processor -- start 2024-09-16T00:03:56.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.307+0000 7fb4bbae6640 1 -- start start 2024-09-16T00:03:56.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.307+0000 7fb4bbae6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b4107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:56.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.307+0000 7fb4bbae6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4107f40 con 0x7fb4b4107600 2024-09-16T00:03:56.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.307+0000 7fb4b9882640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b4107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:56.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.307+0000 7fb4b9882640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b4107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35352/0 (socket says 172.21.15.78:35352) 2024-09-16T00:03:56.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.307+0000 7fb4b9882640 1 -- 172.21.15.78:0/2164150724 learned_addr learned my addr 172.21.15.78:0/2164150724 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:56.335 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.307+0000 7fb4b9882640 1 -- 172.21.15.78:0/2164150724 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b41080c0 con 0x7fb4b4107600 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.308+0000 7fb4b9882640 1 --2- 172.21.15.78:0/2164150724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b4107a00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009820 tx=0x7fb4b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d461fcf408c764df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.308+0000 7fb4b8880640 1 -- 172.21.15.78:0/2164150724 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b0037070 con 0x7fb4b4107600 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.308+0000 7fb4b8880640 1 -- 172.21.15.78:0/2164150724 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b0032920 con 0x7fb4b4107600 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.308+0000 7fb4b8880640 1 -- 172.21.15.78:0/2164150724 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b0030810 con 0x7fb4b4107600 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 -- 172.21.15.78:0/2164150724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 msgr2=0x7fb4b4107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 --2- 172.21.15.78:0/2164150724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b4107a00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009820 tx=0x7fb4b002a270 comp rx=0 tx=0).stop 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 -- 172.21.15.78:0/2164150724 shutdown_connections 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 --2- 172.21.15.78:0/2164150724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b4107a00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 -- 172.21.15.78:0/2164150724 >> 172.21.15.78:0/2164150724 conn(0x7fb4b4102e50 msgr2=0x7fb4b4105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 -- 172.21.15.78:0/2164150724 shutdown_connections 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 -- 172.21.15.78:0/2164150724 wait complete. 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 Processor -- start 2024-09-16T00:03:56.336 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.309+0000 7fb4bbae6640 1 -- start start 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4bbae6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4bbae6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b410ba70 con 0x7fb4b4107600 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4b9882640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4b9882640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35366/0 (socket says 172.21.15.78:35366) 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4b9882640 1 -- 172.21.15.78:0/1664248048 learned_addr learned my addr 172.21.15.78:0/1664248048 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4b9882640 1 -- 172.21.15.78:0/1664248048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b419bae0 con 0x7fb4b4107600 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4b9882640 1 --2- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b419b5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0030380 tx=0x7fb4b0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4aaffd640 1 -- 172.21.15.78:0/1664248048 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b0037040 con 0x7fb4b4107600 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4aaffd640 1 -- 172.21.15.78:0/1664248048 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b0040070 con 0x7fb4b4107600 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b419bd70 con 0x7fb4b4107600 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4aaffd640 1 -- 172.21.15.78:0/1664248048 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4b003a7b0 con 0x7fb4b4107600 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.310+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b419ea60 con 0x7fb4b4107600 2024-09-16T00:03:56.337 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.311+0000 7fb4aaffd640 1 -- 172.21.15.78:0/1664248048 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb4b003aa30 con 0x7fb4b4107600 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.311+0000 7fb4aaffd640 1 --2- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fb4a00391b0 0x7fb4a003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.311+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b4107a80 con 0x7fb4b4107600 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.311+0000 7fb4aaffd640 1 -- 172.21.15.78:0/1664248048 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb4b006cb30 con 0x7fb4b4107600 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.311+0000 7fb4b9081640 1 --2- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fb4a00391b0 0x7fb4a003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.312+0000 7fb4b9081640 1 --2- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fb4a00391b0 0x7fb4a003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40098c0 tx=0x7fb4a4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.316+0000 7fb4aaffd640 1 -- 172.21.15.78:0/1664248048 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4b0044c70 con 0x7fb4b4107600 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.483+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb4b419f090 con 0x7fb4b4107600 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.485+0000 7fb4aaffd640 1 -- 172.21.15.78:0/1664248048 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb4b0041270 con 0x7fb4b4107600 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.487+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fb4a00391b0 msgr2=0x7fb4a003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.487+0000 7fb4bbae6640 1 --2- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fb4a00391b0 0x7fb4a003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40098c0 tx=0x7fb4a4006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.487+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 msgr2=0x7fb4b419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.487+0000 7fb4bbae6640 1 --2- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b419b5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0030380 tx=0x7fb4b0032670 comp rx=0 tx=0).stop 2024-09-16T00:03:56.338 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.487+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 shutdown_connections 2024-09-16T00:03:56.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.487+0000 7fb4bbae6640 1 --2- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7fb4a00391b0 0x7fb4a003b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:56.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.487+0000 7fb4bbae6640 1 --2- 172.21.15.78:0/1664248048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb4b4107600 0x7fb4b419b5a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:56.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.487+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 >> 172.21.15.78:0/1664248048 conn(0x7fb4b4102e50 msgr2=0x7fb4b4103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:56.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.488+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 shutdown_connections 2024-09-16T00:03:56.339 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:55.488+0000 7fb4bbae6640 1 -- 172.21.15.78:0/1664248048 wait complete. 2024-09-16T00:03:56.339 INFO:teuthology.orchestra.run.smithi078.stdout:Enabling the dashboard module... 2024-09-16T00:03:56.763 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:56 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1664248048' entity='client.admin' 2024-09-16T00:03:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:57 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/258981322' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T00:03:57.770 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: ignoring --setuser ceph since I am not root 2024-09-16T00:03:57.770 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: ignoring --setgroup ceph since I am not root 2024-09-16T00:03:57.770 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:57.637+0000 7fe694a52040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:03:57.770 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:57.693+0000 7fe694a52040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:03:58.167 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:57.841+0000 7fe694a52040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:03:58.168 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:57.907+0000 7fe694a52040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:03:58.168 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:58.053+0000 7fe694a52040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.767+0000 7f2d18cc2640 1 Processor -- start 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.767+0000 7f2d18cc2640 1 -- start start 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.767+0000 7f2d18cc2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d14107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.767+0000 7f2d18cc2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14107f40 con 0x7f2d14107600 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.768+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d14107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.768+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d14107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35226/0 (socket says 172.21.15.78:35226) 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.768+0000 7f2d1259c640 1 -- 172.21.15.78:0/2262316231 learned_addr learned my addr 172.21.15.78:0/2262316231 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.768+0000 7f2d1259c640 1 -- 172.21.15.78:0/2262316231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d141080c0 con 0x7f2d14107600 2024-09-16T00:03:58.175 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.769+0000 7f2d1259c640 1 --2- 172.21.15.78:0/2262316231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d14107a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009820 tx=0x7f2d0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462d26cde3e9ae7b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.769+0000 7f2d1159a640 1 -- 172.21.15.78:0/2262316231 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d04037070 con 0x7f2d14107600 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.769+0000 7f2d1159a640 1 -- 172.21.15.78:0/2262316231 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d04032920 con 0x7f2d14107600 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.769+0000 7f2d1159a640 1 -- 172.21.15.78:0/2262316231 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d04030810 con 0x7f2d14107600 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.769+0000 7f2d18cc2640 1 -- 172.21.15.78:0/2262316231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 msgr2=0x7f2d14107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.769+0000 7f2d18cc2640 1 --2- 172.21.15.78:0/2262316231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d14107a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009820 tx=0x7f2d0402a270 comp rx=0 tx=0).stop 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.770+0000 7f2d18cc2640 1 -- 172.21.15.78:0/2262316231 shutdown_connections 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.770+0000 7f2d18cc2640 1 --2- 172.21.15.78:0/2262316231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d14107a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.770+0000 7f2d18cc2640 1 -- 172.21.15.78:0/2262316231 >> 172.21.15.78:0/2262316231 conn(0x7f2d14102e50 msgr2=0x7f2d14105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:58.176 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.770+0000 7f2d18cc2640 1 -- 172.21.15.78:0/2262316231 shutdown_connections 2024-09-16T00:03:58.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.770+0000 7f2d18cc2640 1 -- 172.21.15.78:0/2262316231 wait complete. 2024-09-16T00:03:58.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.770+0000 7f2d18cc2640 1 Processor -- start 2024-09-16T00:03:58.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d18cc2640 1 -- start start 2024-09-16T00:03:58.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d18cc2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d1419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:58.177 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d18cc2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1410ba70 con 0x7f2d14107600 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d1419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d1419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35228/0 (socket says 172.21.15.78:35228) 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d1259c640 1 -- 172.21.15.78:0/258981322 learned_addr learned my addr 172.21.15.78:0/258981322 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d1259c640 1 -- 172.21.15.78:0/258981322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1419bae0 con 0x7f2d14107600 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d1259c640 1 --2- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d1419b5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2d040097f0 tx=0x7f2d040327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.771+0000 7f2d037fe640 1 -- 172.21.15.78:0/258981322 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d04037040 con 0x7f2d14107600 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.772+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1419bd70 con 0x7f2d14107600 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.772+0000 7f2d037fe640 1 -- 172.21.15.78:0/258981322 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d04030cc0 con 0x7f2d14107600 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.772+0000 7f2d037fe640 1 -- 172.21.15.78:0/258981322 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0403a6b0 con 0x7f2d14107600 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.772+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1419ea60 con 0x7f2d14107600 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.772+0000 7f2d037fe640 1 -- 172.21.15.78:0/258981322 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2d0403a8a0 con 0x7f2d14107600 2024-09-16T00:03:58.178 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.773+0000 7f2d037fe640 1 --2- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f2cfc0391b0 0x7f2cfc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:58.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.773+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d14107a80 con 0x7f2d14107600 2024-09-16T00:03:58.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.773+0000 7f2d037fe640 1 -- 172.21.15.78:0/258981322 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d0406c810 con 0x7f2d14107600 2024-09-16T00:03:58.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.773+0000 7f2d11d9b640 1 --2- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f2cfc0391b0 0x7f2cfc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:58.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.773+0000 7f2d11d9b640 1 --2- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f2cfc0391b0 0x7f2cfc03b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2d080098c0 tx=0x7f2d08006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:58.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.778+0000 7f2d037fe640 1 -- 172.21.15.78:0/258981322 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d0402de20 con 0x7f2d14107600 2024-09-16T00:03:58.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:56.988+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f2d1419ec30 con 0x7f2d14107600 2024-09-16T00:03:58.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.488+0000 7f2d037fe640 1 -- 172.21.15.78:0/258981322 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f2d04044920 con 0x7f2d14107600 2024-09-16T00:03:58.179 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.489+0000 7f2d037fe640 1 -- 172.21.15.78:0/258981322 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f2d04044400 con 0x7f2d14107600 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f2cfc0391b0 msgr2=0x7f2cfc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 --2- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f2cfc0391b0 0x7f2cfc03b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2d080098c0 tx=0x7f2d08006db0 comp rx=0 tx=0).stop 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 msgr2=0x7f2d1419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 --2- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d1419b5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2d040097f0 tx=0x7f2d040327b0 comp rx=0 tx=0).stop 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 shutdown_connections 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 --2- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f2cfc0391b0 0x7f2cfc03b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 --2- 172.21.15.78:0/258981322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2d14107600 0x7f2d1419b5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 >> 172.21.15.78:0/258981322 conn(0x7f2d14102e50 msgr2=0x7f2d14103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 shutdown_connections 2024-09-16T00:03:58.180 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:57.491+0000 7f2d18cc2640 1 -- 172.21.15.78:0/258981322 wait complete. 2024-09-16T00:03:58.453 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:58.200+0000 7fe694a52040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:03:58.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:58 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/258981322' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T00:03:58.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:58 smithi078 ceph-mon[31157]: mgrmap e9: x(active, since 19s) 2024-09-16T00:03:58.739 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:58.523+0000 7fe694a52040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:03:58.739 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:58.587+0000 7fe694a52040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:03:59.020 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:58.781+0000 7fe694a52040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:03:59.349 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:03:59.349 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T00:03:59.349 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.532+0000 7f77bf755640 1 Processor -- start 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.532+0000 7f77bf755640 1 -- start start 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.532+0000 7f77bf755640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b810a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.532+0000 7f77bf755640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8075820 con 0x7f77b8108060 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.532+0000 7f77bd4f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b810a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.532+0000 7f77bd4f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b810a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35254/0 (socket says 172.21.15.78:35254) 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.532+0000 7f77bd4f1640 1 -- 172.21.15.78:0/2406378912 learned_addr learned my addr 172.21.15.78:0/2406378912 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.532+0000 7f77bd4f1640 1 -- 172.21.15.78:0/2406378912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b810a990 con 0x7f77b8108060 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.533+0000 7f77bd4f1640 1 --2- 172.21.15.78:0/2406378912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b810a450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009820 tx=0x7f77b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9264e57d729bf07c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:59.350 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.533+0000 7f77affff640 1 -- 172.21.15.78:0/2406378912 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b4037070 con 0x7f77b8108060 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.533+0000 7f77affff640 1 -- 172.21.15.78:0/2406378912 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b4032920 con 0x7f77b8108060 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.533+0000 7f77affff640 1 -- 172.21.15.78:0/2406378912 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b4030810 con 0x7f77b8108060 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.534+0000 7f77bf755640 1 -- 172.21.15.78:0/2406378912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 msgr2=0x7f77b810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.534+0000 7f77bf755640 1 --2- 172.21.15.78:0/2406378912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b810a450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009820 tx=0x7f77b402a270 comp rx=0 tx=0).stop 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.534+0000 7f77bf755640 1 -- 172.21.15.78:0/2406378912 shutdown_connections 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.534+0000 7f77bf755640 1 --2- 172.21.15.78:0/2406378912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b810a450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.534+0000 7f77bf755640 1 -- 172.21.15.78:0/2406378912 >> 172.21.15.78:0/2406378912 conn(0x7f77b8101ca0 msgr2=0x7f77b81040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.534+0000 7f77bf755640 1 -- 172.21.15.78:0/2406378912 shutdown_connections 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.534+0000 7f77bf755640 1 -- 172.21.15.78:0/2406378912 wait complete. 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.535+0000 7f77bf755640 1 Processor -- start 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.535+0000 7f77bf755640 1 -- start start 2024-09-16T00:03:59.351 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.535+0000 7f77bf755640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.535+0000 7f77bf755640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77b810ba10 con 0x7f77b8108060 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77bd4f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77bd4f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b819b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35270/0 (socket says 172.21.15.78:35270) 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77bd4f1640 1 -- 172.21.15.78:0/3990620642 learned_addr learned my addr 172.21.15.78:0/3990620642 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77bd4f1640 1 -- 172.21.15.78:0/3990620642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b819bae0 con 0x7f77b8108060 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77bd4f1640 1 --2- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b819b5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f77b4030380 tx=0x7f77b4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77ae7fc640 1 -- 172.21.15.78:0/3990620642 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b4037040 con 0x7f77b8108060 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77ae7fc640 1 -- 172.21.15.78:0/3990620642 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b4040070 con 0x7f77b8108060 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b819bd70 con 0x7f77b8108060 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77ae7fc640 1 -- 172.21.15.78:0/3990620642 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77b403a7b0 con 0x7f77b8108060 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.536+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b819ea60 con 0x7f77b8108060 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.537+0000 7f77ae7fc640 1 -- 172.21.15.78:0/3990620642 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f77b403aa30 con 0x7f77b8108060 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.537+0000 7f77ae7fc640 1 --2- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f77a4039200 0x7f77a403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:03:59.352 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.537+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b81084c0 con 0x7f77b8108060 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.537+0000 7f77bccf0640 1 -- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f77a4039200 msgr2=0x7f77a403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1052494441 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.537+0000 7f77ae7fc640 1 -- 172.21.15.78:0/3990620642 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f77b406ca40 con 0x7f77b8108060 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.537+0000 7f77bccf0640 1 --2- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f77a4039200 0x7f77a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.542+0000 7f77ae7fc640 1 -- 172.21.15.78:0/3990620642 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77b4044c70 con 0x7f77b8108060 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.738+0000 7f77bccf0640 1 -- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f77a4039200 msgr2=0x7f77a403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1052494441 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.738+0000 7f77bccf0640 1 --2- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f77a4039200 0x7f77a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.761+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f77b819ec30 con 0x7f77b8108060 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.761+0000 7f77ae7fc640 1 -- 172.21.15.78:0/3990620642 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f77b819ec30 con 0x7f77b8108060 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.763+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f77a4039200 msgr2=0x7f77a403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.763+0000 7f77bf755640 1 --2- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f77a4039200 0x7f77a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.763+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 msgr2=0x7f77b819b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.763+0000 7f77bf755640 1 --2- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b819b5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f77b4030380 tx=0x7f77b4032670 comp rx=0 tx=0).stop 2024-09-16T00:03:59.353 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.764+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 shutdown_connections 2024-09-16T00:03:59.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.764+0000 7f77bf755640 1 --2- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f77a4039200 0x7f77a403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:59.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.764+0000 7f77bf755640 1 --2- 172.21.15.78:0/3990620642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77b8108060 0x7f77b819b5a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:03:59.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.764+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 >> 172.21.15.78:0/3990620642 conn(0x7f77b8101ca0 msgr2=0x7f77b81028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:03:59.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.764+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 shutdown_connections 2024-09-16T00:03:59.354 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:58.764+0000 7f77bf755640 1 -- 172.21.15.78:0/3990620642 wait complete. 2024-09-16T00:03:59.354 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for the mgr to restart... 2024-09-16T00:03:59.354 INFO:teuthology.orchestra.run.smithi078.stdout:Waiting for mgr epoch 9... 2024-09-16T00:03:59.603 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:03:59 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3990620642' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:03:59.980 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:59.677+0000 7fe694a52040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:03:59.980 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:03:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:03:59.747+0000 7fe694a52040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:04:00.708 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:00.453+0000 7fe694a52040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:04:00.708 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:00.567+0000 7fe694a52040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:04:00.981 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:00.707+0000 7fe694a52040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:04:00.981 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:00.767+0000 7fe694a52040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:04:00.981 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:00.913+0000 7fe694a52040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:04:00.981 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:00.980+0000 7fe694a52040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:04:01.260 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:01.039+0000 7fe694a52040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:04:01.260 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:01.195+0000 7fe694a52040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:04:01.260 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:01.259+0000 7fe694a52040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:04:01.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:01.324+0000 7fe694a52040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:04:01.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:01.404+0000 7fe694a52040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:04:01.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:01.462+0000 7fe694a52040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:04:02.020 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:01.520+0000 7fe694a52040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:04:02.770 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:02.293+0000 7fe694a52040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:04:02.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: Active manager daemon x restarted 2024-09-16T00:04:02.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: Activating manager daemon x 2024-09-16T00:04:02.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T00:04:02.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: mgrmap e10: x(active, starting, since 0.00346516s) 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: Manager daemon x is now available 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:04:02.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.772+0000 7f5134ed1640 1 Processor -- start 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.772+0000 7f5134ed1640 1 -- start start 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.772+0000 7f5134ed1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513010a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.772+0000 7f5134ed1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5130075800 con 0x7f51301080b0 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.772+0000 7f512e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513010a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:04.036 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.772+0000 7f512e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513010a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35274/0 (socket says 172.21.15.78:35274) 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.773+0000 7f512e59c640 1 -- 172.21.15.78:0/1586286332 learned_addr learned my addr 172.21.15.78:0/1586286332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.773+0000 7f512e59c640 1 -- 172.21.15.78:0/1586286332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513010a9e0 con 0x7f51301080b0 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.773+0000 7f512e59c640 1 --2- 172.21.15.78:0/1586286332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513010a4a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5120009820 tx=0x7f512002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9351690b3e78415 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.774+0000 7f512dd9b640 1 -- 172.21.15.78:0/1586286332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5120037070 con 0x7f51301080b0 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.774+0000 7f512dd9b640 1 -- 172.21.15.78:0/1586286332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5120032920 con 0x7f51301080b0 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.774+0000 7f512dd9b640 1 -- 172.21.15.78:0/1586286332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5120030810 con 0x7f51301080b0 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.775+0000 7f5134ed1640 1 -- 172.21.15.78:0/1586286332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 msgr2=0x7f513010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.775+0000 7f5134ed1640 1 --2- 172.21.15.78:0/1586286332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513010a4a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5120009820 tx=0x7f512002a270 comp rx=0 tx=0).stop 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.775+0000 7f5134ed1640 1 -- 172.21.15.78:0/1586286332 shutdown_connections 2024-09-16T00:04:04.037 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.775+0000 7f5134ed1640 1 --2- 172.21.15.78:0/1586286332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513010a4a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:04.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.775+0000 7f5134ed1640 1 -- 172.21.15.78:0/1586286332 >> 172.21.15.78:0/1586286332 conn(0x7f5130101c70 msgr2=0x7f5130104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:04.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.775+0000 7f5134ed1640 1 -- 172.21.15.78:0/1586286332 shutdown_connections 2024-09-16T00:04:04.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.775+0000 7f5134ed1640 1 -- 172.21.15.78:0/1586286332 wait complete. 2024-09-16T00:04:04.038 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f5134ed1640 1 Processor -- start 2024-09-16T00:04:04.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f5134ed1640 1 -- start start 2024-09-16T00:04:04.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f5134ed1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:04.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f5134ed1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f513010ba00 con 0x7f51301080b0 2024-09-16T00:04:04.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f512e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:04.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f512e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35282/0 (socket says 172.21.15.78:35282) 2024-09-16T00:04:04.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f512e59c640 1 -- 172.21.15.78:0/2542598196 learned_addr learned my addr 172.21.15.78:0/2542598196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:04.039 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f512e59c640 1 -- 172.21.15.78:0/2542598196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513019be70 con 0x7f51301080b0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.776+0000 7f512e59c640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019b930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5120030420 tx=0x7f5120032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.777+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5120037040 con 0x7f51301080b0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.777+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5120040070 con 0x7f51301080b0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.777+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f512003a6e0 con 0x7f51301080b0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.777+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513019c100 con 0x7f51301080b0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.777+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513019eda0 con 0x7f51301080b0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.778+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f512003a990 con 0x7f51301080b0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.778+0000 7f51277fe640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 0x7f511c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.778+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 --> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f511c03be30 con 0x7f511c039260 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.778+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f512006c8f0 con 0x7f51301080b0 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.778+0000 7f5127fff640 1 -- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 msgr2=0x7f511c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1052494441 2024-09-16T00:04:04.040 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.778+0000 7f5127fff640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 0x7f511c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.979+0000 7f5127fff640 1 -- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 msgr2=0x7f511c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1052494441 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:03:59.979+0000 7f5127fff640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 0x7f511c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:00.380+0000 7f5127fff640 1 -- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 msgr2=0x7f511c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1052494441 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:00.380+0000 7f5127fff640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 0x7f511c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:01.181+0000 7f5127fff640 1 -- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 msgr2=0x7f511c03b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.78:6800/1052494441 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:01.181+0000 7f5127fff640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 0x7f511c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:02.296+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f512006b3a0 con 0x7f51301080b0 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:02.296+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 msgr2=0x7f511c03b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:02.296+0000 7f51277fe640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 0x7f511c03b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.298+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f5120039b90 con 0x7f51301080b0 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.299+0000 7f51277fe640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f511c03cd20 0x7f511c03f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.299+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f512005af40 con 0x7f511c03cd20 2024-09-16T00:04:04.041 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.299+0000 7f5127fff640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f511c03cd20 0x7f511c03f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.300+0000 7f5127fff640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f511c03cd20 0x7f511c03f110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5118003980 tx=0x7f51180092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.300+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f512005af40 con 0x7f511c03cd20 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.307+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5130108510 con 0x7f511c03cd20 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.307+0000 7f51277fe640 1 -- 172.21.15.78:0/2542598196 <== mgr.14150 v2:172.21.15.78:6800/119552845 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f5130108510 con 0x7f511c03cd20 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f511c03cd20 msgr2=0x7f511c03f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f511c03cd20 0x7f511c03f110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5118003980 tx=0x7f51180092b0 comp rx=0 tx=0).stop 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 msgr2=0x7f513019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019b930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5120030420 tx=0x7f5120032670 comp rx=0 tx=0).stop 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 shutdown_connections 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f511c03cd20 0x7f511c03f110 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:04.042 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:6800/1052494441,v1:172.21.15.78:6801/1052494441] conn(0x7f511c039260 0x7f511c03b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:04.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 --2- 172.21.15.78:0/2542598196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51301080b0 0x7f513019b930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:04.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 >> 172.21.15.78:0/2542598196 conn(0x7f5130101c70 msgr2=0x7f5130102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:04.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 shutdown_connections 2024-09-16T00:04:04.043 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:03.308+0000 7f5134ed1640 1 -- 172.21.15.78:0/2542598196 wait complete. 2024-09-16T00:04:04.043 INFO:teuthology.orchestra.run.smithi078.stdout:mgr epoch 9 is available 2024-09-16T00:04:04.043 INFO:teuthology.orchestra.run.smithi078.stdout:Generating a dashboard self-signed certificate... 2024-09-16T00:04:04.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-mon[31157]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T00:04:04.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-mon[31157]: mgrmap e11: x(active, since 1.00655s) 2024-09-16T00:04:04.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-mon[31157]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:04.466+0000 7fe5ffd66640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:04:04] ENGINE Error in HTTPServer.serve 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: Traceback (most recent call last): 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: self._connections.run(self.expiration_interval) 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: self._run(expiration_interval) 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:04:04.620 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: s = self.context.wrap_socket( 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: return self.sslsocket_class._create( 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: self.do_handshake() 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: self._sslobj.do_handshake() 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:04:04.621 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:05.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T00:04:05.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.430+0000 7ff422f8e640 1 Processor -- start 2024-09-16T00:04:05.258 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.430+0000 7ff422f8e640 1 -- start start 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.430+0000 7ff422f8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.430+0000 7ff422f8e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c107f40 con 0x7ff41c107600 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.430+0000 7ff420d2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.430+0000 7ff420d2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35336/0 (socket says 172.21.15.78:35336) 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.430+0000 7ff420d2a640 1 -- 172.21.15.78:0/2844811907 learned_addr learned my addr 172.21.15.78:0/2844811907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.430+0000 7ff420d2a640 1 -- 172.21.15.78:0/2844811907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c1080c0 con 0x7ff41c107600 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.431+0000 7ff420d2a640 1 --2- 172.21.15.78:0/2844811907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c107a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff410009820 tx=0x7ff41002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af264130b46d6a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.431+0000 7ff41b7fe640 1 -- 172.21.15.78:0/2844811907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff410037070 con 0x7ff41c107600 2024-09-16T00:04:05.259 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.431+0000 7ff41b7fe640 1 -- 172.21.15.78:0/2844811907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff410032920 con 0x7ff41c107600 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.431+0000 7ff41b7fe640 1 -- 172.21.15.78:0/2844811907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff410030810 con 0x7ff41c107600 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 -- 172.21.15.78:0/2844811907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 msgr2=0x7ff41c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 --2- 172.21.15.78:0/2844811907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c107a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff410009820 tx=0x7ff41002a270 comp rx=0 tx=0).stop 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 -- 172.21.15.78:0/2844811907 shutdown_connections 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 --2- 172.21.15.78:0/2844811907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c107a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 -- 172.21.15.78:0/2844811907 >> 172.21.15.78:0/2844811907 conn(0x7ff41c102e50 msgr2=0x7ff41c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 -- 172.21.15.78:0/2844811907 shutdown_connections 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 -- 172.21.15.78:0/2844811907 wait complete. 2024-09-16T00:04:05.260 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 Processor -- start 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.432+0000 7ff422f8e640 1 -- start start 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff422f8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c19b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff422f8e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c10ba70 con 0x7ff41c107600 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff420d2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c19b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff420d2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c19b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35348/0 (socket says 172.21.15.78:35348) 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff420d2a640 1 -- 172.21.15.78:0/1182362863 learned_addr learned my addr 172.21.15.78:0/1182362863 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff420d2a640 1 -- 172.21.15.78:0/1182362863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c19bae0 con 0x7ff41c107600 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff420d2a640 1 --2- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c19b5a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4100097f0 tx=0x7ff4100327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:05.261 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff419ffb640 1 -- 172.21.15.78:0/1182362863 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff410037040 con 0x7ff41c107600 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff419ffb640 1 -- 172.21.15.78:0/1182362863 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff410030cc0 con 0x7ff41c107600 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff419ffb640 1 -- 172.21.15.78:0/1182362863 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff41003a6b0 con 0x7ff41c107600 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.433+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c19bd70 con 0x7ff41c107600 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.434+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c19ea60 con 0x7ff41c107600 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.434+0000 7ff419ffb640 1 -- 172.21.15.78:0/1182362863 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7ff41003baf0 con 0x7ff41c107600 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.434+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c107a80 con 0x7ff41c107600 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.435+0000 7ff419ffb640 1 --2- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff40c039200 0x7ff40c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.435+0000 7ff419ffb640 1 -- 172.21.15.78:0/1182362863 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff41006c920 con 0x7ff41c107600 2024-09-16T00:04:05.262 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.435+0000 7ff41bfff640 1 --2- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff40c039200 0x7ff40c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:05.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.435+0000 7ff41bfff640 1 --2- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff40c039200 0x7ff40c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4080098c0 tx=0x7ff408006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:05.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.439+0000 7ff419ffb640 1 -- 172.21.15.78:0/1182362863 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff410042e10 con 0x7ff41c107600 2024-09-16T00:04:05.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.618+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff41c1040a0 con 0x7ff40c039200 2024-09-16T00:04:05.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.666+0000 7ff419ffb640 1 -- 172.21.15.78:0/1182362863 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff41c1040a0 con 0x7ff40c039200 2024-09-16T00:04:05.263 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff40c039200 msgr2=0x7ff40c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 --2- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff40c039200 0x7ff40c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4080098c0 tx=0x7ff408006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 msgr2=0x7ff41c19b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 --2- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c19b5a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4100097f0 tx=0x7ff4100327b0 comp rx=0 tx=0).stop 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 shutdown_connections 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 --2- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff40c039200 0x7ff40c03b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 --2- 172.21.15.78:0/1182362863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff41c107600 0x7ff41c19b5a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 >> 172.21.15.78:0/1182362863 conn(0x7ff41c102e50 msgr2=0x7ff41c103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 shutdown_connections 2024-09-16T00:04:05.264 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:04.668+0000 7ff422f8e640 1 -- 172.21.15.78:0/1182362863 wait complete. 2024-09-16T00:04:05.265 INFO:teuthology.orchestra.run.smithi078.stdout:Creating initial admin user... 2024-09-16T00:04:05.477 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:05 smithi078 ceph-mon[31157]: mgrmap e12: x(active, since 2s) 2024-09-16T00:04:05.477 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:05 smithi078 ceph-mon[31157]: [16/Sep/2024:00:04:04] ENGINE Bus STARTING 2024-09-16T00:04:05.477 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:05 smithi078 ceph-mon[31157]: [16/Sep/2024:00:04:04] ENGINE Serving on https://172.21.15.78:7150 2024-09-16T00:04:05.477 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:05 smithi078 ceph-mon[31157]: [16/Sep/2024:00:04:04] ENGINE Error in HTTPServer.serve 2024-09-16T00:04:05.477 INFO:journalctl@ceph.mon.a.smithi078.stdout: Traceback (most recent call last): 2024-09-16T00:04:05.477 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:04:05.477 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:04:05.477 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._run(expiration_interval) 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: s = self.context.wrap_socket( 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: return self.sslsocket_class._create( 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: self.do_handshake() 2024-09-16T00:04:05.478 INFO:journalctl@ceph.mon.a.smithi078.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:04:05.479 INFO:journalctl@ceph.mon.a.smithi078.stdout: self._sslobj.do_handshake() 2024-09-16T00:04:05.479 INFO:journalctl@ceph.mon.a.smithi078.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:04:05.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:05.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:05.479 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:06.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:06 smithi078 ceph-mon[31157]: [16/Sep/2024:00:04:04] ENGINE Serving on http://172.21.15.78:8765 2024-09-16T00:04:06.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:06 smithi078 ceph-mon[31157]: [16/Sep/2024:00:04:04] ENGINE Bus STARTED 2024-09-16T00:04:06.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:06 smithi078 ceph-mon[31157]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:06.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:06.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:07.514 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:07 smithi078 ceph-mon[31157]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:07.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$eJWGHbWJNH2jhPsE2AIRce6g4NfFieyu7QCvTabqlxsINfIqHxrPK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726445046, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T00:04:07.521 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.692+0000 7f63550cb640 1 Processor -- start 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.692+0000 7f63550cb640 1 -- start start 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.692+0000 7f63550cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.692+0000 7f63550cb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f635007c7e0 con 0x7f635007dd30 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.692+0000 7f634ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.692+0000 7f634ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35356/0 (socket says 172.21.15.78:35356) 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.692+0000 7f634ed9d640 1 -- 172.21.15.78:0/3984527449 learned_addr learned my addr 172.21.15.78:0/3984527449 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.693+0000 7f634ed9d640 1 -- 172.21.15.78:0/3984527449 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635007c960 con 0x7f635007dd30 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.693+0000 7f634ed9d640 1 --2- 172.21.15.78:0/3984527449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635007c2a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6340009820 tx=0x7f634002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8887a50060a74b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f634dd9b640 1 -- 172.21.15.78:0/3984527449 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6340037070 con 0x7f635007dd30 2024-09-16T00:04:07.522 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f634dd9b640 1 -- 172.21.15.78:0/3984527449 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6340032920 con 0x7f635007dd30 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f634dd9b640 1 -- 172.21.15.78:0/3984527449 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6340030810 con 0x7f635007dd30 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f63550cb640 1 -- 172.21.15.78:0/3984527449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 msgr2=0x7f635007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f63550cb640 1 --2- 172.21.15.78:0/3984527449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635007c2a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6340009820 tx=0x7f634002a270 comp rx=0 tx=0).stop 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f63550cb640 1 -- 172.21.15.78:0/3984527449 shutdown_connections 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f63550cb640 1 --2- 172.21.15.78:0/3984527449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635007c2a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f63550cb640 1 -- 172.21.15.78:0/3984527449 >> 172.21.15.78:0/3984527449 conn(0x7f6350102e50 msgr2=0x7f6350105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f63550cb640 1 -- 172.21.15.78:0/3984527449 shutdown_connections 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.694+0000 7f63550cb640 1 -- 172.21.15.78:0/3984527449 wait complete. 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.695+0000 7f63550cb640 1 Processor -- start 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.695+0000 7f63550cb640 1 -- start start 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.695+0000 7f63550cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.695+0000 7f63550cb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f635010dba0 con 0x7f635007dd30 2024-09-16T00:04:07.523 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f634ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f634ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35358/0 (socket says 172.21.15.78:35358) 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f634ed9d640 1 -- 172.21.15.78:0/1304299041 learned_addr learned my addr 172.21.15.78:0/1304299041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f634ed9d640 1 -- 172.21.15.78:0/1304299041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635019dff0 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f634ed9d640 1 --2- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635019dab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63400097f0 tx=0x7f63400327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f6337fff640 1 -- 172.21.15.78:0/1304299041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6340037040 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635019e280 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f6337fff640 1 -- 172.21.15.78:0/1304299041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6340030cc0 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f6337fff640 1 -- 172.21.15.78:0/1304299041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f634003a6b0 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.696+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63501a0f70 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.697+0000 7f6337fff640 1 -- 172.21.15.78:0/1304299041 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f634003a8a0 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.697+0000 7f6337fff640 1 --2- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6338039200 0x7f633803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.697+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635007c320 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.698+0000 7f6337fff640 1 -- 172.21.15.78:0/1304299041 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f634006c900 con 0x7f635007dd30 2024-09-16T00:04:07.524 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.698+0000 7f634e59c640 1 --2- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6338039200 0x7f633803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.698+0000 7f634e59c640 1 --2- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6338039200 0x7f633803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f63440098c0 tx=0x7f6344006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.702+0000 7f6337fff640 1 -- 172.21.15.78:0/1304299041 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f634002fd40 con 0x7f635007dd30 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:05.895+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f6350195ee0 con 0x7f6338039200 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.121+0000 7f6337fff640 1 -- 172.21.15.78:0/1304299041 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f6350195ee0 con 0x7f6338039200 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6338039200 msgr2=0x7f633803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 --2- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6338039200 0x7f633803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f63440098c0 tx=0x7f6344006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 msgr2=0x7f635019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 --2- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635019dab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63400097f0 tx=0x7f63400327b0 comp rx=0 tx=0).stop 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 shutdown_connections 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 --2- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6338039200 0x7f633803b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 --2- 172.21.15.78:0/1304299041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f635007dd30 0x7f635019dab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 >> 172.21.15.78:0/1304299041 conn(0x7f6350102e50 msgr2=0x7f6350103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:07.525 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 shutdown_connections 2024-09-16T00:04:07.526 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:06.123+0000 7f63550cb640 1 -- 172.21.15.78:0/1304299041 wait complete. 2024-09-16T00:04:07.526 INFO:teuthology.orchestra.run.smithi078.stdout:Fetching dashboard port number... 2024-09-16T00:04:08.437 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:08 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1057191962' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T00:04:08.680 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T00:04:08.680 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.961+0000 7fbcfa8e9640 1 Processor -- start 2024-09-16T00:04:08.680 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.961+0000 7fbcfa8e9640 1 -- start start 2024-09-16T00:04:08.680 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.961+0000 7fbcfa8e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf4107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:08.680 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.961+0000 7fbcfa8e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4107f60 con 0x7fbcf4107620 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.961+0000 7fbcf3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf4107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.961+0000 7fbcf3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf4107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36794/0 (socket says 172.21.15.78:36794) 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.961+0000 7fbcf3fff640 1 -- 172.21.15.78:0/3156814932 learned_addr learned my addr 172.21.15.78:0/3156814932 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.961+0000 7fbcf3fff640 1 -- 172.21.15.78:0/3156814932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf41080e0 con 0x7fbcf4107620 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.962+0000 7fbcf3fff640 1 --2- 172.21.15.78:0/3156814932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf4107a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009820 tx=0x7fbce402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44263e724503af74 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.962+0000 7fbcf2ffd640 1 -- 172.21.15.78:0/3156814932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce4037070 con 0x7fbcf4107620 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.962+0000 7fbcf2ffd640 1 -- 172.21.15.78:0/3156814932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbce4032920 con 0x7fbcf4107620 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.962+0000 7fbcf2ffd640 1 -- 172.21.15.78:0/3156814932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce4030810 con 0x7fbcf4107620 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/3156814932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 msgr2=0x7fbcf4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:08.681 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 --2- 172.21.15.78:0/3156814932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf4107a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009820 tx=0x7fbce402a270 comp rx=0 tx=0).stop 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/3156814932 shutdown_connections 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 --2- 172.21.15.78:0/3156814932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf4107a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/3156814932 >> 172.21.15.78:0/3156814932 conn(0x7fbcf4102e50 msgr2=0x7fbcf4105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/3156814932 shutdown_connections 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/3156814932 wait complete. 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 Processor -- start 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.963+0000 7fbcfa8e9640 1 -- start start 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcfa8e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:08.682 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcfa8e9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf410ba90 con 0x7fbcf4107620 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcf3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcf3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf419b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36802/0 (socket says 172.21.15.78:36802) 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcf3fff640 1 -- 172.21.15.78:0/1057191962 learned_addr learned my addr 172.21.15.78:0/1057191962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcf3fff640 1 -- 172.21.15.78:0/1057191962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf419be00 con 0x7fbcf4107620 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcf3fff640 1 --2- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf419b8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbce4030380 tx=0x7fbce4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcf17fa640 1 -- 172.21.15.78:0/1057191962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce4037040 con 0x7fbcf4107620 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcf17fa640 1 -- 172.21.15.78:0/1057191962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbce4040070 con 0x7fbcf4107620 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf419c090 con 0x7fbcf4107620 2024-09-16T00:04:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.964+0000 7fbcf17fa640 1 -- 172.21.15.78:0/1057191962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbce403a7b0 con 0x7fbcf4107620 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.965+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf419ed80 con 0x7fbcf4107620 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.965+0000 7fbcf17fa640 1 -- 172.21.15.78:0/1057191962 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fbce403a9e0 con 0x7fbcf4107620 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.965+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf4107aa0 con 0x7fbcf4107620 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.966+0000 7fbcf17fa640 1 --2- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbcdc039260 0x7fbcdc03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.966+0000 7fbcf17fa640 1 -- 172.21.15.78:0/1057191962 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbce406ca10 con 0x7fbcf4107620 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.966+0000 7fbcf37fe640 1 --2- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbcdc039260 0x7fbcdc03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.966+0000 7fbcf37fe640 1 --2- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbcdc039260 0x7fbcdc03b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbce80098c0 tx=0x7fbce8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:07.970+0000 7fbcf17fa640 1 -- 172.21.15.78:0/1057191962 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbce4044c70 con 0x7fbcf4107620 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.136+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fbcf402d7c0 con 0x7fbcf4107620 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.136+0000 7fbcf17fa640 1 -- 172.21.15.78:0/1057191962 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fbce4041270 con 0x7fbcf4107620 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.139+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbcdc039260 msgr2=0x7fbcdc03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:08.684 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.139+0000 7fbcfa8e9640 1 --2- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbcdc039260 0x7fbcdc03b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbce80098c0 tx=0x7fbce8006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:08.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.139+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 msgr2=0x7fbcf419b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:08.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.139+0000 7fbcfa8e9640 1 --2- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf419b8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbce4030380 tx=0x7fbce4032670 comp rx=0 tx=0).stop 2024-09-16T00:04:08.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.139+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 shutdown_connections 2024-09-16T00:04:08.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.139+0000 7fbcfa8e9640 1 --2- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbcdc039260 0x7fbcdc03b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:08.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.139+0000 7fbcfa8e9640 1 --2- 172.21.15.78:0/1057191962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf4107620 0x7fbcf419b8c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:08.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.139+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 >> 172.21.15.78:0/1057191962 conn(0x7fbcf4102e50 msgr2=0x7fbcf4103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:08.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.140+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 shutdown_connections 2024-09-16T00:04:08.685 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:08.140+0000 7fbcfa8e9640 1 -- 172.21.15.78:0/1057191962 wait complete. 2024-09-16T00:04:08.686 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:04:08.686 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout disabled 2024-09-16T00:04:08.694 INFO:teuthology.orchestra.run.smithi078.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:04:08.694 INFO:teuthology.orchestra.run.smithi078.stdout:systemctl: stdout inactive 2024-09-16T00:04:08.694 INFO:teuthology.orchestra.run.smithi078.stdout:firewalld.service is not enabled 2024-09-16T00:04:08.694 INFO:teuthology.orchestra.run.smithi078.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T00:04:08.696 INFO:teuthology.orchestra.run.smithi078.stdout:Ceph Dashboard is now available at: 2024-09-16T00:04:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: URL: https://smithi078.front.sepia.ceph.com:8443/ 2024-09-16T00:04:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: User: admin 2024-09-16T00:04:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: Password: xmqeme074i 2024-09-16T00:04:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:08.697 INFO:teuthology.orchestra.run.smithi078.stdout:Saving cluster configuration to /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config directory 2024-09-16T00:04:09.373 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:09 smithi078 ceph-mon[31157]: mgrmap e13: x(active, since 6s) 2024-09-16T00:04:09.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.109+0000 7f6a4022c640 1 Processor -- start 2024-09-16T00:04:09.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.110+0000 7f6a4022c640 1 -- start start 2024-09-16T00:04:09.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.110+0000 7f6a4022c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a38105cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:09.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.110+0000 7f6a4022c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38106290 con 0x7f6a381058c0 2024-09-16T00:04:09.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.110+0000 7f6a3dfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a38105cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:09.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.110+0000 7f6a3dfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a38105cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36804/0 (socket says 172.21.15.78:36804) 2024-09-16T00:04:09.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.110+0000 7f6a3dfc8640 1 -- 172.21.15.78:0/1187366889 learned_addr learned my addr 172.21.15.78:0/1187366889 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:09.927 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.110+0000 7f6a3dfc8640 1 -- 172.21.15.78:0/1187366889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a381088a0 con 0x7f6a381058c0 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.111+0000 7f6a3dfc8640 1 --2- 172.21.15.78:0/1187366889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a38105cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009820 tx=0x7f6a3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abe7dc7756e31b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.111+0000 7f6a3cfc6640 1 -- 172.21.15.78:0/1187366889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a34037070 con 0x7f6a381058c0 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.111+0000 7f6a3cfc6640 1 -- 172.21.15.78:0/1187366889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a34032920 con 0x7f6a381058c0 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.111+0000 7f6a3cfc6640 1 -- 172.21.15.78:0/1187366889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a34030810 con 0x7f6a381058c0 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 -- 172.21.15.78:0/1187366889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 msgr2=0x7f6a38105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 --2- 172.21.15.78:0/1187366889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a38105cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009820 tx=0x7f6a3402a270 comp rx=0 tx=0).stop 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 -- 172.21.15.78:0/1187366889 shutdown_connections 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 --2- 172.21.15.78:0/1187366889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a38105cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 -- 172.21.15.78:0/1187366889 >> 172.21.15.78:0/1187366889 conn(0x7f6a38101430 msgr2=0x7f6a38103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 -- 172.21.15.78:0/1187366889 shutdown_connections 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 -- 172.21.15.78:0/1187366889 wait complete. 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 Processor -- start 2024-09-16T00:04:09.928 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.112+0000 7f6a4022c640 1 -- start start 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a4022c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a3819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a4022c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3810bad0 con 0x7f6a381058c0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a3dfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a3819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a3dfc8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a3819b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36818/0 (socket says 172.21.15.78:36818) 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a3dfc8640 1 -- 172.21.15.78:0/3840711069 learned_addr learned my addr 172.21.15.78:0/3840711069 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a3dfc8640 1 -- 172.21.15.78:0/3840711069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3819bea0 con 0x7f6a381058c0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a3dfc8640 1 --2- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a3819b960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a34030560 tx=0x7f6a34032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a2affd640 1 -- 172.21.15.78:0/3840711069 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a34037040 con 0x7f6a381058c0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a2affd640 1 -- 172.21.15.78:0/3840711069 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a34040070 con 0x7f6a381058c0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a2affd640 1 -- 172.21.15.78:0/3840711069 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a3403a6d0 con 0x7f6a381058c0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3819c130 con 0x7f6a381058c0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.113+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3819ee20 con 0x7f6a381058c0 2024-09-16T00:04:09.929 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.114+0000 7f6a2affd640 1 -- 172.21.15.78:0/3840711069 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6a3403b420 con 0x7f6a381058c0 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.114+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a38105d40 con 0x7f6a381058c0 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.114+0000 7f6a2affd640 1 --2- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a24039300 0x7f6a2403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.114+0000 7f6a2affd640 1 -- 172.21.15.78:0/3840711069 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6a3406c970 con 0x7f6a381058c0 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.114+0000 7f6a3d7c7640 1 --2- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a24039300 0x7f6a2403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.115+0000 7f6a3d7c7640 1 --2- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a24039300 0x7f6a2403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0098c0 tx=0x7f6a2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.119+0000 7f6a2affd640 1 -- 172.21.15.78:0/3840711069 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a34043360 con 0x7f6a381058c0 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.372+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f6a3802d7c0 con 0x7f6a381058c0 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.374+0000 7f6a2affd640 1 -- 172.21.15.78:0/3840711069 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f6a34042b50 con 0x7f6a381058c0 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.376+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a24039300 msgr2=0x7f6a2403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.376+0000 7f6a4022c640 1 --2- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a24039300 0x7f6a2403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0098c0 tx=0x7f6a2c006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:09.930 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.376+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 msgr2=0x7f6a3819b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.376+0000 7f6a4022c640 1 --2- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a3819b960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a34030560 tx=0x7f6a34032670 comp rx=0 tx=0).stop 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.376+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 shutdown_connections 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.376+0000 7f6a4022c640 1 --2- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a24039300 0x7f6a2403b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.376+0000 7f6a4022c640 1 --2- 172.21.15.78:0/3840711069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a381058c0 0x7f6a3819b960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.376+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 >> 172.21.15.78:0/3840711069 conn(0x7f6a38101430 msgr2=0x7f6a381021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.377+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 shutdown_connections 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/bin/ceph: stderr 2024-09-16T00:04:09.377+0000 7f6a4022c640 1 -- 172.21.15.78:0/3840711069 wait complete. 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:09.931 INFO:teuthology.orchestra.run.smithi078.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout: ceph telemetry on 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout:For more information see: 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:09.932 INFO:teuthology.orchestra.run.smithi078.stdout:Bootstrap complete. 2024-09-16T00:04:09.963 INFO:tasks.cephadm:Fetching config... 2024-09-16T00:04:09.963 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:04:09.964 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T00:04:09.982 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T00:04:09.982 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:04:09.982 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T00:04:10.038 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T00:04:10.038 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:04:10.038 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T00:04:10.104 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T00:04:10.104 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:04:10.104 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T00:04:10.160 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T00:04:10.160 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnv2jWhnxu1NZjNQaifTtjiFZbokoouQu+KvWy6CNYRIZ5eTLAzKCh3mqihFDIpklzON0W9zyvoo4mLj5G32OYkTpsvpjHfYBqA0LnQFGjNfkaHQ3+Pn1SZUImR06qhBGQURdu1uD4AXvLvv+v5pls6/6tjWwWseGiNvYaAR/nGfbgjb5KiXtc5kmxu2vVKsZRpVYqZmctptqhUfN5RMTDsKATfg0C4JIVI/faFGDyFN8qqkkLQfksOXPWWNuPleYpnkC2uLimu1bGNPrnd0mgHiod57K0XWbKN9mLbgng/jGS+im1jzFA5xLwFCXfUUTHQiTAWA8u2YWfapiNm246N20zL0jamGb1thfqDflKzKkHVgcE6E/SFF8dbFhP2NtCs0a33XIh8SsxqZ/K7n1GC2ib8AOVk9/DG9S3BuNzO1y7ZVgD1jRhBgj5yJBm2nKH+z/xr504M52I6iU/L4UdUUw5WKiKRV5n5HF2Y+X0q6L7CKZDKiv83Y45/IbeSD0= ceph-eb73a744-73be-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:04:10.233 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnv2jWhnxu1NZjNQaifTtjiFZbokoouQu+KvWy6CNYRIZ5eTLAzKCh3mqihFDIpklzON0W9zyvoo4mLj5G32OYkTpsvpjHfYBqA0LnQFGjNfkaHQ3+Pn1SZUImR06qhBGQURdu1uD4AXvLvv+v5pls6/6tjWwWseGiNvYaAR/nGfbgjb5KiXtc5kmxu2vVKsZRpVYqZmctptqhUfN5RMTDsKATfg0C4JIVI/faFGDyFN8qqkkLQfksOXPWWNuPleYpnkC2uLimu1bGNPrnd0mgHiod57K0XWbKN9mLbgng/jGS+im1jzFA5xLwFCXfUUTHQiTAWA8u2YWfapiNm246N20zL0jamGb1thfqDflKzKkHVgcE6E/SFF8dbFhP2NtCs0a33XIh8SsxqZ/K7n1GC2ib8AOVk9/DG9S3BuNzO1y7ZVgD1jRhBgj5yJBm2nKH+z/xr504M52I6iU/L4UdUUw5WKiKRV5n5HF2Y+X0q6L7CKZDKiv83Y45/IbeSD0= ceph-eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:10.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnv2jWhnxu1NZjNQaifTtjiFZbokoouQu+KvWy6CNYRIZ5eTLAzKCh3mqihFDIpklzON0W9zyvoo4mLj5G32OYkTpsvpjHfYBqA0LnQFGjNfkaHQ3+Pn1SZUImR06qhBGQURdu1uD4AXvLvv+v5pls6/6tjWwWseGiNvYaAR/nGfbgjb5KiXtc5kmxu2vVKsZRpVYqZmctptqhUfN5RMTDsKATfg0C4JIVI/faFGDyFN8qqkkLQfksOXPWWNuPleYpnkC2uLimu1bGNPrnd0mgHiod57K0XWbKN9mLbgng/jGS+im1jzFA5xLwFCXfUUTHQiTAWA8u2YWfapiNm246N20zL0jamGb1thfqDflKzKkHVgcE6E/SFF8dbFhP2NtCs0a33XIh8SsxqZ/K7n1GC2ib8AOVk9/DG9S3BuNzO1y7ZVgD1jRhBgj5yJBm2nKH+z/xr504M52I6iU/L4UdUUw5WKiKRV5n5HF2Y+X0q6L7CKZDKiv83Y45/IbeSD0= ceph-eb73a744-73be-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:04:10.286 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnv2jWhnxu1NZjNQaifTtjiFZbokoouQu+KvWy6CNYRIZ5eTLAzKCh3mqihFDIpklzON0W9zyvoo4mLj5G32OYkTpsvpjHfYBqA0LnQFGjNfkaHQ3+Pn1SZUImR06qhBGQURdu1uD4AXvLvv+v5pls6/6tjWwWseGiNvYaAR/nGfbgjb5KiXtc5kmxu2vVKsZRpVYqZmctptqhUfN5RMTDsKATfg0C4JIVI/faFGDyFN8qqkkLQfksOXPWWNuPleYpnkC2uLimu1bGNPrnd0mgHiod57K0XWbKN9mLbgng/jGS+im1jzFA5xLwFCXfUUTHQiTAWA8u2YWfapiNm246N20zL0jamGb1thfqDflKzKkHVgcE6E/SFF8dbFhP2NtCs0a33XIh8SsxqZ/K7n1GC2ib8AOVk9/DG9S3BuNzO1y7ZVgD1jRhBgj5yJBm2nKH+z/xr504M52I6iU/L4UdUUw5WKiKRV5n5HF2Y+X0q6L7CKZDKiv83Y45/IbeSD0= ceph-eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:10.299 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnv2jWhnxu1NZjNQaifTtjiFZbokoouQu+KvWy6CNYRIZ5eTLAzKCh3mqihFDIpklzON0W9zyvoo4mLj5G32OYkTpsvpjHfYBqA0LnQFGjNfkaHQ3+Pn1SZUImR06qhBGQURdu1uD4AXvLvv+v5pls6/6tjWwWseGiNvYaAR/nGfbgjb5KiXtc5kmxu2vVKsZRpVYqZmctptqhUfN5RMTDsKATfg0C4JIVI/faFGDyFN8qqkkLQfksOXPWWNuPleYpnkC2uLimu1bGNPrnd0mgHiod57K0XWbKN9mLbgng/jGS+im1jzFA5xLwFCXfUUTHQiTAWA8u2YWfapiNm246N20zL0jamGb1thfqDflKzKkHVgcE6E/SFF8dbFhP2NtCs0a33XIh8SsxqZ/K7n1GC2ib8AOVk9/DG9S3BuNzO1y7ZVgD1jRhBgj5yJBm2nKH+z/xr504M52I6iU/L4UdUUw5WKiKRV5n5HF2Y+X0q6L7CKZDKiv83Y45/IbeSD0= ceph-eb73a744-73be-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:04:10.340 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnv2jWhnxu1NZjNQaifTtjiFZbokoouQu+KvWy6CNYRIZ5eTLAzKCh3mqihFDIpklzON0W9zyvoo4mLj5G32OYkTpsvpjHfYBqA0LnQFGjNfkaHQ3+Pn1SZUImR06qhBGQURdu1uD4AXvLvv+v5pls6/6tjWwWseGiNvYaAR/nGfbgjb5KiXtc5kmxu2vVKsZRpVYqZmctptqhUfN5RMTDsKATfg0C4JIVI/faFGDyFN8qqkkLQfksOXPWWNuPleYpnkC2uLimu1bGNPrnd0mgHiod57K0XWbKN9mLbgng/jGS+im1jzFA5xLwFCXfUUTHQiTAWA8u2YWfapiNm246N20zL0jamGb1thfqDflKzKkHVgcE6E/SFF8dbFhP2NtCs0a33XIh8SsxqZ/K7n1GC2ib8AOVk9/DG9S3BuNzO1y7ZVgD1jRhBgj5yJBm2nKH+z/xr504M52I6iU/L4UdUUw5WKiKRV5n5HF2Y+X0q6L7CKZDKiv83Y45/IbeSD0= ceph-eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:10.353 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T00:04:10.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:10 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3840711069' entity='client.admin' 2024-09-16T00:04:10.607 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:11.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.052+0000 7fb3e9227640 1 -- 172.21.15.78:0/2949048539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 msgr2=0x7fb3e4100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:11.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.052+0000 7fb3e9227640 1 --2- 172.21.15.78:0/2949048539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 0x7fb3e4100940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009880 tx=0x7fb3d402a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.053+0000 7fb3e9227640 1 -- 172.21.15.78:0/2949048539 shutdown_connections 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.053+0000 7fb3e9227640 1 --2- 172.21.15.78:0/2949048539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 0x7fb3e4100940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.053+0000 7fb3e9227640 1 -- 172.21.15.78:0/2949048539 >> 172.21.15.78:0/2949048539 conn(0x7fb3e40fbcd0 msgr2=0x7fb3e40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.053+0000 7fb3e9227640 1 -- 172.21.15.78:0/2949048539 shutdown_connections 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.053+0000 7fb3e9227640 1 -- 172.21.15.78:0/2949048539 wait complete. 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.053+0000 7fb3e9227640 1 Processor -- start 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.053+0000 7fb3e9227640 1 -- start start 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.054+0000 7fb3e9227640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 0x7fb3e418d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.054+0000 7fb3e9227640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e41065f0 con 0x7fb3e4100540 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.054+0000 7fb3e2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 0x7fb3e418d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.054+0000 7fb3e2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 0x7fb3e418d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36836/0 (socket says 172.21.15.78:36836) 2024-09-16T00:04:11.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.054+0000 7fb3e2d9d640 1 -- 172.21.15.78:0/4227700309 learned_addr learned my addr 172.21.15.78:0/4227700309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.054+0000 7fb3e2d9d640 1 -- 172.21.15.78:0/4227700309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e418dc60 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.054+0000 7fb3e2d9d640 1 --2- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 0x7fb3e418d720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4004f60 tx=0x7fb3d40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.054+0000 7fb3cbfff640 1 -- 172.21.15.78:0/4227700309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3d403f070 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.055+0000 7fb3cbfff640 1 -- 172.21.15.78:0/4227700309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d4004d40 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.055+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e418be50 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.055+0000 7fb3cbfff640 1 -- 172.21.15.78:0/4227700309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3d4035030 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.055+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e418c390 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.056+0000 7fb3cbfff640 1 -- 172.21.15.78:0/4227700309 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb3d4042070 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.056+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e4106d70 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.056+0000 7fb3cbfff640 1 --2- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb3cc039300 0x7fb3cc03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.056+0000 7fb3cbfff640 1 -- 172.21.15.78:0/4227700309 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb3d406d250 con 0x7fb3e4100540 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.056+0000 7fb3e259c640 1 --2- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb3cc039300 0x7fb3cc03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:11.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.057+0000 7fb3e259c640 1 --2- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb3cc039300 0x7fb3cc03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80098c0 tx=0x7fb3d8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:11.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.061+0000 7fb3cbfff640 1 -- 172.21.15.78:0/4227700309 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3d4042360 con 0x7fb3e4100540 2024-09-16T00:04:11.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.233+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb3e418c970 con 0x7fb3e4100540 2024-09-16T00:04:11.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.236+0000 7fb3cbfff640 1 -- 172.21.15.78:0/4227700309 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb3d402f300 con 0x7fb3e4100540 2024-09-16T00:04:11.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.238+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb3cc039300 msgr2=0x7fb3cc03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:11.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.238+0000 7fb3e9227640 1 --2- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb3cc039300 0x7fb3cc03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80098c0 tx=0x7fb3d8006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:11.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.238+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 msgr2=0x7fb3e418d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.238+0000 7fb3e9227640 1 --2- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 0x7fb3e418d720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4004f60 tx=0x7fb3d40048f0 comp rx=0 tx=0).stop 2024-09-16T00:04:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.239+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 shutdown_connections 2024-09-16T00:04:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.239+0000 7fb3e9227640 1 --2- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb3cc039300 0x7fb3cc03b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.239+0000 7fb3e9227640 1 --2- 172.21.15.78:0/4227700309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3e4100540 0x7fb3e418d720 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.239+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 >> 172.21.15.78:0/4227700309 conn(0x7fb3e40fbcd0 msgr2=0x7fb3e40fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.239+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 shutdown_connections 2024-09-16T00:04:11.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:11.239+0000 7fb3e9227640 1 -- 172.21.15.78:0/4227700309 wait complete. 2024-09-16T00:04:12.420 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T00:04:12.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T00:04:12.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:12 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4227700309' entity='client.admin' 2024-09-16T00:04:12.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:12 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:12.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:12 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:12.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:12 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:04:12.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:12 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:12.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:12 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:12.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:12 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:12.934 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.418+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3380435293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 msgr2=0x7fed84102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.418+0000 7fed8b3fc640 1 --2- 172.21.15.78:0/3380435293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 0x7fed84102fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed80009880 tx=0x7fed8002a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.418+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3380435293 shutdown_connections 2024-09-16T00:04:13.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.418+0000 7fed8b3fc640 1 --2- 172.21.15.78:0/3380435293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 0x7fed84102fd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.418+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3380435293 >> 172.21.15.78:0/3380435293 conn(0x7fed840fb400 msgr2=0x7fed840fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.418+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3380435293 shutdown_connections 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.418+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3380435293 wait complete. 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.419+0000 7fed8b3fc640 1 Processor -- start 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.419+0000 7fed8b3fc640 1 -- start start 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.419+0000 7fed8b3fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 0x7fed84196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.419+0000 7fed8b3fc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed84107700 con 0x7fed84102bd0 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.419+0000 7fed89198640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 0x7fed84196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.419+0000 7fed89198640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 0x7fed84196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36852/0 (socket says 172.21.15.78:36852) 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.419+0000 7fed89198640 1 -- 172.21.15.78:0/3038835418 learned_addr learned my addr 172.21.15.78:0/3038835418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.420+0000 7fed89198640 1 -- 172.21.15.78:0/3038835418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed84197310 con 0x7fed84102bd0 2024-09-16T00:04:13.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.420+0000 7fed89198640 1 --2- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 0x7fed84196dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed80004f60 tx=0x7fed800048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.420+0000 7fed7a7fc640 1 -- 172.21.15.78:0/3038835418 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed8003f070 con 0x7fed84102bd0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.420+0000 7fed7a7fc640 1 -- 172.21.15.78:0/3038835418 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed80004d40 con 0x7fed84102bd0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.420+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed841975a0 con 0x7fed84102bd0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.420+0000 7fed7a7fc640 1 -- 172.21.15.78:0/3038835418 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed80035030 con 0x7fed84102bd0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.420+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8419a290 con 0x7fed84102bd0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.421+0000 7fed7a7fc640 1 -- 172.21.15.78:0/3038835418 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fed80042070 con 0x7fed84102bd0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.421+0000 7fed7a7fc640 1 --2- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed700392a0 0x7fed7003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.421+0000 7fed7a7fc640 1 -- 172.21.15.78:0/3038835418 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fed8006d1b0 con 0x7fed84102bd0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.421+0000 7fed88997640 1 --2- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed700392a0 0x7fed7003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.421+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed84107e80 con 0x7fed84102bd0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.422+0000 7fed88997640 1 --2- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed700392a0 0x7fed7003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed740098c0 tx=0x7fed74006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:13.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.429+0000 7fed7a7fc640 1 -- 172.21.15.78:0/3038835418 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed8003b550 con 0x7fed84102bd0 2024-09-16T00:04:13.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.605+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fed84062630 con 0x7fed700392a0 2024-09-16T00:04:13.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.608+0000 7fed7a7fc640 1 -- 172.21.15.78:0/3038835418 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fed84062630 con 0x7fed700392a0 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed700392a0 msgr2=0x7fed7003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 --2- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed700392a0 0x7fed7003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed740098c0 tx=0x7fed74006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 msgr2=0x7fed84196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 --2- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 0x7fed84196dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed80004f60 tx=0x7fed800048f0 comp rx=0 tx=0).stop 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 shutdown_connections 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 --2- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed700392a0 0x7fed7003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 --2- 172.21.15.78:0/3038835418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed84102bd0 0x7fed84196dd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 >> 172.21.15.78:0/3038835418 conn(0x7fed840fb400 msgr2=0x7fed840fbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 shutdown_connections 2024-09-16T00:04:13.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:13.611+0000 7fed8b3fc640 1 -- 172.21.15.78:0/3038835418 wait complete. 2024-09-16T00:04:14.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:14.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:14.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:14.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:14.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:14.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: Updating smithi078:/etc/ceph/ceph.conf 2024-09-16T00:04:14.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: Updating smithi078:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:14.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:14.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: Updating smithi078:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:04:14.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:14.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:14.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:14.805 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-09-16T00:04:14.806 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:04:14.806 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:04:14.825 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:04:14.826 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:14.884 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-09-16T00:04:14.884 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch host add smithi148 2024-09-16T00:04:15.123 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.536+0000 7feea7948640 1 -- 172.21.15.78:0/3731053340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 msgr2=0x7feea0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.536+0000 7feea7948640 1 --2- 172.21.15.78:0/3731053340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 0x7feea0106940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009880 tx=0x7fee9c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.536+0000 7feea7948640 1 -- 172.21.15.78:0/3731053340 shutdown_connections 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.536+0000 7feea7948640 1 --2- 172.21.15.78:0/3731053340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 0x7feea0106940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.536+0000 7feea7948640 1 -- 172.21.15.78:0/3731053340 >> 172.21.15.78:0/3731053340 conn(0x7feea00fc1b0 msgr2=0x7feea00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.536+0000 7feea7948640 1 -- 172.21.15.78:0/3731053340 shutdown_connections 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.536+0000 7feea7948640 1 -- 172.21.15.78:0/3731053340 wait complete. 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.537+0000 7feea7948640 1 Processor -- start 2024-09-16T00:04:15.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.537+0000 7feea7948640 1 -- start start 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.537+0000 7feea7948640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 0x7feea0197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.537+0000 7feea7948640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feea010ab90 con 0x7feea0106560 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.537+0000 7feea56e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 0x7feea0197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.537+0000 7feea56e4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 0x7feea0197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36870/0 (socket says 172.21.15.78:36870) 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.537+0000 7feea56e4640 1 -- 172.21.15.78:0/1846570221 learned_addr learned my addr 172.21.15.78:0/1846570221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.538+0000 7feea56e4640 1 -- 172.21.15.78:0/1846570221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea0197640 con 0x7feea0106560 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.538+0000 7feea56e4640 1 --2- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 0x7feea0197100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009230 tx=0x7fee9c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.538+0000 7fee967fc640 1 -- 172.21.15.78:0/1846570221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee9c03f070 con 0x7feea0106560 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.538+0000 7fee967fc640 1 -- 172.21.15.78:0/1846570221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9c03bc80 con 0x7feea0106560 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.538+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea01978d0 con 0x7feea0106560 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.538+0000 7fee967fc640 1 -- 172.21.15.78:0/1846570221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee9c035030 con 0x7feea0106560 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.538+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea019b9d0 con 0x7feea0106560 2024-09-16T00:04:15.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.539+0000 7fee967fc640 1 -- 172.21.15.78:0/1846570221 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fee9c042070 con 0x7feea0106560 2024-09-16T00:04:15.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.539+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea0101bd0 con 0x7feea0106560 2024-09-16T00:04:15.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.539+0000 7fee967fc640 1 --2- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee8c0392a0 0x7fee8c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:15.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.539+0000 7fee967fc640 1 -- 172.21.15.78:0/1846570221 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fee9c06d0f0 con 0x7feea0106560 2024-09-16T00:04:15.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.539+0000 7feea4ee3640 1 --2- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee8c0392a0 0x7fee8c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:15.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.540+0000 7feea4ee3640 1 --2- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee8c0392a0 0x7fee8c03b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee900098c0 tx=0x7fee90006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:15.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.544+0000 7fee967fc640 1 -- 172.21.15.78:0/1846570221 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee9c04c350 con 0x7feea0106560 2024-09-16T00:04:15.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:15.716+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) -- 0x7feea0198460 con 0x7fee8c0392a0 2024-09-16T00:04:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:16 smithi078 ceph-mon[31157]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:18.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:17 smithi078 ceph-mon[31157]: Deploying cephadm binary to smithi148 2024-09-16T00:04:18.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.876+0000 7fee967fc640 1 -- 172.21.15.78:0/1846570221 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7feea0198460 con 0x7fee8c0392a0 2024-09-16T00:04:18.877 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-09-16T00:04:18.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee8c0392a0 msgr2=0x7fee8c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:18.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 --2- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee8c0392a0 0x7fee8c03b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fee900098c0 tx=0x7fee90006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:18.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 msgr2=0x7feea0197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 --2- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 0x7feea0197100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009230 tx=0x7fee9c003d40 comp rx=0 tx=0).stop 2024-09-16T00:04:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 shutdown_connections 2024-09-16T00:04:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 --2- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee8c0392a0 0x7fee8c03b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 --2- 172.21.15.78:0/1846570221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feea0106560 0x7feea0197100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 >> 172.21.15.78:0/1846570221 conn(0x7feea00fc1b0 msgr2=0x7feea0069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 shutdown_connections 2024-09-16T00:04:18.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:18.878+0000 7feea7948640 1 -- 172.21.15.78:0/1846570221 wait complete. 2024-09-16T00:04:19.496 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:04:19.735 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:20.109 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:20.109 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:19 smithi078 ceph-mon[31157]: Added host smithi148 2024-09-16T00:04:20.109 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:20.109 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:20.109 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.183+0000 7f7f8100e640 1 -- 172.21.15.78:0/2536157694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 msgr2=0x7f7f7c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.183+0000 7f7f8100e640 1 --2- 172.21.15.78:0/2536157694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 0x7f7f7c107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009880 tx=0x7f7f6c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.184+0000 7f7f8100e640 1 -- 172.21.15.78:0/2536157694 shutdown_connections 2024-09-16T00:04:20.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.184+0000 7f7f8100e640 1 --2- 172.21.15.78:0/2536157694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 0x7f7f7c107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.184+0000 7f7f8100e640 1 -- 172.21.15.78:0/2536157694 >> 172.21.15.78:0/2536157694 conn(0x7f7f7c0fd290 msgr2=0x7f7f7c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.184+0000 7f7f8100e640 1 -- 172.21.15.78:0/2536157694 shutdown_connections 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.184+0000 7f7f8100e640 1 -- 172.21.15.78:0/2536157694 wait complete. 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.184+0000 7f7f8100e640 1 Processor -- start 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f8100e640 1 -- start start 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f8100e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 0x7f7f7c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f8100e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c10bc90 con 0x7f7f7c107660 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f7ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 0x7f7f7c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f7ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 0x7f7f7c197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53030/0 (socket says 172.21.15.78:53030) 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f7ad9d640 1 -- 172.21.15.78:0/2626469639 learned_addr learned my addr 172.21.15.78:0/2626469639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f7ad9d640 1 -- 172.21.15.78:0/2626469639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c1976c0 con 0x7f7f7c107660 2024-09-16T00:04:20.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f7ad9d640 1 --2- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 0x7f7f7c197180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009230 tx=0x7f7f6c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f63fff640 1 -- 172.21.15.78:0/2626469639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f6c03f070 con 0x7f7f7c107660 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.185+0000 7f7f63fff640 1 -- 172.21.15.78:0/2626469639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f6c03bc80 con 0x7f7f7c107660 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.186+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c197950 con 0x7f7f7c107660 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.186+0000 7f7f63fff640 1 -- 172.21.15.78:0/2626469639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f6c035030 con 0x7f7f7c107660 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.186+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c197db0 con 0x7f7f7c107660 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.186+0000 7f7f63fff640 1 -- 172.21.15.78:0/2626469639 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7f6c042070 con 0x7f7f7c107660 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.186+0000 7f7f63fff640 1 --2- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7f640392a0 0x7f7f6403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.186+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7c102cd0 con 0x7f7f7c107660 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.187+0000 7f7f63fff640 1 -- 172.21.15.78:0/2626469639 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7f6c06cfa0 con 0x7f7f7c107660 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.187+0000 7f7f7a59c640 1 --2- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7f640392a0 0x7f7f6403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.187+0000 7f7f7a59c640 1 --2- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7f640392a0 0x7f7f6403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f700098c0 tx=0x7f7f70006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:20.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.191+0000 7f7f63fff640 1 -- 172.21.15.78:0/2626469639 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f6c04c350 con 0x7f7f7c107660 2024-09-16T00:04:20.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.364+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f7c198510 con 0x7f7f640392a0 2024-09-16T00:04:20.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.365+0000 7f7f63fff640 1 -- 172.21.15.78:0/2626469639 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f7f7c198510 con 0x7f7f640392a0 2024-09-16T00:04:20.366 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:20.366 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-09-16T00:04:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.367+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7f640392a0 msgr2=0x7f7f6403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.367+0000 7f7f8100e640 1 --2- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7f640392a0 0x7f7f6403b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f700098c0 tx=0x7f7f70006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:20.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.367+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 msgr2=0x7f7f7c197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:20.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.367+0000 7f7f8100e640 1 --2- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 0x7f7f7c197180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009230 tx=0x7f7f6c003d40 comp rx=0 tx=0).stop 2024-09-16T00:04:20.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.367+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 shutdown_connections 2024-09-16T00:04:20.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.367+0000 7f7f8100e640 1 --2- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7f640392a0 0x7f7f6403b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:20.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.367+0000 7f7f8100e640 1 --2- 172.21.15.78:0/2626469639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7f7c107660 0x7f7f7c197180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:20.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.367+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 >> 172.21.15.78:0/2626469639 conn(0x7f7f7c0fd290 msgr2=0x7f7f7c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:20.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.368+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 shutdown_connections 2024-09-16T00:04:20.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:20.368+0000 7f7f8100e640 1 -- 172.21.15.78:0/2626469639 wait complete. 2024-09-16T00:04:20.988 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-09-16T00:04:20.988 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:04:20.989 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:04:21.007 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:04:21.007 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:21.065 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-09-16T00:04:21.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch host add smithi193 2024-09-16T00:04:21.306 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:21.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:21 smithi078 ceph-mon[31157]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:04:21.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 -- 172.21.15.78:0/2404516672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 msgr2=0x7fd128106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:21.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 --2- 172.21.15.78:0/2404516672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 0x7fd128106920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd124009810 tx=0x7fd12402a540 comp rx=0 tx=0).stop 2024-09-16T00:04:21.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 -- 172.21.15.78:0/2404516672 shutdown_connections 2024-09-16T00:04:21.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 --2- 172.21.15.78:0/2404516672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 0x7fd128106920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:21.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 -- 172.21.15.78:0/2404516672 >> 172.21.15.78:0/2404516672 conn(0x7fd128077310 msgr2=0x7fd1280fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:21.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 -- 172.21.15.78:0/2404516672 shutdown_connections 2024-09-16T00:04:21.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 -- 172.21.15.78:0/2404516672 wait complete. 2024-09-16T00:04:21.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 Processor -- start 2024-09-16T00:04:21.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.725+0000 7fd12fdfb640 1 -- start start 2024-09-16T00:04:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12fdfb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 0x7fd128197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12fdfb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd12810ab70 con 0x7fd128106540 2024-09-16T00:04:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12db97640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 0x7fd128197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12db97640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 0x7fd128197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53042/0 (socket says 172.21.15.78:53042) 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12db97640 1 -- 172.21.15.78:0/3013076467 learned_addr learned my addr 172.21.15.78:0/3013076467 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12db97640 1 -- 172.21.15.78:0/3013076467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd128197690 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12db97640 1 --2- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 0x7fd128197150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd1240045e0 tx=0x7fd124004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd11effd640 1 -- 172.21.15.78:0/3013076467 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd124030e70 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd11effd640 1 -- 172.21.15.78:0/3013076467 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1240318e0 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd128197920 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd11effd640 1 -- 172.21.15.78:0/3013076467 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1240049c0 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.726+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12819ba20 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.727+0000 7fd11effd640 1 -- 172.21.15.78:0/3013076467 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd124039040 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.727+0000 7fd11effd640 1 --2- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd114039300 0x7fd11403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.727+0000 7fd11effd640 1 -- 172.21.15.78:0/3013076467 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd12406d540 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.727+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd128101bb0 con 0x7fd128106540 2024-09-16T00:04:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.727+0000 7fd12d396640 1 --2- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd114039300 0x7fd11403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.728+0000 7fd12d396640 1 --2- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd114039300 0x7fd11403b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd1180098c0 tx=0x7fd118006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.732+0000 7fd11effd640 1 -- 172.21.15.78:0/3013076467 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd124040030 con 0x7fd128106540 2024-09-16T00:04:21.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:21.899+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) -- 0x7fd1281069a0 con 0x7fd114039300 2024-09-16T00:04:23.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:23.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:23.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:23.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:23.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:23.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:23.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:04:23.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:23.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:24.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:24 smithi078 ceph-mon[31157]: Deploying cephadm binary to smithi193 2024-09-16T00:04:24.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:24 smithi078 ceph-mon[31157]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-16T00:04:24.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:24 smithi078 ceph-mon[31157]: Updating smithi148:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:24.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:24 smithi078 ceph-mon[31157]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-09-16T00:04:25.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.165+0000 7fd11effd640 1 -- 172.21.15.78:0/3013076467 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd1281069a0 con 0x7fd114039300 2024-09-16T00:04:25.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd114039300 msgr2=0x7fd11403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:25.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 --2- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd114039300 0x7fd11403b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd1180098c0 tx=0x7fd118006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:25.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 msgr2=0x7fd128197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:25.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 --2- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 0x7fd128197150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd1240045e0 tx=0x7fd124004610 comp rx=0 tx=0).stop 2024-09-16T00:04:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 shutdown_connections 2024-09-16T00:04:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 --2- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd114039300 0x7fd11403b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 --2- 172.21.15.78:0/3013076467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd128106540 0x7fd128197150 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 >> 172.21.15.78:0/3013076467 conn(0x7fd128077310 msgr2=0x7fd1280696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 shutdown_connections 2024-09-16T00:04:25.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:25.167+0000 7fd12fdfb640 1 -- 172.21.15.78:0/3013076467 wait complete. 2024-09-16T00:04:25.832 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:25 smithi078 ceph-mon[31157]: Updating smithi148:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:04:25.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:25 smithi078 ceph-mon[31157]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:25.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:25.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:25.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:25.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:25.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:25.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:25.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:04:26.097 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:26.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.515+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2264109822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 msgr2=0x7fcc58107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:26.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.515+0000 7fcc5e2d0640 1 --2- 172.21.15.78:0/2264109822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 0x7fcc58107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009880 tx=0x7fcc4802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:26.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.515+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2264109822 shutdown_connections 2024-09-16T00:04:26.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.515+0000 7fcc5e2d0640 1 --2- 172.21.15.78:0/2264109822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 0x7fcc58107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:26.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.515+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2264109822 >> 172.21.15.78:0/2264109822 conn(0x7fcc580fd290 msgr2=0x7fcc580ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:26.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.515+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2264109822 shutdown_connections 2024-09-16T00:04:26.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.515+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2264109822 wait complete. 2024-09-16T00:04:26.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.516+0000 7fcc5e2d0640 1 Processor -- start 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.516+0000 7fcc5e2d0640 1 -- start start 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.516+0000 7fcc5e2d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 0x7fcc58197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.516+0000 7fcc5e2d0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5810bc90 con 0x7fcc58107660 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.516+0000 7fcc57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 0x7fcc58197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.516+0000 7fcc57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 0x7fcc58197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43056/0 (socket says 172.21.15.78:43056) 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.516+0000 7fcc57fff640 1 -- 172.21.15.78:0/2155251952 learned_addr learned my addr 172.21.15.78:0/2155251952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.517+0000 7fcc57fff640 1 -- 172.21.15.78:0/2155251952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc581976c0 con 0x7fcc58107660 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.517+0000 7fcc57fff640 1 --2- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 0x7fcc58197180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009230 tx=0x7fcc48003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.517+0000 7fcc557fa640 1 -- 172.21.15.78:0/2155251952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc4803f070 con 0x7fcc58107660 2024-09-16T00:04:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.517+0000 7fcc557fa640 1 -- 172.21.15.78:0/2155251952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc4803bc80 con 0x7fcc58107660 2024-09-16T00:04:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.517+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc58197950 con 0x7fcc58107660 2024-09-16T00:04:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.517+0000 7fcc557fa640 1 -- 172.21.15.78:0/2155251952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc48035030 con 0x7fcc58107660 2024-09-16T00:04:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.517+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc58197db0 con 0x7fcc58107660 2024-09-16T00:04:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.518+0000 7fcc557fa640 1 -- 172.21.15.78:0/2155251952 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fcc48042070 con 0x7fcc58107660 2024-09-16T00:04:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.518+0000 7fcc557fa640 1 --2- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc440392a0 0x7fcc4403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.518+0000 7fcc557fa640 1 -- 172.21.15.78:0/2155251952 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc480314c0 con 0x7fcc58107660 2024-09-16T00:04:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.518+0000 7fcc577fe640 1 --2- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc440392a0 0x7fcc4403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.518+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc58102cd0 con 0x7fcc58107660 2024-09-16T00:04:26.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.518+0000 7fcc577fe640 1 --2- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc440392a0 0x7fcc4403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc400098c0 tx=0x7fcc40006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:26.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.523+0000 7fcc557fa640 1 -- 172.21.15.78:0/2155251952 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc4804c350 con 0x7fcc58107660 2024-09-16T00:04:26.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.691+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc58198570 con 0x7fcc440392a0 2024-09-16T00:04:26.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.693+0000 7fcc557fa640 1 -- 172.21.15.78:0/2155251952 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fcc58198570 con 0x7fcc440392a0 2024-09-16T00:04:26.694 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:04:26.695 INFO:teuthology.orchestra.run.smithi078.stdout:[{"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-09-16T00:04:26.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.694+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc440392a0 msgr2=0x7fcc4403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:26.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.694+0000 7fcc5e2d0640 1 --2- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc440392a0 0x7fcc4403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc400098c0 tx=0x7fcc40006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.695+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 msgr2=0x7fcc58197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.695+0000 7fcc5e2d0640 1 --2- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 0x7fcc58197180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009230 tx=0x7fcc48003d40 comp rx=0 tx=0).stop 2024-09-16T00:04:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.695+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 shutdown_connections 2024-09-16T00:04:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.695+0000 7fcc5e2d0640 1 --2- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc440392a0 0x7fcc4403b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.695+0000 7fcc5e2d0640 1 --2- 172.21.15.78:0/2155251952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc58107660 0x7fcc58197180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.695+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 >> 172.21.15.78:0/2155251952 conn(0x7fcc580fd290 msgr2=0x7fcc580feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.695+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 shutdown_connections 2024-09-16T00:04:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:26.695+0000 7fcc5e2d0640 1 -- 172.21.15.78:0/2155251952 wait complete. 2024-09-16T00:04:26.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:26 smithi078 ceph-mon[31157]: Added host smithi193 2024-09-16T00:04:26.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:26 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:26.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:26 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:27.468 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T00:04:27.468 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T00:04:27.708 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:27 smithi078 ceph-mon[31157]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:28.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.161+0000 7f1967df0640 1 -- 172.21.15.78:0/3973247891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 msgr2=0x7f1960106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.161+0000 7f1967df0640 1 --2- 172.21.15.78:0/3973247891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 0x7f1960106940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f195c009880 tx=0x7f195c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 -- 172.21.15.78:0/3973247891 shutdown_connections 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 --2- 172.21.15.78:0/3973247891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 0x7f1960106940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 -- 172.21.15.78:0/3973247891 >> 172.21.15.78:0/3973247891 conn(0x7f19600fc1b0 msgr2=0x7f19600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 -- 172.21.15.78:0/3973247891 shutdown_connections 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 -- 172.21.15.78:0/3973247891 wait complete. 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 Processor -- start 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 -- start start 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 0x7f1960197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:28.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.162+0000 7f1967df0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f196010ab90 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1965b8c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 0x7f1960197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1965b8c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 0x7f1960197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43078/0 (socket says 172.21.15.78:43078) 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1965b8c640 1 -- 172.21.15.78:0/1707139311 learned_addr learned my addr 172.21.15.78:0/1707139311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1965b8c640 1 -- 172.21.15.78:0/1707139311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19601976d0 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1965b8c640 1 --2- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 0x7f1960197190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f195c009230 tx=0x7f195c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1956ffd640 1 -- 172.21.15.78:0/1707139311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f195c03f070 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1960197960 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1956ffd640 1 -- 172.21.15.78:0/1707139311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f195c03bc80 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1956ffd640 1 -- 172.21.15.78:0/1707139311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f195c035030 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.163+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1960197dc0 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.164+0000 7f1956ffd640 1 -- 172.21.15.78:0/1707139311 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f195c042070 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.164+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1960101bd0 con 0x7f1960106560 2024-09-16T00:04:28.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.164+0000 7f1956ffd640 1 --2- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f194c0392a0 0x7f194c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:28.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.164+0000 7f1956ffd640 1 -- 172.21.15.78:0/1707139311 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f195c06d080 con 0x7f1960106560 2024-09-16T00:04:28.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.164+0000 7f196538b640 1 --2- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f194c0392a0 0x7f194c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:28.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.165+0000 7f196538b640 1 --2- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f194c0392a0 0x7f194c03b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f19500098c0 tx=0x7f1950006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:28.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.169+0000 7f1956ffd640 1 -- 172.21.15.78:0/1707139311 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f195c03b7a0 con 0x7f1960106560 2024-09-16T00:04:28.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.338+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f19601984f0 con 0x7f1960106560 2024-09-16T00:04:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.547+0000 7f1956ffd640 1 -- 172.21.15.78:0/1707139311 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f195c06f0e0 con 0x7f1960106560 2024-09-16T00:04:28.549 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2024-09-16T00:04:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.549+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f194c0392a0 msgr2=0x7f194c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.549+0000 7f1967df0640 1 --2- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f194c0392a0 0x7f194c03b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f19500098c0 tx=0x7f1950006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.549+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 msgr2=0x7f1960197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.549+0000 7f1967df0640 1 --2- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 0x7f1960197190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f195c009230 tx=0x7f195c003d40 comp rx=0 tx=0).stop 2024-09-16T00:04:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.550+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 shutdown_connections 2024-09-16T00:04:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.550+0000 7f1967df0640 1 --2- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f194c0392a0 0x7f194c03b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.550+0000 7f1967df0640 1 --2- 172.21.15.78:0/1707139311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1960106560 0x7f1960197190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:28.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.550+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 >> 172.21.15.78:0/1707139311 conn(0x7f19600fc1b0 msgr2=0x7f1960069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.550+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 shutdown_connections 2024-09-16T00:04:28.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:28.550+0000 7f1967df0640 1 -- 172.21.15.78:0/1707139311 wait complete. 2024-09-16T00:04:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:28 smithi078 ceph-mon[31157]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:04:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:28 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1707139311' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:04:29.169 INFO:tasks.cephadm:Adding mon.a on smithi078 2024-09-16T00:04:29.169 INFO:tasks.cephadm:Adding mon.b on smithi148 2024-09-16T00:04:29.169 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-09-16T00:04:29.169 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi078:172.21.15.78=a;smithi148:172.21.15.148=b;smithi193:172.21.15.193=c' 2024-09-16T00:04:29.419 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:04:29.465 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:04:29.921 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 -- 172.21.15.193:0/1461189160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 msgr2=0x7f2fac100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:29.921 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 --2- 172.21.15.193:0/1461189160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 0x7f2fac100940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009880 tx=0x7f2f9c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:29.921 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 -- 172.21.15.193:0/1461189160 shutdown_connections 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 --2- 172.21.15.193:0/1461189160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 0x7f2fac100940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 -- 172.21.15.193:0/1461189160 >> 172.21.15.193:0/1461189160 conn(0x7f2fac0fbcd0 msgr2=0x7f2fac0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 -- 172.21.15.193:0/1461189160 shutdown_connections 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 -- 172.21.15.193:0/1461189160 wait complete. 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 Processor -- start 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.880+0000 7f2fb2794640 1 -- start start 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.881+0000 7f2fb2794640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 0x7f2fac06a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.881+0000 7f2fb2794640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1065f0 con 0x7f2fac100540 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.881+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 0x7f2fac06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.881+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 0x7f2fac06a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.193:38798/0 (socket says 172.21.15.193:38798) 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.881+0000 7f2fabfff640 1 -- 172.21.15.193:0/1493811374 learned_addr learned my addr 172.21.15.193:0/1493811374 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.881+0000 7f2fabfff640 1 -- 172.21.15.193:0/1493811374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fac199ab0 con 0x7f2fac100540 2024-09-16T00:04:29.922 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.882+0000 7f2fabfff640 1 --2- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 0x7f2fac06a130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c004f60 tx=0x7f2f9c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.882+0000 7f2fa97fa640 1 -- 172.21.15.193:0/1493811374 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f9c03f070 con 0x7f2fac100540 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.882+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fac199c80 con 0x7f2fac100540 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.882+0000 7f2fa97fa640 1 -- 172.21.15.193:0/1493811374 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f9c004d40 con 0x7f2fac100540 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.882+0000 7f2fa97fa640 1 -- 172.21.15.193:0/1493811374 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f9c035030 con 0x7f2fac100540 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.882+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fac19a020 con 0x7f2fac100540 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.883+0000 7f2fa97fa640 1 -- 172.21.15.193:0/1493811374 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f2f9c042070 con 0x7f2fac100540 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.883+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fac106d70 con 0x7f2fac100540 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.883+0000 7f2fa97fa640 1 --2- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f94039300 0x7f2f9403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.883+0000 7f2fa97fa640 1 -- 172.21.15.193:0/1493811374 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2f9c06d060 con 0x7f2fac100540 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.883+0000 7f2fab7fe640 1 --2- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f94039300 0x7f2f9403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.884+0000 7f2fab7fe640 1 --2- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f94039300 0x7f2f9403b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00098c0 tx=0x7f2fa0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:29.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:29.888+0000 7f2fa97fa640 1 -- 172.21.15.193:0/1493811374 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f9c042360 con 0x7f2fac100540 2024-09-16T00:04:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1707139311' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:04:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:04:30.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:30.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:30.088 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.087+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi148:172.21.15.148=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}) -- 0x7f2fac02db40 con 0x7f2f94039300 2024-09-16T00:04:30.093 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.092+0000 7f2fa97fa640 1 -- 172.21.15.193:0/1493811374 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2fac02db40 con 0x7f2f94039300 2024-09-16T00:04:30.093 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-09-16T00:04:30.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.094+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f94039300 msgr2=0x7f2f9403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.094+0000 7f2fb2794640 1 --2- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f94039300 0x7f2f9403b7c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00098c0 tx=0x7f2fa0006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.094+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 msgr2=0x7f2fac06a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.094+0000 7f2fb2794640 1 --2- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 0x7f2fac06a130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c004f60 tx=0x7f2f9c0048f0 comp rx=0 tx=0).stop 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.095+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 shutdown_connections 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.095+0000 7f2fb2794640 1 --2- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f94039300 0x7f2f9403b7c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.095+0000 7f2fb2794640 1 --2- 172.21.15.193:0/1493811374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fac100540 0x7f2fac06a130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.095+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 >> 172.21.15.193:0/1493811374 conn(0x7f2fac0fbcd0 msgr2=0x7f2fac0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.095+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 shutdown_connections 2024-09-16T00:04:30.096 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:30.095+0000 7f2fb2794640 1 -- 172.21.15.193:0/1493811374 wait complete. 2024-09-16T00:04:30.733 DEBUG:teuthology.orchestra.run.smithi148:mon.b> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T00:04:30.736 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T00:04:30.739 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:04:30.739 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:04:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:30 smithi078 ceph-mon[31157]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T00:04:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:30 smithi078 ceph-mon[31157]: Updating smithi193:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:31.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:31.382 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:31 smithi078 ceph-mon[31157]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:31 smithi078 ceph-mon[31157]: Updating smithi193:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:04:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:31 smithi078 ceph-mon[31157]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi148:172.21.15.148=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:32.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:31 smithi078 ceph-mon[31157]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi148:172.21.15.148=b;smithi193:172.21.15.193=c;count:3 2024-09-16T00:04:32.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:31 smithi078 ceph-mon[31157]: Deploying daemon mon.c on smithi193 2024-09-16T00:04:32.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:31 smithi078 ceph-mon[31157]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:32.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 -- 172.21.15.193:0/4185681998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 msgr2=0x7f9b7c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:32.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 --2- 172.21.15.193:0/4185681998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 0x7f9b7c106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009810 tx=0x7f9b6c02a540 comp rx=0 tx=0).stop 2024-09-16T00:04:32.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 -- 172.21.15.193:0/4185681998 shutdown_connections 2024-09-16T00:04:32.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 --2- 172.21.15.193:0/4185681998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 0x7f9b7c106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:32.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 -- 172.21.15.193:0/4185681998 >> 172.21.15.193:0/4185681998 conn(0x7f9b7c077310 msgr2=0x7f9b7c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:32.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 -- 172.21.15.193:0/4185681998 shutdown_connections 2024-09-16T00:04:32.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 -- 172.21.15.193:0/4185681998 wait complete. 2024-09-16T00:04:32.176 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 Processor -- start 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 -- start start 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.163+0000 7f9b81b37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 0x7f9b7c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b81b37640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c10ab70 con 0x7f9b7c106540 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b7b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 0x7f9b7c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b7b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 0x7f9b7c197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.193:38810/0 (socket says 172.21.15.193:38810) 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b7b7fe640 1 -- 172.21.15.193:0/3564603485 learned_addr learned my addr 172.21.15.193:0/3564603485 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b7b7fe640 1 -- 172.21.15.193:0/3564603485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c198510 con 0x7f9b7c106540 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b7b7fe640 1 --2- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 0x7f9b7c197fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0045e0 tx=0x7f9b6c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b797fa640 1 -- 172.21.15.193:0/3564603485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c030e70 con 0x7f9b7c106540 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c1987a0 con 0x7f9b7c106540 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b797fa640 1 -- 172.21.15.193:0/3564603485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b6c0318e0 con 0x7f9b7c106540 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.164+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c198c00 con 0x7f9b7c106540 2024-09-16T00:04:32.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.165+0000 7f9b797fa640 1 -- 172.21.15.193:0/3564603485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c049060 con 0x7f9b7c106540 2024-09-16T00:04:32.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.165+0000 7f9b797fa640 1 -- 172.21.15.193:0/3564603485 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9b6c039040 con 0x7f9b7c106540 2024-09-16T00:04:32.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.166+0000 7f9b797fa640 1 --2- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b680392a0 0x7f9b6803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:32.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.166+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c101bb0 con 0x7f9b7c106540 2024-09-16T00:04:32.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.166+0000 7f9b797fa640 1 -- 172.21.15.193:0/3564603485 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9b6c06d690 con 0x7f9b7c106540 2024-09-16T00:04:32.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.166+0000 7f9b73fff640 1 --2- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b680392a0 0x7f9b6803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:32.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.166+0000 7f9b73fff640 1 --2- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b680392a0 0x7f9b6803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9b640098c0 tx=0x7f9b64006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:32.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.171+0000 7f9b797fa640 1 -- 172.21.15.193:0/3564603485 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b6c040030 con 0x7f9b7c106540 2024-09-16T00:04:32.408 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.408+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9b7c1069a0 con 0x7f9b7c106540 2024-09-16T00:04:32.409 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.408+0000 7f9b797fa640 1 -- 172.21.15.193:0/3564603485 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f9b6c042310 con 0x7f9b7c106540 2024-09-16T00:04:32.409 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:04:32.409 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","modified":"2024-09-16T00:03:12.254418Z","created":"2024-09-16T00:03:12.254418Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:04:32.410 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.410+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b680392a0 msgr2=0x7f9b6803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.410+0000 7f9b81b37640 1 --2- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b680392a0 0x7f9b6803b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9b640098c0 tx=0x7f9b64006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.411+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 msgr2=0x7f9b7c197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.411+0000 7f9b81b37640 1 --2- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 0x7f9b7c197fd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0045e0 tx=0x7f9b6c004610 comp rx=0 tx=0).stop 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.411+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 shutdown_connections 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.411+0000 7f9b81b37640 1 --2- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b680392a0 0x7f9b6803b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.411+0000 7f9b81b37640 1 --2- 172.21.15.193:0/3564603485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b7c106540 0x7f9b7c197fd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.411+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 >> 172.21.15.193:0/3564603485 conn(0x7f9b7c077310 msgr2=0x7f9b7c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.411+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 shutdown_connections 2024-09-16T00:04:32.412 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:32.411+0000 7f9b81b37640 1 -- 172.21.15.193:0/3564603485 wait complete. 2024-09-16T00:04:33.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:32 smithi078 ceph-mon[31157]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:33.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:32 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/3564603485' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:33.993 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:04:33.994 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:04:34.344 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:04:34.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.963+0000 7f328b4b9640 1 -- 172.21.15.193:0/3378461819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 msgr2=0x7f3284102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:34.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.963+0000 7f328b4b9640 1 --2- 172.21.15.193:0/3378461819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 0x7f3284102c00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3280009c20 tx=0x7f328002ab20 comp rx=0 tx=0).stop 2024-09-16T00:04:34.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.963+0000 7f328b4b9640 1 -- 172.21.15.193:0/3378461819 shutdown_connections 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.963+0000 7f328b4b9640 1 --2- 172.21.15.193:0/3378461819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 0x7f3284102c00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.963+0000 7f328b4b9640 1 -- 172.21.15.193:0/3378461819 >> 172.21.15.193:0/3378461819 conn(0x7f32840fc220 msgr2=0x7f32840fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.963+0000 7f328b4b9640 1 -- 172.21.15.193:0/3378461819 shutdown_connections 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.963+0000 7f328b4b9640 1 -- 172.21.15.193:0/3378461819 wait complete. 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.964+0000 7f328b4b9640 1 Processor -- start 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.964+0000 7f328b4b9640 1 -- start start 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.964+0000 7f328b4b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 0x7f32841972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.964+0000 7f328b4b9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f328410bd70 con 0x7f3284102820 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.964+0000 7f3289255640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 0x7f32841972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.964+0000 7f3289255640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 0x7f32841972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.193:35722/0 (socket says 172.21.15.193:35722) 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.964+0000 7f3289255640 1 -- 172.21.15.193:0/42119712 learned_addr learned my addr 172.21.15.193:0/42119712 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:04:34.969 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.965+0000 7f3289255640 1 -- 172.21.15.193:0/42119712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3284197820 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.965+0000 7f3289255640 1 --2- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 0x7f32841972e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3280030040 tx=0x7f328002eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.965+0000 7f327a7fc640 1 -- 172.21.15.193:0/42119712 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3280047070 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.965+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3284197ab0 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.965+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3284197f10 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.965+0000 7f327a7fc640 1 -- 172.21.15.193:0/42119712 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32800368e0 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.965+0000 7f327a7fc640 1 -- 172.21.15.193:0/42119712 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f328002d3d0 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.966+0000 7f327a7fc640 1 -- 172.21.15.193:0/42119712 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f328004a070 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.966+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3268005290 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.966+0000 7f327a7fc640 1 --2- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3270039300 0x7f327003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.966+0000 7f327a7fc640 1 -- 172.21.15.193:0/42119712 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f328003d030 con 0x7f3284102820 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.967+0000 7f3288a54640 1 --2- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3270039300 0x7f327003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:34.970 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.967+0000 7f3288a54640 1 --2- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3270039300 0x7f327003b7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f32740098c0 tx=0x7f3274006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:34.972 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:34.972+0000 7f327a7fc640 1 -- 172.21.15.193:0/42119712 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3280032030 con 0x7f3284102820 2024-09-16T00:04:35.231 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.230+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3268005580 con 0x7f3284102820 2024-09-16T00:04:35.231 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.230+0000 7f327a7fc640 1 -- 172.21.15.193:0/42119712 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f328004b9f0 con 0x7f3284102820 2024-09-16T00:04:35.232 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:04:35.232 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","modified":"2024-09-16T00:03:12.254418Z","created":"2024-09-16T00:03:12.254418Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:04:35.232 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-09-16T00:04:35.236 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3270039300 msgr2=0x7f327003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:35.236 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 --2- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3270039300 0x7f327003b7c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f32740098c0 tx=0x7f3274006db0 comp rx=0 tx=0).stop 2024-09-16T00:04:35.236 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 msgr2=0x7f32841972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:35.236 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 --2- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 0x7f32841972e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3280030040 tx=0x7f328002eb50 comp rx=0 tx=0).stop 2024-09-16T00:04:35.236 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 shutdown_connections 2024-09-16T00:04:35.237 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 --2- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3270039300 0x7f327003b7c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:35.237 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 --2- 172.21.15.193:0/42119712 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3284102820 0x7f32841972e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:35.237 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 >> 172.21.15.193:0/42119712 conn(0x7f32840fc220 msgr2=0x7f32841055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:35.237 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 shutdown_connections 2024-09-16T00:04:35.237 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:35.232+0000 7f328b4b9640 1 -- 172.21.15.193:0/42119712 wait complete. 2024-09-16T00:04:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:35 smithi078 ceph-mon[31157]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:35 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/42119712' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:36.073 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:04:36.073 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:04:36.073 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:04:36.073 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:04:36.073 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:04:36.073 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:04:36.074 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:04:36.075 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:04:36.076 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.ttl: 2592000 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:04:36.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:04:36.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d7884642-e67d-437c-b055-7b6b1fb186cd 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445076071091, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445076071505, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726445076, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d7884642-e67d-437c-b055-7b6b1fb186cd", "db_session_id": "9AAG71F9EJVC36UZJTJQ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726445076071597, "job": 1, "event": "recovery_finished"} 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564cbbc7ce00 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: DB pointer 0x564cbbd78000 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:04:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:04:36.434 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:04:36.435 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x564cbbc4f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-16T00:04:36.436 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(???) e0 preinit fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).mds e1 new map 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout: btime 2024-09-16T00:03:16:286868+0000 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T00:04:36.437 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: Added host smithi193 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:36.438 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1707139311' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1707139311' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:36.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: Updating smithi193:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: Updating smithi193:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi078:172.21.15.78=a;smithi148:172.21.15.148=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: Saving service mon spec with placement smithi078:172.21.15.78=a;smithi148:172.21.15.148=b;smithi193:172.21.15.193=c;count:3 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: Deploying daemon mon.c on smithi193 2024-09-16T00:04:36.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:36.441 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:36.441 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/3564603485' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:36.441 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:36.441 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/42119712' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:36.441 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 ceph-mon[35484]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T00:04:36.441 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 bash[35448]: 3076c2d30b7bf21cdedcbe3e2d507ab2f4593d1bd667cfbd56e403130c08bdea 2024-09-16T00:04:36.441 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:36 smithi193 systemd[1]: Started Ceph mon.c for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:04:36.825 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:04:36.825 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:04:37.076 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:04:41.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.128+0000 7f1bf9532640 1 -- 172.21.15.193:0/896233243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 msgr2=0x7f1bdc0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:41.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.128+0000 7f1bf9532640 1 --2- 172.21.15.193:0/896233243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 0x7f1bdc0059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1bf41016d0 tx=0x7f1be402c3f0 comp rx=0 tx=0).stop 2024-09-16T00:04:41.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.128+0000 7f1bf9532640 1 -- 172.21.15.193:0/896233243 shutdown_connections 2024-09-16T00:04:41.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.128+0000 7f1bf9532640 1 --2- 172.21.15.193:0/896233243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 0x7f1bdc0059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:41.130 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.128+0000 7f1bf9532640 1 -- 172.21.15.193:0/896233243 >> 172.21.15.193:0/896233243 conn(0x7f1bf40fc250 msgr2=0x7f1bf40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:41.131 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.128+0000 7f1bf9532640 1 -- 172.21.15.193:0/896233243 shutdown_connections 2024-09-16T00:04:41.131 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.128+0000 7f1bf9532640 1 -- 172.21.15.193:0/896233243 wait complete. 2024-09-16T00:04:41.131 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.129+0000 7f1bf9532640 1 Processor -- start 2024-09-16T00:04:41.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf9532640 1 -- start start 2024-09-16T00:04:41.132 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf9532640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 0x7f1bf41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:41.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf9532640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 0x7f1bf419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:41.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf9532640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf41086a0 con 0x7f1bdc003530 2024-09-16T00:04:41.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf9532640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf4108520 con 0x7f1bf4199930 2024-09-16T00:04:41.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 0x7f1bf419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:41.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 0x7f1bf41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:41.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 0x7f1bf419dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:32892/0 (socket says 172.21.15.193:32892) 2024-09-16T00:04:41.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf27fc640 1 -- 172.21.15.193:0/473925351 learned_addr learned my addr 172.21.15.193:0/473925351 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:04:41.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf27fc640 1 -- 172.21.15.193:0/473925351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 msgr2=0x7f1bf419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf27fc640 1 -- 172.21.15.193:0/473925351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 msgr2=0x7f1bf419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf27fc640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 0x7f1bf419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf27fc640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 0x7f1bf419dcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf2ffd640 1 -- 172.21.15.193:0/473925351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 msgr2=0x7f1bf419dcc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf2ffd640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 0x7f1bf419dcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.130+0000 7f1bf2ffd640 1 -- 172.21.15.193:0/473925351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf4199f50 con 0x7f1bdc003530 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.131+0000 7f1bf2ffd640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 0x7f1bf41993f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1be4005f10 tx=0x7f1be4034bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.131+0000 7f1bdbfff640 1 -- 172.21.15.193:0/473925351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1be40051f0 con 0x7f1bdc003530 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.131+0000 7f1bdbfff640 1 -- 172.21.15.193:0/473925351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be4005390 con 0x7f1bdc003530 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.131+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf419e200 con 0x7f1bdc003530 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.131+0000 7f1bdbfff640 1 -- 172.21.15.193:0/473925351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1be40128c0 con 0x7f1bdc003530 2024-09-16T00:04:41.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.131+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf419e7c0 con 0x7f1bdc003530 2024-09-16T00:04:41.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.132+0000 7f1bdbfff640 1 -- 172.21.15.193:0/473925351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f1be4044020 con 0x7f1bdc003530 2024-09-16T00:04:41.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.132+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bf4101cd0 con 0x7f1bdc003530 2024-09-16T00:04:41.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.132+0000 7f1bdbfff640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1be0039090 0x7f1be003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:41.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.132+0000 7f1bdbfff640 1 -- 172.21.15.193:0/473925351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1be4042020 con 0x7f1bdc003530 2024-09-16T00:04:41.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.133+0000 7f1bf27fc640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1be0039090 0x7f1be003b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:41.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.133+0000 7f1bf27fc640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1be0039090 0x7f1be003b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009100 tx=0x7f1be8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:41.139 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.138+0000 7f1bdbfff640 1 -- 172.21.15.193:0/473925351 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1be404b020 con 0x7f1bdc003530 2024-09-16T00:04:41.372 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.371+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1bf419eaa0 con 0x7f1bdc003530 2024-09-16T00:04:41.372 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.371+0000 7f1bdbfff640 1 -- 172.21.15.193:0/473925351 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1be4044570 con 0x7f1bdc003530 2024-09-16T00:04:41.377 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:04:41.377 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","modified":"2024-09-16T00:04:36.100247Z","created":"2024-09-16T00:03:12.254418Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:04:41.378 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-16T00:04:41.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.373+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1be0039090 msgr2=0x7f1be003b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:41.379 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.373+0000 7f1bf9532640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1be0039090 0x7f1be003b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009100 tx=0x7f1be8008040 comp rx=0 tx=0).stop 2024-09-16T00:04:41.379 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.373+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 msgr2=0x7f1bf41993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:41.379 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.373+0000 7f1bf9532640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 0x7f1bf41993f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1be4005f10 tx=0x7f1be4034bb0 comp rx=0 tx=0).stop 2024-09-16T00:04:41.379 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.373+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 shutdown_connections 2024-09-16T00:04:41.379 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.373+0000 7f1bf9532640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1be0039090 0x7f1be003b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:41.379 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.373+0000 7f1bf9532640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bf4199930 0x7f1bf419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:41.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.373+0000 7f1bf9532640 1 --2- 172.21.15.193:0/473925351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1bdc003530 0x7f1bf41993f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:41.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.374+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 >> 172.21.15.193:0/473925351 conn(0x7f1bf40fc250 msgr2=0x7f1bf406a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:41.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.374+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 shutdown_connections 2024-09-16T00:04:41.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:41.374+0000 7f1bf9532640 1 -- 172.21.15.193:0/473925351 wait complete. 2024-09-16T00:04:41.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:04:41.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: mon.a calling monitor election 2024-09-16T00:04:41.384 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: mon.c calling monitor election 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: monmap epoch 2 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: last_changed 2024-09-16T00:04:36.100247+0000 2024-09-16T00:04:41.385 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: min_mon_release 19 (squid) 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: election_strategy: 1 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: fsmap 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: mgrmap e13: x(active, since 38s) 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: overall HEALTH_OK 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:41.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:41.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:04:41.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: mon.a calling monitor election 2024-09-16T00:04:41.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: mon.c calling monitor election 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: monmap epoch 2 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: last_changed 2024-09-16T00:04:36.100247+0000 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:04:41.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: min_mon_release 19 (squid) 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: election_strategy: 1 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: fsmap 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: mgrmap e13: x(active, since 38s) 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: overall HEALTH_OK 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:41.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:42.421 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:42 smithi078 ceph-mon[31157]: Deploying daemon mon.b on smithi148 2024-09-16T00:04:42.421 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:42 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/473925351' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:42.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:42 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:42.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:42 smithi193 ceph-mon[35484]: Deploying daemon mon.b on smithi148 2024-09-16T00:04:42.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:42 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/473925351' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:42.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:42 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:42.958 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:04:42.958 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:04:43.214 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:04:43.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:43 smithi193 ceph-mon[35484]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:43.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:43 smithi078 ceph-mon[31157]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:43.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:04:43 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:04:43.099+0000 7fe63375b640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T00:04:43.597 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.594+0000 7f706420a640 1 -- 172.21.15.193:0/3345514545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c106650 msgr2=0x7f705c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:43.597 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.594+0000 7f706420a640 1 --2- 172.21.15.193:0/3345514545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c106650 0x7f705c106a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7058009880 tx=0x7f705802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:43.597 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.594+0000 7f706420a640 1 -- 172.21.15.193:0/3345514545 shutdown_connections 2024-09-16T00:04:43.597 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.594+0000 7f706420a640 1 --2- 172.21.15.193:0/3345514545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c106650 0x7f705c106a30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:43.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.594+0000 7f706420a640 1 -- 172.21.15.193:0/3345514545 >> 172.21.15.193:0/3345514545 conn(0x7f705c0fc280 msgr2=0x7f705c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:43.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.594+0000 7f706420a640 1 -- 172.21.15.193:0/3345514545 shutdown_connections 2024-09-16T00:04:43.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.594+0000 7f706420a640 1 -- 172.21.15.193:0/3345514545 wait complete. 2024-09-16T00:04:43.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.595+0000 7f706420a640 1 Processor -- start 2024-09-16T00:04:43.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.595+0000 7f706420a640 1 -- start start 2024-09-16T00:04:43.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.595+0000 7f706420a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c106650 0x7f705c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:43.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.595+0000 7f706420a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c196810 0x7f705c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.595+0000 7f706420a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f705c10ac80 con 0x7f705c196810 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.595+0000 7f706420a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f705c10ab00 con 0x7f705c106650 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.595+0000 7f70617a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c196810 0x7f705c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f70617a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c196810 0x7f705c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.193:35816/0 (socket says 172.21.15.193:35816) 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f70617a5640 1 -- 172.21.15.193:0/1418356475 learned_addr learned my addr 172.21.15.193:0/1418356475 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f7061fa6640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c106650 0x7f705c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f70617a5640 1 -- 172.21.15.193:0/1418356475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c106650 msgr2=0x7f705c1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f70617a5640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c106650 0x7f705c1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f70617a5640 1 -- 172.21.15.193:0/1418356475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c196e30 con 0x7f705c196810 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f70617a5640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c196810 0x7f705c19aba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f704c00b4a0 tx=0x7f704c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f7052ffd640 1 -- 172.21.15.193:0/1418356475 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f704c010610 con 0x7f705c196810 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c19b0e0 con 0x7f705c196810 2024-09-16T00:04:43.599 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f7052ffd640 1 -- 172.21.15.193:0/1418356475 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f704c010c90 con 0x7f705c196810 2024-09-16T00:04:43.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f7052ffd640 1 -- 172.21.15.193:0/1418356475 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f704c017e60 con 0x7f705c196810 2024-09-16T00:04:43.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.596+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c19b6a0 con 0x7f705c196810 2024-09-16T00:04:43.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.597+0000 7f7052ffd640 1 -- 172.21.15.193:0/1418356475 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f704c017070 con 0x7f705c196810 2024-09-16T00:04:43.601 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.597+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c101cc0 con 0x7f705c196810 2024-09-16T00:04:43.601 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.597+0000 7f7052ffd640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7048039040 0x7f704803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:43.601 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.597+0000 7f7052ffd640 1 -- 172.21.15.193:0/1418356475 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f704c00d040 con 0x7f705c196810 2024-09-16T00:04:43.601 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.597+0000 7f7061fa6640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7048039040 0x7f704803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:43.601 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.598+0000 7f7061fa6640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7048039040 0x7f704803b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f70580314e0 tx=0x7f7058005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:43.603 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.602+0000 7f7052ffd640 1 -- 172.21.15.193:0/1418356475 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f704c01e020 con 0x7f705c196810 2024-09-16T00:04:43.831 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.830+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f705c19b960 con 0x7f705c196810 2024-09-16T00:04:43.832 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.831+0000 7f7052ffd640 1 -- 172.21.15.193:0/1418356475 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f704c00eb20 con 0x7f705c196810 2024-09-16T00:04:43.832 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:04:43.832 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","modified":"2024-09-16T00:04:36.100247Z","created":"2024-09-16T00:03:12.254418Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:04:43.832 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7048039040 msgr2=0x7f704803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7048039040 0x7f704803b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f70580314e0 tx=0x7f7058005a20 comp rx=0 tx=0).stop 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c196810 msgr2=0x7f705c19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c196810 0x7f705c19aba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f704c00b4a0 tx=0x7f704c00b960 comp rx=0 tx=0).stop 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 shutdown_connections 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7048039040 0x7f704803b500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f705c196810 0x7f705c19aba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 --2- 172.21.15.193:0/1418356475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c106650 0x7f705c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:43.835 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 >> 172.21.15.193:0/1418356475 conn(0x7f705c0fc280 msgr2=0x7f705c0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:43.836 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 shutdown_connections 2024-09-16T00:04:43.836 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:43.833+0000 7f706420a640 1 -- 172.21.15.193:0/1418356475 wait complete. 2024-09-16T00:04:44.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:44 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/1418356475' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:44.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:44 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/1418356475' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:45.102 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:45 smithi148 ceph-mon[35599]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T00:04:45.406 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:04:45.406 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:04:45.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:45 smithi148 bash[35580]: 99365af9c89318611e4dfdbb1261582ee1507eba4b3745e4a3730650f8302ffa 2024-09-16T00:04:45.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:45 smithi148 systemd[1]: Started Ceph mon.b for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:04:45.657 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:04:50.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:04:50.476 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.476 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: mon.c calling monitor election 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: mon.a calling monitor election 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: monmap epoch 3 2024-09-16T00:04:50.477 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: last_changed 2024-09-16T00:04:45.090673+0000 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: min_mon_release 19 (squid) 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: election_strategy: 1 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: fsmap 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: mgrmap e13: x(active, since 47s) 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: overall HEALTH_OK 2024-09-16T00:04:50.478 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:50.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:50.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:50.479 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:50.481 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.479+0000 7fc759c50640 1 -- 172.21.15.193:0/597118693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c003530 msgr2=0x7fc73c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:50.481 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.479+0000 7fc759c50640 1 --2- 172.21.15.193:0/597118693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c003530 0x7fc73c0059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc754102af0 tx=0x7fc74402c490 comp rx=0 tx=0).stop 2024-09-16T00:04:50.481 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.479+0000 7fc759c50640 1 -- 172.21.15.193:0/597118693 shutdown_connections 2024-09-16T00:04:50.481 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.479+0000 7fc759c50640 1 --2- 172.21.15.193:0/597118693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc73c003530 0x7fc73c0059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:50.481 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.479+0000 7fc759c50640 1 -- 172.21.15.193:0/597118693 >> 172.21.15.193:0/597118693 conn(0x7fc7540fd320 msgr2=0x7fc7540ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:50.481 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.479+0000 7fc759c50640 1 -- 172.21.15.193:0/597118693 shutdown_connections 2024-09-16T00:04:50.481 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.479+0000 7fc759c50640 1 -- 172.21.15.193:0/597118693 wait complete. 2024-09-16T00:04:50.482 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.480+0000 7fc759c50640 1 Processor -- start 2024-09-16T00:04:50.482 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.480+0000 7fc759c50640 1 -- start start 2024-09-16T00:04:50.482 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc759c50640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 0x7fc754197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc759c50640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 0x7fc75419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc759c50640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75419c180 0x7fc75419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc759c50640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc754109720 con 0x7fc75419c180 2024-09-16T00:04:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc759c50640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541095a0 con 0x7fc73c003530 2024-09-16T00:04:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc759c50640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541098a0 con 0x7fc7541978b0 2024-09-16T00:04:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc752ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 0x7fc75419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc752ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 0x7fc75419bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:42788/0 (socket says 172.21.15.193:42788) 2024-09-16T00:04:50.483 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc752ffd640 1 -- 172.21.15.193:0/2413137853 learned_addr learned my addr 172.21.15.193:0/2413137853 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:04:50.485 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc753fff640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75419c180 0x7fc75419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:50.485 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc752ffd640 1 -- 172.21.15.193:0/2413137853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 msgr2=0x7fc75419bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T00:04:50.485 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc752ffd640 1 -- 172.21.15.193:0/2413137853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 msgr2=0x7fc75419bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:04:50.485 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc752ffd640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 0x7fc75419bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:04:50.485 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc7537fe640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 0x7fc754197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc752ffd640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 0x7fc75419bc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc7537fe640 1 -- 172.21.15.193:0/2413137853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 msgr2=0x7fc754197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc7537fe640 1 -- 172.21.15.193:0/2413137853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 msgr2=0x7fc754197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc7537fe640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 0x7fc754197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc7537fe640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 0x7fc754197370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc753fff640 1 -- 172.21.15.193:0/2413137853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 msgr2=0x7fc75419bc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc753fff640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 0x7fc75419bc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc753fff640 1 -- 172.21.15.193:0/2413137853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 msgr2=0x7fc754197370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc753fff640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 0x7fc754197370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.481+0000 7fc753fff640 1 -- 172.21.15.193:0/2413137853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7541a3ef0 con 0x7fc75419c180 2024-09-16T00:04:50.486 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.482+0000 7fc753fff640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75419c180 0x7fc75419c630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00bd50 tx=0x7fc74c00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.482+0000 7fc750ff9640 1 -- 172.21.15.193:0/2413137853 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c019070 con 0x7fc75419c180 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.482+0000 7fc750ff9640 1 -- 172.21.15.193:0/2413137853 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74c014920 con 0x7fc75419c180 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.482+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc754074480 con 0x7fc75419c180 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.482+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7540749c0 con 0x7fc75419c180 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.482+0000 7fc750ff9640 1 -- 172.21.15.193:0/2413137853 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c012990 con 0x7fc75419c180 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.483+0000 7fc750ff9640 1 -- 172.21.15.193:0/2413137853 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc74c01c070 con 0x7fc75419c180 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.483+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75410a8d0 con 0x7fc75419c180 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.484+0000 7fc750ff9640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc740039730 0x7fc74003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.484+0000 7fc750ff9640 1 -- 172.21.15.193:0/2413137853 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc74c04c0c0 con 0x7fc75419c180 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.484+0000 7fc7537fe640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc740039730 0x7fc74003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:50.487 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.484+0000 7fc7537fe640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc740039730 0x7fc74003bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc74402ca00 tx=0x7fc74402c990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:50.489 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.488+0000 7fc750ff9640 1 -- 172.21.15.193:0/2413137853 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc74c0139f0 con 0x7fc75419c180 2024-09-16T00:04:50.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:04:50.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: mon.c calling monitor election 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: mon.a calling monitor election 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: monmap epoch 3 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:50.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: last_changed 2024-09-16T00:04:45.090673+0000 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: min_mon_release 19 (squid) 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: election_strategy: 1 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: fsmap 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: mgrmap e13: x(active, since 47s) 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: overall HEALTH_OK 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:50.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:50.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.744+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc754074d40 con 0x7fc75419c180 2024-09-16T00:04:50.746 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.745+0000 7fc750ff9640 1 -- 172.21.15.193:0/2413137853 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fc74c013c50 con 0x7fc75419c180 2024-09-16T00:04:50.746 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:04:50.746 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","modified":"2024-09-16T00:04:45.090673Z","created":"2024-09-16T00:03:12.254418Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:04:50.746 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc740039730 msgr2=0x7fc74003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc740039730 0x7fc74003bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc74402ca00 tx=0x7fc74402c990 comp rx=0 tx=0).stop 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75419c180 msgr2=0x7fc75419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75419c180 0x7fc75419c630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00bd50 tx=0x7fc74c00be40 comp rx=0 tx=0).stop 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 shutdown_connections 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc740039730 0x7fc74003bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75419c180 0x7fc75419c630 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7541978b0 0x7fc75419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:50.748 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 --2- 172.21.15.193:0/2413137853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc73c003530 0x7fc754197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:50.749 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 >> 172.21.15.193:0/2413137853 conn(0x7fc7540fd320 msgr2=0x7fc7540ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:50.749 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 shutdown_connections 2024-09-16T00:04:50.749 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:50.747+0000 7fc759c50640 1 -- 172.21.15.193:0/2413137853 wait complete. 2024-09-16T00:04:51.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:51 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:51.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:51 smithi193 ceph-mon[35484]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:51.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:51 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/2413137853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:51.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:51 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:51.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:51 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:51.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:51 smithi078 ceph-mon[31157]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:51.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:51 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/2413137853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:51.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:51 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:51.929 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T00:04:51.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T00:04:52.182 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:52.430 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:04:52.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:52.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: mon.c calling monitor election 2024-09-16T00:04:52.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: mon.a calling monitor election 2024-09-16T00:04:52.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:04:52.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:52.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:52.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:52.431 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: monmap epoch 3 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: last_changed 2024-09-16T00:04:45.090673+0000 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: min_mon_release 19 (squid) 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: election_strategy: 1 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-16T00:04:52.432 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-09-16T00:04:52.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: fsmap 2024-09-16T00:04:52.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:52.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: mgrmap e13: x(active, since 47s) 2024-09-16T00:04:52.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: overall HEALTH_OK 2024-09-16T00:04:52.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:52.433 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:52.436 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:52.437 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:52.437 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:52.437 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:52.437 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/2413137853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:04:52.437 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:52.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.646+0000 7fcd3c614640 1 -- 172.21.15.78:0/3268216194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34106560 msgr2=0x7fcd34106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.646+0000 7fcd3c614640 1 --2- 172.21.15.78:0/3268216194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34106560 0x7fcd34106940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009880 tx=0x7fcd3002a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.646+0000 7fcd3c614640 1 -- 172.21.15.78:0/3268216194 shutdown_connections 2024-09-16T00:04:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.646+0000 7fcd3c614640 1 --2- 172.21.15.78:0/3268216194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34106560 0x7fcd34106940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.646+0000 7fcd3c614640 1 -- 172.21.15.78:0/3268216194 >> 172.21.15.78:0/3268216194 conn(0x7fcd340fc1b0 msgr2=0x7fcd340fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.646+0000 7fcd3c614640 1 -- 172.21.15.78:0/3268216194 shutdown_connections 2024-09-16T00:04:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.646+0000 7fcd3c614640 1 -- 172.21.15.78:0/3268216194 wait complete. 2024-09-16T00:04:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3c614640 1 Processor -- start 2024-09-16T00:04:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3c614640 1 -- start start 2024-09-16T00:04:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3c614640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd34106560 0x7fcd34197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3c614640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34197890 0x7fcd3419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3c614640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3419c160 0x7fcd3419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3c614640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3410ab40 con 0x7fcd34197890 2024-09-16T00:04:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3c614640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3410a9c0 con 0x7fcd3419c160 2024-09-16T00:04:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3c614640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3410acc0 con 0x7fcd34106560 2024-09-16T00:04:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd39baf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34197890 0x7fcd3419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd39baf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34197890 0x7fcd3419bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50772/0 (socket says 172.21.15.78:50772) 2024-09-16T00:04:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd39baf640 1 -- 172.21.15.78:0/1743810744 learned_addr learned my addr 172.21.15.78:0/1743810744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:04:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3a3b0640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd34106560 0x7fcd34197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd39baf640 1 -- 172.21.15.78:0/1743810744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd34106560 msgr2=0x7fcd34197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3abb1640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3419c160 0x7fcd3419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd39baf640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd34106560 0x7fcd34197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd39baf640 1 -- 172.21.15.78:0/1743810744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3419c160 msgr2=0x7fcd3419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd39baf640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3419c160 0x7fcd3419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd39baf640 1 -- 172.21.15.78:0/1743810744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd341a3ed0 con 0x7fcd34197890 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3a3b0640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd34106560 0x7fcd34197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.647+0000 7fcd3abb1640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3419c160 0x7fcd3419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.648+0000 7fcd39baf640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34197890 0x7fcd3419bc20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400b520 tx=0x7fcd2400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.648+0000 7fcd2b7fe640 1 -- 172.21.15.78:0/1743810744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24004230 con 0x7fcd34197890 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.648+0000 7fcd2b7fe640 1 -- 172.21.15.78:0/1743810744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd240043d0 con 0x7fcd34197890 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.648+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd341a4100 con 0x7fcd34197890 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.648+0000 7fcd2b7fe640 1 -- 172.21.15.78:0/1743810744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24003d60 con 0x7fcd34197890 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.648+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd341a4690 con 0x7fcd34197890 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.649+0000 7fcd2b7fe640 1 -- 172.21.15.78:0/1743810744 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fcd2400e070 con 0x7fcd34197890 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.649+0000 7fcd2b7fe640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd20039730 0x7fcd2003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.649+0000 7fcd2b7fe640 1 -- 172.21.15.78:0/1743810744 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd2404be00 con 0x7fcd34197890 2024-09-16T00:04:52.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.649+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd341069c0 con 0x7fcd34197890 2024-09-16T00:04:52.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.649+0000 7fcd3a3b0640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd20039730 0x7fcd2003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:52.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.649+0000 7fcd3a3b0640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd20039730 0x7fcd2003bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd30031550 tx=0x7fcd30005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:52.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.654+0000 7fcd2b7fe640 1 -- 172.21.15.78:0/1743810744 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd24014020 con 0x7fcd34197890 2024-09-16T00:04:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.822+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fcd341a4a10 con 0x7fcd34197890 2024-09-16T00:04:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.822+0000 7fcd2b7fe640 1 -- 172.21.15.78:0/1743810744 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fcd24010d70 con 0x7fcd34197890 2024-09-16T00:04:52.823 INFO:teuthology.orchestra.run.smithi078.stdout:# minimal ceph.conf for eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:52.823 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-09-16T00:04:52.823 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:52.823 INFO:teuthology.orchestra.run.smithi078.stdout: mon_host = [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-16T00:04:52.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd20039730 msgr2=0x7fcd2003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:52.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd20039730 0x7fcd2003bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd30031550 tx=0x7fcd30005a40 comp rx=0 tx=0).stop 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34197890 msgr2=0x7fcd3419bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34197890 0x7fcd3419bc20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400b520 tx=0x7fcd2400b9e0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 shutdown_connections 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd20039730 0x7fcd2003bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd3419c160 0x7fcd3419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd34197890 0x7fcd3419bc20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 --2- 172.21.15.78:0/1743810744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd34106560 0x7fcd34197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 >> 172.21.15.78:0/1743810744 conn(0x7fcd340fc1b0 msgr2=0x7fcd340fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.824+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 shutdown_connections 2024-09-16T00:04:52.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:04:52.825+0000 7fcd3c614640 1 -- 172.21.15.78:0/1743810744 wait complete. 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: mon.b calling monitor election 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: Updating smithi148:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: Updating smithi193:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: Updating smithi078:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: mon.b calling monitor election 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: mon.c calling monitor election 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: mon.a calling monitor election 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: monmap epoch 3 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:53.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: last_changed 2024-09-16T00:04:45.090673+0000 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: min_mon_release 19 (squid) 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: election_strategy: 1 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: fsmap 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: mgrmap e13: x(active, since 49s) 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: overall HEALTH_OK 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.432 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: Reconfiguring daemon mon.a on smithi078 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1743810744' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:53.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:53.457 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: mon.b calling monitor election 2024-09-16T00:04:53.457 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: Updating smithi148:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.457 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: Updating smithi193:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.457 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: Updating smithi078:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.457 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: mon.b calling monitor election 2024-09-16T00:04:53.457 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: mon.c calling monitor election 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: mon.a calling monitor election 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: monmap epoch 3 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: last_changed 2024-09-16T00:04:45.090673+0000 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: min_mon_release 19 (squid) 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: election_strategy: 1 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: fsmap 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:53.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: mgrmap e13: x(active, since 49s) 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: overall HEALTH_OK 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: Reconfiguring daemon mon.a on smithi078 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1743810744' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:53.459 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: mon.b calling monitor election 2024-09-16T00:04:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: Updating smithi148:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: Updating smithi193:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: Updating smithi078:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: mon.b calling monitor election 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: mon.c calling monitor election 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: mon.a calling monitor election 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: monmap epoch 3 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: last_changed 2024-09-16T00:04:45.090673+0000 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: created 2024-09-16T00:03:12.254418+0000 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: min_mon_release 19 (squid) 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: election_strategy: 1 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: 0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-16T00:04:53.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: 2: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.b 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: fsmap 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: mgrmap e13: x(active, since 49s) 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: overall HEALTH_OK 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:53.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: Reconfiguring daemon mon.a on smithi078 2024-09-16T00:04:53.523 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:53.523 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:53.523 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1743810744' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:53.523 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:04:54.634 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T00:04:54.634 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:04:54.635 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:04:54.663 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:04:54.664 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:54.730 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:04:54.730 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:04:54.758 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:04:54.758 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:54.824 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:04:54.824 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:04:54.857 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:04:54.857 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:04:54.926 INFO:tasks.cephadm:Adding mgr.x on smithi078 2024-09-16T00:04:54.926 INFO:tasks.cephadm:Adding mgr.y on smithi148 2024-09-16T00:04:54.926 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-09-16T00:04:54.926 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi078=x;smithi148=y;smithi193=z' 2024-09-16T00:04:55.213 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:04:55.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:55.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:55.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:55 smithi148 ceph-mon[35599]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:04:55.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:55.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:55.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:55.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:55 smithi148 ceph-mon[35599]: Reconfiguring daemon mon.b on smithi148 2024-09-16T00:04:55.439 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:55 smithi148 ceph-mon[35599]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:55.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:55.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:55.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:55 smithi193 ceph-mon[35484]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:04:55.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:55.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:55.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:55.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:55 smithi193 ceph-mon[35484]: Reconfiguring daemon mon.b on smithi148 2024-09-16T00:04:55.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:55 smithi193 ceph-mon[35484]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:55 smithi078 ceph-mon[31157]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:04:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:55 smithi078 ceph-mon[31157]: Reconfiguring daemon mon.b on smithi148 2024-09-16T00:04:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:55 smithi078 ceph-mon[31157]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.659+0000 7fcd1cd65640 1 -- 172.21.15.193:0/1645982101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd18102170 msgr2=0x7fcd18104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.659+0000 7fcd1cd65640 1 --2- 172.21.15.193:0/1645982101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd18102170 0x7fcd18104560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009880 tx=0x7fcd0802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.659+0000 7fcd1cd65640 1 -- 172.21.15.193:0/1645982101 shutdown_connections 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.659+0000 7fcd1cd65640 1 --2- 172.21.15.193:0/1645982101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd18102170 0x7fcd18104560 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.659+0000 7fcd1cd65640 1 -- 172.21.15.193:0/1645982101 >> 172.21.15.193:0/1645982101 conn(0x7fcd180fbdb0 msgr2=0x7fcd180fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.659+0000 7fcd1cd65640 1 -- 172.21.15.193:0/1645982101 shutdown_connections 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.659+0000 7fcd1cd65640 1 -- 172.21.15.193:0/1645982101 wait complete. 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1cd65640 1 Processor -- start 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1cd65640 1 -- start start 2024-09-16T00:04:55.692 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1cd65640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd18102170 0x7fcd18196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1cd65640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd18197460 0x7fcd1819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1cd65640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd181978e0 0x7fcd1819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1cd65640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18107730 con 0x7fcd181978e0 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1cd65640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd181075b0 con 0x7fcd18197460 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1cd65640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd181078b0 con 0x7fcd18102170 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd18102170 0x7fcd18196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd18102170 0x7fcd18196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:33280/0 (socket says 172.21.15.193:33280) 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd1659c640 1 -- 172.21.15.193:0/3192334858 learned_addr learned my addr 172.21.15.193:0/3192334858 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd16d9d640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd181978e0 0x7fcd1819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:55.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.660+0000 7fcd15d9b640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd18197460 0x7fcd1819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd1659c640 1 -- 172.21.15.193:0/3192334858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd18197460 msgr2=0x7fcd1819c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd1659c640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd18197460 0x7fcd1819c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd1659c640 1 -- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd181978e0 msgr2=0x7fcd1819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd1659c640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd181978e0 0x7fcd1819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd1659c640 1 -- 172.21.15.193:0/3192334858 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1819f320 con 0x7fcd18102170 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd1659c640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd18102170 0x7fcd18196f20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcd08000c00 tx=0x7fcd08004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd077fe640 1 -- 172.21.15.193:0/3192334858 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0803f030 con 0x7fcd18102170 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1819f5b0 con 0x7fcd18102170 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd077fe640 1 -- 172.21.15.193:0/3192334858 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd08003ce0 con 0x7fcd18102170 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd077fe640 1 -- 172.21.15.193:0/3192334858 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08035070 con 0x7fcd18102170 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.661+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1819fbc0 con 0x7fcd18102170 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.662+0000 7fcd077fe640 1 -- 172.21.15.193:0/3192334858 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fcd08042030 con 0x7fcd18102170 2024-09-16T00:04:55.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.662+0000 7fcd077fe640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd00039730 0x7fcd0003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:04:55.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.662+0000 7fcd077fe640 1 -- 172.21.15.193:0/3192334858 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd0806ddb0 con 0x7fcd18102170 2024-09-16T00:04:55.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.662+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd181904f0 con 0x7fcd18102170 2024-09-16T00:04:55.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.662+0000 7fcd15d9b640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd00039730 0x7fcd0003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:04:55.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.663+0000 7fcd15d9b640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd00039730 0x7fcd0003bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd18198440 tx=0x7fcd0c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:04:55.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.667+0000 7fcd077fe640 1 -- 172.21.15.193:0/3192334858 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd0804c350 con 0x7fcd18102170 2024-09-16T00:04:55.861 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.852+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi148=y;smithi193=z", "target": ["mon-mgr", ""]}) -- 0x7fcd180626c0 con 0x7fcd00039730 2024-09-16T00:04:55.862 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.858+0000 7fcd077fe640 1 -- 172.21.15.193:0/3192334858 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcd180626c0 con 0x7fcd00039730 2024-09-16T00:04:55.863 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-09-16T00:04:55.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd00039730 msgr2=0x7fcd0003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:55.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd00039730 0x7fcd0003bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd18198440 tx=0x7fcd0c009190 comp rx=0 tx=0).stop 2024-09-16T00:04:55.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd18102170 msgr2=0x7fcd18196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:04:55.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd18102170 0x7fcd18196f20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcd08000c00 tx=0x7fcd08004430 comp rx=0 tx=0).stop 2024-09-16T00:04:55.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 shutdown_connections 2024-09-16T00:04:55.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd00039730 0x7fcd0003bbf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:55.865 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd181978e0 0x7fcd1819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:55.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd18197460 0x7fcd1819c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:55.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 --2- 172.21.15.193:0/3192334858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd18102170 0x7fcd18196f20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:04:55.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.860+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 >> 172.21.15.193:0/3192334858 conn(0x7fcd180fbdb0 msgr2=0x7fcd180fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:04:55.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.861+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 shutdown_connections 2024-09-16T00:04:55.866 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:04:55.861+0000 7fcd1cd65640 1 -- 172.21.15.193:0/3192334858 wait complete. 2024-09-16T00:04:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:56 smithi078 ceph-mon[31157]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:04:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:56 smithi078 ceph-mon[31157]: Reconfiguring daemon mon.c on smithi193 2024-09-16T00:04:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:56 smithi193 ceph-mon[35484]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:04:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:56 smithi193 ceph-mon[35484]: Reconfiguring daemon mon.c on smithi193 2024-09-16T00:04:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:56.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:56.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:56.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:56 smithi148 ceph-mon[35599]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:04:56.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:04:56.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:04:56.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:56.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:56 smithi148 ceph-mon[35599]: Reconfiguring daemon mon.c on smithi193 2024-09-16T00:04:56.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:57.087 DEBUG:teuthology.orchestra.run.smithi148:mgr.y> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T00:04:57.090 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.z.service 2024-09-16T00:04:57.092 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T00:04:57.092 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:04:57.092 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:04:57.111 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:04:57.111 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-09-16T00:04:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:04:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:57.169 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-16T00:04:57.169 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:57.169 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:57.170 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-16 00:04:13.253202801 +0000 2024-09-16T00:04:57.170 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-15 23:59:53.797840502 +0000 2024-09-16T00:04:57.170 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-15 23:59:53.797840502 +0000 2024-09-16T00:04:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-15 23:59:53.797840502 +0000 2024-09-16T00:04:57.170 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:04:57.235 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-16T00:04:57.235 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-16T00:04:57.236 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000210907 s, 2.4 MB/s 2024-09-16T00:04:57.237 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:04:57.295 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-09-16T00:04:57.354 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:04:57.355 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:57.355 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T00:04:57.355 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:57.355 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:57.355 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-16 00:04:13.286202302 +0000 2024-09-16T00:04:57.355 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-15 23:59:54.130835620 +0000 2024-09-16T00:04:57.355 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-15 23:59:54.130835620 +0000 2024-09-16T00:04:57.355 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-15 23:59:54.130835620 +0000 2024-09-16T00:04:57.355 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:04:57.419 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-16T00:04:57.419 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-16T00:04:57.419 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000203495 s, 2.5 MB/s 2024-09-16T00:04:57.421 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:04:57.477 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-16 00:04:13.319201803 +0000 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-15 23:59:54.461830765 +0000 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-15 23:59:54.461830765 +0000 2024-09-16T00:04:57.534 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-15 23:59:54.461830765 +0000 2024-09-16T00:04:57.534 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:04:57.597 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-16T00:04:57.598 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-16T00:04:57.598 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000246679 s, 2.1 MB/s 2024-09-16T00:04:57.599 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:04:57.654 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-09-16T00:04:57.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:57 smithi193 ceph-mon[35484]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi148=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:57.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:57 smithi193 ceph-mon[35484]: Saving service mgr spec with placement smithi078=x;smithi148=y;smithi193=z;count:3 2024-09-16T00:04:57.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:57 smithi193 ceph-mon[35484]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:57.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:57.709 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:04:57.710 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:57.710 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T00:04:57.710 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:57.710 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:57.710 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-16 00:04:13.351201319 +0000 2024-09-16T00:04:57.710 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-15 23:59:54.782826059 +0000 2024-09-16T00:04:57.710 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-15 23:59:54.782826059 +0000 2024-09-16T00:04:57.710 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-15 23:59:54.782826059 +0000 2024-09-16T00:04:57.710 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:04:57.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:57 smithi078 ceph-mon[31157]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi148=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:57.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:57 smithi078 ceph-mon[31157]: Saving service mgr spec with placement smithi078=x;smithi148=y;smithi193=z;count:3 2024-09-16T00:04:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:57 smithi078 ceph-mon[31157]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-16T00:04:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-16T00:04:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000233195 s, 2.2 MB/s 2024-09-16T00:04:57.774 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:04:57.830 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-16T00:04:57.830 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:04:57.850 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:04:57.850 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-09-16T00:04:57.907 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:04:57.908 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:57.908 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:04:57.908 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:57.908 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:57.908 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-16 00:04:44.546553214 +0000 2024-09-16T00:04:57.908 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-16 00:00:07.094468997 +0000 2024-09-16T00:04:57.908 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-16 00:00:07.094468997 +0000 2024-09-16T00:04:57.908 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-16 00:00:07.094468997 +0000 2024-09-16T00:04:57.908 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:04:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:57 smithi148 ceph-mon[35599]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi078=x;smithi148=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:04:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:57 smithi148 ceph-mon[35599]: Saving service mgr spec with placement smithi078=x;smithi148=y;smithi193=z;count:3 2024-09-16T00:04:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:57 smithi148 ceph-mon[35599]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:57.976 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-16T00:04:57.976 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-16T00:04:57.976 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000333316 s, 1.5 MB/s 2024-09-16T00:04:57.977 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:04:58.036 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-09-16T00:04:58.092 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:04:58.092 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:58.092 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T00:04:58.092 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:58.092 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:58.093 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-16 00:04:44.546553214 +0000 2024-09-16T00:04:58.093 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-16 00:00:07.450460292 +0000 2024-09-16T00:04:58.093 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-16 00:00:07.450460292 +0000 2024-09-16T00:04:58.093 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-16 00:00:07.450460292 +0000 2024-09-16T00:04:58.093 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:04:58.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-16T00:04:58.158 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-16T00:04:58.158 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000282204 s, 1.8 MB/s 2024-09-16T00:04:58.160 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:04:58.217 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-09-16T00:04:58.272 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:04:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-16T00:04:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-16 00:04:44.546553214 +0000 2024-09-16T00:04:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-16 00:00:07.779452247 +0000 2024-09-16T00:04:58.273 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-16 00:00:07.779452247 +0000 2024-09-16T00:04:58.273 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-16 00:00:07.779452247 +0000 2024-09-16T00:04:58.273 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:04:58.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-16T00:04:58.337 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-16T00:04:58.337 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.00018794 s, 2.7 MB/s 2024-09-16T00:04:58.338 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:04:58.395 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-09-16T00:04:58.451 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:04:58.451 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:58.451 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-16T00:04:58.451 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:58.451 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:58.451 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-16 00:04:44.547553191 +0000 2024-09-16T00:04:58.451 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-16 00:00:08.110444153 +0000 2024-09-16T00:04:58.452 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-16 00:00:08.110444153 +0000 2024-09-16T00:04:58.452 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-16 00:00:08.110444153 +0000 2024-09-16T00:04:58.452 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:04:58.515 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-16T00:04:58.515 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-16T00:04:58.515 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000196953 s, 2.6 MB/s 2024-09-16T00:04:58.516 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:04:58.573 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T00:04:58.573 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:04:58.589 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:04:58.589 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-09-16T00:04:58.599 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:58.599 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:58.599 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:58.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:58.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:58.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:04:58.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:04:58.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:04:58.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:58.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:58 smithi193 ceph-mon[35484]: Deploying daemon mgr.z on smithi193 2024-09-16T00:04:58.614 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:04:58.615 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:58.615 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:04:58.615 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:58.615 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:58.615 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-16 00:04:34.678450286 +0000 2024-09-16T00:04:58.615 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-16 00:00:07.309633088 +0000 2024-09-16T00:04:58.615 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-16 00:00:07.309633088 +0000 2024-09-16T00:04:58.615 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-16 00:00:07.309633088 +0000 2024-09-16T00:04:58.615 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:04:58.678 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-16T00:04:58.678 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-16T00:04:58.678 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000263111 s, 1.9 MB/s 2024-09-16T00:04:58.679 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:04:58.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:58.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:58.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:58.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:58.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:58.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:04:58.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:04:58.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:04:58.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:58.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:58 smithi148 ceph-mon[35599]: Deploying daemon mgr.z on smithi193 2024-09-16T00:04:58.735 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:04:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:58 smithi078 ceph-mon[31157]: Deploying daemon mgr.z on smithi193 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-16 00:04:34.678450286 +0000 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-16 00:00:07.637627995 +0000 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-16 00:00:07.637627995 +0000 2024-09-16T00:04:58.791 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-16 00:00:07.637627995 +0000 2024-09-16T00:04:58.791 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:04:58.853 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-16T00:04:58.854 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-16T00:04:58.854 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000202737 s, 2.5 MB/s 2024-09-16T00:04:58.855 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:04:58.911 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-09-16T00:04:58.968 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:04:58.968 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:58.968 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-16T00:04:58.968 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:58.968 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:58.969 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-16 00:04:34.678450286 +0000 2024-09-16T00:04:58.969 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-16 00:00:08.001622342 +0000 2024-09-16T00:04:58.969 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-16 00:00:08.001622342 +0000 2024-09-16T00:04:58.969 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-16 00:00:08.001622342 +0000 2024-09-16T00:04:58.969 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:04:59.033 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-16T00:04:59.033 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-16T00:04:59.033 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000204702 s, 2.5 MB/s 2024-09-16T00:04:59.035 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:04:59.092 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-09-16T00:04:59.147 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:04:59.147 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:04:59.147 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T00:04:59.147 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:04:59.147 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:04:59.148 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-16 00:04:59.136069416 +0000 2024-09-16T00:04:59.148 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-16 00:00:08.325617312 +0000 2024-09-16T00:04:59.148 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-16 00:00:08.325617312 +0000 2024-09-16T00:04:59.148 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-16 00:00:08.325617312 +0000 2024-09-16T00:04:59.148 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:04:59.212 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-16T00:04:59.212 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-16T00:04:59.212 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000198335 s, 2.6 MB/s 2024-09-16T00:04:59.213 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:04:59.270 INFO:tasks.cephadm:Deploying osd.0 on smithi078 with /dev/vg_nvme/lv_4... 2024-09-16T00:04:59.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:04:59.505 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 systemd[1]: Starting Ceph mgr.z for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:04:59.530 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:04:59.651 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:04:59 smithi078 ceph-mon[31157]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:59.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:04:59 smithi193 ceph-mon[35484]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:04:59.813 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 podman[37286]: 2024-09-16 00:04:59.496130561 +0000 UTC m=+0.015694941 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:04:59.813 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 podman[37286]: 2024-09-16 00:04:59.610723653 +0000 UTC m=+0.130288027 container create e3d4387421df7f64fec679205b305d35504a7c441915543cb058199afd3cbb53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T00:04:59.814 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 podman[37286]: 2024-09-16 00:04:59.813067671 +0000 UTC m=+0.332632045 container init e3d4387421df7f64fec679205b305d35504a7c441915543cb058199afd3cbb53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T00:04:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:04:59 smithi148 ceph-mon[35599]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:00.164 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 podman[37286]: 2024-09-16 00:04:59.815890437 +0000 UTC m=+0.335454815 container start e3d4387421df7f64fec679205b305d35504a7c441915543cb058199afd3cbb53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T00:05:00.164 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 ceph-mgr[37305]: -- 172.21.15.193:0/2699805169 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f56fad70e0 con 0x55f56fab4800 2024-09-16T00:05:00.164 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 bash[37286]: e3d4387421df7f64fec679205b305d35504a7c441915543cb058199afd3cbb53 2024-09-16T00:05:00.164 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 systemd[1]: Started Ceph mgr.z for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:05:00.164 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:04:59 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:04:59.956+0000 7f9764dd2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:05:00.164 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:00.014+0000 7f9764dd2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:05:00.164 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:00.163+0000 7f9764dd2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:05:00.430 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:00.229+0000 7f9764dd2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:05:00.430 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:00.375+0000 7f9764dd2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:05:00.829 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:00.515+0000 7f9764dd2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:05:00.829 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:00.828+0000 7f9764dd2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:05:01.180 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:00.888+0000 7f9764dd2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:05:01.180 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:01 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:01.073+0000 7f9764dd2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:05:01.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:05:01.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:05:01.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:05:01.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:01.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: Deploying daemon mgr.y on smithi148 2024-09-16T00:05:01.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:00 smithi193 ceph-mon[35484]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:01.241 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:05:01.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:05:01.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:05:01.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:01.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: Deploying daemon mgr.y on smithi148 2024-09-16T00:05:01.242 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:00 smithi078 ceph-mon[31157]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: Deploying daemon mgr.y on smithi148 2024-09-16T00:05:01.275 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:00 smithi148 ceph-mon[35599]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:01.910 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:05:01.911 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:01 smithi148 systemd[1]: Starting Ceph mgr.y for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:05:01.938 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_4 2024-09-16T00:05:02.179 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:01 smithi148 podman[36952]: 2024-09-16 00:05:01.909840432 +0000 UTC m=+0.108602359 container create 151c57f5ff20e9f4c1a4f6b272a11a506bf7aa1e2d6571b2351979fcd2259897 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T00:05:02.179 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:01 smithi148 podman[36952]: 2024-09-16 00:05:01.816763741 +0000 UTC m=+0.015525673 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:05:02.180 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 podman[36952]: 2024-09-16 00:05:02.104072844 +0000 UTC m=+0.302834785 container init 151c57f5ff20e9f4c1a4f6b272a11a506bf7aa1e2d6571b2351979fcd2259897 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T00:05:02.180 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 podman[36952]: 2024-09-16 00:05:02.107325271 +0000 UTC m=+0.306087197 container start 151c57f5ff20e9f4c1a4f6b272a11a506bf7aa1e2d6571b2351979fcd2259897 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T00:05:02.180 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 ceph-mgr[36971]: -- 172.21.15.148:0/2106086406 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abe155f0e0 con 0x55abe153c800 2024-09-16T00:05:02.180 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 bash[36952]: 151c57f5ff20e9f4c1a4f6b272a11a506bf7aa1e2d6571b2351979fcd2259897 2024-09-16T00:05:02.180 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:05:02.430 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:01 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:01.949+0000 7f9764dd2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:05:02.430 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:02.015+0000 7f9764dd2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:05:02.447 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 systemd[1]: Started Ceph mgr.y for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:05:02.447 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:02.229+0000 7f1aed3f6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:05:02.447 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:02.289+0000 7f1aed3f6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:05:02.447 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:02.445+0000 7f1aed3f6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 -- 172.21.15.78:0/2491393356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390104fd0 msgr2=0x7f13901073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 --2- 172.21.15.78:0/2491393356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390104fd0 0x7f13901073c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1378009900 tx=0x7f137802a410 comp rx=0 tx=0).stop 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 -- 172.21.15.78:0/2491393356 shutdown_connections 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 --2- 172.21.15.78:0/2491393356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1390107900 0x7f1390109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 --2- 172.21.15.78:0/2491393356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390104fd0 0x7f13901073c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 --2- 172.21.15.78:0/2491393356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13901026a0 0x7f1390104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 -- 172.21.15.78:0/2491393356 >> 172.21.15.78:0/2491393356 conn(0x7f13900fc2a0 msgr2=0x7f13900fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 -- 172.21.15.78:0/2491393356 shutdown_connections 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.636+0000 7f1396312640 1 -- 172.21.15.78:0/2491393356 wait complete. 2024-09-16T00:05:02.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1396312640 1 Processor -- start 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1396312640 1 -- start start 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1396312640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13901026a0 0x7f1390102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1396312640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1390104fd0 0x7f1390100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1396312640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390107900 0x7f1390100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1396312640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f139010d880 con 0x7f1390107900 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1396312640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f139010d700 con 0x7f1390104fd0 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1396312640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f139010da00 con 0x7f13901026a0 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f13948af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390107900 0x7f1390100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f138ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13901026a0 0x7f1390102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f13948af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390107900 0x7f1390100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49902/0 (socket says 172.21.15.78:49902) 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f13948af640 1 -- 172.21.15.78:0/398804609 learned_addr learned my addr 172.21.15.78:0/398804609 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f1387fff640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1390104fd0 0x7f1390100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:02.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f138ffff640 1 -- 172.21.15.78:0/398804609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1390104fd0 msgr2=0x7f1390100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f138ffff640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1390104fd0 0x7f1390100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f138ffff640 1 -- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390107900 msgr2=0x7f1390100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f138ffff640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390107900 0x7f1390100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f138ffff640 1 -- 172.21.15.78:0/398804609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1390101450 con 0x7f13901026a0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.637+0000 7f13948af640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390107900 0x7f1390100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.638+0000 7f138ffff640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13901026a0 0x7f1390102000 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9f0 tx=0x7f138000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.638+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138000ccc0 con 0x7f13901026a0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.638+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13901a2120 con 0x7f13901026a0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.638+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1380010910 con 0x7f13901026a0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.638+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380018960 con 0x7f13901026a0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.638+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13901a2680 con 0x7f13901026a0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.639+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13900fdd90 con 0x7f13901026a0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.639+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f1380018b00 con 0x7f13901026a0 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.639+0000 7f138dffb640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f137c0394d0 0x7f137c03b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:02.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.639+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f138004ce50 con 0x7f13901026a0 2024-09-16T00:05:02.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.639+0000 7f1387fff640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f137c0394d0 0x7f137c03b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:02.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.640+0000 7f1387fff640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f137c0394d0 0x7f137c03b990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1378005ab0 tx=0x7f1378000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:05:02.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.649+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1380016330 con 0x7f13901026a0 2024-09-16T00:05:02.704 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:02.515+0000 7f1aed3f6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:05:02.704 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:02.675+0000 7f1aed3f6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:05:02.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:02.823+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1390062630 con 0x7f137c0394d0 2024-09-16T00:05:03.044 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:02.776+0000 7f9764dd2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:05:03.045 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:02.892+0000 7f9764dd2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:05:03.045 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.043+0000 7f9764dd2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:05:03.154 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:02.823+0000 7f1aed3f6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:05:03.154 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:03.153+0000 7f1aed3f6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:05:03.331 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.108+0000 7f9764dd2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.261+0000 7f9764dd2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.330+0000 7f9764dd2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.332 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:03.333 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:03.333 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:03.333 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:03.409 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:03.216+0000 7f1aed3f6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:03.410 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:03.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:03.637 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.397+0000 7f9764dd2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:05:03.637 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.571+0000 7f9764dd2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:05:03.637 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.635+0000 7f9764dd2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:05:03.704 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:03.408+0000 7f1aed3f6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:05:03.903 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.701+0000 7f9764dd2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:05:03.903 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.783+0000 7f9764dd2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:05:03.904 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.843+0000 7f9764dd2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:05:03.904 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:03.901+0000 7f9764dd2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:05:04.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:05:04.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:04.704 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:04.293+0000 7f1aed3f6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:05:04.704 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:04.359+0000 7f1aed3f6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:05:04.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:04.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:05:04.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:04.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:05:04.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:05:04.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:05.180 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:05:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z[37301]: 2024-09-16T00:05:04.682+0000 7f9764dd2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:05:05.332 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.075+0000 7f1aed3f6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:05:05.332 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.190+0000 7f1aed3f6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:05:05.332 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.330+0000 7f1aed3f6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:05:05.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:05.472+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f1380017c70 con 0x7f13901026a0 2024-09-16T00:05:05.612 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.391+0000 7f1aed3f6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:05:05.612 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.541+0000 7f1aed3f6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.611+0000 7f1aed3f6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: Reconfiguring daemon mgr.x on smithi078 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: Standby manager daemon z started 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:05.613 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: Reconfiguring daemon mgr.x on smithi078 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: Standby manager daemon z started 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:05.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:05.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:05.893 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.672+0000 7f1aed3f6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:05:05.893 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.828+0000 7f1aed3f6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:05:05.893 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.892+0000 7f1aed3f6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: Reconfiguring daemon mgr.x on smithi078 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: Standby manager daemon z started 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: from='mgr.? 172.21.15.193:0/1114108659' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:05:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:05.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:05.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:06.160 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:05.957+0000 7f1aed3f6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:05:06.160 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:06.040+0000 7f1aed3f6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:05:06.160 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:06.100+0000 7f1aed3f6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:05:06.160 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:06.159+0000 7f1aed3f6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2690739629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576"}]: dispatch 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2690739629' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576"}]': finished 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:05:06.641 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2690739629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576"}]: dispatch 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2690739629' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576"}]': finished 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:05:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:06.942 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-16T00:05:06.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:05:06.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:06.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:06.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:06.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2690739629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576"}]: dispatch 2024-09-16T00:05:06.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2690739629' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576"}]': finished 2024-09-16T00:05:06.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:05:06.943 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:07.204 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:05:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y[36967]: 2024-09-16T00:05:06.941+0000 7f1aed3f6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:05:07.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:07.483+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mon.1 v2:172.21.15.193:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f138001f310 con 0x7f13901026a0 2024-09-16T00:05:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:07 smithi078 ceph-mon[31157]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:07 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1229073320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:07 smithi078 ceph-mon[31157]: Standby manager daemon y started 2024-09-16T00:05:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:07 smithi078 ceph-mon[31157]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:05:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:07 smithi078 ceph-mon[31157]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:05:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:07 smithi078 ceph-mon[31157]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:05:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:07 smithi078 ceph-mon[31157]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:05:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:07 smithi193 ceph-mon[35484]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:07 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1229073320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:07 smithi193 ceph-mon[35484]: Standby manager daemon y started 2024-09-16T00:05:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:07 smithi193 ceph-mon[35484]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:05:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:07 smithi193 ceph-mon[35484]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:05:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:07 smithi193 ceph-mon[35484]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:05:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:07 smithi193 ceph-mon[35484]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:05:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:07 smithi148 ceph-mon[35599]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:07 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1229073320' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:07 smithi148 ceph-mon[35599]: Standby manager daemon y started 2024-09-16T00:05:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:07 smithi148 ceph-mon[35599]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:05:07.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:07 smithi148 ceph-mon[35599]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:05:07.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:07 smithi148 ceph-mon[35599]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:05:07.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:07 smithi148 ceph-mon[35599]: from='mgr.? 172.21.15.148:0/1713122977' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:05:08.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:08 smithi078 ceph-mon[31157]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-16T00:05:08.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:08 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:05:08.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:08 smithi193 ceph-mon[35484]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-16T00:05:08.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:08 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:05:08.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:08 smithi148 ceph-mon[35599]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-16T00:05:08.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:08 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:05:09.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:09 smithi078 ceph-mon[31157]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:09 smithi193 ceph-mon[35484]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:09 smithi148 ceph-mon[35599]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:11.747 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:11 smithi193 ceph-mon[35484]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:11.756 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:11 smithi148 ceph-mon[35599]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:11.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:11 smithi078 ceph-mon[31157]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:13.710 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:13 smithi078 ceph-mon[31157]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:13 smithi193 ceph-mon[35484]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:13 smithi148 ceph-mon[35599]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:15.738 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:15 smithi078 ceph-mon[31157]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:15.738 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:05:15.738 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:15 smithi193 ceph-mon[35484]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:05:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:15 smithi148 ceph-mon[35599]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:05:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:16.579 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:16 smithi078 ceph-mon[31157]: Deploying daemon osd.0 on smithi078 2024-09-16T00:05:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:16 smithi193 ceph-mon[35484]: Deploying daemon osd.0 on smithi078 2024-09-16T00:05:16.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:16 smithi148 ceph-mon[35599]: Deploying daemon osd.0 on smithi078 2024-09-16T00:05:17.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:17 smithi078 ceph-mon[31157]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:17 smithi193 ceph-mon[35484]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:17 smithi148 ceph-mon[35599]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:19 smithi078 ceph-mon[31157]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:19 smithi193 ceph-mon[35484]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:19 smithi148 ceph-mon[35599]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:21.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:21 smithi078 ceph-mon[31157]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:21.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:21.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:21.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:21 smithi193 ceph-mon[35484]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:21 smithi148 ceph-mon[35599]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:23 smithi078 ceph-mon[31157]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:23 smithi193 ceph-mon[35484]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:23 smithi148 ceph-mon[35599]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:24.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.670+0000 7f138dffb640 1 -- 172.21.15.78:0/398804609 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1390062630 con 0x7f137c0394d0 2024-09-16T00:05:24.671 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 0 on host 'smithi078' 2024-09-16T00:05:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f137c0394d0 msgr2=0x7f137c03b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f137c0394d0 0x7f137c03b990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1378005ab0 tx=0x7f1378000bc0 comp rx=0 tx=0).stop 2024-09-16T00:05:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13901026a0 msgr2=0x7f1390102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13901026a0 0x7f1390102000 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9f0 tx=0x7f138000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:05:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 shutdown_connections 2024-09-16T00:05:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f137c0394d0 0x7f137c03b990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1390107900 0x7f1390100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1390104fd0 0x7f1390100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 --2- 172.21.15.78:0/398804609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13901026a0 0x7f1390102000 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 >> 172.21.15.78:0/398804609 conn(0x7f13900fc2a0 msgr2=0x7f1390103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:05:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 shutdown_connections 2024-09-16T00:05:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:05:24.673+0000 7f1396312640 1 -- 172.21.15.78:0/398804609 wait complete. 2024-09-16T00:05:25.415 DEBUG:teuthology.orchestra.run.smithi078:osd.0> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T00:05:25.418 INFO:tasks.cephadm:Deploying osd.1 on smithi148 with /dev/vg_nvme/lv_4... 2024-09-16T00:05:25.418 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:05:25.674 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:05:25.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:25 smithi148 ceph-mon[35599]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:25.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:25 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:25.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:25 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:25.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:25 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:25.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:25 smithi078 ceph-mon[31157]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:25.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:25.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:25.754 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:25.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:25 smithi193 ceph-mon[35484]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:25.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:25 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:25.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:25 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:25.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:25 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:27.520 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:05:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0[45158]: 2024-09-16T00:05:27.249+0000 7f685232d640 -1 osd.0 0 log_to_monitors true 2024-09-16T00:05:27.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:27 smithi148 ceph-mon[35599]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:27 smithi148 ceph-mon[35599]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:05:27.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:27 smithi078 ceph-mon[31157]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:27.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:27 smithi078 ceph-mon[31157]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:05:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:27 smithi193 ceph-mon[35484]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:27 smithi193 ceph-mon[35484]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:05:28.039 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:05:28.064 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-09-16T00:05:28.305 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:05:28.688 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:05:28.688 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:05:28.688 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:05:28.689 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.770+0000 7eff328bc640 1 -- 172.21.15.148:0/3460309156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c104fd0 msgr2=0x7eff2c1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.770+0000 7eff328bc640 1 --2- 172.21.15.148:0/3460309156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c104fd0 0x7eff2c1073c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff20009900 tx=0x7eff2002a410 comp rx=0 tx=0).stop 2024-09-16T00:05:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.770+0000 7eff328bc640 1 -- 172.21.15.148:0/3460309156 shutdown_connections 2024-09-16T00:05:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.770+0000 7eff328bc640 1 --2- 172.21.15.148:0/3460309156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff2c107900 0x7eff2c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.770+0000 7eff328bc640 1 --2- 172.21.15.148:0/3460309156 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c104fd0 0x7eff2c1073c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.770+0000 7eff328bc640 1 --2- 172.21.15.148:0/3460309156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff2c1026a0 0x7eff2c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:28.774 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.770+0000 7eff328bc640 1 -- 172.21.15.148:0/3460309156 >> 172.21.15.148:0/3460309156 conn(0x7eff2c0fc2a0 msgr2=0x7eff2c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.770+0000 7eff328bc640 1 -- 172.21.15.148:0/3460309156 shutdown_connections 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 -- 172.21.15.148:0/3460309156 wait complete. 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 Processor -- start 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 -- start start 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c1026a0 0x7eff2c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff2c104fd0 0x7eff2c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff2c107900 0x7eff2c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c10d8d0 con 0x7eff2c107900 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c10d750 con 0x7eff2c1026a0 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.771+0000 7eff328bc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c10da50 con 0x7eff2c104fd0 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c1026a0 0x7eff2c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c1026a0 0x7eff2c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37506/0 (socket says 172.21.15.148:37506) 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 -- 172.21.15.148:0/3584066278 learned_addr learned my addr 172.21.15.148:0/3584066278 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-16T00:05:28.775 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2b7fe640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff2c104fd0 0x7eff2c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff30e59640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff2c107900 0x7eff2c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 -- 172.21.15.148:0/3584066278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff2c104fd0 msgr2=0x7eff2c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff2c104fd0 0x7eff2c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 -- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff2c107900 msgr2=0x7eff2c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff2c107900 0x7eff2c19f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 -- 172.21.15.148:0/3584066278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c19fbc0 con 0x7eff2c1026a0 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff30e59640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff2c107900 0x7eff2c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff2bfff640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c1026a0 0x7eff2c197f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00d950 tx=0x7eff1c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff297fa640 1 -- 172.21.15.148:0/3584066278 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c017070 con 0x7eff2c1026a0 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c19feb0 con 0x7eff2c1026a0 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff297fa640 1 -- 172.21.15.148:0/3584066278 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1c009c30 con 0x7eff2c1026a0 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff297fa640 1 -- 172.21.15.148:0/3584066278 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c00f6b0 con 0x7eff2c1026a0 2024-09-16T00:05:28.776 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.772+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c1a0300 con 0x7eff2c1026a0 2024-09-16T00:05:28.777 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.773+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c0fdd90 con 0x7eff2c1026a0 2024-09-16T00:05:28.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.774+0000 7eff297fa640 1 -- 172.21.15.148:0/3584066278 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7eff1c0109d0 con 0x7eff2c1026a0 2024-09-16T00:05:28.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.775+0000 7eff297fa640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7eff140a31e0 0x7eff140a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:28.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.775+0000 7eff297fa640 1 -- 172.21.15.148:0/3584066278 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7eff1c0cd910 con 0x7eff2c1026a0 2024-09-16T00:05:28.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.775+0000 7eff2b7fe640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7eff140a31e0 0x7eff140a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:28.778 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.776+0000 7eff2b7fe640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7eff140a31e0 0x7eff140a56a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff2002aa00 tx=0x7eff200315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:05:28.780 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.780+0000 7eff297fa640 1 -- 172.21.15.148:0/3584066278 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff1c09c1c0 con 0x7eff2c1026a0 2024-09-16T00:05:28.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:05:28.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:05:28.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:05:28.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:28.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:05:28.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:28.956 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:28.955+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7eff2c062630 con 0x7eff140a31e0 2024-09-16T00:05:29.019 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0[45158]: 2024-09-16T00:05:28.526+0000 7f684e2b0640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T00:05:29.020 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0[45158]: 2024-09-16T00:05:28.529+0000 7f684a0d8640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:05:29.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:05:29.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:05:29.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:05:29.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:29.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:29.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:29.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:05:29.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:29.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:29.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:29.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:29.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:05:29.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:05:29.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:29.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:29 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi078 2024-09-16T00:05:29.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:29 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-16T00:05:29.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:29 smithi148 ceph-mon[35599]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:29.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:29 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:29.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:29 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:29.667 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:29 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:29.667 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:29 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:29.667 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:29 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:29 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi078 2024-09-16T00:05:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:29 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-16T00:05:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:29 smithi193 ceph-mon[35484]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:29 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:29 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:29 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:29 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:29.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:29 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:29 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi078 2024-09-16T00:05:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:29 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi078 to 17088M 2024-09-16T00:05:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:29 smithi078 ceph-mon[31157]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:30.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:29 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:30.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:30 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:05:30.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:30 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:05:30.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:30 smithi148 ceph-mon[35599]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:30.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:30 smithi148 ceph-mon[35599]: osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] boot 2024-09-16T00:05:30.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:30 smithi148 ceph-mon[35599]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:05:30.874 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:30 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:05:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:30 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:05:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:30 smithi193 ceph-mon[35484]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:30 smithi193 ceph-mon[35484]: osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] boot 2024-09-16T00:05:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:30 smithi193 ceph-mon[35484]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:05:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:30 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:05:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:30 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:05:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:30 smithi078 ceph-mon[31157]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:30 smithi078 ceph-mon[31157]: osd.0 [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] boot 2024-09-16T00:05:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:30 smithi078 ceph-mon[31157]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:05:31.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:05:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:31 smithi148 ceph-mon[35599]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:31 smithi148 ceph-mon[35599]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:05:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:31 smithi148 ceph-mon[35599]: from='client.? 172.21.15.148:0/772968203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]: dispatch 2024-09-16T00:05:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:31 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]: dispatch 2024-09-16T00:05:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:31 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]': finished 2024-09-16T00:05:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:31 smithi148 ceph-mon[35599]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:05:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:31 smithi193 ceph-mon[35484]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:31 smithi193 ceph-mon[35484]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:05:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:31 smithi193 ceph-mon[35484]: from='client.? 172.21.15.148:0/772968203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]: dispatch 2024-09-16T00:05:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:31 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]: dispatch 2024-09-16T00:05:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:31 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]': finished 2024-09-16T00:05:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:31 smithi193 ceph-mon[35484]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:05:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:31 smithi078 ceph-mon[31157]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:05:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:31 smithi078 ceph-mon[31157]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:05:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:31 smithi078 ceph-mon[31157]: from='client.? 172.21.15.148:0/772968203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]: dispatch 2024-09-16T00:05:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:31 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]: dispatch 2024-09-16T00:05:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:31 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2638a47-f838-4300-848b-e731ac3cce78"}]': finished 2024-09-16T00:05:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:31 smithi078 ceph-mon[31157]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:05:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:32.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:32 smithi193 ceph-mon[35484]: from='client.? 172.21.15.148:0/1082588856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:32.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:32 smithi148 ceph-mon[35599]: from='client.? 172.21.15.148:0/1082588856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:33.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:32 smithi078 ceph-mon[31157]: from='client.? 172.21.15.148:0/1082588856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:33.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:33 smithi078 ceph-mon[31157]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:33 smithi193 ceph-mon[35484]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:33 smithi148 ceph-mon[35599]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:35 smithi193 ceph-mon[35484]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:35 smithi148 ceph-mon[35599]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:36.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:35 smithi078 ceph-mon[31157]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:37.629 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:37 smithi148 ceph-mon[35599]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:37 smithi193 ceph-mon[35484]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:38.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:37 smithi078 ceph-mon[31157]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:39 smithi148 ceph-mon[35599]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:05:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:39 smithi193 ceph-mon[35484]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:05:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:39 smithi078 ceph-mon[31157]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:05:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:40.816 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:40 smithi148 ceph-mon[35599]: Deploying daemon osd.1 on smithi148 2024-09-16T00:05:40.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:40 smithi193 ceph-mon[35484]: Deploying daemon osd.1 on smithi148 2024-09-16T00:05:41.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:40 smithi078 ceph-mon[31157]: Deploying daemon osd.1 on smithi148 2024-09-16T00:05:41.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:41 smithi148 ceph-mon[35599]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:41.831 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:41 smithi193 ceph-mon[35484]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:41 smithi078 ceph-mon[31157]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:42.721 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:42 smithi148 ceph-mon[35599]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:42.921 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:42 smithi078 ceph-mon[31157]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:42.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:42 smithi193 ceph-mon[35484]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:45.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:45 smithi078 ceph-mon[31157]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:45.563 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:45 smithi148 ceph-mon[35599]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:45.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:45 smithi193 ceph-mon[35484]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:46.581 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:46 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:46.581 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:46 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:46.581 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:46 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:46.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:46 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:46.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:46 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:46.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:46 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:46 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:46 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:46 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:47.495 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:47 smithi148 ceph-mon[35599]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:47.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:47 smithi193 ceph-mon[35484]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:47.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:47 smithi078 ceph-mon[31157]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:49.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:49.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:49.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:49 smithi078 ceph-mon[31157]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:49.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:49.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:49.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:49 smithi193 ceph-mon[35484]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:49.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:49.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:49.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:49.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:49 smithi148 ceph-mon[35599]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:50.027 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.025+0000 7eff297fa640 1 -- 172.21.15.148:0/3584066278 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff2c062630 con 0x7eff140a31e0 2024-09-16T00:05:50.027 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 1 on host 'smithi148' 2024-09-16T00:05:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7eff140a31e0 msgr2=0x7eff140a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7eff140a31e0 0x7eff140a56a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff2002aa00 tx=0x7eff200315b0 comp rx=0 tx=0).stop 2024-09-16T00:05:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c1026a0 msgr2=0x7eff2c197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c1026a0 0x7eff2c197f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00d950 tx=0x7eff1c00de10 comp rx=0 tx=0).stop 2024-09-16T00:05:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 shutdown_connections 2024-09-16T00:05:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7eff140a31e0 0x7eff140a56a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff2c107900 0x7eff2c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:50.029 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff2c104fd0 0x7eff2c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 --2- 172.21.15.148:0/3584066278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff2c1026a0 0x7eff2c197f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 >> 172.21.15.148:0/3584066278 conn(0x7eff2c0fc2a0 msgr2=0x7eff2c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:05:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 shutdown_connections 2024-09-16T00:05:50.030 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:05:50.028+0000 7eff328bc640 1 -- 172.21.15.148:0/3584066278 wait complete. 2024-09-16T00:05:50.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:50.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:50.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:50 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:50.654 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:50 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:50.654 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:50 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:50.654 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:50 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:50.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:05:50.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:50.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:50 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:50.682 DEBUG:teuthology.orchestra.run.smithi148:osd.1> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T00:05:50.685 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-09-16T00:05:50.685 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:05:50.955 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:05:51.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:51 smithi193 ceph-mon[35484]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:51.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:51 smithi078 ceph-mon[31157]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:51.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:51 smithi148 ceph-mon[35599]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:52.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:52 smithi148 ceph-mon[35599]: from='osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:05:52.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:52 smithi148 ceph-mon[35599]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:05:52.455 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:05:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1[40051]: 2024-09-16T00:05:52.120+0000 7ffaa682a640 -1 osd.1 0 log_to_monitors true 2024-09-16T00:05:52.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:52 smithi078 ceph-mon[31157]: from='osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:05:52.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:52 smithi078 ceph-mon[31157]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:05:52.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:52 smithi193 ceph-mon[35484]: from='osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:05:52.601 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:52 smithi193 ceph-mon[35484]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:05:53.339 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:05:53.365 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-09-16T00:05:53.391 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:53 smithi193 ceph-mon[35484]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:05:53.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:53 smithi193 ceph-mon[35484]: from='osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:05:53.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:53 smithi193 ceph-mon[35484]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:05:53.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:53.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:53 smithi193 ceph-mon[35484]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:05:53.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:53 smithi193 ceph-mon[35484]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:53.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:53 smithi193 ceph-mon[35484]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:05:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:53 smithi078 ceph-mon[31157]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:05:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:53 smithi078 ceph-mon[31157]: from='osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:05:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:53 smithi078 ceph-mon[31157]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:05:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:53 smithi078 ceph-mon[31157]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:05:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:53 smithi078 ceph-mon[31157]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:53.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:53 smithi078 ceph-mon[31157]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:05:53.551 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1[40051]: 2024-09-16T00:05:53.248+0000 7ffaa27ad640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T00:05:53.552 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1[40051]: 2024-09-16T00:05:53.253+0000 7ffa9e5d5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:05:53.552 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-mon[35599]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:05:53.552 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-mon[35599]: from='osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:05:53.552 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-mon[35599]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:05:53.552 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:53.552 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-mon[35599]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:05:53.552 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-mon[35599]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:53.552 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:53 smithi148 ceph-mon[35599]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:05:53.615 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:05:54.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 -- 172.21.15.193:0/2332521443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8101c00 msgr2=0x7f48a8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 --2- 172.21.15.193:0/2332521443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8101c00 0x7f48a8102000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f489c009900 tx=0x7f489c02a410 comp rx=0 tx=0).stop 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 -- 172.21.15.193:0/2332521443 shutdown_connections 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 --2- 172.21.15.193:0/2332521443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a81037c0 0x7f48a810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 --2- 172.21.15.193:0/2332521443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a8102e00 0x7f48a8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 --2- 172.21.15.193:0/2332521443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8101c00 0x7f48a8102000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 -- 172.21.15.193:0/2332521443 >> 172.21.15.193:0/2332521443 conn(0x7f48a80fd390 msgr2=0x7f48a80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 -- 172.21.15.193:0/2332521443 shutdown_connections 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 -- 172.21.15.193:0/2332521443 wait complete. 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 Processor -- start 2024-09-16T00:05:54.081 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.073+0000 7f48acec4640 1 -- start start 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48acec4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a8101c00 0x7f48a8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48acec4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8102e00 0x7f48a81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48acec4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a81037c0 0x7f48a819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48acec4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48a810d8e0 con 0x7f48a81037c0 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48acec4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48a810d760 con 0x7f48a8101c00 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48acec4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48a810da60 con 0x7f48a8102e00 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8102e00 0x7f48a81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8102e00 0x7f48a81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:38882/0 (socket says 172.21.15.193:38882) 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 -- 172.21.15.193:0/2003634918 learned_addr learned my addr 172.21.15.193:0/2003634918 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a6d9d640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a81037c0 0x7f48a819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a659c640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a8101c00 0x7f48a8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:54.082 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 -- 172.21.15.193:0/2003634918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a8101c00 msgr2=0x7f48a8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a8101c00 0x7f48a8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 -- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a81037c0 msgr2=0x7f48a819f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a81037c0 0x7f48a819f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 -- 172.21.15.193:0/2003634918 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a819fbe0 con 0x7f48a8102e00 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a659c640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a8101c00 0x7f48a8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48a5d9b640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8102e00 0x7f48a81984f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f489800b4a0 tx=0x7f489800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48977fe640 1 -- 172.21.15.193:0/2003634918 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898014070 con 0x7f48a8102e00 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48977fe640 1 -- 172.21.15.193:0/2003634918 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4898010af0 con 0x7f48a8102e00 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a819fed0 con 0x7f48a8102e00 2024-09-16T00:05:54.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.074+0000 7f48977fe640 1 -- 172.21.15.193:0/2003634918 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898017d10 con 0x7f48a8102e00 2024-09-16T00:05:54.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.075+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a81a03b0 con 0x7f48a8102e00 2024-09-16T00:05:54.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.076+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a8108ff0 con 0x7f48a8102e00 2024-09-16T00:05:54.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.077+0000 7f48977fe640 1 -- 172.21.15.193:0/2003634918 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f4898017070 con 0x7f48a8102e00 2024-09-16T00:05:54.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.078+0000 7f48977fe640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f48900a31e0 0x7f48900a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:05:54.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.078+0000 7f48977fe640 1 -- 172.21.15.193:0/2003634918 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f48980cd390 con 0x7f48a8102e00 2024-09-16T00:05:54.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.078+0000 7f48a659c640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f48900a31e0 0x7f48900a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:05:54.084 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.078+0000 7f48a659c640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f48900a31e0 0x7f48900a56a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f489c006010 tx=0x7f489c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:05:54.085 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.082+0000 7f48977fe640 1 -- 172.21.15.193:0/2003634918 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f489809b980 con 0x7f48a8102e00 2024-09-16T00:05:54.266 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:05:54.265+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f48a8062630 con 0x7f48900a31e0 2024-09-16T00:05:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:05:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi148 2024-09-16T00:05:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:05:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi148 to 17088M 2024-09-16T00:05:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi148 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi148 to 17088M 2024-09-16T00:05:54.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:54.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:54.632 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi148 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi148 to 17088M 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:05:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] boot 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:55.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:55.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:55.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:55 smithi078 ceph-mon[31157]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:55.577 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:05:55.577 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] boot 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:55.578 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:55 smithi193 ceph-mon[35484]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: osd.1 [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] boot 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: from='client.24142 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:05:55.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:55 smithi148 ceph-mon[35599]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:05:56.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:56 smithi078 ceph-mon[31157]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:05:56.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:56 smithi193 ceph-mon[35484]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:05:56.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:56 smithi148 ceph-mon[35599]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:05:57.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:57 smithi193 ceph-mon[35484]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:05:57.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:57 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/2017116921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]: dispatch 2024-09-16T00:05:57.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:57 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]: dispatch 2024-09-16T00:05:57.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:57 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]': finished 2024-09-16T00:05:57.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:57 smithi193 ceph-mon[35484]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:05:57.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:05:57.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:57 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/492968922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:57.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:57 smithi078 ceph-mon[31157]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:05:57.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:57 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/2017116921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]: dispatch 2024-09-16T00:05:57.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:57 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]: dispatch 2024-09-16T00:05:57.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:57 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]': finished 2024-09-16T00:05:57.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:57 smithi078 ceph-mon[31157]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:05:57.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:05:57.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:57 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/492968922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:57.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:57 smithi148 ceph-mon[35599]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:05:57.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:57 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/2017116921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]: dispatch 2024-09-16T00:05:57.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:57 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]: dispatch 2024-09-16T00:05:57.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:57 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "863c25a6-76c4-4097-9c6f-20e838ddb3e9"}]': finished 2024-09-16T00:05:57.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:57 smithi148 ceph-mon[35599]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:05:57.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:05:57.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:57 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/492968922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:05:59.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:05:59 smithi193 ceph-mon[35484]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:05:59.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:05:59 smithi148 ceph-mon[35599]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:05:59.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:05:59 smithi078 ceph-mon[31157]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:01.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:01 smithi193 ceph-mon[35484]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:01.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:01 smithi148 ceph-mon[35599]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:01.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:01 smithi078 ceph-mon[31157]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:03.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:03 smithi193 ceph-mon[35484]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:03.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:03 smithi148 ceph-mon[35599]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:03.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:03 smithi078 ceph-mon[31157]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:05.500 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:05 smithi193 ceph-mon[35484]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:05.500 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:06:05.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:05.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:05 smithi148 ceph-mon[35599]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:05.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:06:05.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:05 smithi078 ceph-mon[31157]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:06:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:06.648 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:06 smithi193 ceph-mon[35484]: Deploying daemon osd.2 on smithi193 2024-09-16T00:06:06.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:06 smithi148 ceph-mon[35599]: Deploying daemon osd.2 on smithi193 2024-09-16T00:06:06.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:06 smithi078 ceph-mon[31157]: Deploying daemon osd.2 on smithi193 2024-09-16T00:06:07.652 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:07 smithi193 ceph-mon[35484]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:07.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:07 smithi148 ceph-mon[35599]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:07 smithi078 ceph-mon[31157]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:09 smithi193 ceph-mon[35484]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:09.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:09 smithi148 ceph-mon[35599]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:09.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:09 smithi078 ceph-mon[31157]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:11.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:11 smithi193 ceph-mon[35484]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:11.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:11.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:11.666 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:11.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:11 smithi148 ceph-mon[35599]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:11.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:11.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:11.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:11.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:11 smithi078 ceph-mon[31157]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:11.772 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:11.772 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:11.773 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:13.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:13 smithi193 ceph-mon[35484]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:13.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:13 smithi148 ceph-mon[35599]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:13.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:13 smithi078 ceph-mon[31157]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:14.691 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.688+0000 7f48977fe640 1 -- 172.21.15.193:0/2003634918 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f48a8062630 con 0x7f48900a31e0 2024-09-16T00:06:14.691 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-09-16T00:06:14.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f48900a31e0 msgr2=0x7f48900a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:14.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f48900a31e0 0x7f48900a56a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f489c006010 tx=0x7f489c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:06:14.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8102e00 msgr2=0x7f48a81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:14.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8102e00 0x7f48a81984f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f489800b4a0 tx=0x7f489800b960 comp rx=0 tx=0).stop 2024-09-16T00:06:14.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 shutdown_connections 2024-09-16T00:06:14.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f48900a31e0 0x7f48900a56a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:14.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48a81037c0 0x7f48a819f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:14.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48a8102e00 0x7f48a81984f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:14.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 --2- 172.21.15.193:0/2003634918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48a8101c00 0x7f48a8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:14.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 >> 172.21.15.193:0/2003634918 conn(0x7f48a80fd390 msgr2=0x7f48a81061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:06:14.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 shutdown_connections 2024-09-16T00:06:14.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:06:14.691+0000 7f48acec4640 1 -- 172.21.15.193:0/2003634918 wait complete. 2024-09-16T00:06:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:14.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:14.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:15.383 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T00:06:15.426 INFO:tasks.cephadm:Deploying osd.3 on smithi078 with /dev/vg_nvme/lv_3... 2024-09-16T00:06:15.426 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:06:15.685 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:06:17.385 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2[40459]: 2024-09-16T00:06:17.073+0000 7f47e1073640 -1 osd.2 0 log_to_monitors true 2024-09-16T00:06:17.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:17 smithi078 ceph-mon[31157]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:17.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:17 smithi078 ceph-mon[31157]: from='osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:06:17.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:17 smithi078 ceph-mon[31157]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:06:17.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:17 smithi193 ceph-mon[35484]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:17.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:17 smithi193 ceph-mon[35484]: from='osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:06:17.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:17 smithi193 ceph-mon[35484]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:06:17.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:17 smithi148 ceph-mon[35599]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:17.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:17 smithi148 ceph-mon[35599]: from='osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:06:17.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:17 smithi148 ceph-mon[35599]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:06:18.270 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:06:18.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_3 2024-09-16T00:06:18.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:06:18.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:06:18.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:18.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:06:18.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:06:18.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:18.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:18.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:18.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:06:18.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.556 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:06:18.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:06:18.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:18.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 -- 172.21.15.78:0/2029251093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660100ad0 msgr2=0x7fb660100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 --2- 172.21.15.78:0/2029251093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660100ad0 0x7fb660100ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb658009900 tx=0x7fb65802a410 comp rx=0 tx=0).stop 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 -- 172.21.15.78:0/2029251093 shutdown_connections 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 --2- 172.21.15.78:0/2029251093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb660102690 0x7fb660108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 --2- 172.21.15.78:0/2029251093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb660101cd0 0x7fb660102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 --2- 172.21.15.78:0/2029251093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660100ad0 0x7fb660100ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 -- 172.21.15.78:0/2029251093 >> 172.21.15.78:0/2029251093 conn(0x7fb6600fc280 msgr2=0x7fb6600fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 -- 172.21.15.78:0/2029251093 shutdown_connections 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 -- 172.21.15.78:0/2029251093 wait complete. 2024-09-16T00:06:19.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 Processor -- start 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 -- start start 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb660100ad0 0x7fb66019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660101cd0 0x7fb66019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.989+0000 7fb667b18640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb660102690 0x7fb6601a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb667b18640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010c7d0 con 0x7fb660101cd0 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb667b18640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010c650 con 0x7fb660102690 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb667b18640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb66010c950 con 0x7fb660100ad0 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660101cd0 0x7fb66019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660101cd0 0x7fb66019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60608/0 (socket says 172.21.15.78:60608) 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 -- 172.21.15.78:0/1351260701 learned_addr learned my addr 172.21.15.78:0/1351260701 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6658b4640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb660100ad0 0x7fb66019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6660b5640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb660102690 0x7fb6601a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:06:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 -- 172.21.15.78:0/1351260701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb660100ad0 msgr2=0x7fb66019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb660100ad0 0x7fb66019a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 -- 172.21.15.78:0/1351260701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb660102690 msgr2=0x7fb6601a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb660102690 0x7fb6601a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 -- 172.21.15.78:0/1351260701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6601a1e00 con 0x7fb660101cd0 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6660b5640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb660102690 0x7fb6601a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb6650b3640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660101cd0 0x7fb66019a710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb65400c890 tx=0x7fb65400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb65effd640 1 -- 172.21.15.78:0/1351260701 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654004220 con 0x7fb660101cd0 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb65effd640 1 -- 172.21.15.78:0/1351260701 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6540043c0 con 0x7fb660101cd0 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6601a20f0 con 0x7fb660101cd0 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb65effd640 1 -- 172.21.15.78:0/1351260701 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6540049e0 con 0x7fb660101cd0 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.990+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6601a26b0 con 0x7fb660101cd0 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.991+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6601a2970 con 0x7fb660101cd0 2024-09-16T00:06:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.993+0000 7fb65effd640 1 -- 172.21.15.78:0/1351260701 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fb6540079b0 con 0x7fb660101cd0 2024-09-16T00:06:19.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.993+0000 7fb65effd640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6480a31e0 0x7fb6480a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:06:19.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.994+0000 7fb6658b4640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6480a31e0 0x7fb6480a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:06:19.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.994+0000 7fb65effd640 1 -- 172.21.15.78:0/1351260701 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fb6540ce200 con 0x7fb660101cd0 2024-09-16T00:06:19.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.994+0000 7fb6658b4640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6480a31e0 0x7fb6480a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb6580314e0 tx=0x7fb658000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:06:19.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:18.997+0000 7fb65effd640 1 -- 172.21.15.78:0/1351260701 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb654014030 con 0x7fb660101cd0 2024-09-16T00:06:19.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:19.178+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb66006a750 con 0x7fb6480a31e0 2024-09-16T00:06:19.389 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2[40459]: 2024-09-16T00:06:19.026+0000 7f47dd80a640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T00:06:19.390 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2[40459]: 2024-09-16T00:06:19.031+0000 7f47d4e16640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi193 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: from='osd.2 ' entity='osd.2' 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:06:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi193 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: from='osd.2 ' entity='osd.2' 2024-09-16T00:06:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:06:19.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:06:19.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:19.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi193 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: from='osd.2 ' entity='osd.2' 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:06:19.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:20.897 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:20 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:06:20.897 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:20 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:06:20.897 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:20 smithi078 ceph-mon[31157]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:06:20.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:20 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:20.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:20 smithi078 ceph-mon[31157]: osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] boot 2024-09-16T00:06:20.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:20 smithi078 ceph-mon[31157]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:06:20.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:20 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:20.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:20 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:06:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:20 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:06:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:20 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:06:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:20 smithi193 ceph-mon[35484]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:06:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:20 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:20.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:20 smithi193 ceph-mon[35484]: osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] boot 2024-09-16T00:06:20.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:20 smithi193 ceph-mon[35484]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:06:20.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:20 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:20.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:20 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:06:20.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:20 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:06:20.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:20 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:06:20.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:20 smithi148 ceph-mon[35599]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:06:20.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:20 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:20.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:20 smithi148 ceph-mon[35599]: osd.2 [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] boot 2024-09-16T00:06:20.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:20 smithi148 ceph-mon[35599]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:06:20.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:20 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:06:20.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:20 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:06:21.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:21 smithi078 ceph-mon[31157]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:06:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:21 smithi078 ceph-mon[31157]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:06:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:06:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:21 smithi193 ceph-mon[35484]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:06:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:21 smithi193 ceph-mon[35484]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:06:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:06:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:21 smithi148 ceph-mon[35599]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:06:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:06:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:21 smithi148 ceph-mon[35599]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:06:21.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:06:22.269 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:06:22 smithi078 sudo[50013]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:06:22.270 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:06:22 smithi078 sudo[50013]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:22.270 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:06:22 smithi078 sudo[50013]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:22.270 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:06:22 smithi078 sudo[50013]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:22.614 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:06:22 smithi078 sudo[50019]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:06:22.615 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:06:22 smithi078 sudo[50019]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:22.615 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:06:22 smithi078 sudo[50019]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:22.615 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:06:22 smithi078 sudo[50019]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:22.737+0000 7fb65effd640 1 -- 172.21.15.78:0/1351260701 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb6540985e0 con 0x7fb660101cd0 2024-09-16T00:06:22.858 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:06:22 smithi148 sudo[43743]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:06:22.858 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:06:22 smithi148 sudo[43743]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:22.858 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:06:22 smithi148 sudo[43743]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:22.858 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:06:22 smithi148 sudo[43743]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:22.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:22 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/629398922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34ec822-59f0-44e6-a33b-ae877c648c67"}]: dispatch 2024-09-16T00:06:22.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:06:22.858 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:22 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/629398922' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f34ec822-59f0-44e6-a33b-ae877c648c67"}]': finished 2024-09-16T00:06:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:22 smithi148 ceph-mon[35599]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:06:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:22 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3700506891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:06:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:22 smithi148 ceph-mon[35599]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:22.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:22 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/629398922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34ec822-59f0-44e6-a33b-ae877c648c67"}]: dispatch 2024-09-16T00:06:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:06:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:22 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/629398922' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f34ec822-59f0-44e6-a33b-ae877c648c67"}]': finished 2024-09-16T00:06:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:22 smithi193 ceph-mon[35484]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:06:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:22 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3700506891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:06:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:22 smithi193 ceph-mon[35484]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:23.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:22 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/629398922' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f34ec822-59f0-44e6-a33b-ae877c648c67"}]: dispatch 2024-09-16T00:06:23.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:06:23.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:22 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/629398922' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f34ec822-59f0-44e6-a33b-ae877c648c67"}]': finished 2024-09-16T00:06:23.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:22 smithi078 ceph-mon[31157]: osdmap e20: 4 total, 3 up, 4 in 2024-09-16T00:06:23.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:23.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:22 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3700506891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:06:23.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:22 smithi078 ceph-mon[31157]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:23.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:23.204 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:06:22 smithi148 sudo[43747]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:06:23.204 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:06:22 smithi148 sudo[43747]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:23.204 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:06:22 smithi148 sudo[43747]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:23.205 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:06:22 smithi148 sudo[43747]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:23.378 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:23 smithi193 sudo[44548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:06:23.378 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:23 smithi193 sudo[44548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:23.378 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:23 smithi193 sudo[44548]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:23.378 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:23 smithi193 sudo[44548]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:23.680 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:23 smithi193 sudo[44551]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:06:23.680 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:23 smithi193 sudo[44551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:23.681 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:23 smithi193 sudo[44551]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:23.681 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:06:23 smithi193 sudo[44551]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:23.898 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 sudo[50251]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:06:23.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 sudo[50251]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:23.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 sudo[50251]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:23.899 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 sudo[50251]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:24.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:23 smithi193 ceph-mon[35484]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:06:24.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:23 smithi193 ceph-mon[35484]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:06:24.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:24.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:24.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:24.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:23 smithi193 ceph-mon[35484]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:23 smithi148 ceph-mon[35599]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:06:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:23 smithi148 ceph-mon[35599]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:06:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:24.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:23 smithi148 ceph-mon[35599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:24.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 sudo[50437]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 sudo[50437]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 sudo[50437]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 sudo[50437]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 ceph-mon[31157]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 ceph-mon[31157]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:24.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:23 smithi078 ceph-mon[31157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:24.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 sudo[44554]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:06:24.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 sudo[44554]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:24.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 sudo[44554]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:24.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 sudo[44554]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:24.927 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 sudo[43750]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:06:24.927 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 sudo[43750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:24.927 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 sudo[43750]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:24.927 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 sudo[43750]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:24.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 sudo[44557]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:06:24.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 sudo[44557]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:24.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 sudo[44557]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:24.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 sudo[44557]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 sudo[43753]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 sudo[43753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 sudo[43753]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 sudo[43753]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:25.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:25.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:25.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:25.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:25.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:25.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:25.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:25.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:25.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:25.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:25.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:25.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:25.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:06:25.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:06:25.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:06:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:25 smithi148 ceph-mon[35599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:25 smithi148 ceph-mon[35599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:26.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:25 smithi078 ceph-mon[31157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:26.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:25 smithi078 ceph-mon[31157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:26.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:25 smithi193 ceph-mon[35484]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:06:26.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:25 smithi193 ceph-mon[35484]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:06:27.203 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:26 smithi078 ceph-mon[31157]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:26 smithi148 ceph-mon[35599]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:27.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:26 smithi193 ceph-mon[35484]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:29.500 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:29 smithi078 ceph-mon[31157]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:29.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:29 smithi193 ceph-mon[35484]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:29.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:29 smithi148 ceph-mon[35599]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:30.673 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:06:30.673 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:30.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:06:30.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:30.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:06:30.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:31 smithi193 ceph-mon[35484]: Deploying daemon osd.3 on smithi078 2024-09-16T00:06:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:31 smithi193 ceph-mon[35484]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:31 smithi148 ceph-mon[35599]: Deploying daemon osd.3 on smithi078 2024-09-16T00:06:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:31 smithi148 ceph-mon[35599]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:31.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:31 smithi078 ceph-mon[31157]: Deploying daemon osd.3 on smithi078 2024-09-16T00:06:31.746 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:31 smithi078 ceph-mon[31157]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:33.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:33 smithi078 ceph-mon[31157]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:33 smithi193 ceph-mon[35484]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:33.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:33 smithi148 ceph-mon[35599]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:35.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:35 smithi078 ceph-mon[31157]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:35.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:35 smithi193 ceph-mon[35484]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:35.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:35 smithi148 ceph-mon[35599]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:37.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:37 smithi078 ceph-mon[31157]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:37.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:37 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:37.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:37 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:37.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:37 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:37.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:37 smithi193 ceph-mon[35484]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:37.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:37 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:37.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:37 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:37.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:37 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:37.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:37 smithi148 ceph-mon[35599]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:37.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:37 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:37 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:37 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:39 smithi078 ceph-mon[31157]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:39.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:39.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:39.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:39 smithi193 ceph-mon[35484]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:39 smithi148 ceph-mon[35599]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:39.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:40.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.727+0000 7fb65effd640 1 -- 172.21.15.78:0/1351260701 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb66006a750 con 0x7fb6480a31e0 2024-09-16T00:06:40.729 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 3 on host 'smithi078' 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6480a31e0 msgr2=0x7fb6480a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6480a31e0 0x7fb6480a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb6580314e0 tx=0x7fb658000bc0 comp rx=0 tx=0).stop 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660101cd0 msgr2=0x7fb66019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660101cd0 0x7fb66019a710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb65400c890 tx=0x7fb65400cd50 comp rx=0 tx=0).stop 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 shutdown_connections 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6480a31e0 0x7fb6480a56a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb660102690 0x7fb6601a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb660101cd0 0x7fb66019a710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 --2- 172.21.15.78:0/1351260701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb660100ad0 0x7fb66019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:40.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.729+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 >> 172.21.15.78:0/1351260701 conn(0x7fb6600fc280 msgr2=0x7fb66006a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:06:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.730+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 shutdown_connections 2024-09-16T00:06:40.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:06:40.730+0000 7fb667b18640 1 -- 172.21.15.78:0/1351260701 wait complete. 2024-09-16T00:06:41.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:41 smithi078 ceph-mon[31157]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:41.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:41.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:41.471 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:41.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:41 smithi193 ceph-mon[35484]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:41.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:41.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:41.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:41.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:41 smithi148 ceph-mon[35599]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:41.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:06:41.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:41.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:42.146 DEBUG:teuthology.orchestra.run.smithi078:osd.3> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T00:06:42.148 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_3... 2024-09-16T00:06:42.149 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:06:42.405 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:06:42.831 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:06:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:06:42.584+0000 7fb8f6fb4640 -1 osd.3 0 log_to_monitors true 2024-09-16T00:06:43.537 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:43 smithi078 ceph-mon[31157]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:43.537 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:43 smithi078 ceph-mon[31157]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:06:43.546 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:43 smithi148 ceph-mon[35599]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:43.547 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:43 smithi148 ceph-mon[35599]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:06:43.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:43 smithi193 ceph-mon[35484]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:43.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:43 smithi193 ceph-mon[35484]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:06:44.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:06:44.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:06:44.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:44 smithi148 ceph-mon[35599]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:06:44.520 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:06:44.412+0000 7fb8f2f37640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T00:06:44.520 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:06:44.416+0000 7fb8eed5f640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:06:44.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:06:44.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:06:44.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:06:44.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:44.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:44.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:44.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:44.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:44.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:44 smithi078 ceph-mon[31157]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:06:44.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:06:44.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:06:44.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:06:44.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:44.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:44.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:06:44.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:44.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:06:44.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:06:44.683 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:44 smithi193 ceph-mon[35484]: from='osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:06:45.074 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:06:45.105 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-09-16T00:06:45.352 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:06:45.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:45 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi078 2024-09-16T00:06:45.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:45 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-16T00:06:45.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:45 smithi193 ceph-mon[35484]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:45.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:45 smithi193 ceph-mon[35484]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:06:45.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:45 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:45.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:45 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:45.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:45 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi078 2024-09-16T00:06:45.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:45 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-16T00:06:45.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:45 smithi148 ceph-mon[35599]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:45.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:45 smithi148 ceph-mon[35599]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:06:45.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:45 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:45.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:45 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:45 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi078 2024-09-16T00:06:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:45 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi078 to 8544M 2024-09-16T00:06:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:45 smithi078 ceph-mon[31157]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:45 smithi078 ceph-mon[31157]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:06:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:45 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:45 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:45.817 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 -- 172.21.15.148:0/1362907590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001017a0 msgr2=0x7fd500109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 --2- 172.21.15.148:0/1362907590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001017a0 0x7fd500109040 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009ca0 tx=0x7fd4f802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 -- 172.21.15.148:0/1362907590 shutdown_connections 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 --2- 172.21.15.148:0/1362907590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001017a0 0x7fd500109040 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 --2- 172.21.15.148:0/1362907590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd500100de0 0x7fd500101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 --2- 172.21.15.148:0/1362907590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5001020a0 0x7fd5001024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 -- 172.21.15.148:0/1362907590 >> 172.21.15.148:0/1362907590 conn(0x7fd5000fba40 msgr2=0x7fd5000fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 -- 172.21.15.148:0/1362907590 shutdown_connections 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.790+0000 7fd50717c640 1 -- 172.21.15.148:0/1362907590 wait complete. 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd50717c640 1 Processor -- start 2024-09-16T00:06:45.818 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd50717c640 1 -- start start 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd50717c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd500100de0 0x7fd500197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd50717c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5001017a0 0x7fd500198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd50717c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001020a0 0x7fd50019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd50717c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd50010d8d0 con 0x7fd5001017a0 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd50717c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd50010d750 con 0x7fd5001020a0 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd50717c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd50010da50 con 0x7fd500100de0 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd505719640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001020a0 0x7fd50019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd505719640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001020a0 0x7fd50019f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58100/0 (socket says 172.21.15.148:58100) 2024-09-16T00:06:45.819 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.791+0000 7fd505719640 1 -- 172.21.15.148:0/3957826812 learned_addr learned my addr 172.21.15.148:0/3957826812 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd504f18640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd500100de0 0x7fd500197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd505719640 1 -- 172.21.15.148:0/3957826812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd500100de0 msgr2=0x7fd500197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd505719640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd500100de0 0x7fd500197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd505719640 1 -- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5001017a0 msgr2=0x7fd500198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd505719640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5001017a0 0x7fd500198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd505719640 1 -- 172.21.15.148:0/3957826812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50019fb70 con 0x7fd5001020a0 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd505719640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001020a0 0x7fd50019f500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8031ee0 tx=0x7fd4f8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd4fdffb640 1 -- 172.21.15.148:0/3957826812 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8003c30 con 0x7fd5001020a0 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50019fe00 con 0x7fd5001020a0 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd4fdffb640 1 -- 172.21.15.148:0/3957826812 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f8003dd0 con 0x7fd5001020a0 2024-09-16T00:06:45.820 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd4fdffb640 1 -- 172.21.15.148:0/3957826812 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f802fb30 con 0x7fd5001020a0 2024-09-16T00:06:45.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.792+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5001a0360 con 0x7fd5001020a0 2024-09-16T00:06:45.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.793+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd500108060 con 0x7fd5001020a0 2024-09-16T00:06:45.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.795+0000 7fd4fdffb640 1 -- 172.21.15.148:0/3957826812 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd4f802fd70 con 0x7fd5001020a0 2024-09-16T00:06:45.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.795+0000 7fd4fdffb640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4f00a3270 0x7fd4f00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:06:45.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.795+0000 7fd4fdffb640 1 -- 172.21.15.148:0/3957826812 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7fd4f80efb30 con 0x7fd5001020a0 2024-09-16T00:06:45.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.796+0000 7fd504f18640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4f00a3270 0x7fd4f00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:06:45.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.796+0000 7fd504f18640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4f00a3270 0x7fd4f00a5730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4005fd0 tx=0x7fd4f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:06:45.821 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.799+0000 7fd4fdffb640 1 -- 172.21.15.148:0/3957826812 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4f80bdb70 con 0x7fd5001020a0 2024-09-16T00:06:45.973 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:06:45.972+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd500062630 con 0x7fd4f00a3270 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] boot 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:46.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:46 smithi193 ceph-mon[35484]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:06:46.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:06:46.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:06:46.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:46.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] boot 2024-09-16T00:06:46.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T00:06:46.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:46.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:06:46.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:06:46.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:46.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:46 smithi148 ceph-mon[35599]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: osd.3 [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] boot 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: osdmap e24: 4 total, 4 up, 4 in 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:06:46.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:46.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:46 smithi078 ceph-mon[31157]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:06:47.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:47 smithi193 ceph-mon[35484]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:06:47.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:47 smithi193 ceph-mon[35484]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:47.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:47 smithi193 ceph-mon[35484]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:06:47.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:47 smithi148 ceph-mon[35599]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:06:47.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:47 smithi148 ceph-mon[35599]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:47.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:47 smithi148 ceph-mon[35599]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:06:47.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:47 smithi078 ceph-mon[31157]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:06:47.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:47 smithi078 ceph-mon[31157]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:06:47.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:47 smithi078 ceph-mon[31157]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:06:48.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:48 smithi148 ceph-mon[35599]: from='client.? 172.21.15.148:0/1707667212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]: dispatch 2024-09-16T00:06:48.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:48 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]: dispatch 2024-09-16T00:06:48.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:48 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]': finished 2024-09-16T00:06:48.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:48 smithi148 ceph-mon[35599]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:06:48.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:06:48.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:48 smithi078 ceph-mon[31157]: from='client.? 172.21.15.148:0/1707667212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]: dispatch 2024-09-16T00:06:48.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:48 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]: dispatch 2024-09-16T00:06:48.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:48 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]': finished 2024-09-16T00:06:48.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:48 smithi078 ceph-mon[31157]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:06:48.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:06:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:48 smithi193 ceph-mon[35484]: from='client.? 172.21.15.148:0/1707667212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]: dispatch 2024-09-16T00:06:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:48 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]: dispatch 2024-09-16T00:06:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:48 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "928fd68c-669c-4de2-aa53-11d764c071ed"}]': finished 2024-09-16T00:06:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:48 smithi193 ceph-mon[35484]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:06:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:06:49.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:49 smithi148 ceph-mon[35599]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:06:49.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:49 smithi148 ceph-mon[35599]: from='client.? 172.21.15.148:0/1087427844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:06:49.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:49 smithi148 ceph-mon[35599]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:06:49.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:06:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:49 smithi078 ceph-mon[31157]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:06:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:49 smithi078 ceph-mon[31157]: from='client.? 172.21.15.148:0/1087427844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:06:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:49 smithi078 ceph-mon[31157]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:06:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:06:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:49 smithi193 ceph-mon[35484]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:06:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:49 smithi193 ceph-mon[35484]: from='client.? 172.21.15.148:0/1087427844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:06:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:49 smithi193 ceph-mon[35484]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:06:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:06:51.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:51 smithi148 ceph-mon[35599]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-16T00:06:51.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:51 smithi078 ceph-mon[31157]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-16T00:06:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:51 smithi193 ceph-mon[35484]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-16T00:06:53.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:53 smithi148 ceph-mon[35599]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:06:53.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:53 smithi078 ceph-mon[31157]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:06:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:53 smithi193 ceph-mon[35484]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:06:55.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:55 smithi148 ceph-mon[35599]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T00:06:55.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:55 smithi078 ceph-mon[31157]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T00:06:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:55 smithi193 ceph-mon[35484]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-16T00:06:57.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:57 smithi148 ceph-mon[35599]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:06:57.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:06:57.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:57 smithi078 ceph-mon[31157]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:06:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:06:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:57 smithi193 ceph-mon[35484]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:06:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:06:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:06:58.745 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:58 smithi148 ceph-mon[35599]: Deploying daemon osd.4 on smithi148 2024-09-16T00:06:58.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:58 smithi078 ceph-mon[31157]: Deploying daemon osd.4 on smithi148 2024-09-16T00:06:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:58 smithi193 ceph-mon[35484]: Deploying daemon osd.4 on smithi148 2024-09-16T00:06:59.678 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:06:59 smithi148 ceph-mon[35599]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:06:59.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:06:59 smithi078 ceph-mon[31157]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:06:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:06:59 smithi193 ceph-mon[35484]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:07:01.746 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:01 smithi148 ceph-mon[35599]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-16T00:07:01.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:01 smithi078 ceph-mon[31157]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-16T00:07:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:01 smithi193 ceph-mon[35484]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-16T00:07:03.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:03 smithi148 ceph-mon[35599]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:03.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:03.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:03.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:03.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:03 smithi078 ceph-mon[31157]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:03.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:03.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:03.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:03 smithi193 ceph-mon[35484]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:03.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:03.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:05.589 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:05 smithi148 ceph-mon[35599]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:05 smithi078 ceph-mon[31157]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:05 smithi193 ceph-mon[35484]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:06.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:06 smithi148 ceph-mon[35599]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:07.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:07.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:07.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:07.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:07.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:07.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:06 smithi078 ceph-mon[31157]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:07.050 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.048+0000 7fd4fdffb640 1 -- 172.21.15.148:0/3957826812 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd500062630 con 0x7fd4f00a3270 2024-09-16T00:07:07.050 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2024-09-16T00:07:07.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4f00a3270 msgr2=0x7fd4f00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:07.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4f00a3270 0x7fd4f00a5730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4005fd0 tx=0x7fd4f400b390 comp rx=0 tx=0).stop 2024-09-16T00:07:07.052 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001020a0 msgr2=0x7fd50019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001020a0 0x7fd50019f500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8031ee0 tx=0x7fd4f8004ab0 comp rx=0 tx=0).stop 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 shutdown_connections 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4f00a3270 0x7fd4f00a5730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5001020a0 0x7fd50019f500 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5001017a0 0x7fd500198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 --2- 172.21.15.148:0/3957826812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd500100de0 0x7fd500197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 >> 172.21.15.148:0/3957826812 conn(0x7fd5000fba40 msgr2=0x7fd500105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 shutdown_connections 2024-09-16T00:07:07.053 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:07:07.051+0000 7fd50717c640 1 -- 172.21.15.148:0/3957826812 wait complete. 2024-09-16T00:07:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:06 smithi193 ceph-mon[35484]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:07.748 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T00:07:07.750 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-09-16T00:07:07.750 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:07:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:08.008 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:07:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:08.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:08.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:08.137 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:08.890 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:08 smithi148 ceph-mon[35599]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:08.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:08 smithi193 ceph-mon[35484]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:09.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:08 smithi078 ceph-mon[31157]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:09.172 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:07:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4[46507]: 2024-09-16T00:07:09.023+0000 7f5256b5a640 -1 osd.4 0 log_to_monitors true 2024-09-16T00:07:09.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:09 smithi078 ceph-mon[31157]: from='osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:07:09.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:09 smithi078 ceph-mon[31157]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:07:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:09 smithi193 ceph-mon[35484]: from='osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:07:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:09 smithi193 ceph-mon[35484]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:07:10.184 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:09 smithi148 ceph-mon[35599]: from='osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:07:10.184 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:09 smithi148 ceph-mon[35599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:07:10.582 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:07:10.608 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-09-16T00:07:10.861 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:07:10.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:07:10.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi148 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi148 to 8544M 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:10.891 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:10.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:10.954 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4[46507]: 2024-09-16T00:07:10.728+0000 7f5252add640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T00:07:10.954 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4[46507]: 2024-09-16T00:07:10.734+0000 7f524e905640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:10.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi148 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi148 to 8544M 2024-09-16T00:07:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:10.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:10.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:10.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:11.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:07:11.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:07:11.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:07:11.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:11.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:07:11.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:11.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi148 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi148 to 8544M 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:11.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:11.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 -- 172.21.15.193:0/4080010764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102df0 msgr2=0x7f35e8103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 --2- 172.21.15.193:0/4080010764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102df0 0x7f35e8103270 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009900 tx=0x7f35d802a410 comp rx=0 tx=0).stop 2024-09-16T00:07:11.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 -- 172.21.15.193:0/4080010764 shutdown_connections 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 --2- 172.21.15.193:0/4080010764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e81037b0 0x7f35e810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 --2- 172.21.15.193:0/4080010764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8102df0 0x7f35e8103270 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 --2- 172.21.15.193:0/4080010764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e8101bf0 0x7f35e8101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 -- 172.21.15.193:0/4080010764 >> 172.21.15.193:0/4080010764 conn(0x7f35e80fd3c0 msgr2=0x7f35e80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 -- 172.21.15.193:0/4080010764 shutdown_connections 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.350+0000 7f35ed61f640 1 -- 172.21.15.193:0/4080010764 wait complete. 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35ed61f640 1 Processor -- start 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35ed61f640 1 -- start start 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35ed61f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8101bf0 0x7f35e8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35ed61f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e8102df0 0x7f35e8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.358 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35ed61f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81037b0 0x7f35e819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35ed61f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35e810d970 con 0x7f35e81037b0 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35ed61f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35e810d7f0 con 0x7f35e8102df0 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35ed61f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35e810daf0 con 0x7f35e8101bf0 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35e6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8101bf0 0x7f35e8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35e6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8101bf0 0x7f35e8197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:56134/0 (socket says 172.21.15.193:56134) 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35e6ffd640 1 -- 172.21.15.193:0/1293185494 learned_addr learned my addr 172.21.15.193:0/1293185494 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.351+0000 7f35e67fc640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e8102df0 0x7f35e8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35e6ffd640 1 -- 172.21.15.193:0/1293185494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e8102df0 msgr2=0x7f35e8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35e6ffd640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e8102df0 0x7f35e8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35e6ffd640 1 -- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81037b0 msgr2=0x7f35e819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35e6ffd640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81037b0 0x7f35e819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.359 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35e6ffd640 1 -- 172.21.15.193:0/1293185494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e819fb70 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35e6ffd640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8101bf0 0x7f35e8197f40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00b450 tx=0x7f35dc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35cffff640 1 -- 172.21.15.193:0/1293185494 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc015020 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35cffff640 1 -- 172.21.15.193:0/1293185494 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35dc011d70 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819fe60 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35cffff640 1 -- 172.21.15.193:0/1293185494 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00f340 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.352+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a0330 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.353+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e8108f20 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.354+0000 7f35cffff640 1 -- 172.21.15.193:0/1293185494 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f35dc018020 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.355+0000 7f35cffff640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35d00a3270 0x7f35d00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.355+0000 7f35cffff640 1 -- 172.21.15.193:0/1293185494 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f35dc0ce7a0 con 0x7f35e8101bf0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.355+0000 7f35e67fc640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35d00a3270 0x7f35d00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.360 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.356+0000 7f35e67fc640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35d00a3270 0x7f35d00a5730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006010 tx=0x7f35d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.363 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.362+0000 7f35cffff640 1 -- 172.21.15.193:0/1293185494 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35dc01f020 con 0x7f35e8101bf0 2024-09-16T00:07:11.543 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:11.542+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f35e8062630 con 0x7f35d00a3270 2024-09-16T00:07:11.888 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:07:11.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:07:11.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:11.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:11.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:07:11.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:07:11.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:07:11.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:11.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:12.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:07:12.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:07:12.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:12.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:12.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:07:12.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:07:12.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:07:12.123 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:12.123 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:12.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:12 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:07:12.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:12 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:07:12.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:12 smithi193 ceph-mon[35484]: osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] boot 2024-09-16T00:07:12.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:12 smithi193 ceph-mon[35484]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T00:07:12.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:12 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:12.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:12 smithi193 ceph-mon[35484]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:12 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:07:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:12 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:07:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:12 smithi078 ceph-mon[31157]: osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] boot 2024-09-16T00:07:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:12 smithi078 ceph-mon[31157]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T00:07:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:12 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:12 smithi078 ceph-mon[31157]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:13.074 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:12 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:07:13.074 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:12 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:07:13.074 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:12 smithi148 ceph-mon[35599]: osd.4 [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] boot 2024-09-16T00:07:13.074 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:12 smithi148 ceph-mon[35599]: osdmap e31: 5 total, 5 up, 5 in 2024-09-16T00:07:13.074 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:12 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:07:13.074 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:12 smithi148 ceph-mon[35599]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:07:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:13 smithi193 ceph-mon[35484]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:07:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:13 smithi078 ceph-mon[31157]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:07:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:13 smithi148 ceph-mon[35599]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:07:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:14 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/3840904201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]: dispatch 2024-09-16T00:07:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:14 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]: dispatch 2024-09-16T00:07:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:14 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]': finished 2024-09-16T00:07:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:14 smithi193 ceph-mon[35484]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T00:07:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:14 smithi193 ceph-mon[35484]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:14.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:14 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/2863391739' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:07:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:14 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/3840904201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]: dispatch 2024-09-16T00:07:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:14 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]: dispatch 2024-09-16T00:07:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:14 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]': finished 2024-09-16T00:07:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:14 smithi078 ceph-mon[31157]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T00:07:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:14 smithi078 ceph-mon[31157]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:15.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:14 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/2863391739' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:07:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:14 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/3840904201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]: dispatch 2024-09-16T00:07:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:14 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]: dispatch 2024-09-16T00:07:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:14 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "538b1b72-96f3-41ed-bfc6-9796b5ae0578"}]': finished 2024-09-16T00:07:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:14 smithi148 ceph-mon[35599]: osdmap e33: 6 total, 5 up, 6 in 2024-09-16T00:07:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:15.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:14 smithi148 ceph-mon[35599]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:15.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:14 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/2863391739' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:07:17.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:17 smithi193 ceph-mon[35484]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:17.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:17 smithi148 ceph-mon[35599]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:17.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:17 smithi078 ceph-mon[31157]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:19 smithi193 ceph-mon[35484]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:19.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:19 smithi148 ceph-mon[35599]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:19 smithi078 ceph-mon[31157]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:21.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:21 smithi193 ceph-mon[35484]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:21.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:21 smithi148 ceph-mon[35599]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:21 smithi078 ceph-mon[31157]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:22.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:07:22.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:22.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:07:22.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:07:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:23.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:23 smithi193 ceph-mon[35484]: Deploying daemon osd.5 on smithi193 2024-09-16T00:07:23.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:23 smithi193 ceph-mon[35484]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:23.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:23 smithi148 ceph-mon[35599]: Deploying daemon osd.5 on smithi193 2024-09-16T00:07:23.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:23 smithi148 ceph-mon[35599]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:23 smithi078 ceph-mon[31157]: Deploying daemon osd.5 on smithi193 2024-09-16T00:07:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:23 smithi078 ceph-mon[31157]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:25.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:25 smithi078 ceph-mon[31157]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:25.541 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:25 smithi193 ceph-mon[35484]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:25.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:25 smithi148 ceph-mon[35599]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:27 smithi193 ceph-mon[35484]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:27.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:27 smithi148 ceph-mon[35599]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:27 smithi078 ceph-mon[31157]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:28.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:28.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:28.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:28.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:28.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:28.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:29.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:29 smithi193 ceph-mon[35484]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:29.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:29 smithi148 ceph-mon[35599]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:29.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:29 smithi078 ceph-mon[31157]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:31 smithi193 ceph-mon[35484]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:31.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:31.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:31 smithi148 ceph-mon[35599]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:31.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:31 smithi078 ceph-mon[31157]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:32.142 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.141+0000 7f35cffff640 1 -- 172.21.15.193:0/1293185494 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f35e8062630 con 0x7f35d00a3270 2024-09-16T00:07:32.143 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35d00a3270 msgr2=0x7f35d00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35d00a3270 0x7f35d00a5730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006010 tx=0x7f35d8000bc0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8101bf0 msgr2=0x7f35e8197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8101bf0 0x7f35e8197f40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00b450 tx=0x7f35dc00b910 comp rx=0 tx=0).stop 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 shutdown_connections 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35d00a3270 0x7f35d00a5730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35e81037b0 0x7f35e819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35e8102df0 0x7f35e8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.145 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 --2- 172.21.15.193:0/1293185494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35e8101bf0 0x7f35e8197f40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.146 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 >> 172.21.15.193:0/1293185494 conn(0x7f35e80fd3c0 msgr2=0x7f35e81060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:32.146 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 shutdown_connections 2024-09-16T00:07:32.146 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:07:32.144+0000 7f35ed61f640 1 -- 172.21.15.193:0/1293185494 wait complete. 2024-09-16T00:07:32.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:32 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:32.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:32 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:32.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:32 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:32.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:32 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:32.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:32 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:32.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:32 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:32.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:32 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:32.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:32 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:32.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:32 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:32.871 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T00:07:32.874 INFO:tasks.cephadm:Deploying osd.6 on smithi078 with /dev/vg_nvme/lv_2... 2024-09-16T00:07:32.875 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:07:33.130 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:07:33.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:33 smithi078 ceph-mon[31157]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:33 smithi193 ceph-mon[35484]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:33.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:33 smithi148 ceph-mon[35599]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:34.003 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:07:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:07:33.842+0000 7f1b83465640 -1 osd.5 0 log_to_monitors true 2024-09-16T00:07:34.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:34 smithi078 ceph-mon[31157]: from='osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:07:34.621 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:34 smithi078 ceph-mon[31157]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:07:34.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:34 smithi193 ceph-mon[35484]: from='osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:07:34.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:34 smithi193 ceph-mon[35484]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:07:34.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:34 smithi148 ceph-mon[35599]: from='osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:07:34.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:34 smithi148 ceph-mon[35599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:07:35.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:35 smithi193 ceph-mon[35484]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:35.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:35 smithi193 ceph-mon[35484]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:07:35.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:35 smithi193 ceph-mon[35484]: from='osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:07:35.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:35 smithi193 ceph-mon[35484]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:07:35.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:35 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:35.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:35 smithi193 ceph-mon[35484]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:07:35.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:35 smithi148 ceph-mon[35599]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:35.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:35 smithi148 ceph-mon[35599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:07:35.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:35 smithi148 ceph-mon[35599]: from='osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:07:35.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:35 smithi148 ceph-mon[35599]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:07:35.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:35 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:35.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:35 smithi148 ceph-mon[35599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:07:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:35 smithi078 ceph-mon[31157]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:35 smithi078 ceph-mon[31157]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:07:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:35 smithi078 ceph-mon[31157]: from='osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:07:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:35 smithi078 ceph-mon[31157]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:07:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:35 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:35 smithi078 ceph-mon[31157]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:07:35.818 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:07:35.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_2 2024-09-16T00:07:36.110 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:07:36.430 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:07:36.087+0000 7f1b7f3e8640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T00:07:36.430 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:07:36.093+0000 7f1b77208640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:36.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:07:36.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:07:36.483 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:36.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:36.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:36.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:36.484 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:36.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:36.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='osd.5 ' entity='osd.5' 2024-09-16T00:07:36.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] boot 2024-09-16T00:07:36.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T00:07:36.485 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:36.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 -- 172.21.15.78:0/1694057276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f70101c00 msgr2=0x7f8f70102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/1694057276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f70101c00 0x7f8f70102000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009900 tx=0x7f8f6002a410 comp rx=0 tx=0).stop 2024-09-16T00:07:36.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 -- 172.21.15.78:0/1694057276 shutdown_connections 2024-09-16T00:07:36.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/1694057276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f701037c0 0x7f8f7010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/1694057276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f70102e00 0x7f8f70103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/1694057276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f70101c00 0x7f8f70102000 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 -- 172.21.15.78:0/1694057276 >> 172.21.15.78:0/1694057276 conn(0x7f8f700fd390 msgr2=0x7f8f700ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 -- 172.21.15.78:0/1694057276 shutdown_connections 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.687+0000 7f8f77dcb640 1 -- 172.21.15.78:0/1694057276 wait complete. 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f77dcb640 1 Processor -- start 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f77dcb640 1 -- start start 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f77dcb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f70101c00 0x7f8f70197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f77dcb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f70102e00 0x7f8f70198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f77dcb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f701037c0 0x7f8f7019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f77dcb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7010d860 con 0x7f8f701037c0 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f77dcb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7010d6e0 con 0x7f8f70102e00 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f77dcb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7010d9e0 con 0x7f8f70101c00 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f76368640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f701037c0 0x7f8f7019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f76368640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f701037c0 0x7f8f7019f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57034/0 (socket says 172.21.15.78:57034) 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f76368640 1 -- 172.21.15.78:0/676861936 learned_addr learned my addr 172.21.15.78:0/676861936 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f76368640 1 -- 172.21.15.78:0/676861936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f70101c00 msgr2=0x7f8f70197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f76368640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f70101c00 0x7f8f70197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f76368640 1 -- 172.21.15.78:0/676861936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f70102e00 msgr2=0x7f8f70198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f76368640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f70102e00 0x7f8f70198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.688+0000 7f8f76368640 1 -- 172.21.15.78:0/676861936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7019fb30 con 0x7f8f701037c0 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.689+0000 7f8f76368640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f701037c0 0x7f8f7019f4c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800eaa0 tx=0x7f8f6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.689+0000 7f8f66ffd640 1 -- 172.21.15.78:0/676861936 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6800cdb0 con 0x7f8f701037c0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.689+0000 7f8f66ffd640 1 -- 172.21.15.78:0/676861936 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f68005020 con 0x7f8f701037c0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.689+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7019fe20 con 0x7f8f701037c0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.689+0000 7f8f66ffd640 1 -- 172.21.15.78:0/676861936 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f680116b0 con 0x7f8f701037c0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.689+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f701a02e0 con 0x7f8f701037c0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.690+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f70109000 con 0x7f8f701037c0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.691+0000 7f8f66ffd640 1 -- 172.21.15.78:0/676861936 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8f68005560 con 0x7f8f701037c0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.692+0000 7f8f66ffd640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8f5c0a3270 0x7f8f5c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.692+0000 7f8f75b67640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8f5c0a3270 0x7f8f5c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.692+0000 7f8f66ffd640 1 -- 172.21.15.78:0/676861936 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4566+0+0 (secure 0 0 0) 0x7f8f680cf050 con 0x7f8f701037c0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.692+0000 7f8f75b67640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8f5c0a3270 0x7f8f5c0a5730 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f600096c0 tx=0x7f8f60031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.695+0000 7f8f66ffd640 1 -- 172.21.15.78:0/676861936 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f68016030 con 0x7f8f701037c0 2024-09-16T00:07:36.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:07:36.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='osd.5 ' entity='osd.5' 2024-09-16T00:07:36.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] boot 2024-09-16T00:07:36.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T00:07:36.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:36.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:36.872+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8f70062630 con 0x7f8f5c0a3270 2024-09-16T00:07:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:07:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:07:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='osd.5 ' entity='osd.5' 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: osd.5 [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] boot 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: osdmap e36: 6 total, 6 up, 6 in 2024-09-16T00:07:36.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:07:37.559 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:07:37.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:07:37.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi193 2024-09-16T00:07:37.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-16T00:07:37.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:37.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:07:37.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:07:37.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:37.560 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:37 smithi078 ceph-mon[31157]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:07:37.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:07:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:07:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi193 2024-09-16T00:07:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-16T00:07:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:07:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:07:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:37.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:37 smithi148 ceph-mon[35599]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:07:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:07:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:07:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi193 2024-09-16T00:07:37.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-16T00:07:37.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:07:37.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:07:37.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:07:37.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:37.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:37 smithi193 ceph-mon[35484]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:07:38.622 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:38 smithi078 ceph-mon[31157]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:07:38.622 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:38 smithi078 ceph-mon[31157]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:07:38.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:38 smithi148 ceph-mon[35599]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:07:38.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:38 smithi148 ceph-mon[35599]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:07:38.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:38 smithi193 ceph-mon[35484]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:07:38.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:38 smithi193 ceph-mon[35484]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:07:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:39 smithi148 ceph-mon[35599]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:39 smithi148 ceph-mon[35599]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:07:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:39 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1548581310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d19221b9-f4f0-42ca-ae31-7b1935b52a50"}]: dispatch 2024-09-16T00:07:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:39 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1548581310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d19221b9-f4f0-42ca-ae31-7b1935b52a50"}]': finished 2024-09-16T00:07:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:39 smithi148 ceph-mon[35599]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:07:39.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:07:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:39 smithi078 ceph-mon[31157]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:39 smithi078 ceph-mon[31157]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:07:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:39 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1548581310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d19221b9-f4f0-42ca-ae31-7b1935b52a50"}]: dispatch 2024-09-16T00:07:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:39 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1548581310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d19221b9-f4f0-42ca-ae31-7b1935b52a50"}]': finished 2024-09-16T00:07:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:39 smithi078 ceph-mon[31157]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:07:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:07:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:39 smithi193 ceph-mon[35484]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:39 smithi193 ceph-mon[35484]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:07:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:39 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1548581310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d19221b9-f4f0-42ca-ae31-7b1935b52a50"}]: dispatch 2024-09-16T00:07:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:39 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1548581310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d19221b9-f4f0-42ca-ae31-7b1935b52a50"}]': finished 2024-09-16T00:07:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:39 smithi193 ceph-mon[35484]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:07:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:07:40.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:40 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1568547336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:07:40.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:40 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1568547336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:07:40.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:40 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1568547336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:07:41.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:41 smithi148 ceph-mon[35599]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:07:41.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:41 smithi078 ceph-mon[31157]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:07:41.831 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:41 smithi193 ceph-mon[35484]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:07:43.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:43 smithi148 ceph-mon[35599]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:07:43.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:43 smithi078 ceph-mon[31157]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:07:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:43 smithi193 ceph-mon[35484]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-09-16T00:07:45.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:45 smithi148 ceph-mon[35599]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:07:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:45 smithi078 ceph-mon[31157]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:07:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:45 smithi193 ceph-mon[35484]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-16T00:07:47.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:47 smithi078 ceph-mon[31157]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:07:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:47 smithi193 ceph-mon[35484]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:07:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:47 smithi148 ceph-mon[35599]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:07:48.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:07:48.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:48.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:48 smithi078 ceph-mon[31157]: Deploying daemon osd.6 on smithi078 2024-09-16T00:07:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:07:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:48 smithi193 ceph-mon[35484]: Deploying daemon osd.6 on smithi078 2024-09-16T00:07:48.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:07:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:48 smithi148 ceph-mon[35599]: Deploying daemon osd.6 on smithi078 2024-09-16T00:07:49.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:49 smithi078 ceph-mon[31157]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:07:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:49 smithi193 ceph-mon[35484]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:07:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:49 smithi148 ceph-mon[35599]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-16T00:07:51.528 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:51 smithi078 ceph-mon[31157]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:07:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:51 smithi193 ceph-mon[35484]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:07:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:51 smithi148 ceph-mon[35599]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:07:53.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:53 smithi148 ceph-mon[35599]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:53.737 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:53 smithi078 ceph-mon[31157]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:53 smithi193 ceph-mon[35484]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:54.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:54.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:54.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:55.772 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:55 smithi078 ceph-mon[31157]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:55 smithi193 ceph-mon[35484]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:55.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:55 smithi148 ceph-mon[35599]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:57.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:57 smithi078 ceph-mon[31157]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:57.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.852+0000 7f8f66ffd640 1 -- 172.21.15.78:0/676861936 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f70062630 con 0x7f8f5c0a3270 2024-09-16T00:07:57.854 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 6 on host 'smithi078' 2024-09-16T00:07:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8f5c0a3270 msgr2=0x7f8f5c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8f5c0a3270 0x7f8f5c0a5730 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f600096c0 tx=0x7f8f60031400 comp rx=0 tx=0).stop 2024-09-16T00:07:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f701037c0 msgr2=0x7f8f7019f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f701037c0 0x7f8f7019f4c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800eaa0 tx=0x7f8f6800ef60 comp rx=0 tx=0).stop 2024-09-16T00:07:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 shutdown_connections 2024-09-16T00:07:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8f5c0a3270 0x7f8f5c0a5730 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8f701037c0 0x7f8f7019f4c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8f70102e00 0x7f8f70198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 --2- 172.21.15.78:0/676861936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f70101c00 0x7f8f70197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 >> 172.21.15.78:0/676861936 conn(0x7f8f700fd390 msgr2=0x7f8f701061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 shutdown_connections 2024-09-16T00:07:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:07:57.854+0000 7f8f77dcb640 1 -- 172.21.15.78:0/676861936 wait complete. 2024-09-16T00:07:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:57 smithi193 ceph-mon[35484]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:57.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:57.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:57.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:57.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:57.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:57 smithi148 ceph-mon[35599]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:57.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:57.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:07:57.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:07:57.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:58.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:58.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:58.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:58.601 DEBUG:teuthology.orchestra.run.smithi078:osd.6> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T00:07:58.604 INFO:tasks.cephadm:Deploying osd.7 on smithi148 with /dev/vg_nvme/lv_2... 2024-09-16T00:07:58.604 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:07:58.861 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:07:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:58.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:07:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:07:59.477 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:07:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:07:59.146+0000 7f58d04c6640 -1 osd.6 0 log_to_monitors true 2024-09-16T00:07:59.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:59 smithi148 ceph-mon[35599]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:59.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:07:59 smithi148 ceph-mon[35599]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:07:59.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:59 smithi078 ceph-mon[31157]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:59.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:07:59 smithi078 ceph-mon[31157]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:07:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:59 smithi193 ceph-mon[35484]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:07:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:07:59 smithi193 ceph-mon[35484]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:08:00.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:00 smithi148 ceph-mon[35599]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:08:00.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:00 smithi148 ceph-mon[35599]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:08:00.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:00 smithi148 ceph-mon[35599]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:08:00.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:00.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:00 smithi148 ceph-mon[35599]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:08:01.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:00 smithi193 ceph-mon[35484]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:08:01.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:00 smithi193 ceph-mon[35484]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:08:01.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:00 smithi193 ceph-mon[35484]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:08:01.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:01.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:00 smithi193 ceph-mon[35484]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:08:01.214 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:00 smithi078 ceph-mon[31157]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:08:01.214 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:00 smithi078 ceph-mon[31157]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:08:01.214 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:00 smithi078 ceph-mon[31157]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:08:01.214 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:01.214 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:00 smithi078 ceph-mon[31157]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:08:01.215 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:08:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:08:00.861+0000 7f58cc449640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T00:08:01.215 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:08:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:08:00.873+0000 7f58c8271640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:08:01.366 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:08:01.392 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-09-16T00:08:01.647 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:08:01.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:08:01.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:08:01.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:01.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:01.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi078 2024-09-16T00:08:01.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:01.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:01.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:01.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:01.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:01.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-16T00:08:01.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:01.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:01.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:01.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:01.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:02.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3839606249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4081078c0 msgr2=0x7fd408109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3839606249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4081078c0 0x7fd408109d10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd400009ca0 tx=0x7fd40002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3839606249 shutdown_connections 2024-09-16T00:08:02.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3839606249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4081078c0 0x7fd408109d10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3839606249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd408104f90 0x7fd408107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3839606249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd408102660 0x7fd408104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3839606249 >> 172.21.15.148:0/3839606249 conn(0x7fd4080fc280 msgr2=0x7fd4080fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3839606249 shutdown_connections 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.109+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3839606249 wait complete. 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40fcbf640 1 Processor -- start 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40fcbf640 1 -- start start 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40fcbf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd408102660 0x7fd408197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40fcbf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd408104f90 0x7fd408198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.126 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40fcbf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4081078c0 0x7fd40819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40fcbf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810d860 con 0x7fd408104f90 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40fcbf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810d6e0 con 0x7fd408102660 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40fcbf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd40810d9e0 con 0x7fd4081078c0 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40da5b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd408102660 0x7fd408197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40da5b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd408102660 0x7fd408197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:47182/0 (socket says 172.21.15.148:47182) 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40da5b640 1 -- 172.21.15.148:0/3796952161 learned_addr learned my addr 172.21.15.148:0/3796952161 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.110+0000 7fd40d25a640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd408104f90 0x7fd408198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40e25c640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4081078c0 0x7fd40819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40da5b640 1 -- 172.21.15.148:0/3796952161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4081078c0 msgr2=0x7fd40819f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.127 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40da5b640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4081078c0 0x7fd40819f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40da5b640 1 -- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd408104f90 msgr2=0x7fd408198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40da5b640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd408104f90 0x7fd408198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40da5b640 1 -- 172.21.15.148:0/3796952161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40819fa60 con 0x7fd408102660 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40da5b640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd408102660 0x7fd408197e30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c920 tx=0x7fd40400cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd3feffd640 1 -- 172.21.15.148:0/3796952161 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404007c70 con 0x7fd408102660 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd3feffd640 1 -- 172.21.15.148:0/3796952161 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd404007e10 con 0x7fd408102660 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40819fd50 con 0x7fd408102660 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd3feffd640 1 -- 172.21.15.148:0/3796952161 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd404018890 con 0x7fd408102660 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.111+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4081a0200 con 0x7fd408102660 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.112+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4080fdd70 con 0x7fd408102660 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.114+0000 7fd3feffd640 1 -- 172.21.15.148:0/3796952161 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd404018a30 con 0x7fd408102660 2024-09-16T00:08:02.128 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.115+0000 7fd3feffd640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a32f0 0x7fd3f40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.115+0000 7fd3feffd640 1 -- 172.21.15.148:0/3796952161 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7fd4040cef40 con 0x7fd408102660 2024-09-16T00:08:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.115+0000 7fd40d25a640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a32f0 0x7fd3f40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.116+0000 7fd40d25a640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a32f0 0x7fd3f40a57b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd408199350 tx=0x7fd3f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.129 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.118+0000 7fd3feffd640 1 -- 172.21.15.148:0/3796952161 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd404014030 con 0x7fd408102660 2024-09-16T00:08:02.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:08:02.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:08:02.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:02.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:02.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi078 2024-09-16T00:08:02.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:02.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:02.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:08:02.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:08:02.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:02.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:02.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi078 2024-09-16T00:08:02.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:02.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi078 to 5696M 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:02.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:02.301 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:02.299+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd408062630 con 0x7fd3f40a32f0 2024-09-16T00:08:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:08:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:08:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] boot 2024-09-16T00:08:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: osdmap e43: 7 total, 7 up, 7 in 2024-09-16T00:08:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:02 smithi148 ceph-mon[35599]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:08:03.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:08:03.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:08:03.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] boot 2024-09-16T00:08:03.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: osdmap e43: 7 total, 7 up, 7 in 2024-09-16T00:08:03.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:03.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:03.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:03.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:03.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:03.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:02 smithi193 ceph-mon[35484]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:08:03.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:08:03.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:08:03.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: osd.6 [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] boot 2024-09-16T00:08:03.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: osdmap e43: 7 total, 7 up, 7 in 2024-09-16T00:08:03.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:08:03.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:03.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:03.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:03.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:03.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:02 smithi078 ceph-mon[31157]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:08:04.114 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:03 smithi148 ceph-mon[35599]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:08:04.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:03 smithi193 ceph-mon[35484]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:08:04.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:03 smithi078 ceph-mon[31157]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:08:05.130 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:04 smithi148 ceph-mon[35599]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:05.131 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:04 smithi148 ceph-mon[35599]: from='client.? 172.21.15.148:0/2263805474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]: dispatch 2024-09-16T00:08:05.131 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:04 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]: dispatch 2024-09-16T00:08:05.131 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:04 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]': finished 2024-09-16T00:08:05.131 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:04 smithi148 ceph-mon[35599]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T00:08:05.131 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:05.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:04 smithi193 ceph-mon[35484]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:05.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:04 smithi193 ceph-mon[35484]: from='client.? 172.21.15.148:0/2263805474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]: dispatch 2024-09-16T00:08:05.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:04 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]: dispatch 2024-09-16T00:08:05.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:04 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]': finished 2024-09-16T00:08:05.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:04 smithi193 ceph-mon[35484]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T00:08:05.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:05.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:04 smithi078 ceph-mon[31157]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:05.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:04 smithi078 ceph-mon[31157]: from='client.? 172.21.15.148:0/2263805474' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]: dispatch 2024-09-16T00:08:05.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:04 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]: dispatch 2024-09-16T00:08:05.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:04 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7efa41b-836b-419a-b2c0-8e2861cddeaa"}]': finished 2024-09-16T00:08:05.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:04 smithi078 ceph-mon[31157]: osdmap e45: 8 total, 7 up, 8 in 2024-09-16T00:08:05.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:06.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:05 smithi193 ceph-mon[35484]: from='client.? 172.21.15.148:0/2552412655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:05 smithi148 ceph-mon[35599]: from='client.? 172.21.15.148:0/2552412655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:05 smithi078 ceph-mon[31157]: from='client.? 172.21.15.148:0/2552412655' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:06 smithi193 ceph-mon[35484]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:07.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:06 smithi148 ceph-mon[35599]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:07.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:06 smithi078 ceph-mon[31157]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:09 smithi193 ceph-mon[35484]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:09.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:09 smithi148 ceph-mon[35599]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:09.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:09 smithi078 ceph-mon[31157]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:11.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:11 smithi193 ceph-mon[35484]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:11.680 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:11 smithi148 ceph-mon[35599]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:11.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:11 smithi078 ceph-mon[31157]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:13.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:13 smithi148 ceph-mon[35599]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:13.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:13 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:08:13.666 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:13 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:13.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:13 smithi193 ceph-mon[35484]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:13.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:13 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:08:13.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:13 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:13.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:13 smithi078 ceph-mon[31157]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:13.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:13 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:08:13.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:13 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:14.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:14 smithi193 ceph-mon[35484]: Deploying daemon osd.7 on smithi148 2024-09-16T00:08:14.686 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:14 smithi148 ceph-mon[35599]: Deploying daemon osd.7 on smithi148 2024-09-16T00:08:14.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:14 smithi078 ceph-mon[31157]: Deploying daemon osd.7 on smithi148 2024-09-16T00:08:15.543 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:15 smithi148 ceph-mon[35599]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:15.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:15 smithi193 ceph-mon[35484]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:15.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:15 smithi078 ceph-mon[31157]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:17.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:17 smithi193 ceph-mon[35484]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:17.685 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:17 smithi148 ceph-mon[35599]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:17.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:17 smithi078 ceph-mon[31157]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:19 smithi193 ceph-mon[35484]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:19.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:19.700 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:19 smithi148 ceph-mon[35599]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:19.700 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:19.700 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:19.700 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:19 smithi078 ceph-mon[31157]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:21.677 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:21 smithi148 ceph-mon[35599]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:21.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:21 smithi193 ceph-mon[35484]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:21 smithi078 ceph-mon[31157]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:22.899 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.898+0000 7fd3feffd640 1 -- 172.21.15.148:0/3796952161 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd408062630 con 0x7fd3f40a32f0 2024-09-16T00:08:22.899 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 7 on host 'smithi148' 2024-09-16T00:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a32f0 msgr2=0x7fd3f40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a32f0 0x7fd3f40a57b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd408199350 tx=0x7fd3f800b390 comp rx=0 tx=0).stop 2024-09-16T00:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd408102660 msgr2=0x7fd408197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.901 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd408102660 0x7fd408197e30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd40400c920 tx=0x7fd40400cde0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 shutdown_connections 2024-09-16T00:08:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a32f0 0x7fd3f40a57b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4081078c0 0x7fd40819f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd408104f90 0x7fd408198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 --2- 172.21.15.148:0/3796952161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd408102660 0x7fd408197e30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 >> 172.21.15.148:0/3796952161 conn(0x7fd4080fc280 msgr2=0x7fd408102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 shutdown_connections 2024-09-16T00:08:22.902 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:08:22.900+0000 7fd40fcbf640 1 -- 172.21.15.148:0/3796952161 wait complete. 2024-09-16T00:08:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:23.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:23.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:23.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:23.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:23.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:23.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:23.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:23.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:23.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:23.621 DEBUG:teuthology.orchestra.run.smithi148:osd.7> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T00:08:23.625 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-09-16T00:08:23.626 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:08:23.877 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:08:24.872 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:08:24 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:08:24.578+0000 7f4e8a24e640 -1 osd.7 0 log_to_monitors true 2024-09-16T00:08:25.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:25 smithi148 ceph-mon[35599]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:25.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:25 smithi148 ceph-mon[35599]: from='osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:08:25.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:25 smithi148 ceph-mon[35599]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:08:25.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:25 smithi193 ceph-mon[35484]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:25.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:25 smithi193 ceph-mon[35484]: from='osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:08:25.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:25 smithi193 ceph-mon[35484]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:08:25.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:25 smithi078 ceph-mon[31157]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:25.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:25 smithi078 ceph-mon[31157]: from='osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:08:25.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:25 smithi078 ceph-mon[31157]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:08:26.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:08:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:08:26.412+0000 7f4e861d1640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T00:08:26.609 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:08:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:08:26.421+0000 7f4e81ff9640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:08:26.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:26 smithi148 ceph-mon[35599]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:08:26.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:26 smithi148 ceph-mon[35599]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:08:26.609 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:26 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:26.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:26 smithi148 ceph-mon[35599]: from='osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:08:26.610 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:26 smithi148 ceph-mon[35599]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:08:26.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:26 smithi193 ceph-mon[35484]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:08:26.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:26 smithi193 ceph-mon[35484]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:08:26.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:26 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:26.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:26 smithi193 ceph-mon[35484]: from='osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:08:26.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:26 smithi193 ceph-mon[35484]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:08:26.751 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:08:26.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:26 smithi078 ceph-mon[31157]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:08:26.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:26 smithi078 ceph-mon[31157]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:08:26.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:26 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:26.902 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:26 smithi078 ceph-mon[31157]: from='osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:08:26.902 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:26 smithi078 ceph-mon[31157]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:08:26.903 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-09-16T00:08:27.146 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:08:27.635 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.636 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:27.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:27.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:27.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:27.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.728 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 -- 172.21.15.193:0/2982091776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20501037c0 msgr2=0x7f205010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:27.728 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 --2- 172.21.15.193:0/2982091776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20501037c0 0x7f205010a050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2048009ca0 tx=0x7f204802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.728 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 -- 172.21.15.193:0/2982091776 shutdown_connections 2024-09-16T00:08:27.728 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 --2- 172.21.15.193:0/2982091776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20501037c0 0x7f205010a050 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.728 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 --2- 172.21.15.193:0/2982091776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050102e00 0x7f2050103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.728 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 --2- 172.21.15.193:0/2982091776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2050101c00 0x7f2050102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 -- 172.21.15.193:0/2982091776 >> 172.21.15.193:0/2982091776 conn(0x7f20500fd390 msgr2=0x7f20500ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 -- 172.21.15.193:0/2982091776 shutdown_connections 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.717+0000 7f2057d64640 1 -- 172.21.15.193:0/2982091776 wait complete. 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2057d64640 1 Processor -- start 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2057d64640 1 -- start start 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2057d64640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2050101c00 0x7f2050197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2057d64640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050102e00 0x7f2050198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2057d64640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20501037c0 0x7f205019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2057d64640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f205010d880 con 0x7f2050102e00 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2057d64640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f205010d700 con 0x7f20501037c0 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2057d64640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f205010da00 con 0x7f2050101c00 2024-09-16T00:08:27.729 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2055b00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2050101c00 0x7f2050197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2055b00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2050101c00 0x7f2050197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:33704/0 (socket says 172.21.15.193:33704) 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2055b00640 1 -- 172.21.15.193:0/3177628756 learned_addr learned my addr 172.21.15.193:0/3177628756 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2056301640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20501037c0 0x7f205019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2055b00640 1 -- 172.21.15.193:0/3177628756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20501037c0 msgr2=0x7f205019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2055b00640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20501037c0 0x7f205019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.718+0000 7f2055b00640 1 -- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050102e00 msgr2=0x7f2050198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.719+0000 7f2055b00640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050102e00 0x7f2050198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.719+0000 7f2055b00640 1 -- 172.21.15.193:0/3177628756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205019fa80 con 0x7f2050101c00 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.719+0000 7f2055b00640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2050101c00 0x7f2050197e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f204c00d950 tx=0x7f204c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.719+0000 7f2046ffd640 1 -- 172.21.15.193:0/3177628756 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c017070 con 0x7f2050101c00 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.719+0000 7f2046ffd640 1 -- 172.21.15.193:0/3177628756 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f204c009c30 con 0x7f2050101c00 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.719+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205019fd70 con 0x7f2050101c00 2024-09-16T00:08:27.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.719+0000 7f2046ffd640 1 -- 172.21.15.193:0/3177628756 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c00f600 con 0x7f2050101c00 2024-09-16T00:08:27.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.719+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20501a0200 con 0x7f2050101c00 2024-09-16T00:08:27.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.720+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2050108fb0 con 0x7f2050101c00 2024-09-16T00:08:27.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.721+0000 7f2046ffd640 1 -- 172.21.15.193:0/3177628756 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f204c01a070 con 0x7f2050101c00 2024-09-16T00:08:27.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.722+0000 7f2046ffd640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f203c0a3270 0x7f203c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.722+0000 7f2046ffd640 1 -- 172.21.15.193:0/3177628756 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5430+0+0 (secure 0 0 0) 0x7f204c0ce8b0 con 0x7f2050101c00 2024-09-16T00:08:27.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.723+0000 7f20552ff640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f203c0a3270 0x7f203c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.731 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.727+0000 7f20552ff640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f203c0a3270 0x7f203c0a5730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2040005fd0 tx=0x7f2040005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.732 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.727+0000 7f2046ffd640 1 -- 172.21.15.193:0/3177628756 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f204c09c110 con 0x7f2050101c00 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:27.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:27.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:27.905 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:27.903+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2050062630 con 0x7f203c0a3270 2024-09-16T00:08:28.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:08:28.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:08:28.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi148 2024-09-16T00:08:28.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi148 to 5696M 2024-09-16T00:08:28.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:28.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] boot 2024-09-16T00:08:28.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: osdmap e48: 8 total, 8 up, 8 in 2024-09-16T00:08:28.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:28.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:28.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:28.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:28.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:08:28.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi148 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi148 to 5696M 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] boot 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: osdmap e48: 8 total, 8 up, 8 in 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi148 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi148 to 5696M 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: osd.7 [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] boot 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: osdmap e48: 8 total, 8 up, 8 in 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:08:28.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:28.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:28.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:29.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:29 smithi193 ceph-mon[35484]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:29.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:29 smithi193 ceph-mon[35484]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:29.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:29 smithi193 ceph-mon[35484]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:08:29.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:29 smithi148 ceph-mon[35599]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:29.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:29 smithi148 ceph-mon[35599]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:29.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:29 smithi148 ceph-mon[35599]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:08:29.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:29 smithi078 ceph-mon[31157]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:29.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:29 smithi078 ceph-mon[31157]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:08:29.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:29 smithi078 ceph-mon[31157]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:08:30.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:30 smithi148 ceph-mon[35599]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:08:30.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:30 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/2059934225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]: dispatch 2024-09-16T00:08:30.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:30 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]: dispatch 2024-09-16T00:08:30.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:30 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]': finished 2024-09-16T00:08:30.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:30 smithi148 ceph-mon[35599]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T00:08:30.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:30.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:30 smithi193 ceph-mon[35484]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:08:30.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:30 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/2059934225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]: dispatch 2024-09-16T00:08:30.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:30 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]: dispatch 2024-09-16T00:08:30.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:30 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]': finished 2024-09-16T00:08:30.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:30 smithi193 ceph-mon[35484]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T00:08:30.715 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:30.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:30 smithi078 ceph-mon[31157]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:08:30.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:30 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/2059934225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]: dispatch 2024-09-16T00:08:30.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:30 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]: dispatch 2024-09-16T00:08:30.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:30 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8e53741-a657-48d0-a667-fb8a28f6f922"}]': finished 2024-09-16T00:08:30.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:30 smithi078 ceph-mon[31157]: osdmap e51: 9 total, 8 up, 9 in 2024-09-16T00:08:30.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:31 smithi148 ceph-mon[35599]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:31 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/2167649404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:31 smithi148 ceph-mon[35599]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T00:08:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:31 smithi078 ceph-mon[31157]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:31 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/2167649404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:31 smithi078 ceph-mon[31157]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T00:08:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:31 smithi193 ceph-mon[35484]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:31 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/2167649404' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:31 smithi193 ceph-mon[35484]: osdmap e52: 9 total, 8 up, 9 in 2024-09-16T00:08:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:33.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:33 smithi148 ceph-mon[35599]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:33.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:33 smithi078 ceph-mon[31157]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:33.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:33 smithi193 ceph-mon[35484]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:35.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:35 smithi193 ceph-mon[35484]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:35.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:35 smithi078 ceph-mon[31157]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:35 smithi148 ceph-mon[35599]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:37.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:37 smithi193 ceph-mon[35484]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:37.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:37 smithi078 ceph-mon[31157]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:37 smithi148 ceph-mon[35599]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:39 smithi193 ceph-mon[35484]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:08:39.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:39 smithi078 ceph-mon[31157]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:08:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:39 smithi148 ceph-mon[35599]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:08:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:40.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:40 smithi193 ceph-mon[35484]: Deploying daemon osd.8 on smithi193 2024-09-16T00:08:40.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:40 smithi078 ceph-mon[31157]: Deploying daemon osd.8 on smithi193 2024-09-16T00:08:40.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:40 smithi148 ceph-mon[35599]: Deploying daemon osd.8 on smithi193 2024-09-16T00:08:41.657 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:41 smithi193 ceph-mon[35484]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:41.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:41 smithi078 ceph-mon[31157]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:41.855 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:41 smithi148 ceph-mon[35599]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:43.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:43 smithi078 ceph-mon[31157]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:43.854 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:43 smithi193 ceph-mon[35484]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:43 smithi148 ceph-mon[35599]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:45.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:45 smithi078 ceph-mon[31157]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:45 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:45 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:45 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:45.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:45 smithi193 ceph-mon[35484]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:45.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:45 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:45.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:45 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:45.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:45 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:45 smithi148 ceph-mon[35599]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:45 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:45 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:45 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:47.718 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:47 smithi193 ceph-mon[35484]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:47.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:47 smithi078 ceph-mon[31157]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:47 smithi148 ceph-mon[35599]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:48.743 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.741+0000 7f2046ffd640 1 -- 172.21.15.193:0/3177628756 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2050062630 con 0x7f203c0a3270 2024-09-16T00:08:48.743 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-09-16T00:08:48.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f203c0a3270 msgr2=0x7f203c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:48.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f203c0a3270 0x7f203c0a5730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2040005fd0 tx=0x7f2040005850 comp rx=0 tx=0).stop 2024-09-16T00:08:48.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2050101c00 msgr2=0x7f2050197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:48.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2050101c00 0x7f2050197e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f204c00d950 tx=0x7f204c00de10 comp rx=0 tx=0).stop 2024-09-16T00:08:48.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 shutdown_connections 2024-09-16T00:08:48.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f203c0a3270 0x7f203c0a5730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:48.745 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20501037c0 0x7f205019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:48.746 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050102e00 0x7f2050198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:48.746 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 --2- 172.21.15.193:0/3177628756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2050101c00 0x7f2050197e50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:48.746 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 >> 172.21.15.193:0/3177628756 conn(0x7f20500fd390 msgr2=0x7f2050106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:48.746 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 shutdown_connections 2024-09-16T00:08:48.746 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:08:48.744+0000 7f2057d64640 1 -- 172.21.15.193:0/3177628756 wait complete. 2024-09-16T00:08:49.411 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.8.service 2024-09-16T00:08:49.414 INFO:tasks.cephadm:Deploying osd.9 on smithi078 with /dev/vg_nvme/lv_1... 2024-09-16T00:08:49.414 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:08:49.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:49 smithi193 ceph-mon[35484]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:49.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:49 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.674 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:08:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:49 smithi078 ceph-mon[31157]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:49 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:49 smithi148 ceph-mon[35599]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:49.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:49.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:49 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:51.181 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:08:50 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:08:50.985+0000 7fb6e60d6640 -1 osd.8 0 log_to_monitors true 2024-09-16T00:08:51.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:51 smithi078 ceph-mon[31157]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:51.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:51 smithi078 ceph-mon[31157]: from='osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:08:51.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:51 smithi078 ceph-mon[31157]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:08:51.776 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:51 smithi193 ceph-mon[35484]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:51.776 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:51 smithi193 ceph-mon[35484]: from='osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:08:51.776 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:51 smithi193 ceph-mon[35484]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:08:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:51 smithi148 ceph-mon[35599]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:51 smithi148 ceph-mon[35599]: from='osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:08:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:51 smithi148 ceph-mon[35599]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:08:52.680 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:08:52.509+0000 7fb6e2059640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T00:08:52.680 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:08:52.518+0000 7fb6dde81640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:08:52.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:08:52.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:08:52.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:52.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:08:52.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:08:52.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:52.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:52.682 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:52.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:52.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.806 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:08:52.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi078:vg_nvme/lv_1 2024-09-16T00:08:52.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:52.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:52.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:08:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.551+0000 7fd411be9640 1 -- 172.21.15.78:0/2077257429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c101c00 msgr2=0x7fd40c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.551+0000 7fd411be9640 1 --2- 172.21.15.78:0/2077257429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c101c00 0x7fd40c102000 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009900 tx=0x7fd3fc02a410 comp rx=0 tx=0).stop 2024-09-16T00:08:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.551+0000 7fd411be9640 1 -- 172.21.15.78:0/2077257429 shutdown_connections 2024-09-16T00:08:53.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.551+0000 7fd411be9640 1 --2- 172.21.15.78:0/2077257429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c1037c0 0x7fd40c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.551+0000 7fd411be9640 1 --2- 172.21.15.78:0/2077257429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd40c102e00 0x7fd40c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.551+0000 7fd411be9640 1 --2- 172.21.15.78:0/2077257429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c101c00 0x7fd40c102000 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.551+0000 7fd411be9640 1 -- 172.21.15.78:0/2077257429 >> 172.21.15.78:0/2077257429 conn(0x7fd40c0fd390 msgr2=0x7fd40c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.552+0000 7fd411be9640 1 -- 172.21.15.78:0/2077257429 shutdown_connections 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.552+0000 7fd411be9640 1 -- 172.21.15.78:0/2077257429 wait complete. 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.552+0000 7fd411be9640 1 Processor -- start 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.552+0000 7fd411be9640 1 -- start start 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd411be9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c101c00 0x7fd40c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd411be9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c102e00 0x7fd40c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd411be9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd40c1037c0 0x7fd40c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd411be9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c10d8e0 con 0x7fd40c102e00 2024-09-16T00:08:53.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd411be9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c10d760 con 0x7fd40c101c00 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd411be9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c10da60 con 0x7fd40c1037c0 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c102e00 0x7fd40c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c102e00 0x7fd40c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40156/0 (socket says 172.21.15.78:40156) 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 -- 172.21.15.78:0/784944230 learned_addr learned my addr 172.21.15.78:0/784944230 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 -- 172.21.15.78:0/784944230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd40c1037c0 msgr2=0x7fd40c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd40c1037c0 0x7fd40c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 -- 172.21.15.78:0/784944230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c101c00 msgr2=0x7fd40c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c101c00 0x7fd40c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 -- 172.21.15.78:0/784944230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c19fb90 con 0x7fd40c102e00 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd40affd640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c102e00 0x7fd40c1984a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b4a0 tx=0x7fd40000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd408ff9640 1 -- 172.21.15.78:0/784944230 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400014070 con 0x7fd40c102e00 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c19fe80 con 0x7fd40c102e00 2024-09-16T00:08:53.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd408ff9640 1 -- 172.21.15.78:0/784944230 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd400010af0 con 0x7fd40c102e00 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd408ff9640 1 -- 172.21.15.78:0/784944230 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400017d10 con 0x7fd40c102e00 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.553+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c1a03b0 con 0x7fd40c102e00 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.554+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c109070 con 0x7fd40c102e00 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.555+0000 7fd408ff9640 1 -- 172.21.15.78:0/784944230 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd400017070 con 0x7fd40c102e00 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.556+0000 7fd408ff9640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a3270 0x7fd3f40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.556+0000 7fd40b7fe640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a3270 0x7fd3f40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.557+0000 7fd408ff9640 1 -- 172.21.15.78:0/784944230 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5862+0+0 (secure 0 0 0) 0x7fd4000ce3c0 con 0x7fd40c102e00 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.557+0000 7fd40b7fe640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a3270 0x7fd3f40a5730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0096c0 tx=0x7fd3fc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.561+0000 7fd408ff9640 1 -- 172.21.15.78:0/784944230 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd40009ba70 con 0x7fd40c102e00 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:08:53.737+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd40c062630 con 0x7fd3f40a3270 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi193 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:53.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:53.740 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:53.740 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:53 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi193 2024-09-16T00:08:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-16T00:08:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:08:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:08:53.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:53.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:53.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:53.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:53 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:53.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi193 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:53.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:53 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] boot 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:54.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:08:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:08:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] boot 2024-09-16T00:08:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T00:08:54.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:54.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:54.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:54.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:54.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:08:54.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:08:54.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:54.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: osd.8 [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] boot 2024-09-16T00:08:54.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: osdmap e55: 9 total, 9 up, 9 in 2024-09-16T00:08:54.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:08:54.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:08:54.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:08:54.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:08:55.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:55 smithi078 ceph-mon[31157]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:55.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:55 smithi078 ceph-mon[31157]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:55.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:55 smithi078 ceph-mon[31157]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T00:08:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:55 smithi193 ceph-mon[35484]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:55.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:55 smithi193 ceph-mon[35484]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:55.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:55 smithi193 ceph-mon[35484]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T00:08:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:55 smithi148 ceph-mon[35599]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:08:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:55 smithi148 ceph-mon[35599]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:08:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:55 smithi148 ceph-mon[35599]: osdmap e56: 9 total, 9 up, 9 in 2024-09-16T00:08:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:56 smithi078 ceph-mon[31157]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:08:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:56 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1207296186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e15eca2-aea8-4693-807d-bfa2b2439c48"}]: dispatch 2024-09-16T00:08:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:56 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1207296186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e15eca2-aea8-4693-807d-bfa2b2439c48"}]': finished 2024-09-16T00:08:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:56 smithi078 ceph-mon[31157]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T00:08:56.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:08:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:56 smithi193 ceph-mon[35484]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:08:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:56 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1207296186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e15eca2-aea8-4693-807d-bfa2b2439c48"}]: dispatch 2024-09-16T00:08:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:56 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1207296186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e15eca2-aea8-4693-807d-bfa2b2439c48"}]': finished 2024-09-16T00:08:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:56 smithi193 ceph-mon[35484]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T00:08:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:08:56.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:56 smithi148 ceph-mon[35599]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:08:56.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:56 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1207296186' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e15eca2-aea8-4693-807d-bfa2b2439c48"}]: dispatch 2024-09-16T00:08:56.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:56 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1207296186' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e15eca2-aea8-4693-807d-bfa2b2439c48"}]': finished 2024-09-16T00:08:56.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:56 smithi148 ceph-mon[35599]: osdmap e58: 10 total, 9 up, 10 in 2024-09-16T00:08:56.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:08:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:57 smithi193 ceph-mon[35484]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:08:58.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:57 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2542823359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:58.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:57 smithi193 ceph-mon[35484]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T00:08:58.005 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:08:58.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:57 smithi148 ceph-mon[35599]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:08:58.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:57 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2542823359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:58.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:57 smithi148 ceph-mon[35599]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T00:08:58.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:08:58.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:57 smithi078 ceph-mon[31157]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:08:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:57 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2542823359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:08:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:57 smithi078 ceph-mon[31157]: osdmap e59: 10 total, 9 up, 10 in 2024-09-16T00:08:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:08:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:08:59 smithi193 ceph-mon[35484]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:08:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:08:59 smithi148 ceph-mon[35599]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:00.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:08:59 smithi078 ceph-mon[31157]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:01 smithi193 ceph-mon[35484]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-16T00:09:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:01 smithi148 ceph-mon[35599]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-16T00:09:02.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:01 smithi078 ceph-mon[31157]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-16T00:09:03.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:03 smithi078 ceph-mon[31157]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T00:09:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:03 smithi193 ceph-mon[35484]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T00:09:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:03 smithi148 ceph-mon[35599]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-16T00:09:05.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:05 smithi078 ceph-mon[31157]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T00:09:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:09:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:05 smithi193 ceph-mon[35484]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T00:09:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:09:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:05 smithi148 ceph-mon[35599]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-16T00:09:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:09:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:06.737 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:06 smithi078 ceph-mon[31157]: Deploying daemon osd.9 on smithi078 2024-09-16T00:09:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:06 smithi193 ceph-mon[35484]: Deploying daemon osd.9 on smithi078 2024-09-16T00:09:06.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:06 smithi148 ceph-mon[35599]: Deploying daemon osd.9 on smithi078 2024-09-16T00:09:07.876 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:07 smithi078 ceph-mon[31157]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:09:07.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:07 smithi193 ceph-mon[35484]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:09:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:07 smithi148 ceph-mon[35599]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:09:09.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:09 smithi078 ceph-mon[31157]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:09:09.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:09 smithi193 ceph-mon[35484]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:09:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:09 smithi148 ceph-mon[35599]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-16T00:09:11.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:11 smithi193 ceph-mon[35484]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:09:11.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:11.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:11.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:11 smithi148 ceph-mon[35599]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:09:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:11.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:11.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:11 smithi078 ceph-mon[31157]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:09:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:13 smithi193 ceph-mon[35484]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:13 smithi148 ceph-mon[35599]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:13.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:13 smithi078 ceph-mon[31157]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:14.464 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 9 on host 'smithi078' 2024-09-16T00:09:14.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.462+0000 7fd408ff9640 1 -- 172.21.15.78:0/784944230 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd40c062630 con 0x7fd3f40a3270 2024-09-16T00:09:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a3270 msgr2=0x7fd3f40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a3270 0x7fd3f40a5730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0096c0 tx=0x7fd3fc031400 comp rx=0 tx=0).stop 2024-09-16T00:09:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c102e00 msgr2=0x7fd40c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c102e00 0x7fd40c1984a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b4a0 tx=0x7fd40000b960 comp rx=0 tx=0).stop 2024-09-16T00:09:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 shutdown_connections 2024-09-16T00:09:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3f40a3270 0x7fd3f40a5730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd40c1037c0 0x7fd40c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd40c102e00 0x7fd40c1984a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 --2- 172.21.15.78:0/784944230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd40c101c00 0x7fd40c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 >> 172.21.15.78:0/784944230 conn(0x7fd40c0fd390 msgr2=0x7fd40c106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 shutdown_connections 2024-09-16T00:09:14.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:09:14.465+0000 7fd411be9640 1 -- 172.21.15.78:0/784944230 wait complete. 2024-09-16T00:09:15.199 DEBUG:teuthology.orchestra.run.smithi078:osd.9> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.9.service 2024-09-16T00:09:15.201 INFO:tasks.cephadm:Deploying osd.10 on smithi148 with /dev/vg_nvme/lv_1... 2024-09-16T00:09:15.201 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:09:15.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:15 smithi148 ceph-mon[35599]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:15.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.579 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.580 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:15 smithi078 ceph-mon[31157]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:15.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:15 smithi193 ceph-mon[35484]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:17.182 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:09:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:09:16.927+0000 7fa17fa93640 -1 osd.9 0 log_to_monitors true 2024-09-16T00:09:17.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:17 smithi078 ceph-mon[31157]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:17.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:17 smithi078 ceph-mon[31157]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:09:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:17 smithi193 ceph-mon[35484]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:17 smithi193 ceph-mon[35484]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:09:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:17 smithi148 ceph-mon[35599]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:17 smithi148 ceph-mon[35599]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:09:18.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-16T00:09:18.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_1 2024-09-16T00:09:18.292 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:09:18.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:09:18 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:09:18.467+0000 7fa17ba16640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T00:09:18.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:09:18 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:09:18.482+0000 7fa17783e640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:09:18.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:18 smithi078 ceph-mon[31157]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:09:18.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:18 smithi078 ceph-mon[31157]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:09:18.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:18 smithi078 ceph-mon[31157]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:09:18.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:18 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:18.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:18 smithi078 ceph-mon[31157]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:09:18.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:18 smithi148 ceph-mon[35599]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:09:18.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:18 smithi148 ceph-mon[35599]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:09:18.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:18 smithi148 ceph-mon[35599]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:09:18.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:18 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:18.684 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:18 smithi148 ceph-mon[35599]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:09:18.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 -- 172.21.15.148:0/691122113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60100ad0 msgr2=0x7fbb60100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 --2- 172.21.15.148:0/691122113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60100ad0 0x7fbb60100ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009900 tx=0x7fbb5002a410 comp rx=0 tx=0).stop 2024-09-16T00:09:18.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 -- 172.21.15.148:0/691122113 shutdown_connections 2024-09-16T00:09:18.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 --2- 172.21.15.148:0/691122113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb60102690 0x7fbb60108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.792 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 --2- 172.21.15.148:0/691122113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb60101cd0 0x7fbb60102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 --2- 172.21.15.148:0/691122113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60100ad0 0x7fbb60100ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 -- 172.21.15.148:0/691122113 >> 172.21.15.148:0/691122113 conn(0x7fbb600fc280 msgr2=0x7fbb600fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 -- 172.21.15.148:0/691122113 shutdown_connections 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.768+0000 7fbb651b7640 1 -- 172.21.15.148:0/691122113 wait complete. 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb651b7640 1 Processor -- start 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb651b7640 1 -- start start 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb651b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb60100ad0 0x7fbb60197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb651b7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb60101cd0 0x7fbb60198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb651b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60102690 0x7fbb6019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb651b7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6010c780 con 0x7fbb60101cd0 2024-09-16T00:09:18.793 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb651b7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6010c600 con 0x7fbb60102690 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb651b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6010c900 con 0x7fbb60100ad0 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb5f59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60102690 0x7fbb6019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb5f59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60102690 0x7fbb6019f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:45942/0 (socket says 172.21.15.148:45942) 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb5f59e640 1 -- 172.21.15.148:0/148239213 learned_addr learned my addr 172.21.15.148:0/148239213 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.769+0000 7fbb57fff640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb60101cd0 0x7fbb60198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5ed9d640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb60100ad0 0x7fbb60197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5f59e640 1 -- 172.21.15.148:0/148239213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb60100ad0 msgr2=0x7fbb60197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5f59e640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb60100ad0 0x7fbb60197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5f59e640 1 -- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb60101cd0 msgr2=0x7fbb60198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5f59e640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb60101cd0 0x7fbb60198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.794 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5f59e640 1 -- 172.21.15.148:0/148239213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6019fb70 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5f59e640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60102690 0x7fbb6019f500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800eb10 tx=0x7fbb5800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5cd99640 1 -- 172.21.15.148:0/148239213 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58004b20 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6019fe60 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5cd99640 1 -- 172.21.15.148:0/148239213 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb58004cc0 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb5cd99640 1 -- 172.21.15.148:0/148239213 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58010760 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.770+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601a02c0 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.771+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb601a0580 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.773+0000 7fbb5cd99640 1 -- 172.21.15.148:0/148239213 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbb58010900 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.774+0000 7fbb5cd99640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbb4c0a3270 0x7fbb4c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.774+0000 7fbb5cd99640 1 -- 172.21.15.148:0/148239213 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5981+0+0 (secure 0 0 0) 0x7fbb580cf7d0 con 0x7fbb60102690 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.774+0000 7fbb5ed9d640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbb4c0a3270 0x7fbb4c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.774+0000 7fbb5ed9d640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbb4c0a3270 0x7fbb4c0a5730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb5002aa00 tx=0x7fbb50031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.777+0000 7fbb5cd99640 1 -- 172.21.15.148:0/148239213 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb580541b0 con 0x7fbb60102690 2024-09-16T00:09:18.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:18 smithi193 ceph-mon[35484]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:09:18.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:18 smithi193 ceph-mon[35484]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:09:18.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:18 smithi193 ceph-mon[35484]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-09-16T00:09:18.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:18 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:18.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:18 smithi193 ceph-mon[35484]: from='osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-09-16T00:09:18.961 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:18.960+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbb601a0750 con 0x7fbb4c0a3270 2024-09-16T00:09:19.617 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:19.617 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:09:19.617 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:19.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:19.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:19.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:19.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:19.622 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.622 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:09:19.622 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:09:19.622 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.623 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:09:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:09:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:19.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:19.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:19.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:19.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:09:19.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:09:19.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:19.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:20.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:20 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:09:20.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:20 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:09:20.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:20 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi078 2024-09-16T00:09:20.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:20 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-16T00:09:20.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:20 smithi148 ceph-mon[35599]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:09:20.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:20 smithi148 ceph-mon[35599]: osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] boot 2024-09-16T00:09:20.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:20 smithi148 ceph-mon[35599]: osdmap e62: 10 total, 10 up, 10 in 2024-09-16T00:09:20.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:20 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:20.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:20 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:09:20.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:20 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:09:20.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:20 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi078 2024-09-16T00:09:20.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:20 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-16T00:09:20.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:20 smithi078 ceph-mon[31157]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:09:20.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:20 smithi078 ceph-mon[31157]: osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] boot 2024-09-16T00:09:20.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:20 smithi078 ceph-mon[31157]: osdmap e62: 10 total, 10 up, 10 in 2024-09-16T00:09:20.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:20 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:20 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:09:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:20 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:09:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:20 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi078 2024-09-16T00:09:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:20 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi078 to 4272M 2024-09-16T00:09:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:20 smithi193 ceph-mon[35484]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:09:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:20 smithi193 ceph-mon[35484]: osd.9 [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] boot 2024-09-16T00:09:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:20 smithi193 ceph-mon[35484]: osdmap e62: 10 total, 10 up, 10 in 2024-09-16T00:09:20.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:20 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:09:21.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:21 smithi148 ceph-mon[35599]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:21.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:21 smithi148 ceph-mon[35599]: osdmap e63: 10 total, 10 up, 10 in 2024-09-16T00:09:21.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:21 smithi148 ceph-mon[35599]: from='client.? 172.21.15.148:0/1641123437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]: dispatch 2024-09-16T00:09:21.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:21 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]: dispatch 2024-09-16T00:09:21.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:21 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]': finished 2024-09-16T00:09:21.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:21 smithi148 ceph-mon[35599]: osdmap e64: 11 total, 10 up, 11 in 2024-09-16T00:09:21.754 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:21 smithi078 ceph-mon[31157]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:21 smithi078 ceph-mon[31157]: osdmap e63: 10 total, 10 up, 10 in 2024-09-16T00:09:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:21 smithi078 ceph-mon[31157]: from='client.? 172.21.15.148:0/1641123437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]: dispatch 2024-09-16T00:09:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:21 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]: dispatch 2024-09-16T00:09:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:21 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]': finished 2024-09-16T00:09:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:21 smithi078 ceph-mon[31157]: osdmap e64: 11 total, 10 up, 11 in 2024-09-16T00:09:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:21 smithi193 ceph-mon[35484]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:09:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:21 smithi193 ceph-mon[35484]: osdmap e63: 10 total, 10 up, 10 in 2024-09-16T00:09:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:21 smithi193 ceph-mon[35484]: from='client.? 172.21.15.148:0/1641123437' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]: dispatch 2024-09-16T00:09:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:21 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]: dispatch 2024-09-16T00:09:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:21 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92a86462-ac20-4b68-b8eb-d18ba5c604b3"}]': finished 2024-09-16T00:09:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:21 smithi193 ceph-mon[35484]: osdmap e64: 11 total, 10 up, 11 in 2024-09-16T00:09:21.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:22 smithi078 ceph-mon[31157]: from='client.? 172.21.15.148:0/2914212689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:09:22.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:22 smithi193 ceph-mon[35484]: from='client.? 172.21.15.148:0/2914212689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:09:22.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:22 smithi148 ceph-mon[35599]: from='client.? 172.21.15.148:0/2914212689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:09:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:23 smithi078 ceph-mon[31157]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:23 smithi193 ceph-mon[35484]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:23.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:23 smithi148 ceph-mon[35599]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:25.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:25 smithi078 ceph-mon[31157]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:25.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:25 smithi193 ceph-mon[35484]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:25 smithi148 ceph-mon[35599]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:27.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:27 smithi148 ceph-mon[35599]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:27.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:27 smithi078 ceph-mon[31157]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:27 smithi193 ceph-mon[35484]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:29.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:29 smithi148 ceph-mon[35599]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:29.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:29 smithi078 ceph-mon[31157]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:29 smithi193 ceph-mon[35484]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:30.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:09:30.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:30.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:09:30.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:09:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:31 smithi148 ceph-mon[35599]: Deploying daemon osd.10 on smithi148 2024-09-16T00:09:31.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:31 smithi148 ceph-mon[35599]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:31 smithi078 ceph-mon[31157]: Deploying daemon osd.10 on smithi148 2024-09-16T00:09:31.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:31 smithi078 ceph-mon[31157]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:31 smithi193 ceph-mon[35484]: Deploying daemon osd.10 on smithi148 2024-09-16T00:09:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:31 smithi193 ceph-mon[35484]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:33.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:33 smithi148 ceph-mon[35599]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:33.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:33 smithi078 ceph-mon[31157]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:33 smithi193 ceph-mon[35484]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:35.677 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:35 smithi148 ceph-mon[35599]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:35 smithi193 ceph-mon[35484]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:35 smithi078 ceph-mon[31157]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:36.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:36.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:36.649 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:36.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:37.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:37.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:37.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:37.644 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:37 smithi148 ceph-mon[35599]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:37 smithi193 ceph-mon[35484]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:37 smithi078 ceph-mon[31157]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:39.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:39 smithi148 ceph-mon[35599]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:39.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:39.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:39.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:39.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:39.658 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:39.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.698+0000 7fbb5cd99640 1 -- 172.21.15.148:0/148239213 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbb601a0750 con 0x7fbb4c0a3270 2024-09-16T00:09:39.699 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 10 on host 'smithi148' 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbb4c0a3270 msgr2=0x7fbb4c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbb4c0a3270 0x7fbb4c0a5730 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb5002aa00 tx=0x7fbb50031400 comp rx=0 tx=0).stop 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60102690 msgr2=0x7fbb6019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60102690 0x7fbb6019f500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800eb10 tx=0x7fbb5800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 shutdown_connections 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbb4c0a3270 0x7fbb4c0a5730 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb60102690 0x7fbb6019f500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb60101cd0 0x7fbb60198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.705 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 --2- 172.21.15.148:0/148239213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb60100ad0 0x7fbb60197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 >> 172.21.15.148:0/148239213 conn(0x7fbb600fc280 msgr2=0x7fbb6006a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:39.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 shutdown_connections 2024-09-16T00:09:39.706 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-16T00:09:39.700+0000 7fbb651b7640 1 -- 172.21.15.148:0/148239213 wait complete. 2024-09-16T00:09:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:39 smithi193 ceph-mon[35484]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:39 smithi078 ceph-mon[31157]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:40.356 DEBUG:teuthology.orchestra.run.smithi148:osd.10> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.10.service 2024-09-16T00:09:40.359 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-09-16T00:09:40.359 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:09:40.616 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:09:40.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:40 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:40.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:40 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:40.632 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:40 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:40.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:40 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:40.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:40 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:40.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:40 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:41.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:40 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:09:41.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:40 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:41.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:40 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:41.719 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:41 smithi148 ceph-mon[35599]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:41 smithi193 ceph-mon[35484]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:41 smithi078 ceph-mon[31157]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:42.455 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:09:42 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:09:42.236+0000 7f0de9d87640 -1 osd.10 0 log_to_monitors true 2024-09-16T00:09:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:42 smithi193 ceph-mon[35484]: from='osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:09:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:42 smithi193 ceph-mon[35484]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:09:42.948 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:42 smithi148 ceph-mon[35599]: from='osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:09:42.948 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:42 smithi148 ceph-mon[35599]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:09:43.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:42 smithi078 ceph-mon[31157]: from='osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:09:43.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:42 smithi078 ceph-mon[31157]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:09:43.483 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T00:09:43.510 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-09-16T00:09:43.770 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:09:43.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:43.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:09:43.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T00:09:43.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:43.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:09:43.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:09:43.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:43.792 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:43 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:43.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:09:43.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T00:09:43.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:43.923 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:09:43.923 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:09:43.923 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.923 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.923 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.923 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.924 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.924 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.924 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.924 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:43.924 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:43.924 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:43 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.927 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:09:43.549+0000 7f0de5d0a640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T00:09:43.928 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:09:43.559+0000 7f0de1b32640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:09:43.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:43.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:09:43.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: osdmap e65: 11 total, 10 up, 11 in 2024-09-16T00:09:43.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:43.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:09:43.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-16T00:09:43.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:09:43.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:43.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:43.931 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:09:43.931 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:43 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 -- 172.21.15.193:0/3976877468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218100ad0 msgr2=0x7f1218100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 --2- 172.21.15.193:0/3976877468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218100ad0 0x7f1218100ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1208009ca0 tx=0x7f120802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 -- 172.21.15.193:0/3976877468 shutdown_connections 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 --2- 172.21.15.193:0/3976877468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1218102690 0x7f1218108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 --2- 172.21.15.193:0/3976877468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218101cd0 0x7f1218102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 --2- 172.21.15.193:0/3976877468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218100ad0 0x7f1218100ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 -- 172.21.15.193:0/3976877468 >> 172.21.15.193:0/3976877468 conn(0x7f12180fc280 msgr2=0x7f12180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 -- 172.21.15.193:0/3976877468 shutdown_connections 2024-09-16T00:09:44.301 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.260+0000 7f121f284640 1 -- 172.21.15.193:0/3976877468 wait complete. 2024-09-16T00:09:44.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121f284640 1 Processor -- start 2024-09-16T00:09:44.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121f284640 1 -- start start 2024-09-16T00:09:44.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121f284640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1218100ad0 0x7f121806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121f284640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218101cd0 0x7f121819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121f284640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218102690 0x7f121819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121f284640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f121810c7a0 con 0x7f1218100ad0 2024-09-16T00:09:44.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121f284640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f121810c620 con 0x7f1218102690 2024-09-16T00:09:44.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121f284640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f121810c920 con 0x7f1218101cd0 2024-09-16T00:09:44.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218101cd0 0x7f121819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218101cd0 0x7f121819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:32862/0 (socket says 172.21.15.193:32862) 2024-09-16T00:09:44.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 -- 172.21.15.193:0/1059708803 learned_addr learned my addr 172.21.15.193:0/1059708803 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:09:44.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121d821640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218102690 0x7f121819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121d020640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1218100ad0 0x7f121806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 -- 172.21.15.193:0/1059708803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218102690 msgr2=0x7f121819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:44.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218102690 0x7f121819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 -- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1218100ad0 msgr2=0x7f121806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:44.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1218100ad0 0x7f121806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 -- 172.21.15.193:0/1059708803 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121819fbb0 con 0x7f1218101cd0 2024-09-16T00:09:44.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121d020640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1218100ad0 0x7f121806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:09:44.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.261+0000 7f121c81f640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218101cd0 0x7f121819cc80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f120c00d770 tx=0x7f120c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:44.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.262+0000 7f12167fc640 1 -- 172.21.15.193:0/1059708803 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c017070 con 0x7f1218101cd0 2024-09-16T00:09:44.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.262+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121819fe40 con 0x7f1218101cd0 2024-09-16T00:09:44.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.262+0000 7f12167fc640 1 -- 172.21.15.193:0/1059708803 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f120c005500 con 0x7f1218101cd0 2024-09-16T00:09:44.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.262+0000 7f12167fc640 1 -- 172.21.15.193:0/1059708803 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c004e00 con 0x7f1218101cd0 2024-09-16T00:09:44.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.262+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181a02f0 con 0x7f1218101cd0 2024-09-16T00:09:44.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.263+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1218107f20 con 0x7f1218101cd0 2024-09-16T00:09:44.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.264+0000 7f12167fc640 1 -- 172.21.15.193:0/1059708803 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f120c004fa0 con 0x7f1218101cd0 2024-09-16T00:09:44.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.264+0000 7f12167fc640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12000a3240 0x7f12000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.307 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.265+0000 7f12167fc640 1 -- 172.21.15.193:0/1059708803 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6414+0+0 (secure 0 0 0) 0x7f120c0cebd0 con 0x7f1218101cd0 2024-09-16T00:09:44.307 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.265+0000 7f121d020640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12000a3240 0x7f12000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.307 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.266+0000 7f121d020640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12000a3240 0x7f12000a5700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f1208000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:44.307 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.269+0000 7f12167fc640 1 -- 172.21.15.193:0/1059708803 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f120c09c060 con 0x7f1218101cd0 2024-09-16T00:09:44.451 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:09:44.450+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1218062630 con 0x7f12000a3240 2024-09-16T00:09:44.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi148 2024-09-16T00:09:44.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi148 to 4272M 2024-09-16T00:09:44.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:09:44.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:09:44.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:44.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:44.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:09:44.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:09:44.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:44.939 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi148 2024-09-16T00:09:44.939 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi148 to 4272M 2024-09-16T00:09:44.939 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:09:44.939 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:09:44.939 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:44.939 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:44.940 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:09:44.940 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:09:44.940 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi148 2024-09-16T00:09:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi148 to 4272M 2024-09-16T00:09:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-16T00:09:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:09:44.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:44.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:44.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:09:44.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:09:44.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:45.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:45 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:09:45.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:45 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:09:45.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:45 smithi193 ceph-mon[35484]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:45.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:45 smithi193 ceph-mon[35484]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:09:45.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:45 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:45.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:45 smithi193 ceph-mon[35484]: osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] boot 2024-09-16T00:09:45.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:45 smithi193 ceph-mon[35484]: osdmap e67: 11 total, 11 up, 11 in 2024-09-16T00:09:45.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:45 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:45 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:09:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:45 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:09:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:45 smithi148 ceph-mon[35599]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:45 smithi148 ceph-mon[35599]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:09:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:45 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:45 smithi148 ceph-mon[35599]: osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] boot 2024-09-16T00:09:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:45 smithi148 ceph-mon[35599]: osdmap e67: 11 total, 11 up, 11 in 2024-09-16T00:09:45.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:45 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:46.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:45 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:09:46.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:45 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:09:46.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:45 smithi078 ceph-mon[31157]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 284 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:09:46.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:45 smithi078 ceph-mon[31157]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:09:46.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:45 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:46.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:45 smithi078 ceph-mon[31157]: osd.10 [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] boot 2024-09-16T00:09:46.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:45 smithi078 ceph-mon[31157]: osdmap e67: 11 total, 11 up, 11 in 2024-09-16T00:09:46.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:45 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:09:46.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:46 smithi193 ceph-mon[35484]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T00:09:46.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:46 smithi148 ceph-mon[35599]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T00:09:47.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:46 smithi078 ceph-mon[31157]: osdmap e68: 11 total, 11 up, 11 in 2024-09-16T00:09:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:47 smithi193 ceph-mon[35484]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:09:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:47 smithi193 ceph-mon[35484]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:09:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:47 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/2719193686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]: dispatch 2024-09-16T00:09:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:47 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]: dispatch 2024-09-16T00:09:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:47 smithi193 ceph-mon[35484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]': finished 2024-09-16T00:09:47.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:47 smithi193 ceph-mon[35484]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T00:09:47.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:47 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:09:47.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:47 smithi193 ceph-mon[35484]: from='client.? 172.21.15.193:0/2125605433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:09:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:47 smithi148 ceph-mon[35599]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:09:47.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:47 smithi148 ceph-mon[35599]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:09:47.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:47 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/2719193686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]: dispatch 2024-09-16T00:09:47.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:47 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]: dispatch 2024-09-16T00:09:47.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:47 smithi148 ceph-mon[35599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]': finished 2024-09-16T00:09:47.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:47 smithi148 ceph-mon[35599]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T00:09:47.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:47 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:09:47.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:47 smithi148 ceph-mon[35599]: from='client.? 172.21.15.193:0/2125605433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:09:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:47 smithi078 ceph-mon[31157]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:09:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:47 smithi078 ceph-mon[31157]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:09:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:47 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/2719193686' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]: dispatch 2024-09-16T00:09:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:47 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]: dispatch 2024-09-16T00:09:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:47 smithi078 ceph-mon[31157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41c5e26a-ddcf-4933-95f1-5c868521acfd"}]': finished 2024-09-16T00:09:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:47 smithi078 ceph-mon[31157]: osdmap e70: 12 total, 11 up, 12 in 2024-09-16T00:09:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:47 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:09:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:47 smithi078 ceph-mon[31157]: from='client.? 172.21.15.193:0/2125605433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:09:49.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:48 smithi193 ceph-mon[35484]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:09:49.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:09:49.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:48 smithi193 ceph-mon[35484]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:09:49.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:48 smithi148 ceph-mon[35599]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:09:49.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:09:49.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:48 smithi148 ceph-mon[35599]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:09:49.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:48 smithi078 ceph-mon[31157]: osdmap e71: 12 total, 11 up, 12 in 2024-09-16T00:09:49.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:09:49.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:48 smithi078 ceph-mon[31157]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:09:51.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:51 smithi193 ceph-mon[35484]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-16T00:09:51.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:51 smithi148 ceph-mon[35599]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-16T00:09:51.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:51 smithi078 ceph-mon[31157]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 311 MiB used, 983 GiB / 983 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-16T00:09:53.605 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:53 smithi193 ceph-mon[35484]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:09:53.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:53 smithi148 ceph-mon[35599]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:09:53.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:53 smithi078 ceph-mon[31157]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:09:55.679 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:55 smithi078 ceph-mon[31157]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:09:55.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:55 smithi193 ceph-mon[35484]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:09:55.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:55 smithi148 ceph-mon[35599]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:09:56.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:09:56.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:56.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:09:56.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:56.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:09:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:09:57.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:57 smithi193 ceph-mon[35484]: Deploying daemon osd.11 on smithi193 2024-09-16T00:09:57.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:57 smithi193 ceph-mon[35484]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-16T00:09:57.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:57 smithi148 ceph-mon[35599]: Deploying daemon osd.11 on smithi193 2024-09-16T00:09:57.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:57 smithi148 ceph-mon[35599]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-16T00:09:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:57 smithi078 ceph-mon[31157]: Deploying daemon osd.11 on smithi193 2024-09-16T00:09:57.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:57 smithi078 ceph-mon[31157]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-16T00:09:59.665 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:09:59 smithi193 ceph-mon[35484]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-16T00:09:59.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:09:59 smithi148 ceph-mon[35599]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-16T00:09:59.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:09:59 smithi078 ceph-mon[31157]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-16T00:10:00.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:00 smithi148 ceph-mon[35599]: overall HEALTH_OK 2024-09-16T00:10:00.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:00 smithi078 ceph-mon[31157]: overall HEALTH_OK 2024-09-16T00:10:00.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:00 smithi193 ceph-mon[35484]: overall HEALTH_OK 2024-09-16T00:10:01.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:01 smithi193 ceph-mon[35484]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:10:01.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:01 smithi148 ceph-mon[35599]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:10:01.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:01 smithi078 ceph-mon[31157]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T00:10:02.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:02.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:02.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:02.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:02.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:02.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:02.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:02.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:02.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:03.693 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:03 smithi193 ceph-mon[35484]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:03.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:03 smithi148 ceph-mon[35599]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:03.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:03 smithi078 ceph-mon[31157]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:05.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:05 smithi193 ceph-mon[35484]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:05.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:05.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:05.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:05.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:05.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:05.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:05 smithi148 ceph-mon[35599]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:05.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:05.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:05.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:05.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:05.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:05.764 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.763+0000 7f12167fc640 1 -- 172.21.15.193:0/1059708803 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1218062630 con 0x7f12000a3240 2024-09-16T00:10:05.765 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-09-16T00:10:05.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.765+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12000a3240 msgr2=0x7f12000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.765+0000 7f121f284640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12000a3240 0x7f12000a5700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f1208000f40 comp rx=0 tx=0).stop 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.765+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218101cd0 msgr2=0x7f121819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.765+0000 7f121f284640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218101cd0 0x7f121819cc80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f120c00d770 tx=0x7f120c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.766+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 shutdown_connections 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.766+0000 7f121f284640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12000a3240 0x7f12000a5700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.766+0000 7f121f284640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1218102690 0x7f121819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.766+0000 7f121f284640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1218101cd0 0x7f121819cc80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.766+0000 7f121f284640 1 --2- 172.21.15.193:0/1059708803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1218100ad0 0x7f121806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.766+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 >> 172.21.15.193:0/1059708803 conn(0x7f12180fc280 msgr2=0x7f12181050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.766+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 shutdown_connections 2024-09-16T00:10:05.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:05.766+0000 7f121f284640 1 -- 172.21.15.193:0/1059708803 wait complete. 2024-09-16T00:10:05.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:05 smithi078 ceph-mon[31157]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:05.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:06.479 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.11.service 2024-09-16T00:10:06.481 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-16T00:10:06.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:10:06.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:06.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:06.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:06.748 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:06.749 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:06.749 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:06.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:06.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:06.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:07.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 -- 172.21.15.78:0/694874561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1741037c0 msgr2=0x7ff17410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 --2- 172.21.15.78:0/694874561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1741037c0 0x7ff17410a050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009ca0 tx=0x7ff16c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 -- 172.21.15.78:0/694874561 shutdown_connections 2024-09-16T00:10:07.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 --2- 172.21.15.78:0/694874561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1741037c0 0x7ff17410a050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 --2- 172.21.15.78:0/694874561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff174102e00 0x7ff174103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 --2- 172.21.15.78:0/694874561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff174101c00 0x7ff174102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 -- 172.21.15.78:0/694874561 >> 172.21.15.78:0/694874561 conn(0x7ff1740fd390 msgr2=0x7ff1740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 -- 172.21.15.78:0/694874561 shutdown_connections 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.237+0000 7ff1793b1640 1 -- 172.21.15.78:0/694874561 wait complete. 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1793b1640 1 Processor -- start 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1793b1640 1 -- start start 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1793b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff174101c00 0x7ff174197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1793b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff174102e00 0x7ff1741983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1793b1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1741037c0 0x7ff17419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1793b1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff17410d880 con 0x7ff174102e00 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1793b1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff17410d700 con 0x7ff174101c00 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1793b1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff17410da00 con 0x7ff1741037c0 2024-09-16T00:10:07.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1727fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff174102e00 0x7ff1741983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1727fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff174102e00 0x7ff1741983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32834/0 (socket says 172.21.15.78:32834) 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1727fc640 1 -- 172.21.15.78:0/2085341402 learned_addr learned my addr 172.21.15.78:0/2085341402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff1737fe640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1741037c0 0x7ff17419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.238+0000 7ff172ffd640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff174101c00 0x7ff174197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff1727fc640 1 -- 172.21.15.78:0/2085341402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1741037c0 msgr2=0x7ff17419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff1727fc640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1741037c0 0x7ff17419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff1727fc640 1 -- 172.21.15.78:0/2085341402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff174101c00 msgr2=0x7ff174197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff1727fc640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff174101c00 0x7ff174197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff1727fc640 1 -- 172.21.15.78:0/2085341402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17419fad0 con 0x7ff174102e00 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff1727fc640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff174102e00 0x7ff1741983e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff16800d950 tx=0x7ff16800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff15bfff640 1 -- 172.21.15.78:0/2085341402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168017070 con 0x7ff174102e00 2024-09-16T00:10:07.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17419fdc0 con 0x7ff174102e00 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff15bfff640 1 -- 172.21.15.78:0/2085341402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff168009c30 con 0x7ff174102e00 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff15bfff640 1 -- 172.21.15.78:0/2085341402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16800f6b0 con 0x7ff174102e00 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.239+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1741a0280 con 0x7ff174102e00 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.240+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff174109000 con 0x7ff174102e00 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.241+0000 7ff15bfff640 1 -- 172.21.15.78:0/2085341402 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff1680109d0 con 0x7ff174102e00 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.242+0000 7ff15bfff640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff15c0a3270 0x7ff15c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.242+0000 7ff172ffd640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff15c0a3270 0x7ff15c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.242+0000 7ff15bfff640 1 -- 172.21.15.78:0/2085341402 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7ff1680cef00 con 0x7ff174102e00 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.243+0000 7ff172ffd640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff15c0a3270 0x7ff15c0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff174102c60 tx=0x7ff164005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:07.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.247+0000 7ff15bfff640 1 -- 172.21.15.78:0/2085341402 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff168009de0 con 0x7ff174102e00 2024-09-16T00:10:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.420+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff17402da10 con 0x7ff174102e00 2024-09-16T00:10:07.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.420+0000 7ff15bfff640 1 -- 172.21.15.78:0/2085341402 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7ff16809bb10 con 0x7ff174102e00 2024-09-16T00:10:07.422 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.422+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff15c0a3270 msgr2=0x7ff15c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.422+0000 7ff1793b1640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff15c0a3270 0x7ff15c0a5730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff174102c60 tx=0x7ff164005f90 comp rx=0 tx=0).stop 2024-09-16T00:10:07.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.422+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff174102e00 msgr2=0x7ff1741983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.422+0000 7ff1793b1640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff174102e00 0x7ff1741983e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff16800d950 tx=0x7ff16800de10 comp rx=0 tx=0).stop 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.422+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 shutdown_connections 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.422+0000 7ff1793b1640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff15c0a3270 0x7ff15c0a5730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.422+0000 7ff1793b1640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1741037c0 0x7ff17419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.423+0000 7ff1793b1640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff174102e00 0x7ff1741983e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.423+0000 7ff1793b1640 1 --2- 172.21.15.78:0/2085341402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff174101c00 0x7ff174197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.423+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 >> 172.21.15.78:0/2085341402 conn(0x7ff1740fd390 msgr2=0x7ff1741061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.423+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 shutdown_connections 2024-09-16T00:10:07.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:07.423+0000 7ff1793b1640 1 -- 172.21.15.78:0/2085341402 wait complete. 2024-09-16T00:10:07.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:07 smithi193 ceph-mon[35484]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:07.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:07 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2085341402' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:07.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:07 smithi148 ceph-mon[35599]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:07.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:07 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2085341402' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:07 smithi078 ceph-mon[31157]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:07.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:07 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2085341402' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:08.214 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:10:08 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:10:08.018+0000 7ff2fa0cb640 -1 osd.11 0 log_to_monitors true 2024-09-16T00:10:08.247 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726445384,"num_in_osds":12,"osd_in_since":1726445386,"num_remapped_pgs":0} 2024-09-16T00:10:08.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:08 smithi078 ceph-mon[31157]: from='osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:10:08.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:08 smithi078 ceph-mon[31157]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:10:08.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:08 smithi193 ceph-mon[35484]: from='osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:10:08.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:08 smithi193 ceph-mon[35484]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:10:08.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:08 smithi148 ceph-mon[35599]: from='osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:10:08.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:08 smithi148 ceph-mon[35599]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:10:09.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:10:09.506 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:09.627 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:09 smithi078 ceph-mon[31157]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:09.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:09 smithi078 ceph-mon[31157]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:10:09.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:09 smithi078 ceph-mon[31157]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:10:09.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:09 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:09.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:09 smithi078 ceph-mon[31157]: from='osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:10:09.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:09 smithi078 ceph-mon[31157]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:10:09.628 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:09 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:09.629 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:09 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:09 smithi193 ceph-mon[35484]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:09 smithi193 ceph-mon[35484]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:10:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:09 smithi193 ceph-mon[35484]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:10:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:09 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:09 smithi193 ceph-mon[35484]: from='osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:10:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:09 smithi193 ceph-mon[35484]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:10:09.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:09 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:09.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:09 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:09 smithi148 ceph-mon[35599]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:09.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:09 smithi148 ceph-mon[35599]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:10:09.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:09 smithi148 ceph-mon[35599]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:10:09.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:09 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:09.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:09 smithi148 ceph-mon[35599]: from='osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:10:09.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:09 smithi148 ceph-mon[35599]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T00:10:09.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:09 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:09.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:09 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 -- 172.21.15.78:0/830167352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780101c00 msgr2=0x7f2780102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 --2- 172.21.15.78:0/830167352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780101c00 0x7f2780102000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2774009900 tx=0x7f277402a410 comp rx=0 tx=0).stop 2024-09-16T00:10:10.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 -- 172.21.15.78:0/830167352 shutdown_connections 2024-09-16T00:10:10.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 --2- 172.21.15.78:0/830167352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27801037c0 0x7f278010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 --2- 172.21.15.78:0/830167352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2780102e00 0x7f2780103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 --2- 172.21.15.78:0/830167352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780101c00 0x7f2780102000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 -- 172.21.15.78:0/830167352 >> 172.21.15.78:0/830167352 conn(0x7f27800fd390 msgr2=0x7f27800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 -- 172.21.15.78:0/830167352 shutdown_connections 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.061+0000 7f2786a6b640 1 -- 172.21.15.78:0/830167352 wait complete. 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2786a6b640 1 Processor -- start 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2786a6b640 1 -- start start 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2786a6b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2780101c00 0x7f2780075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2786a6b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780102e00 0x7f27800745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2786a6b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27801037c0 0x7f2780074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2786a6b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f278010d8e0 con 0x7f2780102e00 2024-09-16T00:10:10.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2786a6b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f278010d760 con 0x7f27801037c0 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2786a6b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f278010da60 con 0x7f2780101c00 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f277ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780102e00 0x7f27800745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f277ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780102e00 0x7f27800745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32852/0 (socket says 172.21.15.78:32852) 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f277ffff640 1 -- 172.21.15.78:0/2088528608 learned_addr learned my addr 172.21.15.78:0/2088528608 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.062+0000 7f2784807640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2780101c00 0x7f2780075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f2785008640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27801037c0 0x7f2780074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277ffff640 1 -- 172.21.15.78:0/2088528608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2780101c00 msgr2=0x7f2780075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277ffff640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2780101c00 0x7f2780075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277ffff640 1 -- 172.21.15.78:0/2088528608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27801037c0 msgr2=0x7f2780074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277ffff640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27801037c0 0x7f2780074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277ffff640 1 -- 172.21.15.78:0/2088528608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27800753e0 con 0x7f2780102e00 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f2785008640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27801037c0 0x7f2780074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277ffff640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780102e00 0x7f27800745e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f276c00d950 tx=0x7f276c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277dffb640 1 -- 172.21.15.78:0/2088528608 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c017030 con 0x7f2780102e00 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277dffb640 1 -- 172.21.15.78:0/2088528608 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276c004020 con 0x7f2780102e00 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27801a20f0 con 0x7f2780102e00 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f277dffb640 1 -- 172.21.15.78:0/2088528608 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c0047a0 con 0x7f2780102e00 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.063+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27801a2600 con 0x7f2780102e00 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.064+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2780109020 con 0x7f2780102e00 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.065+0000 7f277dffb640 1 -- 172.21.15.78:0/2088528608 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f276c0041d0 con 0x7f2780102e00 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.066+0000 7f277dffb640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27700a3240 0x7f27700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.066+0000 7f277dffb640 1 -- 172.21.15.78:0/2088528608 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7f276c0ceda0 con 0x7f2780102e00 2024-09-16T00:10:10.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.066+0000 7f2784807640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27700a3240 0x7f27700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.066+0000 7f2784807640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27700a3240 0x7f27700a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2774005ab0 tx=0x7f2774000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:10.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.071+0000 7f277dffb640 1 -- 172.21.15.78:0/2088528608 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f276c09c060 con 0x7f2780102e00 2024-09-16T00:10:10.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.238+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f278002d970 con 0x7f2780102e00 2024-09-16T00:10:10.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.238+0000 7f277dffb640 1 -- 172.21.15.78:0/2088528608 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f276c09b960 con 0x7f2780102e00 2024-09-16T00:10:10.240 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27700a3240 msgr2=0x7f27700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27700a3240 0x7f27700a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2774005ab0 tx=0x7f2774000bc0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780102e00 msgr2=0x7f27800745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780102e00 0x7f27800745e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f276c00d950 tx=0x7f276c00de10 comp rx=0 tx=0).stop 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 shutdown_connections 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27700a3240 0x7f27700a5700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f27801037c0 0x7f2780074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2780102e00 0x7f27800745e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 --2- 172.21.15.78:0/2088528608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2780101c00 0x7f2780075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 >> 172.21.15.78:0/2088528608 conn(0x7f27800fd390 msgr2=0x7f27801061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 shutdown_connections 2024-09-16T00:10:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:10.241+0000 7f2786a6b640 1 -- 172.21.15.78:0/2088528608 wait complete. 2024-09-16T00:10:10.430 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:10:10.142+0000 7ff2f604e640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T00:10:10.430 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:10:10.153+0000 7ff2ede6e640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: Detected new or changed devices on smithi193 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:10.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:10.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:10.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:10.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='osd.11 ' entity='osd.11' 2024-09-16T00:10:10.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2088528608' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:10.522 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:10 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: Detected new or changed devices on smithi193 2024-09-16T00:10:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:10:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:10:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='osd.11 ' entity='osd.11' 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2088528608' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:10.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:10 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: Detected new or changed devices on smithi193 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:10.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:10.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:10.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:10.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='osd.11 ' entity='osd.11' 2024-09-16T00:10:10.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2088528608' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:10.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:10 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:10.968 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726445384,"num_in_osds":12,"osd_in_since":1726445386,"num_remapped_pgs":0} 2024-09-16T00:10:11.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:11 smithi078 ceph-mon[31157]: purged_snaps scrub starts 2024-09-16T00:10:11.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:11 smithi078 ceph-mon[31157]: purged_snaps scrub ok 2024-09-16T00:10:11.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:11 smithi078 ceph-mon[31157]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:11.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:11 smithi078 ceph-mon[31157]: osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] boot 2024-09-16T00:10:11.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:11 smithi078 ceph-mon[31157]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T00:10:11.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:11 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:11.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:11 smithi193 ceph-mon[35484]: purged_snaps scrub starts 2024-09-16T00:10:11.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:11 smithi193 ceph-mon[35484]: purged_snaps scrub ok 2024-09-16T00:10:11.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:11 smithi193 ceph-mon[35484]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:11.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:11 smithi193 ceph-mon[35484]: osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] boot 2024-09-16T00:10:11.834 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:11 smithi193 ceph-mon[35484]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T00:10:11.834 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:11 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:11.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:11 smithi148 ceph-mon[35599]: purged_snaps scrub starts 2024-09-16T00:10:11.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:11 smithi148 ceph-mon[35599]: purged_snaps scrub ok 2024-09-16T00:10:11.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:11 smithi148 ceph-mon[35599]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:10:11.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:11 smithi148 ceph-mon[35599]: osd.11 [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] boot 2024-09-16T00:10:11.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:11 smithi148 ceph-mon[35599]: osdmap e74: 12 total, 12 up, 12 in 2024-09-16T00:10:11.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:11 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:10:11.969 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:10:12.229 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:12.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.704+0000 7f0f9c874640 1 -- 172.21.15.78:0/2668489834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98100ad0 msgr2=0x7f0f98100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.704+0000 7f0f9c874640 1 --2- 172.21.15.78:0/2668489834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98100ad0 0x7f0f98100ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009900 tx=0x7f0f8002a410 comp rx=0 tx=0).stop 2024-09-16T00:10:12.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 -- 172.21.15.78:0/2668489834 shutdown_connections 2024-09-16T00:10:12.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 --2- 172.21.15.78:0/2668489834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98102690 0x7f0f98108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 --2- 172.21.15.78:0/2668489834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f98101cd0 0x7f0f98102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 --2- 172.21.15.78:0/2668489834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98100ad0 0x7f0f98100ed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 -- 172.21.15.78:0/2668489834 >> 172.21.15.78:0/2668489834 conn(0x7f0f980fc280 msgr2=0x7f0f980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 -- 172.21.15.78:0/2668489834 shutdown_connections 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 -- 172.21.15.78:0/2668489834 wait complete. 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 Processor -- start 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 -- start start 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f98100ad0 0x7f0f98197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.705+0000 7f0f9c874640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98101cd0 0x7f0f98198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f9c874640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98102690 0x7f0f9819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f9c874640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810c760 con 0x7f0f98101cd0 2024-09-16T00:10:12.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f9c874640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810c5e0 con 0x7f0f98100ad0 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f9c874640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9810c8e0 con 0x7f0f98102690 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98101cd0 0x7f0f98198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98101cd0 0x7f0f98198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32874/0 (socket says 172.21.15.78:32874) 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 -- 172.21.15.78:0/3143694246 learned_addr learned my addr 172.21.15.78:0/3143694246 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f96d9d640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98102690 0x7f0f9819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f9659c640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f98100ad0 0x7f0f98197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 -- 172.21.15.78:0/3143694246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98102690 msgr2=0x7f0f9819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98102690 0x7f0f9819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 -- 172.21.15.78:0/3143694246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f98100ad0 msgr2=0x7f0f98197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f98100ad0 0x7f0f98197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 -- 172.21.15.78:0/3143694246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9819fb10 con 0x7f0f98101cd0 2024-09-16T00:10:12.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f95d9b640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98101cd0 0x7f0f98198420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800e9f0 tx=0x7f0f8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f8f7fe640 1 -- 172.21.15.78:0/3143694246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800cd80 con 0x7f0f98101cd0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f8f7fe640 1 -- 172.21.15.78:0/3143694246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f88010910 con 0x7f0f98101cd0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9819fe00 con 0x7f0f98101cd0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f8f7fe640 1 -- 172.21.15.78:0/3143694246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88018990 con 0x7f0f98101cd0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.706+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f981a0300 con 0x7f0f98101cd0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.707+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f981a0530 con 0x7f0f98101cd0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.709+0000 7f0f8f7fe640 1 -- 172.21.15.78:0/3143694246 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0f88010ac0 con 0x7f0f98101cd0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.710+0000 7f0f8f7fe640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0f840a32f0 0x7f0f840a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.710+0000 7f0f9659c640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0f840a32f0 0x7f0f840a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.710+0000 7f0f8f7fe640 1 -- 172.21.15.78:0/3143694246 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f0f880cf670 con 0x7f0f98101cd0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.710+0000 7f0f9659c640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0f840a32f0 0x7f0f840a57b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f80005ab0 tx=0x7f0f80000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:12.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.714+0000 7f0f8f7fe640 1 -- 172.21.15.78:0/3143694246 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f8809c1e0 con 0x7f0f98101cd0 2024-09-16T00:10:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.879+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0f981a08c0 con 0x7f0f98101cd0 2024-09-16T00:10:12.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.880+0000 7f0f8f7fe640 1 -- 172.21.15.78:0/3143694246 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f0f880a04e0 con 0x7f0f98101cd0 2024-09-16T00:10:12.881 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:12.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0f840a32f0 msgr2=0x7f0f840a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0f840a32f0 0x7f0f840a57b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0f80005ab0 tx=0x7f0f80000bc0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98101cd0 msgr2=0x7f0f98198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98101cd0 0x7f0f98198420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800e9f0 tx=0x7f0f8800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 shutdown_connections 2024-09-16T00:10:12.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0f840a32f0 0x7f0f840a57b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98102690 0x7f0f9819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f98101cd0 0x7f0f98198420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 --2- 172.21.15.78:0/3143694246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f98100ad0 0x7f0f98197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 >> 172.21.15.78:0/3143694246 conn(0x7f0f980fc280 msgr2=0x7f0f98069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 shutdown_connections 2024-09-16T00:10:12.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:12.882+0000 7f0f9c874640 1 -- 172.21.15.78:0/3143694246 wait complete. 2024-09-16T00:10:13.765 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:13 smithi078 ceph-mon[31157]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:13.765 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:13 smithi078 ceph-mon[31157]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T00:10:13.765 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:13 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3143694246' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:13.794 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726445411,"num_in_osds":12,"osd_in_since":1726445386,"num_remapped_pgs":0} 2024-09-16T00:10:13.795 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:10:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:13 smithi193 ceph-mon[35484]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:13 smithi193 ceph-mon[35484]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T00:10:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:13 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3143694246' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:13 smithi148 ceph-mon[35599]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:13 smithi148 ceph-mon[35599]: osdmap e75: 12 total, 12 up, 12 in 2024-09-16T00:10:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:13 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3143694246' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 -- 172.21.15.78:0/3450908635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4380103710 msgr2=0x7f438010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 --2- 172.21.15.78:0/3450908635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4380103710 0x7f438010bc00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4368009900 tx=0x7f436802a480 comp rx=0 tx=0).stop 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 -- 172.21.15.78:0/3450908635 shutdown_connections 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 --2- 172.21.15.78:0/3450908635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f438010c190 0x7f438010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 --2- 172.21.15.78:0/3450908635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4380103710 0x7f438010bc00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 --2- 172.21.15.78:0/3450908635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4380102df0 0x7f43801031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 -- 172.21.15.78:0/3450908635 >> 172.21.15.78:0/3450908635 conn(0x7f43800fc7d0 msgr2=0x7f43800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 -- 172.21.15.78:0/3450908635 shutdown_connections 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.530+0000 7f4387119640 1 -- 172.21.15.78:0/3450908635 wait complete. 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f4387119640 1 Processor -- start 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f4387119640 1 -- start start 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f4387119640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4380102df0 0x7f4380198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f4387119640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4380103710 0x7f4380198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f4387119640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f438010c190 0x7f438019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f4387119640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4380112040 con 0x7f438010c190 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f4387119640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4380111ec0 con 0x7f4380103710 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f4387119640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43801121c0 con 0x7f4380102df0 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.531+0000 7f43856b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f438010c190 0x7f438019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f43856b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f438010c190 0x7f438019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32902/0 (socket says 172.21.15.78:32902) 2024-09-16T00:10:14.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f43856b6640 1 -- 172.21.15.78:0/1558128375 learned_addr learned my addr 172.21.15.78:0/1558128375 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f437ffff640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4380103710 0x7f4380198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f4384eb5640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4380102df0 0x7f4380198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f43856b6640 1 -- 172.21.15.78:0/1558128375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4380102df0 msgr2=0x7f4380198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f43856b6640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4380102df0 0x7f4380198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f43856b6640 1 -- 172.21.15.78:0/1558128375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4380103710 msgr2=0x7f4380198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f43856b6640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4380103710 0x7f4380198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f43856b6640 1 -- 172.21.15.78:0/1558128375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438019d530 con 0x7f438010c190 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f43856b6640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f438010c190 0x7f438019ce50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4378009fd0 tx=0x7f437800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f437dffb640 1 -- 172.21.15.78:0/1558128375 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378011830 con 0x7f438010c190 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f437dffb640 1 -- 172.21.15.78:0/1558128375 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4378016070 con 0x7f438010c190 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438019d820 con 0x7f438010c190 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f437dffb640 1 -- 172.21.15.78:0/1558128375 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437800f2e0 con 0x7f438010c190 2024-09-16T00:10:14.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.532+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43801a5100 con 0x7f438010c190 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.533+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4380103250 con 0x7f438010c190 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.534+0000 7f437dffb640 1 -- 172.21.15.78:0/1558128375 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f43780119d0 con 0x7f438010c190 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.535+0000 7f437dffb640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f436c0a3240 0x7f436c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.535+0000 7f4384eb5640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f436c0a3240 0x7f436c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.535+0000 7f437dffb640 1 -- 172.21.15.78:0/1558128375 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f43780cebd0 con 0x7f438010c190 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.536+0000 7f4384eb5640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f436c0a3240 0x7f436c0a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4370007dc0 tx=0x7f43700072f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:14.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.539+0000 7f437dffb640 1 -- 172.21.15.78:0/1558128375 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f437809bd90 con 0x7f438010c190 2024-09-16T00:10:14.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.706+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4380106c80 con 0x7f438010c190 2024-09-16T00:10:14.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.707+0000 7f437dffb640 1 -- 172.21.15.78:0/1558128375 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18469 (secure 0 0 0) 0x7f437809b690 con 0x7f438010c190 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:14.709 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":75,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:10:12.474724+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f436c0a3240 msgr2=0x7f436c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f436c0a3240 0x7f436c0a5700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4370007dc0 tx=0x7f43700072f0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f438010c190 msgr2=0x7f438019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f438010c190 0x7f438019ce50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4378009fd0 tx=0x7f437800b8e0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 shutdown_connections 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f436c0a3240 0x7f436c0a5700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f438010c190 0x7f438019ce50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4380103710 0x7f4380198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 --2- 172.21.15.78:0/1558128375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4380102df0 0x7f4380198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 >> 172.21.15.78:0/1558128375 conn(0x7f43800fc7d0 msgr2=0x7f43800fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 shutdown_connections 2024-09-16T00:10:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:14.710+0000 7f4387119640 1 -- 172.21.15.78:0/1558128375 wait complete. 2024-09-16T00:10:15.527 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:06:20.608203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T00:10:15.527 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T00:10:15.769 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:15.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:15 smithi078 ceph-mon[31157]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:15.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:15 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1558128375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:15 smithi193 ceph-mon[35484]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:15 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1558128375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:15 smithi148 ceph-mon[35599]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:15 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1558128375' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:10:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 -- 172.21.15.78:0/1634503298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40106f10 msgr2=0x7f6e4010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 --2- 172.21.15.78:0/1634503298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40106f10 0x7f6e4010ea90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009900 tx=0x7f6e2802a410 comp rx=0 tx=0).stop 2024-09-16T00:10:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 -- 172.21.15.78:0/1634503298 shutdown_connections 2024-09-16T00:10:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 --2- 172.21.15.78:0/1634503298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40106f10 0x7f6e4010ea90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 --2- 172.21.15.78:0/1634503298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e40102790 0x7f6e401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 --2- 172.21.15.78:0/1634503298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e40101de0 0x7f6e401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 -- 172.21.15.78:0/1634503298 >> 172.21.15.78:0/1634503298 conn(0x7f6e400fd8c0 msgr2=0x7f6e400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 -- 172.21.15.78:0/1634503298 shutdown_connections 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.259+0000 7f6e47200640 1 -- 172.21.15.78:0/1634503298 wait complete. 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e47200640 1 Processor -- start 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e47200640 1 -- start start 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e47200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e40101de0 0x7f6e401983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e47200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40102790 0x7f6e40198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e47200640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e40106f10 0x7f6e4019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e47200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40111f70 con 0x7f6e40102790 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e47200640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40111df0 con 0x7f6e40106f10 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e47200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401120f0 con 0x7f6e40101de0 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.260+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40102790 0x7f6e40198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40102790 0x7f6e40198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40472/0 (socket says 172.21.15.78:40472) 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3ffff640 1 -- 172.21.15.78:0/3870787455 learned_addr learned my addr 172.21.15.78:0/3870787455 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e44f9c640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e40101de0 0x7f6e401983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e4579d640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e40106f10 0x7f6e4019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3ffff640 1 -- 172.21.15.78:0/3870787455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e40101de0 msgr2=0x7f6e401983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3ffff640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e40101de0 0x7f6e401983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3ffff640 1 -- 172.21.15.78:0/3870787455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e40106f10 msgr2=0x7f6e4019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3ffff640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e40106f10 0x7f6e4019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3ffff640 1 -- 172.21.15.78:0/3870787455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4019d380 con 0x7f6e40102790 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3ffff640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40102790 0x7f6e40198910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000cce0 tx=0x7f6e3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3dffb640 1 -- 172.21.15.78:0/3870787455 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30004b20 con 0x7f6e40102790 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3dffb640 1 -- 172.21.15.78:0/3870787455 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e30004cc0 con 0x7f6e40102790 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4019d670 con 0x7f6e40102790 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e3dffb640 1 -- 172.21.15.78:0/3870787455 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3000f7d0 con 0x7f6e40102790 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.261+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e401a4f50 con 0x7f6e40102790 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.262+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e401021c0 con 0x7f6e40102790 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.263+0000 7f6e3dffb640 1 -- 172.21.15.78:0/3870787455 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6e3000f970 con 0x7f6e40102790 2024-09-16T00:10:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.264+0000 7f6e3dffb640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6e2c0a32c0 0x7f6e2c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:16.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.264+0000 7f6e3dffb640 1 -- 172.21.15.78:0/3870787455 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f6e300cf500 con 0x7f6e40102790 2024-09-16T00:10:16.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.264+0000 7f6e44f9c640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6e2c0a32c0 0x7f6e2c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:16.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.264+0000 7f6e44f9c640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6e2c0a32c0 0x7f6e2c0a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e380058c0 tx=0x7f6e3800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:16.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.268+0000 7f6e3dffb640 1 -- 172.21.15.78:0/3870787455 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e30014030 con 0x7f6e40102790 2024-09-16T00:10:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.434+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6e4010a150 con 0x7f6e40102790 2024-09-16T00:10:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.435+0000 7f6e3dffb640 1 -- 172.21.15.78:0/3870787455 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f6e3009c620 con 0x7f6e40102790 2024-09-16T00:10:16.436 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-16T00:10:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.436+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6e2c0a32c0 msgr2=0x7f6e2c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:16.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.436+0000 7f6e47200640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6e2c0a32c0 0x7f6e2c0a5780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e380058c0 tx=0x7f6e3800a250 comp rx=0 tx=0).stop 2024-09-16T00:10:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40102790 msgr2=0x7f6e40198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40102790 0x7f6e40198910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000cce0 tx=0x7f6e3000a620 comp rx=0 tx=0).stop 2024-09-16T00:10:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 shutdown_connections 2024-09-16T00:10:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6e2c0a32c0 0x7f6e2c0a5780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6e40106f10 0x7f6e4019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e40102790 0x7f6e40198910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 --2- 172.21.15.78:0/3870787455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e40101de0 0x7f6e401983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 >> 172.21.15.78:0/3870787455 conn(0x7f6e400fd8c0 msgr2=0x7f6e400feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:16.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 shutdown_connections 2024-09-16T00:10:16.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:16.437+0000 7f6e47200640 1 -- 172.21.15.78:0/3870787455 wait complete. 2024-09-16T00:10:16.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:16 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3870787455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:10:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:16 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3870787455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:10:16.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:16 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3870787455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:10:17.235 INFO:tasks.cephadm:Adding mds.a on smithi078 2024-09-16T00:10:17.235 INFO:tasks.cephadm:Adding mds.d on smithi078 2024-09-16T00:10:17.235 INFO:tasks.cephadm:Adding mds.g on smithi078 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.j on smithi078 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.b on smithi148 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.e on smithi148 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.h on smithi148 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.k on smithi148 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-09-16T00:10:17.236 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-09-16T00:10:17.236 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-09-16T00:10:17.496 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:10:17.617 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:17 smithi193 ceph-mon[35484]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:17.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:17 smithi078 ceph-mon[31157]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:17 smithi148 ceph-mon[35599]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:18.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 -- 172.21.15.193:0/2842368279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 msgr2=0x7f9744103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 --2- 172.21.15.193:0/2842368279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 0x7f9744103280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9740009900 tx=0x7f974002a410 comp rx=0 tx=0).stop 2024-09-16T00:10:18.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 -- 172.21.15.193:0/2842368279 shutdown_connections 2024-09-16T00:10:18.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 --2- 172.21.15.193:0/2842368279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97441037c0 0x7f974410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 --2- 172.21.15.193:0/2842368279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 0x7f9744103280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 --2- 172.21.15.193:0/2842368279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9744101c00 0x7f9744102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 -- 172.21.15.193:0/2842368279 >> 172.21.15.193:0/2842368279 conn(0x7f97440fd390 msgr2=0x7f97440ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:18.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 -- 172.21.15.193:0/2842368279 shutdown_connections 2024-09-16T00:10:18.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.988+0000 7f974bc3b640 1 -- 172.21.15.193:0/2842368279 wait complete. 2024-09-16T00:10:18.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f974bc3b640 1 Processor -- start 2024-09-16T00:10:18.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f974bc3b640 1 -- start start 2024-09-16T00:10:18.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f974bc3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9744101c00 0x7f9744197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:18.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f974bc3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 0x7f9744198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:18.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f974bc3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97441037c0 0x7f974419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:18.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f974bc3b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f974410d880 con 0x7f9744101c00 2024-09-16T00:10:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f974bc3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f974410d700 con 0x7f97441037c0 2024-09-16T00:10:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f974bc3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f974410da00 con 0x7f9744102e00 2024-09-16T00:10:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f97491d6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 0x7f9744198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f97491d6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 0x7f9744198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:41796/0 (socket says 172.21.15.193:41796) 2024-09-16T00:10:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f97491d6640 1 -- 172.21.15.193:0/1544692045 learned_addr learned my addr 172.21.15.193:0/1544692045 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-16T00:10:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f97499d7640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9744101c00 0x7f9744197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f97491d6640 1 -- 172.21.15.193:0/1544692045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97441037c0 msgr2=0x7f974419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.989+0000 7f97491d6640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97441037c0 0x7f974419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f97491d6640 1 -- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9744101c00 msgr2=0x7f9744197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f97491d6640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9744101c00 0x7f9744197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f97491d6640 1 -- 172.21.15.193:0/1544692045 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974419fb40 con 0x7f9744102e00 2024-09-16T00:10:18.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f97491d6640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 0x7f9744198450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9740009230 tx=0x7f9740004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:18.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f9736ffd640 1 -- 172.21.15.193:0/1544692045 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974003f030 con 0x7f9744102e00 2024-09-16T00:10:18.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f9736ffd640 1 -- 172.21.15.193:0/1544692045 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9740003ce0 con 0x7f9744102e00 2024-09-16T00:10:18.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974419fdd0 con 0x7f9744102e00 2024-09-16T00:10:18.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f9736ffd640 1 -- 172.21.15.193:0/1544692045 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740035070 con 0x7f9744102e00 2024-09-16T00:10:18.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.990+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97441a0270 con 0x7f9744102e00 2024-09-16T00:10:18.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.991+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9744108ff0 con 0x7f9744102e00 2024-09-16T00:10:18.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.992+0000 7f9736ffd640 1 -- 172.21.15.193:0/1544692045 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9740042030 con 0x7f9744102e00 2024-09-16T00:10:18.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.993+0000 7f9736ffd640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97300a32f0 0x7f97300a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:18.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.993+0000 7f9736ffd640 1 -- 172.21.15.193:0/1544692045 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f97400efb80 con 0x7f9744102e00 2024-09-16T00:10:18.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.993+0000 7f97499d7640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97300a32f0 0x7f97300a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:18.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.994+0000 7f97499d7640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97300a32f0 0x7f97300a57b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97380058c0 tx=0x7f9738005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:18.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:17.997+0000 7f9736ffd640 1 -- 172.21.15.193:0/1544692045 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97400bcd40 con 0x7f9744102e00 2024-09-16T00:10:18.175 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.173+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7f97440627f0 con 0x7f97300a32f0 2024-09-16T00:10:18.182 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.182+0000 7f9736ffd640 1 -- 172.21.15.193:0/1544692045 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f97440627f0 con 0x7f97300a32f0 2024-09-16T00:10:18.183 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-09-16T00:10:18.185 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97300a32f0 msgr2=0x7f97300a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.185 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97300a32f0 0x7f97300a57b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f97380058c0 tx=0x7f9738005c80 comp rx=0 tx=0).stop 2024-09-16T00:10:18.185 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 msgr2=0x7f9744198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.185 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 0x7f9744198450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9740009230 tx=0x7f9740004430 comp rx=0 tx=0).stop 2024-09-16T00:10:18.185 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 shutdown_connections 2024-09-16T00:10:18.185 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97300a32f0 0x7f97300a57b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.185 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f97441037c0 0x7f974419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.186 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9744102e00 0x7f9744198450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.186 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 --2- 172.21.15.193:0/1544692045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9744101c00 0x7f9744197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.186 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 >> 172.21.15.193:0/1544692045 conn(0x7f97440fd390 msgr2=0x7f97441061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:18.186 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 shutdown_connections 2024-09-16T00:10:18.186 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-16T00:10:18.184+0000 7f974bc3b640 1 -- 172.21.15.193:0/1544692045 wait complete. 2024-09-16T00:10:18.890 DEBUG:teuthology.orchestra.run.smithi078:mds.a> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.a.service 2024-09-16T00:10:18.893 DEBUG:teuthology.orchestra.run.smithi078:mds.d> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.d.service 2024-09-16T00:10:18.937 DEBUG:teuthology.orchestra.run.smithi078:mds.g> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.g.service 2024-09-16T00:10:18.980 DEBUG:teuthology.orchestra.run.smithi078:mds.j> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.j.service 2024-09-16T00:10:19.023 DEBUG:teuthology.orchestra.run.smithi148:mds.b> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.b.service 2024-09-16T00:10:19.027 DEBUG:teuthology.orchestra.run.smithi148:mds.e> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.e.service 2024-09-16T00:10:19.072 DEBUG:teuthology.orchestra.run.smithi148:mds.h> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.h.service 2024-09-16T00:10:19.115 DEBUG:teuthology.orchestra.run.smithi148:mds.k> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.k.service 2024-09-16T00:10:19.158 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.c.service 2024-09-16T00:10:19.160 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.f.service 2024-09-16T00:10:19.204 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.i.service 2024-09-16T00:10:19.246 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.l.service 2024-09-16T00:10:19.290 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-16T00:10:19.290 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:19.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:19.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-16T00:10:19.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:19.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:19 smithi193 ceph-mon[35484]: Deploying daemon mds.l on smithi193 2024-09-16T00:10:19.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:19.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:19 smithi148 ceph-mon[35599]: Deploying daemon mds.l on smithi193 2024-09-16T00:10:19.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: Saving service mds.all spec with placement smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:19.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:19 smithi078 ceph-mon[31157]: Deploying daemon mds.l on smithi193 2024-09-16T00:10:19.544 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:20.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.023+0000 7fd36bcc3640 1 -- 172.21.15.78:0/4228727676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364101420 msgr2=0x7fd36410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.023+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/4228727676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364101420 0x7fd36410d950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009cd0 tx=0x7fd35c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.023+0000 7fd36bcc3640 1 -- 172.21.15.78:0/4228727676 shutdown_connections 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.023+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/4228727676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364101420 0x7fd36410d950 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.023+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/4228727676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364100a80 0x7fd364100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.023+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/4228727676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd364106a80 0x7fd364106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.023+0000 7fd36bcc3640 1 -- 172.21.15.78:0/4228727676 >> 172.21.15.78:0/4228727676 conn(0x7fd3640fc7a0 msgr2=0x7fd3640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.023+0000 7fd36bcc3640 1 -- 172.21.15.78:0/4228727676 shutdown_connections 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 -- 172.21.15.78:0/4228727676 wait complete. 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 Processor -- start 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 -- start start 2024-09-16T00:10:20.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364100a80 0x7fd364197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd364101420 0x7fd364197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364106a80 0x7fd36419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd364110e80 con 0x7fd364106a80 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd364110d00 con 0x7fd364100a80 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36bcc3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd364111000 con 0x7fd364101420 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36a260640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364106a80 0x7fd36419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36a260640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364106a80 0x7fd36419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40488/0 (socket says 172.21.15.78:40488) 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36a260640 1 -- 172.21.15.78:0/2389292231 learned_addr learned my addr 172.21.15.78:0/2389292231 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.024+0000 7fd36925e640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd364101420 0x7fd364197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd369a5f640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364100a80 0x7fd364197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd36a260640 1 -- 172.21.15.78:0/2389292231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd364101420 msgr2=0x7fd364197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd36a260640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd364101420 0x7fd364197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd36a260640 1 -- 172.21.15.78:0/2389292231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364100a80 msgr2=0x7fd364197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd36a260640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364100a80 0x7fd364197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd36a260640 1 -- 172.21.15.78:0/2389292231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36419c2c0 con 0x7fd364106a80 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd36a260640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364106a80 0x7fd36419bbe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005cc0 tx=0x7fd35c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd35affd640 1 -- 172.21.15.78:0/2389292231 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c004040 con 0x7fd364106a80 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd35affd640 1 -- 172.21.15.78:0/2389292231 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c0046c0 con 0x7fd364106a80 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36419c550 con 0x7fd364106a80 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd35affd640 1 -- 172.21.15.78:0/2389292231 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c046480 con 0x7fd364106a80 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.025+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36406a790 con 0x7fd364106a80 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.026+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3641021c0 con 0x7fd364106a80 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.027+0000 7fd35affd640 1 -- 172.21.15.78:0/2389292231 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd35c0305a0 con 0x7fd364106a80 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.028+0000 7fd35affd640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3500a5450 0x7fd3500a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.028+0000 7fd369a5f640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3500a5450 0x7fd3500a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.028+0000 7fd35affd640 1 -- 172.21.15.78:0/2389292231 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd35c02e030 con 0x7fd364106a80 2024-09-16T00:10:20.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.028+0000 7fd369a5f640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3500a5450 0x7fd3500a7910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd354005d00 tx=0x7fd35400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.032+0000 7fd35affd640 1 -- 172.21.15.78:0/2389292231 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd35c04b060 con 0x7fd364106a80 2024-09-16T00:10:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.231+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd36406b1b0 con 0x7fd364106a80 2024-09-16T00:10:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.231+0000 7fd35affd640 1 -- 172.21.15.78:0/2389292231 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fd35c0bddb0 con 0x7fd364106a80 2024-09-16T00:10:20.232 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:20.233 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":1,"btime":"2024-09-16T00:03:16:286868+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-16T00:10:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 1 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3500a5450 msgr2=0x7fd3500a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3500a5450 0x7fd3500a7910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd354005d00 tx=0x7fd35400b410 comp rx=0 tx=0).stop 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364106a80 msgr2=0x7fd36419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364106a80 0x7fd36419bbe0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd35c005cc0 tx=0x7fd35c003970 comp rx=0 tx=0).stop 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 shutdown_connections 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3500a5450 0x7fd3500a7910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd364106a80 0x7fd36419bbe0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd364101420 0x7fd364197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 --2- 172.21.15.78:0/2389292231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364100a80 0x7fd364197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 >> 172.21.15.78:0/2389292231 conn(0x7fd3640fc7a0 msgr2=0x7fd3640fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 shutdown_connections 2024-09-16T00:10:20.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:20.234+0000 7fd36bcc3640 1 -- 172.21.15.78:0/2389292231 wait complete. 2024-09-16T00:10:20.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:20 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2389292231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:20.538 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:20 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2389292231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:20.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:20 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2389292231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:20.926 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:20 smithi193 systemd[1]: Starting Ceph mds.l for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:21.180 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:20 smithi193 podman[65581]: 2024-09-16 00:10:20.8286241 +0000 UTC m=+0.015993441 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:21.180 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:20 smithi193 podman[65581]: 2024-09-16 00:10:20.966090297 +0000 UTC m=+0.153459631 container create 6b6940ac814d1fa7074e65666c2b0f25c5dc3212e23ade81675e873bdf534dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:10:21.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:21 smithi193 ceph-mon[35484]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:21.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:21 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6832/1629662299,v1:172.21.15.193:6833/1629662299] up:boot 2024-09-16T00:10:21.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:21 smithi193 ceph-mon[35484]: fsmap 1 up:standby 2024-09-16T00:10:21.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:10:21.680 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:21 smithi193 podman[65581]: 2024-09-16 00:10:21.209042624 +0000 UTC m=+0.396411959 container init 6b6940ac814d1fa7074e65666c2b0f25c5dc3212e23ade81675e873bdf534dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T00:10:21.680 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:21 smithi193 podman[65581]: 2024-09-16 00:10:21.212023361 +0000 UTC m=+0.399392697 container start 6b6940ac814d1fa7074e65666c2b0f25c5dc3212e23ade81675e873bdf534dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:10:21.681 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:21 smithi193 ceph-mds[65600]: -- 172.21.15.193:0/1293278748 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb530f9860 con 0x55cb530d7000 2024-09-16T00:10:21.681 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:21 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l[65596]: starting mds.l at 2024-09-16T00:10:21.681 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:21 smithi193 bash[65581]: 6b6940ac814d1fa7074e65666c2b0f25c5dc3212e23ade81675e873bdf534dd3 2024-09-16T00:10:21.681 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:10:21 smithi193 systemd[1]: Started Ceph mds.l for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:21.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:21 smithi148 ceph-mon[35599]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:21.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:21 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6832/1629662299,v1:172.21.15.193:6833/1629662299] up:boot 2024-09-16T00:10:21.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:21 smithi148 ceph-mon[35599]: fsmap 1 up:standby 2024-09-16T00:10:21.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:10:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:21 smithi078 ceph-mon[31157]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:21 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6832/1629662299,v1:172.21.15.193:6833/1629662299] up:boot 2024-09-16T00:10:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:21 smithi078 ceph-mon[31157]: fsmap 1 up:standby 2024-09-16T00:10:21.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:10:22.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:22.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:22.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:22.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:22 smithi148 ceph-mon[35599]: Deploying daemon mds.k on smithi148 2024-09-16T00:10:22.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:22.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:22.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:22.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:22 smithi193 ceph-mon[35484]: Deploying daemon mds.k on smithi148 2024-09-16T00:10:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:22.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:22 smithi078 ceph-mon[31157]: Deploying daemon mds.k on smithi148 2024-09-16T00:10:22.996 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:23.255 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:23.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:23 smithi148 ceph-mon[35599]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:23.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:23 smithi078 ceph-mon[31157]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:23.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:23 smithi193 ceph-mon[35484]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 -- 172.21.15.78:0/2867454415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100460 msgr2=0x7fd3a8100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 --2- 172.21.15.78:0/2867454415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100460 0x7fd3a8100840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd39800aa10 tx=0x7fd39802b720 comp rx=0 tx=0).stop 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 -- 172.21.15.78:0/2867454415 shutdown_connections 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 --2- 172.21.15.78:0/2867454415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3a8105590 0x7fd3a810d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 --2- 172.21.15.78:0/2867454415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a8100e10 0x7fd3a8104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 --2- 172.21.15.78:0/2867454415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100460 0x7fd3a8100840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 -- 172.21.15.78:0/2867454415 >> 172.21.15.78:0/2867454415 conn(0x7fd3a80fbee0 msgr2=0x7fd3a80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 -- 172.21.15.78:0/2867454415 shutdown_connections 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.723+0000 7fd3ae814640 1 -- 172.21.15.78:0/2867454415 wait complete. 2024-09-16T00:10:23.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3ae814640 1 Processor -- start 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3ae814640 1 -- start start 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3ae814640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a8100460 0x7fd3a8101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3ae814640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100e10 0x7fd3a8102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3ae814640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3a8105590 0x7fd3a8102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3ae814640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a810fd70 con 0x7fd3a8100e10 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3ae814640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a810fbf0 con 0x7fd3a8100460 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3ae814640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a810fef0 con 0x7fd3a8105590 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100e10 0x7fd3a8102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100e10 0x7fd3a8102440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40494/0 (socket says 172.21.15.78:40494) 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.724+0000 7fd3a77fe640 1 -- 172.21.15.78:0/3192397860 learned_addr learned my addr 172.21.15.78:0/3192397860 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3acdb1640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3a8105590 0x7fd3a8102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a77fe640 1 -- 172.21.15.78:0/3192397860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3a8105590 msgr2=0x7fd3a8102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a77fe640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3a8105590 0x7fd3a8102980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a77fe640 1 -- 172.21.15.78:0/3192397860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a8100460 msgr2=0x7fd3a8101f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a77fe640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a8100460 0x7fd3a8101f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a77fe640 1 -- 172.21.15.78:0/3192397860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a81a6910 con 0x7fd3a8100e10 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3acdb1640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3a8105590 0x7fd3a8102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a77fe640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100e10 0x7fd3a8102440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00d950 tx=0x7fd39c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a57fa640 1 -- 172.21.15.78:0/3192397860 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c017070 con 0x7fd3a8100e10 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a57fa640 1 -- 172.21.15.78:0/3192397860 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c009c30 con 0x7fd3a8100e10 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a81a6c00 con 0x7fd3a8100e10 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3a57fa640 1 -- 172.21.15.78:0/3192397860 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c00f6b0 con 0x7fd3a8100e10 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.725+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a81a7140 con 0x7fd3a8100e10 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.726+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a8100840 con 0x7fd3a8100e10 2024-09-16T00:10:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.727+0000 7fd3a57fa640 1 -- 172.21.15.78:0/3192397860 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd39c0109d0 con 0x7fd3a8100e10 2024-09-16T00:10:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.728+0000 7fd3a57fa640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3900a3240 0x7fd3900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.728+0000 7fd3a7fff640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3900a3240 0x7fd3900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.728+0000 7fd3a57fa640 1 -- 172.21.15.78:0/3192397860 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd39c0cf1b0 con 0x7fd3a8100e10 2024-09-16T00:10:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.728+0000 7fd3a7fff640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3900a3240 0x7fd3900a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd39802bc40 tx=0x7fd398000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.732+0000 7fd3a57fa640 1 -- 172.21.15.78:0/3192397860 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd39c009de0 con 0x7fd3a8100e10 2024-09-16T00:10:23.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.933+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3a81a7410 con 0x7fd3a8100e10 2024-09-16T00:10:23.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.933+0000 7fd3a57fa640 1 -- 172.21.15.78:0/3192397860 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fd39c09c370 con 0x7fd3a8100e10 2024-09-16T00:10:23.934 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:23.934 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"btime":"2024-09-16T00:10:21:240148+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T00:10:23.934 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 2 2024-09-16T00:10:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3900a3240 msgr2=0x7fd3900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3900a3240 0x7fd3900a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd39802bc40 tx=0x7fd398000bc0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100e10 msgr2=0x7fd3a8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100e10 0x7fd3a8102440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00d950 tx=0x7fd39c00de10 comp rx=0 tx=0).stop 2024-09-16T00:10:23.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 shutdown_connections 2024-09-16T00:10:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3900a3240 0x7fd3900a5700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3a8105590 0x7fd3a8102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3a8100e10 0x7fd3a8102440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 --2- 172.21.15.78:0/3192397860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd3a8100460 0x7fd3a8101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 >> 172.21.15.78:0/3192397860 conn(0x7fd3a80fbee0 msgr2=0x7fd3a80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 shutdown_connections 2024-09-16T00:10:23.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:23.935+0000 7fd3ae814640 1 -- 172.21.15.78:0/3192397860 wait complete. 2024-09-16T00:10:24.204 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:10:24 smithi148 systemd[1]: Started Ceph mds.k for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:24.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:24 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3192397860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:24.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:24.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:24.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:24 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:24.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:24 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3192397860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:24.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:24.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:24.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:24 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:24.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:24 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3192397860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:24.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.727 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.728 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:24.728 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:24.728 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:24.728 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:24 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:25.644 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:25 smithi078 ceph-mon[31157]: Deploying daemon mds.j on smithi078 2024-09-16T00:10:25.644 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:25 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6832/3203333506,v1:172.21.15.148:6833/3203333506] up:boot 2024-09-16T00:10:25.644 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:25 smithi078 ceph-mon[31157]: fsmap 2 up:standby 2024-09-16T00:10:25.644 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:10:25.644 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:25 smithi078 ceph-mon[31157]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:25.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:25 smithi193 ceph-mon[35484]: Deploying daemon mds.j on smithi078 2024-09-16T00:10:25.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:25 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6832/3203333506,v1:172.21.15.148:6833/3203333506] up:boot 2024-09-16T00:10:25.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:25 smithi193 ceph-mon[35484]: fsmap 2 up:standby 2024-09-16T00:10:25.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:25 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:10:25.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:25 smithi193 ceph-mon[35484]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:25.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:25 smithi148 ceph-mon[35599]: Deploying daemon mds.j on smithi078 2024-09-16T00:10:25.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:25 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6832/3203333506,v1:172.21.15.148:6833/3203333506] up:boot 2024-09-16T00:10:25.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:25 smithi148 ceph-mon[35599]: fsmap 2 up:standby 2024-09-16T00:10:25.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:25 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:10:25.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:25 smithi148 ceph-mon[35599]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:26.736 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:26 smithi078 systemd[1]: Starting Ceph mds.j for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:26.758 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:27.020 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:26 smithi078 podman[70865]: 2024-09-16 00:10:26.637861449 +0000 UTC m=+0.015835400 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:27.020 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:26 smithi078 podman[70865]: 2024-09-16 00:10:26.785743038 +0000 UTC m=+0.163716982 container create 980434cf6e6ddea37056e83169b46c00e6d2262c789ea3ac8e560b93b0d49c4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T00:10:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:27.322 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:27 smithi078 podman[70865]: 2024-09-16 00:10:27.024818902 +0000 UTC m=+0.402792840 container init 980434cf6e6ddea37056e83169b46c00e6d2262c789ea3ac8e560b93b0d49c4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T00:10:27.322 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:27 smithi078 podman[70865]: 2024-09-16 00:10:27.027855646 +0000 UTC m=+0.405829586 container start 980434cf6e6ddea37056e83169b46c00e6d2262c789ea3ac8e560b93b0d49c4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T00:10:27.322 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-mds[70913]: -- 172.21.15.78:0/1974031382 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5555ed167860 con 0x5555ed144800 2024-09-16T00:10:27.322 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j[70898]: starting mds.j at 2024-09-16T00:10:27.322 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:27 smithi078 bash[70865]: 980434cf6e6ddea37056e83169b46c00e6d2262c789ea3ac8e560b93b0d49c4d 2024-09-16T00:10:27.322 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:10:27 smithi078 systemd[1]: Started Ceph mds.j for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:27.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-mon[31157]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:27.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:27.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:27.613 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:27 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:27.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:27 smithi193 ceph-mon[35484]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:27.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:27.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:27.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:27 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 -- 172.21.15.78:0/1152539010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8102db0 msgr2=0x7f14a8103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 --2- 172.21.15.78:0/1152539010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8102db0 0x7f14a8103190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1498009930 tx=0x7f149802a420 comp rx=0 tx=0).stop 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 -- 172.21.15.78:0/1152539010 shutdown_connections 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 --2- 172.21.15.78:0/1152539010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a81082e0 0x7f14a810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 --2- 172.21.15.78:0/1152539010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14a8103760 0x7f14a8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 --2- 172.21.15.78:0/1152539010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8102db0 0x7f14a8103190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 -- 172.21.15.78:0/1152539010 >> 172.21.15.78:0/1152539010 conn(0x7f14a80fc7d0 msgr2=0x7f14a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 -- 172.21.15.78:0/1152539010 shutdown_connections 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 -- 172.21.15.78:0/1152539010 wait complete. 2024-09-16T00:10:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 Processor -- start 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.694+0000 7f14ad871640 1 -- start start 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14ad871640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14a8102db0 0x7f14a8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14ad871640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8103760 0x7f14a81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14ad871640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a81082e0 0x7f14a819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14ad871640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8112090 con 0x7f14a8103760 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14ad871640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8111f10 con 0x7f14a8102db0 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14ad871640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8112210 con 0x7f14a81082e0 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8103760 0x7f14a81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8103760 0x7f14a81989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57770/0 (socket says 172.21.15.78:57770) 2024-09-16T00:10:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a67fc640 1 -- 172.21.15.78:0/3134816507 learned_addr learned my addr 172.21.15.78:0/3134816507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a6ffd640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14a8102db0 0x7f14a8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a77fe640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a81082e0 0x7f14a819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a67fc640 1 -- 172.21.15.78:0/3134816507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a81082e0 msgr2=0x7f14a819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a67fc640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a81082e0 0x7f14a819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a67fc640 1 -- 172.21.15.78:0/3134816507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14a8102db0 msgr2=0x7f14a8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a67fc640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14a8102db0 0x7f14a8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.695+0000 7f14a67fc640 1 -- 172.21.15.78:0/3134816507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a819d430 con 0x7f14a8103760 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.696+0000 7f14a67fc640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8103760 0x7f14a81989c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f149c00c8c0 tx=0x7f149c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.696+0000 7f14ac86f640 1 -- 172.21.15.78:0/3134816507 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c007bb0 con 0x7f14a8103760 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.696+0000 7f14ac86f640 1 -- 172.21.15.78:0/3134816507 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149c007d50 con 0x7f14a8103760 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.696+0000 7f14ac86f640 1 -- 172.21.15.78:0/3134816507 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c018830 con 0x7f14a8103760 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.696+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a819d720 con 0x7f14a8103760 2024-09-16T00:10:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.696+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a81a5000 con 0x7f14a8103760 2024-09-16T00:10:27.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.697+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a806a7d0 con 0x7f14a8103760 2024-09-16T00:10:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.698+0000 7f14ac86f640 1 -- 172.21.15.78:0/3134816507 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f149c0189f0 con 0x7f14a8103760 2024-09-16T00:10:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.699+0000 7f14ac86f640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f14900a32c0 0x7f14900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.699+0000 7f14a6ffd640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f14900a32c0 0x7f14900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.699+0000 7f14ac86f640 1 -- 172.21.15.78:0/3134816507 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f149c0cf780 con 0x7f14a8103760 2024-09-16T00:10:27.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.700+0000 7f14a6ffd640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f14900a32c0 0x7f14900a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1498006010 tx=0x7f14980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:27.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.702+0000 7f14ac86f640 1 -- 172.21.15.78:0/3134816507 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f149c014030 con 0x7f14a8103760 2024-09-16T00:10:27.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:27 smithi148 ceph-mon[35599]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:27.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:27.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:27 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:27.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.910+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14a8103190 con 0x7f14a8103760 2024-09-16T00:10:27.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.911+0000 7f14ac86f640 1 -- 172.21.15.78:0/3134816507 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f149c096e20 con 0x7f14a8103760 2024-09-16T00:10:27.912 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:27.912 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":4,"btime":"2024-09-16T00:10:27:370661+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T00:10:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 4 2024-09-16T00:10:27.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f14900a32c0 msgr2=0x7f14900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:27.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f14900a32c0 0x7f14900a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1498006010 tx=0x7f14980058d0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8103760 msgr2=0x7f14a81989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:27.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8103760 0x7f14a81989c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f149c00c8c0 tx=0x7f149c00cd80 comp rx=0 tx=0).stop 2024-09-16T00:10:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 shutdown_connections 2024-09-16T00:10:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f14900a32c0 0x7f14900a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a81082e0 0x7f14a819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a8103760 0x7f14a81989c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 --2- 172.21.15.78:0/3134816507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f14a8102db0 0x7f14a8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 >> 172.21.15.78:0/3134816507 conn(0x7f14a80fc7d0 msgr2=0x7f14a80fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 shutdown_connections 2024-09-16T00:10:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:27.913+0000 7f14ad871640 1 -- 172.21.15.78:0/3134816507 wait complete. 2024-09-16T00:10:28.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:28 smithi193 ceph-mon[35484]: Deploying daemon mds.d on smithi078 2024-09-16T00:10:28.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:28 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.78:6834/4001113137,v1:172.21.15.78:6835/4001113137] up:boot 2024-09-16T00:10:28.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:28 smithi193 ceph-mon[35484]: fsmap 3 up:standby 2024-09-16T00:10:28.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:10:28.681 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:28 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3134816507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:28.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:28 smithi148 ceph-mon[35599]: Deploying daemon mds.d on smithi078 2024-09-16T00:10:28.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:28 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.78:6834/4001113137,v1:172.21.15.78:6835/4001113137] up:boot 2024-09-16T00:10:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:28 smithi148 ceph-mon[35599]: fsmap 3 up:standby 2024-09-16T00:10:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:10:28.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:28 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3134816507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:28.714 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:28 smithi078 ceph-mon[31157]: Deploying daemon mds.d on smithi078 2024-09-16T00:10:28.714 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:28 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.78:6834/4001113137,v1:172.21.15.78:6835/4001113137] up:boot 2024-09-16T00:10:28.714 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:28 smithi078 ceph-mon[31157]: fsmap 3 up:standby 2024-09-16T00:10:28.714 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:10:28.714 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:28 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3134816507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:29.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:29 smithi078 ceph-mon[31157]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:29.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:29 smithi193 ceph-mon[35484]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:29.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:29 smithi148 ceph-mon[35599]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:30.245 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 systemd[1]: Starting Ceph mds.d for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:30.520 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 podman[71277]: 2024-09-16 00:10:30.146399162 +0000 UTC m=+0.015820078 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:30.520 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 podman[71277]: 2024-09-16 00:10:30.281238216 +0000 UTC m=+0.150659125 container create 1d0226a243e8a89277ea176c4ff39bbffc0049de9a3f7c479532a0cd92fd243a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T00:10:30.743 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:31.087 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 podman[71277]: 2024-09-16 00:10:30.567848693 +0000 UTC m=+0.437269604 container init 1d0226a243e8a89277ea176c4ff39bbffc0049de9a3f7c479532a0cd92fd243a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:10:31.087 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 podman[71277]: 2024-09-16 00:10:30.570917301 +0000 UTC m=+0.440338208 container start 1d0226a243e8a89277ea176c4ff39bbffc0049de9a3f7c479532a0cd92fd243a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-16T00:10:31.087 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 ceph-mds[71296]: -- 172.21.15.78:0/3396334477 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b1fdd05860 con 0x55b1fdce3400 2024-09-16T00:10:31.087 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d[71292]: starting mds.d at 2024-09-16T00:10:31.087 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 bash[71277]: 1d0226a243e8a89277ea176c4ff39bbffc0049de9a3f7c479532a0cd92fd243a 2024-09-16T00:10:31.087 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:10:30 smithi078 systemd[1]: Started Ceph mds.d for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:31.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:31 smithi148 ceph-mon[35599]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:31.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:31.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:31.411 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:31 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:31.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 -- 172.21.15.78:0/940378433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01036e0 msgr2=0x7fe0b0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:31.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/940378433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01036e0 0x7fe0b0107b90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe098009930 tx=0x7fe09802a420 comp rx=0 tx=0).stop 2024-09-16T00:10:31.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 -- 172.21.15.78:0/940378433 shutdown_connections 2024-09-16T00:10:31.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/940378433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b01082a0 0x7fe0b010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/940378433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01036e0 0x7fe0b0107b90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/940378433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0b0102d30 0x7fe0b0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 -- 172.21.15.78:0/940378433 >> 172.21.15.78:0/940378433 conn(0x7fe0b00fc7d0 msgr2=0x7fe0b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 -- 172.21.15.78:0/940378433 shutdown_connections 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.486+0000 7fe0b5e70640 1 -- 172.21.15.78:0/940378433 wait complete. 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0b5e70640 1 Processor -- start 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0b5e70640 1 -- start start 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0b5e70640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0102d30 0x7fe0b0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0b5e70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0b01036e0 0x7fe0b0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0b5e70640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01082a0 0x7fe0b019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0b5e70640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0112140 con 0x7fe0b01082a0 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0b5e70640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0111fc0 con 0x7fe0b01036e0 2024-09-16T00:10:31.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0b5e70640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b01122c0 con 0x7fe0b0102d30 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01082a0 0x7fe0b019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01082a0 0x7fe0b019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57802/0 (socket says 172.21.15.78:57802) 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 -- 172.21.15.78:0/1167634738 learned_addr learned my addr 172.21.15.78:0/1167634738 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0a7fff640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0b01036e0 0x7fe0b0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0102d30 0x7fe0b0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 -- 172.21.15.78:0/1167634738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0102d30 msgr2=0x7fe0b0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0102d30 0x7fe0b0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 -- 172.21.15.78:0/1167634738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0b01036e0 msgr2=0x7fe0b0198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0b01036e0 0x7fe0b0198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 -- 172.21.15.78:0/1167634738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b019d6f0 con 0x7fe0b01082a0 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0a7fff640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0b01036e0 0x7fe0b0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:31.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.487+0000 7fe0affff640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01082a0 0x7fe0b019d010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b970 tx=0x7fe0a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.488+0000 7fe0ad7fa640 1 -- 172.21.15.78:0/1167634738 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8019070 con 0x7fe0b01082a0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.488+0000 7fe0ad7fa640 1 -- 172.21.15.78:0/1167634738 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a800cb40 con 0x7fe0b01082a0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.488+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b019d9e0 con 0x7fe0b01082a0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.488+0000 7fe0ad7fa640 1 -- 172.21.15.78:0/1167634738 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8007650 con 0x7fe0b01082a0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.488+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b01a52c0 con 0x7fe0b01082a0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.489+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b0103190 con 0x7fe0b01082a0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.490+0000 7fe0ad7fa640 1 -- 172.21.15.78:0/1167634738 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe0a801c070 con 0x7fe0b01082a0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.490+0000 7fe0ad7fa640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe09c0a32c0 0x7fe09c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.490+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe09c0a32c0 0x7fe09c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.491+0000 7fe0ad7fa640 1 -- 172.21.15.78:0/1167634738 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe0a80cea50 con 0x7fe0b01082a0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.491+0000 7fe0af7fe640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe09c0a32c0 0x7fe09c0a5780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009890 tx=0x7fe0a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:31.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.495+0000 7fe0ad7fa640 1 -- 172.21.15.78:0/1167634738 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0a809bc10 con 0x7fe0b01082a0 2024-09-16T00:10:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:31 smithi193 ceph-mon[35484]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:31.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:31 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:31.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.694+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0b010a2b0 con 0x7fe0b01082a0 2024-09-16T00:10:31.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:31 smithi078 ceph-mon[31157]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:31.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:31.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:31.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:31.696 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:31 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:31.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.695+0000 7fe0ad7fa640 1 -- 172.21.15.78:0/1167634738 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7fe0a809b510 con 0x7fe0b01082a0 2024-09-16T00:10:31.697 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:31.697 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":5,"btime":"2024-09-16T00:10:31:387345+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T00:10:31.697 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 5 2024-09-16T00:10:31.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe09c0a32c0 msgr2=0x7fe09c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:31.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe09c0a32c0 0x7fe09c0a5780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009890 tx=0x7fe0a0008040 comp rx=0 tx=0).stop 2024-09-16T00:10:31.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01082a0 msgr2=0x7fe0b019d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:31.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01082a0 0x7fe0b019d010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b970 tx=0x7fe0a800be30 comp rx=0 tx=0).stop 2024-09-16T00:10:31.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 shutdown_connections 2024-09-16T00:10:31.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe09c0a32c0 0x7fe09c0a5780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0b01082a0 0x7fe0b019d010 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0b01036e0 0x7fe0b0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 --2- 172.21.15.78:0/1167634738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0b0102d30 0x7fe0b0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 >> 172.21.15.78:0/1167634738 conn(0x7fe0b00fc7d0 msgr2=0x7fe0b0105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:31.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 shutdown_connections 2024-09-16T00:10:31.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:31.697+0000 7fe0b5e70640 1 -- 172.21.15.78:0/1167634738 wait complete. 2024-09-16T00:10:32.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:32 smithi148 ceph-mon[35599]: Deploying daemon mds.e on smithi148 2024-09-16T00:10:32.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:32 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.78:6836/3192121522,v1:172.21.15.78:6837/3192121522] up:boot 2024-09-16T00:10:32.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:32 smithi148 ceph-mon[35599]: fsmap 4 up:standby 2024-09-16T00:10:32.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:32 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:10:32.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:32 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1167634738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:32.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:32 smithi193 ceph-mon[35484]: Deploying daemon mds.e on smithi148 2024-09-16T00:10:32.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:32 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.78:6836/3192121522,v1:172.21.15.78:6837/3192121522] up:boot 2024-09-16T00:10:32.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:32 smithi193 ceph-mon[35484]: fsmap 4 up:standby 2024-09-16T00:10:32.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:32 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:10:32.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:32 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1167634738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:32.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:32 smithi078 ceph-mon[31157]: Deploying daemon mds.e on smithi148 2024-09-16T00:10:32.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:32 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.78:6836/3192121522,v1:172.21.15.78:6837/3192121522] up:boot 2024-09-16T00:10:32.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:32 smithi078 ceph-mon[31157]: fsmap 4 up:standby 2024-09-16T00:10:32.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:32 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:10:32.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:32 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1167634738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:32.954 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:32 smithi148 systemd[1]: Starting Ceph mds.e for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:33.212 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:32 smithi148 podman[64432]: 2024-09-16 00:10:32.85661472 +0000 UTC m=+0.015737936 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:33.212 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:32 smithi148 podman[64432]: 2024-09-16 00:10:32.967157366 +0000 UTC m=+0.126280570 container create 4052c5131330d6e9d1163f0b3856c28dc49a4ab7ca6445436d16a1ceb4cdf458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T00:10:33.212 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:33 smithi148 podman[64432]: 2024-09-16 00:10:33.211604698 +0000 UTC m=+0.370727905 container init 4052c5131330d6e9d1163f0b3856c28dc49a4ab7ca6445436d16a1ceb4cdf458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T00:10:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:33 smithi193 ceph-mon[35484]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:33 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:33 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:33 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:33 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:33.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:33 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:33.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:33 smithi148 ceph-mon[35599]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:33 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:33 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:33 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:33 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:33 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:33 smithi148 podman[64432]: 2024-09-16 00:10:33.214374033 +0000 UTC m=+0.373497239 container start 4052c5131330d6e9d1163f0b3856c28dc49a4ab7ca6445436d16a1ceb4cdf458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e) 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:33 smithi148 ceph-mds[64452]: -- 172.21.15.148:0/1220195327 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555622b37860 con 0x555622b14800 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:33 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e[64448]: starting mds.e at 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:33 smithi148 bash[64432]: 4052c5131330d6e9d1163f0b3856c28dc49a4ab7ca6445436d16a1ceb4cdf458 2024-09-16T00:10:33.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:10:33 smithi148 systemd[1]: Started Ceph mds.e for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:33.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:33 smithi078 ceph-mon[31157]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:33.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:33 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:33 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:33 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:33.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:33 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:33.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:33 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:34.398 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:34.666 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:34.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:34 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:34.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:34 smithi193 ceph-mon[35484]: Deploying daemon mds.f on smithi193 2024-09-16T00:10:34.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:34 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6834/903408403,v1:172.21.15.148:6835/903408403] up:boot 2024-09-16T00:10:34.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:34 smithi193 ceph-mon[35484]: fsmap 5 up:standby 2024-09-16T00:10:34.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:34 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:10:34.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:34 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:34.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:34 smithi148 ceph-mon[35599]: Deploying daemon mds.f on smithi193 2024-09-16T00:10:34.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:34 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6834/903408403,v1:172.21.15.148:6835/903408403] up:boot 2024-09-16T00:10:34.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:34 smithi148 ceph-mon[35599]: fsmap 5 up:standby 2024-09-16T00:10:34.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:34 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:10:34.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:34 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:34.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:34 smithi078 ceph-mon[31157]: Deploying daemon mds.f on smithi193 2024-09-16T00:10:34.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:34 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6834/903408403,v1:172.21.15.148:6835/903408403] up:boot 2024-09-16T00:10:34.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:34 smithi078 ceph-mon[31157]: fsmap 5 up:standby 2024-09-16T00:10:34.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:34 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.133+0000 7f72f57f6640 1 -- 172.21.15.78:0/3376037086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102ef0 msgr2=0x7f72f010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.133+0000 7f72f57f6640 1 --2- 172.21.15.78:0/3376037086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102ef0 0x7f72f010da40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009900 tx=0x7f72e802a480 comp rx=0 tx=0).stop 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 -- 172.21.15.78:0/3376037086 shutdown_connections 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 --2- 172.21.15.78:0/3376037086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102ef0 0x7f72f010da40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 --2- 172.21.15.78:0/3376037086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102570 0x7f72f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 --2- 172.21.15.78:0/3376037086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72f0100e20 0x7f72f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 -- 172.21.15.78:0/3376037086 >> 172.21.15.78:0/3376037086 conn(0x7f72f00fc7d0 msgr2=0x7f72f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 -- 172.21.15.78:0/3376037086 shutdown_connections 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 -- 172.21.15.78:0/3376037086 wait complete. 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 Processor -- start 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.134+0000 7f72f57f6640 1 -- start start 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72f57f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72f0100e20 0x7f72f01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72f57f6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102570 0x7f72f0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72f57f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102ef0 0x7f72f019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72f57f6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0110f80 con 0x7f72f0102570 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72f57f6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0110e00 con 0x7f72f0102ef0 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72f57f6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0111100 con 0x7f72f0100e20 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102570 0x7f72f0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102570 0x7f72f0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57822/0 (socket says 172.21.15.78:57822) 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 -- 172.21.15.78:0/1665011219 learned_addr learned my addr 172.21.15.78:0/1665011219 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72eeffd640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72f0100e20 0x7f72f01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ef7fe640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102ef0 0x7f72f019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 -- 172.21.15.78:0/1665011219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72f0100e20 msgr2=0x7f72f01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72f0100e20 0x7f72f01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 -- 172.21.15.78:0/1665011219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102ef0 msgr2=0x7f72f019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102ef0 0x7f72f019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 -- 172.21.15.78:0/1665011219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f019d490 con 0x7f72f0102570 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72eeffd640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72f0100e20 0x7f72f01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72ee7fc640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102570 0x7f72f0198a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72e400d950 tx=0x7f72e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72d7fff640 1 -- 172.21.15.78:0/1665011219 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4017070 con 0x7f72f0102570 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f019d780 con 0x7f72f0102570 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72d7fff640 1 -- 172.21.15.78:0/1665011219 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e4009c30 con 0x7f72f0102570 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.135+0000 7f72d7fff640 1 -- 172.21.15.78:0/1665011219 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e400f6b0 con 0x7f72f0102570 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.136+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f01a5060 con 0x7f72f0102570 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.136+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f0101200 con 0x7f72f0102570 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.138+0000 7f72d7fff640 1 -- 172.21.15.78:0/1665011219 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f72e40109d0 con 0x7f72f0102570 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.138+0000 7f72d7fff640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f72d80a3240 0x7f72d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.138+0000 7f72eeffd640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f72d80a3240 0x7f72d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.138+0000 7f72d7fff640 1 -- 172.21.15.78:0/1665011219 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f72e40cef80 con 0x7f72f0102570 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.139+0000 7f72eeffd640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f72d80a3240 0x7f72d80a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005fd0 tx=0x7f72e0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.141+0000 7f72d7fff640 1 -- 172.21.15.78:0/1665011219 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72e4009de0 con 0x7f72f0102570 2024-09-16T00:10:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.341+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72f0199840 con 0x7f72f0102570 2024-09-16T00:10:35.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.341+0000 7f72d7fff640 1 -- 172.21.15.78:0/1665011219 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7f72f0199840 con 0x7f72f0102570 2024-09-16T00:10:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:35.343 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":6,"btime":"2024-09-16T00:10:33:400926+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-16T00:10:35.343 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 6 2024-09-16T00:10:35.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.343+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f72d80a3240 msgr2=0x7f72d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.343+0000 7f72f57f6640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f72d80a3240 0x7f72d80a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005fd0 tx=0x7f72e0009500 comp rx=0 tx=0).stop 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102570 msgr2=0x7f72f0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102570 0x7f72f0198a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f72e400d950 tx=0x7f72e400de10 comp rx=0 tx=0).stop 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 shutdown_connections 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f72d80a3240 0x7f72d80a5700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f72f0102ef0 0x7f72f019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72f0102570 0x7f72f0198a20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 --2- 172.21.15.78:0/1665011219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72f0100e20 0x7f72f01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 >> 172.21.15.78:0/1665011219 conn(0x7f72f00fc7d0 msgr2=0x7f72f006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 shutdown_connections 2024-09-16T00:10:35.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:35.344+0000 7f72f57f6640 1 -- 172.21.15.78:0/1665011219 wait complete. 2024-09-16T00:10:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:35 smithi193 ceph-mon[35484]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:35 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1665011219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:35.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:35 smithi148 ceph-mon[35599]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:35.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:35 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1665011219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:35.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:35 smithi078 ceph-mon[31157]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:35 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1665011219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:35.849 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:10:35 smithi193 podman[65880]: 2024-09-16 00:10:35.825510617 +0000 UTC m=+0.359943007 container start 3e9c74b0cac859ea087f88c6f613cd1e62d7ada02a07ee725d4c76a57cc488ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T00:10:35.849 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:10:35 smithi193 ceph-mds[65899]: -- 172.21.15.193:0/316639617 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561e71dbc20 con 0x5561e71b9000 2024-09-16T00:10:36.180 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:10:35 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f[65895]: starting mds.f at 2024-09-16T00:10:36.180 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:10:35 smithi193 bash[65880]: 3e9c74b0cac859ea087f88c6f613cd1e62d7ada02a07ee725d4c76a57cc488ed 2024-09-16T00:10:36.180 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:10:35 smithi193 systemd[1]: Started Ceph mds.f for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:37.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:37.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:37.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:37.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: Deploying daemon mds.c on smithi193 2024-09-16T00:10:37.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:37.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] up:boot 2024-09-16T00:10:37.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: fsmap 6 up:standby 2024-09-16T00:10:37.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:36 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:10:37.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:37.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:37.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:37.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: Deploying daemon mds.c on smithi193 2024-09-16T00:10:37.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:37.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] up:boot 2024-09-16T00:10:37.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: fsmap 6 up:standby 2024-09-16T00:10:37.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:36 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:10:37.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:37.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:37.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:37.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:37.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: Deploying daemon mds.c on smithi193 2024-09-16T00:10:37.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:37.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] up:boot 2024-09-16T00:10:37.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: fsmap 6 up:standby 2024-09-16T00:10:37.455 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:36 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:10:38.041 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:38.161 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:37 smithi193 systemd[1]: Starting Ceph mds.c for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:38.301 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:38.430 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:38 smithi193 podman[66181]: 2024-09-16 00:10:38.063298016 +0000 UTC m=+0.016403627 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:38.430 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:38 smithi193 podman[66181]: 2024-09-16 00:10:38.183858809 +0000 UTC m=+0.136964419 container create f4a64c18669736473c08f5c8604861f6608e66a0c5005793b0c3a175e29429d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 -- 172.21.15.78:0/3328372012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac103660 msgr2=0x7f2dac10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 --2- 172.21.15.78:0/3328372012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac103660 0x7f2dac10bb50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009900 tx=0x7f2d9c02a480 comp rx=0 tx=0).stop 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 -- 172.21.15.78:0/3328372012 shutdown_connections 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 --2- 172.21.15.78:0/3328372012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dac10c170 0x7f2dac10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 --2- 172.21.15.78:0/3328372012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac103660 0x7f2dac10bb50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 --2- 172.21.15.78:0/3328372012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dac102d40 0x7f2dac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 -- 172.21.15.78:0/3328372012 >> 172.21.15.78:0/3328372012 conn(0x7f2dac0fc7a0 msgr2=0x7f2dac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 -- 172.21.15.78:0/3328372012 shutdown_connections 2024-09-16T00:10:38.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.794+0000 7f2db1030640 1 -- 172.21.15.78:0/3328372012 wait complete. 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2db1030640 1 Processor -- start 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2db1030640 1 -- start start 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2db1030640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dac102d40 0x7f2dac1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2db1030640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dac103660 0x7f2dac1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2db1030640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac10c170 0x7f2dac19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2db1030640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac111fb0 con 0x7f2dac10c170 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2db1030640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac111e30 con 0x7f2dac102d40 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2db1030640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac112130 con 0x7f2dac103660 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2dab59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac10c170 0x7f2dac19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2dab59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac10c170 0x7f2dac19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39846/0 (socket says 172.21.15.78:39846) 2024-09-16T00:10:38.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2dab59e640 1 -- 172.21.15.78:0/2542304281 learned_addr learned my addr 172.21.15.78:0/2542304281 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2daad9d640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dac102d40 0x7f2dac1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2daa59c640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dac103660 0x7f2dac1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2dab59e640 1 -- 172.21.15.78:0/2542304281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dac103660 msgr2=0x7f2dac1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2dab59e640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dac103660 0x7f2dac1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2dab59e640 1 -- 172.21.15.78:0/2542304281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dac102d40 msgr2=0x7f2dac1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2dab59e640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dac102d40 0x7f2dac1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.795+0000 7f2dab59e640 1 -- 172.21.15.78:0/2542304281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac19d460 con 0x7f2dac10c170 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.796+0000 7f2dab59e640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac10c170 0x7f2dac19cd80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2da400eaa0 tx=0x7f2da400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.796+0000 7f2da3fff640 1 -- 172.21.15.78:0/2542304281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4004d20 con 0x7f2dac10c170 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.796+0000 7f2da3fff640 1 -- 172.21.15.78:0/2542304281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da4005020 con 0x7f2dac10c170 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.796+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac19d750 con 0x7f2dac10c170 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.796+0000 7f2da3fff640 1 -- 172.21.15.78:0/2542304281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da40116d0 con 0x7f2dac10c170 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.796+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac1a5030 con 0x7f2dac10c170 2024-09-16T00:10:38.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.797+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dac103120 con 0x7f2dac10c170 2024-09-16T00:10:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.798+0000 7f2da3fff640 1 -- 172.21.15.78:0/2542304281 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2da40051d0 con 0x7f2dac10c170 2024-09-16T00:10:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.799+0000 7f2da3fff640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2d980a32c0 0x7f2d980a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.799+0000 7f2daad9d640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2d980a32c0 0x7f2d980a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.799+0000 7f2da3fff640 1 -- 172.21.15.78:0/2542304281 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f2da40cf680 con 0x7f2dac10c170 2024-09-16T00:10:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.799+0000 7f2daad9d640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2d980a32c0 0x7f2d980a5780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009990 tx=0x7f2d94009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:38.803+0000 7f2da3fff640 1 -- 172.21.15.78:0/2542304281 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2da4016030 con 0x7f2dac10c170 2024-09-16T00:10:38.930 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:38 smithi193 podman[66181]: 2024-09-16 00:10:38.436824165 +0000 UTC m=+0.389929778 container init f4a64c18669736473c08f5c8604861f6608e66a0c5005793b0c3a175e29429d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T00:10:38.930 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:38 smithi193 podman[66181]: 2024-09-16 00:10:38.439583589 +0000 UTC m=+0.392689200 container start f4a64c18669736473c08f5c8604861f6608e66a0c5005793b0c3a175e29429d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T00:10:38.930 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:38 smithi193 ceph-mds[66200]: -- 172.21.15.193:0/1789294083 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1c2e69c20 con 0x55a1c2e47000 2024-09-16T00:10:38.930 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c[66196]: starting mds.c at 2024-09-16T00:10:38.930 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:38 smithi193 bash[66181]: f4a64c18669736473c08f5c8604861f6608e66a0c5005793b0c3a175e29429d5 2024-09-16T00:10:38.930 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:10:38 smithi193 systemd[1]: Started Ceph mds.c for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:39.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.009+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dac02d7c0 con 0x7f2dac10c170 2024-09-16T00:10:39.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.009+0000 7f2da3fff640 1 -- 172.21.15.78:0/2542304281 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7f2da409c840 con 0x7f2dac10c170 2024-09-16T00:10:39.010 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:39.011 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":7,"btime":"2024-09-16T00:10:36:410790+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-16T00:10:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 7 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2d980a32c0 msgr2=0x7f2d980a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2d980a32c0 0x7f2d980a5780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009990 tx=0x7f2d94009110 comp rx=0 tx=0).stop 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac10c170 msgr2=0x7f2dac19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac10c170 0x7f2dac19cd80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2da400eaa0 tx=0x7f2da400eea0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 shutdown_connections 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2d980a32c0 0x7f2d980a5780 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2dac10c170 0x7f2dac19cd80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dac103660 0x7f2dac1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 --2- 172.21.15.78:0/2542304281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2dac102d40 0x7f2dac1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 >> 172.21.15.78:0/2542304281 conn(0x7f2dac0fc7a0 msgr2=0x7f2dac0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 shutdown_connections 2024-09-16T00:10:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:39.013+0000 7f2db1030640 1 -- 172.21.15.78:0/2542304281 wait complete. 2024-09-16T00:10:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: Deploying daemon mds.h on smithi148 2024-09-16T00:10:39.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2542304281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:39.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] up:boot 2024-09-16T00:10:39.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: fsmap 7 up:standby 2024-09-16T00:10:39.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:39 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: Deploying daemon mds.h on smithi148 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2542304281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] up:boot 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: fsmap 7 up:standby 2024-09-16T00:10:39.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:39 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: Deploying daemon mds.h on smithi148 2024-09-16T00:10:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2542304281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:40.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] up:boot 2024-09-16T00:10:40.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: fsmap 7 up:standby 2024-09-16T00:10:40.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:39 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:10:40.705 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:40 smithi148 systemd[1]: Starting Ceph mds.h for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:41.042 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:40 smithi148 podman[64733]: 2024-09-16 00:10:40.668933356 +0000 UTC m=+0.016031995 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:41.042 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:40 smithi148 podman[64733]: 2024-09-16 00:10:40.805674564 +0000 UTC m=+0.152773235 container create 2744b0beb61239049ef1426a181ccd7a139c5e9b3d8092eb81ece14ba98c04d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:10:41.042 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:41 smithi148 podman[64733]: 2024-09-16 00:10:41.041576559 +0000 UTC m=+0.388675185 container init 2744b0beb61239049ef1426a181ccd7a139c5e9b3d8092eb81ece14ba98c04d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T00:10:41.454 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:41 smithi148 podman[64733]: 2024-09-16 00:10:41.044553064 +0000 UTC m=+0.391651691 container start 2744b0beb61239049ef1426a181ccd7a139c5e9b3d8092eb81ece14ba98c04d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:10:41.455 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-mds[64752]: -- 172.21.15.148:0/1996815155 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564f984b860 con 0x5564f9829400 2024-09-16T00:10:41.455 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h[64748]: starting mds.h at 2024-09-16T00:10:41.455 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:41 smithi148 bash[64733]: 2744b0beb61239049ef1426a181ccd7a139c5e9b3d8092eb81ece14ba98c04d6 2024-09-16T00:10:41.455 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:10:41 smithi148 systemd[1]: Started Ceph mds.h for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:41.718 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:41.916 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:41 smithi193 ceph-mon[35484]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:41.916 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:41.916 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:41.916 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:41.916 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:41.916 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:41.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:41 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:41.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-mon[35599]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:41.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:41.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:41.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:41.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:41.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:41.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:41 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:41 smithi078 ceph-mon[31157]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:42.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:41 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 -- 172.21.15.78:0/4286826796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8101de0 msgr2=0x7fbdb81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 --2- 172.21.15.78:0/4286826796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8101de0 0x7fbdb81021c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009900 tx=0x7fbda802a410 comp rx=0 tx=0).stop 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 -- 172.21.15.78:0/4286826796 shutdown_connections 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 --2- 172.21.15.78:0/4286826796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb8106f10 0x7fbdb810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 --2- 172.21.15.78:0/4286826796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdb8102790 0x7fbdb81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 --2- 172.21.15.78:0/4286826796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8101de0 0x7fbdb81021c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 -- 172.21.15.78:0/4286826796 >> 172.21.15.78:0/4286826796 conn(0x7fbdb80fd8c0 msgr2=0x7fbdb80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 -- 172.21.15.78:0/4286826796 shutdown_connections 2024-09-16T00:10:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.471+0000 7fbdbd874640 1 -- 172.21.15.78:0/4286826796 wait complete. 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdbd874640 1 Processor -- start 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdbd874640 1 -- start start 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdbd874640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdb8101de0 0x7fbdb8198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdbd874640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb8102790 0x7fbdb8198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdbd874640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8106f10 0x7fbdb819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdbd874640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8111f30 con 0x7fbdb8106f10 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdbd874640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb8111db0 con 0x7fbdb8102790 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdbd874640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb81120b0 con 0x7fbdb8101de0 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8106f10 0x7fbdb819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8106f10 0x7fbdb819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39866/0 (socket says 172.21.15.78:39866) 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 -- 172.21.15.78:0/636077831 learned_addr learned my addr 172.21.15.78:0/636077831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:42.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdae7fc640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb8102790 0x7fbdb8198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb6ffd640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdb8101de0 0x7fbdb8198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 -- 172.21.15.78:0/636077831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdb8101de0 msgr2=0x7fbdb8198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdb8101de0 0x7fbdb8198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 -- 172.21.15.78:0/636077831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb8102790 msgr2=0x7fbdb8198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb8102790 0x7fbdb8198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 -- 172.21.15.78:0/636077831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb819d4c0 con 0x7fbdb8106f10 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.472+0000 7fbdb77fe640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8106f10 0x7fbdb819cde0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000eaa0 tx=0x7fbdb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.473+0000 7fbdbc872640 1 -- 172.21.15.78:0/636077831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb000cdb0 con 0x7fbdb8106f10 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.473+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb819d7b0 con 0x7fbdb8106f10 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.473+0000 7fbdbc872640 1 -- 172.21.15.78:0/636077831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb0005020 con 0x7fbdb8106f10 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.473+0000 7fbdbc872640 1 -- 172.21.15.78:0/636077831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00116b0 con 0x7fbdb8106f10 2024-09-16T00:10:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.473+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb81a5090 con 0x7fbdb8106f10 2024-09-16T00:10:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.474+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb81021c0 con 0x7fbdb8106f10 2024-09-16T00:10:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.475+0000 7fbdbc872640 1 -- 172.21.15.78:0/636077831 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbdb0005560 con 0x7fbdb8106f10 2024-09-16T00:10:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.476+0000 7fbdbc872640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbda40a3240 0x7fbda40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.476+0000 7fbdb6ffd640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbda40a3240 0x7fbda40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.476+0000 7fbdbc872640 1 -- 172.21.15.78:0/636077831 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fbdb00cf970 con 0x7fbdb8106f10 2024-09-16T00:10:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.476+0000 7fbdb6ffd640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbda40a3240 0x7fbda40a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009230 tx=0x7fbda80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:42.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.479+0000 7fbdbc872640 1 -- 172.21.15.78:0/636077831 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbdb0016030 con 0x7fbdb8106f10 2024-09-16T00:10:42.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.681+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbdb810a0f0 con 0x7fbdb8106f10 2024-09-16T00:10:42.682 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:42 smithi078 ceph-mon[31157]: Deploying daemon mds.i on smithi193 2024-09-16T00:10:42.682 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:42 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] up:boot 2024-09-16T00:10:42.682 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:42 smithi078 ceph-mon[31157]: fsmap 8 up:standby 2024-09-16T00:10:42.683 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:42 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:10:42.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.681+0000 7fbdbc872640 1 -- 172.21.15.78:0/636077831 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7fbdb009c4e0 con 0x7fbdb8106f10 2024-09-16T00:10:42.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:42.684 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":9,"btime":"2024-09-16T00:10:41:590498+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 9 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.683+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbda40a3240 msgr2=0x7fbda40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.683+0000 7fbdbd874640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbda40a3240 0x7fbda40a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009230 tx=0x7fbda80058d0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.683+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8106f10 msgr2=0x7fbdb819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.683+0000 7fbdbd874640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8106f10 0x7fbdb819cde0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000eaa0 tx=0x7fbdb000ef60 comp rx=0 tx=0).stop 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.684+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 shutdown_connections 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.684+0000 7fbdbd874640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbda40a3240 0x7fbda40a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.684+0000 7fbdbd874640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbdb8106f10 0x7fbdb819cde0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.684+0000 7fbdbd874640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbdb8102790 0x7fbdb8198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.684+0000 7fbdbd874640 1 --2- 172.21.15.78:0/636077831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdb8101de0 0x7fbdb8198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.684+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 >> 172.21.15.78:0/636077831 conn(0x7fbdb80fd8c0 msgr2=0x7fbdb80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:42.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.684+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 shutdown_connections 2024-09-16T00:10:42.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:42.684+0000 7fbdbd874640 1 -- 172.21.15.78:0/636077831 wait complete. 2024-09-16T00:10:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:42 smithi193 ceph-mon[35484]: Deploying daemon mds.i on smithi193 2024-09-16T00:10:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:42 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] up:boot 2024-09-16T00:10:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:42 smithi193 ceph-mon[35484]: fsmap 8 up:standby 2024-09-16T00:10:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:42 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:10:42.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:42 smithi148 ceph-mon[35599]: Deploying daemon mds.i on smithi193 2024-09-16T00:10:42.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:42 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] up:boot 2024-09-16T00:10:42.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:42 smithi148 ceph-mon[35599]: fsmap 8 up:standby 2024-09-16T00:10:42.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:42 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:10:43.349 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 systemd[1]: Starting Ceph mds.i for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:43.607 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 podman[66493]: 2024-09-16 00:10:43.251215422 +0000 UTC m=+0.015814177 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:43.607 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 podman[66493]: 2024-09-16 00:10:43.37328166 +0000 UTC m=+0.137880400 container create 39672d44980d49f7ed827a3b51f7232494faa74e54f3542e8e0b9c603669e1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T00:10:43.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:43 smithi148 ceph-mon[35599]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:43.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:43 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/636077831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:43 smithi193 ceph-mon[35484]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:43 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/636077831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:43.930 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 podman[66493]: 2024-09-16 00:10:43.64436604 +0000 UTC m=+0.408964778 container init 39672d44980d49f7ed827a3b51f7232494faa74e54f3542e8e0b9c603669e1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-16T00:10:43.930 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 podman[66493]: 2024-09-16 00:10:43.647256235 +0000 UTC m=+0.411854974 container start 39672d44980d49f7ed827a3b51f7232494faa74e54f3542e8e0b9c603669e1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:10:43.930 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 ceph-mds[66512]: -- 172.21.15.193:0/1241915912 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612b2523860 con 0x5612b2501000 2024-09-16T00:10:43.931 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i[66508]: starting mds.i at 2024-09-16T00:10:43.931 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 bash[66493]: 39672d44980d49f7ed827a3b51f7232494faa74e54f3542e8e0b9c603669e1da 2024-09-16T00:10:43.931 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:10:43 smithi193 systemd[1]: Started Ceph mds.i for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:44.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:43 smithi078 ceph-mon[31157]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:44.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:43 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/636077831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:45.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:45.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:45.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:45.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: Deploying daemon mds.g on smithi078 2024-09-16T00:10:45.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:45.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] up:boot 2024-09-16T00:10:45.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: fsmap 9 up:standby 2024-09-16T00:10:45.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:44 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:10:45.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: Deploying daemon mds.g on smithi078 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] up:boot 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: fsmap 9 up:standby 2024-09-16T00:10:45.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:44 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:10:45.246 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: Deploying daemon mds.g on smithi078 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] up:boot 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: fsmap 9 up:standby 2024-09-16T00:10:45.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:44 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:10:45.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:45.759 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:45.882 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:45 smithi078 systemd[1]: Starting Ceph mds.g for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 -- 172.21.15.78:0/2447308969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 msgr2=0x7fe55810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 --2- 172.21.15.78:0/2447308969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 0x7fe55810d950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe550009cd0 tx=0x7fe55002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 -- 172.21.15.78:0/2447308969 shutdown_connections 2024-09-16T00:10:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 --2- 172.21.15.78:0/2447308969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 0x7fe55810d950 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 --2- 172.21.15.78:0/2447308969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe558100a80 0x7fe558100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 --2- 172.21.15.78:0/2447308969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe558106a80 0x7fe558106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 -- 172.21.15.78:0/2447308969 >> 172.21.15.78:0/2447308969 conn(0x7fe5580fc7a0 msgr2=0x7fe5580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 -- 172.21.15.78:0/2447308969 shutdown_connections 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.281+0000 7fe5606d6640 1 -- 172.21.15.78:0/2447308969 wait complete. 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe5606d6640 1 Processor -- start 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe5606d6640 1 -- start start 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe5606d6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe558100a80 0x7fe5581984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe5606d6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 0x7fe558198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe5606d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe558106a80 0x7fe55819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe5606d6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe558111fc0 con 0x7fe558101420 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe5606d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe558111e40 con 0x7fe558106a80 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe5606d6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe558112140 con 0x7fe558100a80 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe55dc71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 0x7fe558198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:46.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe55dc71640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 0x7fe558198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59064/0 (socket says 172.21.15.78:59064) 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe55dc71640 1 -- 172.21.15.78:0/1595155864 learned_addr learned my addr 172.21.15.78:0/1595155864 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe55dc71640 1 -- 172.21.15.78:0/1595155864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe558100a80 msgr2=0x7fe5581984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe55dc71640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe558100a80 0x7fe5581984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe55dc71640 1 -- 172.21.15.78:0/1595155864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe558106a80 msgr2=0x7fe55819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe55dc71640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe558106a80 0x7fe55819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.282+0000 7fe55dc71640 1 -- 172.21.15.78:0/1595155864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55819d470 con 0x7fe558101420 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.283+0000 7fe55dc71640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 0x7fe558198a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe55400b4a0 tx=0x7fe55400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.283+0000 7fe54b7fe640 1 -- 172.21.15.78:0/1595155864 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554014070 con 0x7fe558101420 2024-09-16T00:10:46.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.283+0000 7fe54b7fe640 1 -- 172.21.15.78:0/1595155864 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe554010af0 con 0x7fe558101420 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.283+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55819d760 con 0x7fe558101420 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.283+0000 7fe54b7fe640 1 -- 172.21.15.78:0/1595155864 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554017d10 con 0x7fe558101420 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.283+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5581a5040 con 0x7fe558101420 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.284+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe558100ee0 con 0x7fe558101420 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.285+0000 7fe54b7fe640 1 -- 172.21.15.78:0/1595155864 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe554017070 con 0x7fe558101420 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.285+0000 7fe54b7fe640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe5440a3240 0x7fe5440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.285+0000 7fe55e472640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe5440a3240 0x7fe5440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.285+0000 7fe54b7fe640 1 -- 172.21.15.78:0/1595155864 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe5540ce8f0 con 0x7fe558101420 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.286+0000 7fe55e472640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe5440a3240 0x7fe5440a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe54c005fd0 tx=0x7fe54c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:46.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.290+0000 7fe54b7fe640 1 -- 172.21.15.78:0/1595155864 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe55409bab0 con 0x7fe558101420 2024-09-16T00:10:46.453 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:46 smithi078 podman[72119]: 2024-09-16 00:10:46.326357636 +0000 UTC m=+0.249487935 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:46.454 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:46 smithi078 podman[72119]: 2024-09-16 00:10:46.453407473 +0000 UTC m=+0.376537775 container create 0560a45c8536ca0b3e8f784f2d1a2c9f7e3a9c39a2ee16d1534f70fc75d82f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-16T00:10:46.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.498+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe55802d7c0 con 0x7fe558101420 2024-09-16T00:10:46.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.498+0000 7fe54b7fe640 1 -- 172.21.15.78:0/1595155864 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7fe55409b3b0 con 0x7fe558101420 2024-09-16T00:10:46.499 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:46.499 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":10,"btime":"2024-09-16T00:10:44:606000+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T00:10:46.500 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 10 2024-09-16T00:10:46.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe5440a3240 msgr2=0x7fe5440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe5440a3240 0x7fe5440a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe54c005fd0 tx=0x7fe54c005850 comp rx=0 tx=0).stop 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 msgr2=0x7fe558198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 0x7fe558198a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe55400b4a0 tx=0x7fe55400b960 comp rx=0 tx=0).stop 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 shutdown_connections 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe5440a3240 0x7fe5440a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe558106a80 0x7fe55819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe558101420 0x7fe558198a00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 --2- 172.21.15.78:0/1595155864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe558100a80 0x7fe5581984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.500+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 >> 172.21.15.78:0/1595155864 conn(0x7fe5580fc7a0 msgr2=0x7fe5580fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.501+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 shutdown_connections 2024-09-16T00:10:46.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:46.501+0000 7fe5606d6640 1 -- 172.21.15.78:0/1595155864 wait complete. 2024-09-16T00:10:47.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:47 smithi078 ceph-mon[31157]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:47.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:47 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1595155864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:47.520 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:47 smithi078 podman[72119]: 2024-09-16 00:10:47.420051577 +0000 UTC m=+1.343181886 container init 0560a45c8536ca0b3e8f784f2d1a2c9f7e3a9c39a2ee16d1534f70fc75d82f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T00:10:47.520 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:47 smithi078 podman[72119]: 2024-09-16 00:10:47.423374126 +0000 UTC m=+1.346504428 container start 0560a45c8536ca0b3e8f784f2d1a2c9f7e3a9c39a2ee16d1534f70fc75d82f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:10:47.521 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:47 smithi078 ceph-mds[72176]: -- 172.21.15.78:0/4090771244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bad453b860 con 0x55bad4518800 2024-09-16T00:10:47.521 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:47 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g[72169]: starting mds.g at 2024-09-16T00:10:47.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:47 smithi193 ceph-mon[35484]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:47.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:47 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1595155864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:47.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:47 smithi148 ceph-mon[35599]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:47.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:47 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1595155864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:48.010 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:47 smithi078 bash[72119]: 0560a45c8536ca0b3e8f784f2d1a2c9f7e3a9c39a2ee16d1534f70fc75d82f1f 2024-09-16T00:10:48.010 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:10:47 smithi078 systemd[1]: Started Ceph mds.g for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:48.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:48.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:48.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:48.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:48.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: Deploying daemon mds.a on smithi078 2024-09-16T00:10:48.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.78:6838/1764484583,v1:172.21.15.78:6839/1764484583] up:boot 2024-09-16T00:10:48.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: fsmap 10 up:standby 2024-09-16T00:10:48.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:48 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:10:48.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: Deploying daemon mds.a on smithi078 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.78:6838/1764484583,v1:172.21.15.78:6839/1764484583] up:boot 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: fsmap 10 up:standby 2024-09-16T00:10:48.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:48 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: Deploying daemon mds.a on smithi078 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.78:6838/1764484583,v1:172.21.15.78:6839/1764484583] up:boot 2024-09-16T00:10:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: fsmap 10 up:standby 2024-09-16T00:10:49.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:48 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:10:49.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:49.558 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:49.602 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:49 smithi078 systemd[1]: Starting Ceph mds.a for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:10:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:49 smithi193 ceph-mon[35484]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:49.952 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:49 smithi078 ceph-mon[31157]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:49 smithi148 ceph-mon[35599]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.032+0000 7f9687b7f640 1 -- 172.21.15.78:0/46725434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 msgr2=0x7f968010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.032+0000 7f9687b7f640 1 --2- 172.21.15.78:0/46725434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 0x7f968010ea90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9678009ca0 tx=0x7f967802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.032+0000 7f9687b7f640 1 -- 172.21.15.78:0/46725434 shutdown_connections 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.032+0000 7f9687b7f640 1 --2- 172.21.15.78:0/46725434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 0x7f968010ea90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.032+0000 7f9687b7f640 1 --2- 172.21.15.78:0/46725434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9680102790 0x7f96801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.032+0000 7f9687b7f640 1 --2- 172.21.15.78:0/46725434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9680101de0 0x7f96801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.032+0000 7f9687b7f640 1 -- 172.21.15.78:0/46725434 >> 172.21.15.78:0/46725434 conn(0x7f96800fd8c0 msgr2=0x7f96800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 -- 172.21.15.78:0/46725434 shutdown_connections 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 -- 172.21.15.78:0/46725434 wait complete. 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 Processor -- start 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 -- start start 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9680101de0 0x7f96800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9680102790 0x7f9680074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 0x7f9680074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:50.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9680111f90 con 0x7f9680106f10 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9680111e10 con 0x7f9680101de0 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f9687b7f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9680112110 con 0x7f9680102790 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.033+0000 7f968611c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 0x7f9680074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968611c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 0x7f9680074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59100/0 (socket says 172.21.15.78:59100) 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968611c640 1 -- 172.21.15.78:0/3890864274 learned_addr learned my addr 172.21.15.78:0/3890864274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968591b640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9680101de0 0x7f96800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968511a640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9680102790 0x7f9680074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968611c640 1 -- 172.21.15.78:0/3890864274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9680102790 msgr2=0x7f9680074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968611c640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9680102790 0x7f9680074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968611c640 1 -- 172.21.15.78:0/3890864274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9680101de0 msgr2=0x7f96800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:50.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968611c640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9680101de0 0x7f96800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968611c640 1 -- 172.21.15.78:0/3890864274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96801a68d0 con 0x7f9680106f10 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f968611c640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 0x7f9680074f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9678031a60 tx=0x7f9678031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f9676ffd640 1 -- 172.21.15.78:0/3890864274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678003b70 con 0x7f9680106f10 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f9676ffd640 1 -- 172.21.15.78:0/3890864274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9678003d10 con 0x7f9680106f10 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96801a6b60 con 0x7f9680106f10 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f9676ffd640 1 -- 172.21.15.78:0/3890864274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678004770 con 0x7f9680106f10 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.034+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96801a70a0 con 0x7f9680106f10 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.035+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96801021c0 con 0x7f9680106f10 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.036+0000 7f9676ffd640 1 -- 172.21.15.78:0/3890864274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f967802f790 con 0x7f9680106f10 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.037+0000 7f9676ffd640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f966c0a3240 0x7f966c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.037+0000 7f968591b640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f966c0a3240 0x7f966c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:50.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.037+0000 7f9676ffd640 1 -- 172.21.15.78:0/3890864274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f96780f07a0 con 0x7f9680106f10 2024-09-16T00:10:50.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.037+0000 7f968591b640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f966c0a3240 0x7f966c0a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f967c00abe0 tx=0x7f967c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:50.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.041+0000 7f9676ffd640 1 -- 172.21.15.78:0/3890864274 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96780bd960 con 0x7f9680106f10 2024-09-16T00:10:50.212 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:50 smithi078 podman[72533]: 2024-09-16 00:10:49.960013407 +0000 UTC m=+0.120840679 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:10:50.212 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:50 smithi078 podman[72533]: 2024-09-16 00:10:50.211730339 +0000 UTC m=+0.372557604 container create 38e369dd5f437043043fcad742558b16546b22d838b73fe9a860fb7d6e70a2fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T00:10:50.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.248+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96801a72d0 con 0x7f9680106f10 2024-09-16T00:10:50.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.249+0000 7f9676ffd640 1 -- 172.21.15.78:0/3890864274 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7f96780bd260 con 0x7f9680106f10 2024-09-16T00:10:50.250 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:50.250 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":11,"btime":"2024-09-16T00:10:48:426899+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T00:10:50.250 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 11 2024-09-16T00:10:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f966c0a3240 msgr2=0x7f966c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f966c0a3240 0x7f966c0a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f967c00abe0 tx=0x7f967c00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 msgr2=0x7f9680074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 0x7f9680074f70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9678031a60 tx=0x7f9678031a90 comp rx=0 tx=0).stop 2024-09-16T00:10:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 shutdown_connections 2024-09-16T00:10:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f966c0a3240 0x7f966c0a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9680106f10 0x7f9680074f70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9680102790 0x7f9680074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 --2- 172.21.15.78:0/3890864274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9680101de0 0x7f96800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:50.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 >> 172.21.15.78:0/3890864274 conn(0x7f96800fd8c0 msgr2=0x7f96800ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:50.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 shutdown_connections 2024-09-16T00:10:50.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:50.251+0000 7f9687b7f640 1 -- 172.21.15.78:0/3890864274 wait complete. 2024-09-16T00:10:50.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:50 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3890864274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:50.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:50 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3890864274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:51.011 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:50 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3890864274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:51.270 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:51 smithi078 podman[72533]: 2024-09-16 00:10:51.203881516 +0000 UTC m=+1.364708784 container init 38e369dd5f437043043fcad742558b16546b22d838b73fe9a860fb7d6e70a2fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T00:10:51.270 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:51 smithi078 podman[72533]: 2024-09-16 00:10:51.207417285 +0000 UTC m=+1.368244551 container start 38e369dd5f437043043fcad742558b16546b22d838b73fe9a860fb7d6e70a2fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-16T00:10:51.270 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-mds[72591]: -- 172.21.15.78:0/3992452817 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb6471d860 con 0x55bb646fb000 2024-09-16T00:10:51.270 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a[72583]: starting mds.a at 2024-09-16T00:10:51.608 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:51 smithi078 bash[72533]: 38e369dd5f437043043fcad742558b16546b22d838b73fe9a860fb7d6e70a2fa 2024-09-16T00:10:51.609 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:10:51 smithi078 systemd[1]: Started Ceph mds.a for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:51.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:51 smithi148 ceph-mon[35599]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:51.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:51 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:51.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:51 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:51.928 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:51 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:51.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:51 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:51.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:51 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:51.929 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:51 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:51 smithi193 ceph-mon[35484]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:51 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:51 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:51 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:51 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:51 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:51 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:52.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-mon[31157]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:52.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:52.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:52.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:52.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:10:52.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:10:52.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:51 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:52.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:52 smithi193 ceph-mon[35484]: Deploying daemon mds.b on smithi148 2024-09-16T00:10:52.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:52 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.78:6840/1822173762,v1:172.21.15.78:6841/1822173762] up:boot 2024-09-16T00:10:52.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:52 smithi193 ceph-mon[35484]: fsmap 11 up:standby 2024-09-16T00:10:52.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:52 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:10:52.930 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:52 smithi148 ceph-mon[35599]: Deploying daemon mds.b on smithi148 2024-09-16T00:10:52.931 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:52 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.78:6840/1822173762,v1:172.21.15.78:6841/1822173762] up:boot 2024-09-16T00:10:52.931 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:52 smithi148 ceph-mon[35599]: fsmap 11 up:standby 2024-09-16T00:10:52.931 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:52 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:10:53.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:52 smithi078 ceph-mon[31157]: Deploying daemon mds.b on smithi148 2024-09-16T00:10:53.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:52 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.78:6840/1822173762,v1:172.21.15.78:6841/1822173762] up:boot 2024-09-16T00:10:53.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:52 smithi078 ceph-mon[31157]: fsmap 11 up:standby 2024-09-16T00:10:53.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:52 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:10:53.043 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:53.304 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:53.622 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:10:53 smithi148 podman[65049]: 2024-09-16 00:10:53.543998428 +0000 UTC m=+0.124367546 container create 3899f630a044e7e011125c469409d4c6caf099039cb6841dbd9e8ebdbf156fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T00:10:53.708 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:53 smithi078 ceph-mon[31157]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 -- 172.21.15.78:0/1886441957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740102ef0 msgr2=0x7fb74010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 --2- 172.21.15.78:0/1886441957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740102ef0 0x7fb74010da40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb738009ca0 tx=0x7fb73802a950 comp rx=0 tx=0).stop 2024-09-16T00:10:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 -- 172.21.15.78:0/1886441957 shutdown_connections 2024-09-16T00:10:53.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 --2- 172.21.15.78:0/1886441957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740102ef0 0x7fb74010da40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 --2- 172.21.15.78:0/1886441957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740102570 0x7fb7401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 --2- 172.21.15.78:0/1886441957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb740100e20 0x7fb740101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 -- 172.21.15.78:0/1886441957 >> 172.21.15.78:0/1886441957 conn(0x7fb7400fc7d0 msgr2=0x7fb7400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 -- 172.21.15.78:0/1886441957 shutdown_connections 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.787+0000 7fb747356640 1 -- 172.21.15.78:0/1886441957 wait complete. 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb747356640 1 Processor -- start 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb747356640 1 -- start start 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb747356640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740100e20 0x7fb740198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb747356640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb740102570 0x7fb740198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb747356640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740102ef0 0x7fb74019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb747356640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb740110fa0 con 0x7fb740100e20 2024-09-16T00:10:53.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb747356640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb740110e20 con 0x7fb740102ef0 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb747356640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb740111120 con 0x7fb740102570 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb7450f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740100e20 0x7fb740198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb7450f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740100e20 0x7fb740198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59112/0 (socket says 172.21.15.78:59112) 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb7450f2640 1 -- 172.21.15.78:0/354680108 learned_addr learned my addr 172.21.15.78:0/354680108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb7450f2640 1 -- 172.21.15.78:0/354680108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb740102570 msgr2=0x7fb740198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb7450f2640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb740102570 0x7fb740198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb7450f2640 1 -- 172.21.15.78:0/354680108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740102ef0 msgr2=0x7fb74019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb7450f2640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740102ef0 0x7fb74019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.788+0000 7fb7450f2640 1 -- 172.21.15.78:0/354680108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74019d530 con 0x7fb740100e20 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.789+0000 7fb7450f2640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740100e20 0x7fb740198580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb73400b4a0 tx=0x7fb73400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.789+0000 7fb7327fc640 1 -- 172.21.15.78:0/354680108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734014070 con 0x7fb740100e20 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.789+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74019d820 con 0x7fb740100e20 2024-09-16T00:10:53.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.789+0000 7fb7327fc640 1 -- 172.21.15.78:0/354680108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb734010af0 con 0x7fb740100e20 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.789+0000 7fb7327fc640 1 -- 172.21.15.78:0/354680108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734017d10 con 0x7fb740100e20 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.789+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7400fcbb0 con 0x7fb740100e20 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.790+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb740101200 con 0x7fb740100e20 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.791+0000 7fb7327fc640 1 -- 172.21.15.78:0/354680108 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb734017070 con 0x7fb740100e20 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.791+0000 7fb7327fc640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb72c0a32c0 0x7fb72c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.792+0000 7fb7448f1640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb72c0a32c0 0x7fb72c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.792+0000 7fb7327fc640 1 -- 172.21.15.78:0/354680108 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb7340cec30 con 0x7fb740100e20 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.792+0000 7fb7448f1640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb72c0a32c0 0x7fb72c0a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7400fef90 tx=0x7fb73c005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:53.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:53.795+0000 7fb7327fc640 1 -- 172.21.15.78:0/354680108 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb73409bdf0 con 0x7fb740100e20 2024-09-16T00:10:53.905 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:53 smithi148 ceph-mon[35599]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:53.905 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:10:53 smithi148 podman[65049]: 2024-09-16 00:10:53.805064379 +0000 UTC m=+0.385433498 container init 3899f630a044e7e011125c469409d4c6caf099039cb6841dbd9e8ebdbf156fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:10:53.906 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:10:53 smithi148 podman[65049]: 2024-09-16 00:10:53.807763993 +0000 UTC m=+0.388133122 container start 3899f630a044e7e011125c469409d4c6caf099039cb6841dbd9e8ebdbf156fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T00:10:53.906 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:10:53 smithi148 ceph-mds[65068]: -- 172.21.15.148:0/2899519087 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56411b223c20 con 0x56411b201400 2024-09-16T00:10:53.906 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:10:53 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: starting mds.b at 2024-09-16T00:10:53.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:53 smithi193 ceph-mon[35484]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:54.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.009+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7401998e0 con 0x7fb740100e20 2024-09-16T00:10:54.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.009+0000 7fb7327fc640 1 -- 172.21.15.78:0/354680108 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9707 (secure 0 0 0) 0x7fb7401998e0 con 0x7fb740100e20 2024-09-16T00:10:54.010 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:54.010 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":12,"btime":"2024-09-16T00:10:51:607513+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-16T00:10:54.011 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 12 2024-09-16T00:10:54.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.011+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb72c0a32c0 msgr2=0x7fb72c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:54.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.011+0000 7fb747356640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb72c0a32c0 0x7fb72c0a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb7400fef90 tx=0x7fb73c005bf0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.011+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740100e20 msgr2=0x7fb740198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.011+0000 7fb747356640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740100e20 0x7fb740198580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb73400b4a0 tx=0x7fb73400b960 comp rx=0 tx=0).stop 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.011+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 shutdown_connections 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.011+0000 7fb747356640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb72c0a32c0 0x7fb72c0a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.011+0000 7fb747356640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb740102ef0 0x7fb74019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.012+0000 7fb747356640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb740102570 0x7fb740198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.012+0000 7fb747356640 1 --2- 172.21.15.78:0/354680108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb740100e20 0x7fb740198580 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.012+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 >> 172.21.15.78:0/354680108 conn(0x7fb7400fc7d0 msgr2=0x7fb74006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.012+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 shutdown_connections 2024-09-16T00:10:54.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:54.012+0000 7fb747356640 1 -- 172.21.15.78:0/354680108 wait complete. 2024-09-16T00:10:54.204 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:10:53 smithi148 bash[65049]: 3899f630a044e7e011125c469409d4c6caf099039cb6841dbd9e8ebdbf156fbf 2024-09-16T00:10:54.204 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:10:53 smithi148 systemd[1]: Started Ceph mds.b for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/354680108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:55.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] up:boot 2024-09-16T00:10:55.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: fsmap 12 up:standby 2024-09-16T00:10:55.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:10:55.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/354680108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:55.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:55.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:55.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] up:boot 2024-09-16T00:10:55.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: fsmap 12 up:standby 2024-09-16T00:10:55.431 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:10:55.434 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.434 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.434 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.435 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.435 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:55.435 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/354680108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:55.435 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:55.435 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:55.435 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] up:boot 2024-09-16T00:10:55.435 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: fsmap 12 up:standby 2024-09-16T00:10:55.435 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:10:56.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:56 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:56.726 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:56.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.845 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:56 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:56.853 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:56 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:10:57.313 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:57.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:57 smithi078 ceph-mon[31157]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:57.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:57 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 -- 172.21.15.78:0/3647737071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 msgr2=0x7fe100103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 --2- 172.21.15.78:0/3647737071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 0x7fe100103170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009930 tx=0x7fe0f002a420 comp rx=0 tx=0).stop 2024-09-16T00:10:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 -- 172.21.15.78:0/3647737071 shutdown_connections 2024-09-16T00:10:57.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 --2- 172.21.15.78:0/3647737071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe100108290 0x7fe10010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 --2- 172.21.15.78:0/3647737071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe100103740 0x7fe100107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 --2- 172.21.15.78:0/3647737071 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 0x7fe100103170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 -- 172.21.15.78:0/3647737071 >> 172.21.15.78:0/3647737071 conn(0x7fe1000fc7d0 msgr2=0x7fe1000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:57.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 -- 172.21.15.78:0/3647737071 shutdown_connections 2024-09-16T00:10:57.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.831+0000 7fe10622b640 1 -- 172.21.15.78:0/3647737071 wait complete. 2024-09-16T00:10:57.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe10622b640 1 Processor -- start 2024-09-16T00:10:57.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe10622b640 1 -- start start 2024-09-16T00:10:57.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe10622b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 0x7fe1001009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:57.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe10622b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe100103740 0x7fe100100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:57.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe10622b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe100108290 0x7fe100101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:57.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe10622b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe100112090 con 0x7fe100102d90 2024-09-16T00:10:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe10622b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe100111f10 con 0x7fe100103740 2024-09-16T00:10:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe10622b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe100112210 con 0x7fe100108290 2024-09-16T00:10:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 0x7fe1001009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 0x7fe1001009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52576/0 (socket says 172.21.15.78:52576) 2024-09-16T00:10:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0ff7fe640 1 -- 172.21.15.78:0/509484670 learned_addr learned my addr 172.21.15.78:0/509484670 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:57.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0feffd640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe100103740 0x7fe100100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0fffff640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe100108290 0x7fe100101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0ff7fe640 1 -- 172.21.15.78:0/509484670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe100108290 msgr2=0x7fe100101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0ff7fe640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe100108290 0x7fe100101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0ff7fe640 1 -- 172.21.15.78:0/509484670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe100103740 msgr2=0x7fe100100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0ff7fe640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe100103740 0x7fe100100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.832+0000 7fe0ff7fe640 1 -- 172.21.15.78:0/509484670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1001a6ab0 con 0x7fe100102d90 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.833+0000 7fe0ff7fe640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 0x7fe1001009f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009230 tx=0x7fe0f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.833+0000 7fe0fcff9640 1 -- 172.21.15.78:0/509484670 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f003f030 con 0x7fe100102d90 2024-09-16T00:10:57.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.833+0000 7fe0fcff9640 1 -- 172.21.15.78:0/509484670 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f0003ad0 con 0x7fe100102d90 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.833+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1001a6d40 con 0x7fe100102d90 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.833+0000 7fe0fcff9640 1 -- 172.21.15.78:0/509484670 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0035070 con 0x7fe100102d90 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.833+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1001a7210 con 0x7fe100102d90 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.834+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10006a7f0 con 0x7fe100102d90 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.835+0000 7fe0fcff9640 1 -- 172.21.15.78:0/509484670 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe0f0042030 con 0x7fe100102d90 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.836+0000 7fe0fcff9640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe0e80a32c0 0x7fe0e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.836+0000 7fe0feffd640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe0e80a32c0 0x7fe0e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.836+0000 7fe0fcff9640 1 -- 172.21.15.78:0/509484670 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe0f00ef810 con 0x7fe100102d90 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.836+0000 7fe0feffd640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe0e80a32c0 0x7fe0e80a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe100102480 tx=0x7fe0f400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:57.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:57.840+0000 7fe0fcff9640 1 -- 172.21.15.78:0/509484670 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0f00bc380 con 0x7fe100102d90 2024-09-16T00:10:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:57 smithi193 ceph-mon[35484]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:57 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:57 smithi148 ceph-mon[35599]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:57 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.064+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe10010a2a0 con 0x7fe100102d90 2024-09-16T00:10:58.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.066+0000 7fe0fcff9640 1 -- 172.21.15.78:0/509484670 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7fe0f00c0680 con 0x7fe100102d90 2024-09-16T00:10:58.067 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:58.067 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":13,"btime":"2024-09-16T00:10:54:621135+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-16T00:10:58.068 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 13 2024-09-16T00:10:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe0e80a32c0 msgr2=0x7fe0e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe0e80a32c0 0x7fe0e80a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe100102480 tx=0x7fe0f400b410 comp rx=0 tx=0).stop 2024-09-16T00:10:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 msgr2=0x7fe1001009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 0x7fe1001009f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009230 tx=0x7fe0f0004270 comp rx=0 tx=0).stop 2024-09-16T00:10:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 shutdown_connections 2024-09-16T00:10:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe0e80a32c0 0x7fe0e80a5780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe100108290 0x7fe100101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe100103740 0x7fe100100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 --2- 172.21.15.78:0/509484670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe100102d90 0x7fe1001009f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 >> 172.21.15.78:0/509484670 conn(0x7fe1000fc7d0 msgr2=0x7fe100106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 shutdown_connections 2024-09-16T00:10:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:58.069+0000 7fe10622b640 1 -- 172.21.15.78:0/509484670 wait complete. 2024-09-16T00:10:58.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.725 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:58.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:58.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.726 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:58 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/509484670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:58.754 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-16T00:10:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:58.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:58.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:58 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/509484670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:58.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:10:58.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:58 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/509484670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:10:58.998 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:10:59.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 -- 172.21.15.78:0/4229497305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 msgr2=0x7fe0a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/4229497305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 0x7fe0a010da40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe098009ca0 tx=0x7fe09802a950 comp rx=0 tx=0).stop 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 -- 172.21.15.78:0/4229497305 shutdown_connections 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/4229497305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 0x7fe0a010da40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/4229497305 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a0102570 0x7fe0a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/4229497305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a0100e20 0x7fe0a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 -- 172.21.15.78:0/4229497305 >> 172.21.15.78:0/4229497305 conn(0x7fe0a00fc7d0 msgr2=0x7fe0a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 -- 172.21.15.78:0/4229497305 shutdown_connections 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.482+0000 7fe0a66a5640 1 -- 172.21.15.78:0/4229497305 wait complete. 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a66a5640 1 Processor -- start 2024-09-16T00:10:59.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a66a5640 1 -- start start 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a66a5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a0100e20 0x7fe0a0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a66a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a0102570 0x7fe0a0197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a66a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 0x7fe0a019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a66a5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a0110f80 con 0x7fe0a0102ef0 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a66a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a0110e00 con 0x7fe0a0102570 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a66a5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a0111100 con 0x7fe0a0100e20 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a4c42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 0x7fe0a019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a4c42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 0x7fe0a019bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52594/0 (socket says 172.21.15.78:52594) 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a4c42640 1 -- 172.21.15.78:0/3914233787 learned_addr learned my addr 172.21.15.78:0/3914233787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe097fff640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a0102570 0x7fe0a0197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe09ffff640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a0100e20 0x7fe0a0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a4c42640 1 -- 172.21.15.78:0/3914233787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a0100e20 msgr2=0x7fe0a0197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a4c42640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a0100e20 0x7fe0a0197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a4c42640 1 -- 172.21.15.78:0/3914233787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a0102570 msgr2=0x7fe0a0197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a4c42640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a0102570 0x7fe0a0197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe0a4c42640 1 -- 172.21.15.78:0/3914233787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a019c2b0 con 0x7fe0a0102ef0 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.483+0000 7fe09ffff640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a0100e20 0x7fe0a0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.484+0000 7fe0a4c42640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 0x7fe0a019bbd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe098006010 tx=0x7fe09802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.484+0000 7fe09dffb640 1 -- 172.21.15.78:0/3914233787 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098034040 con 0x7fe0a0102ef0 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.484+0000 7fe09dffb640 1 -- 172.21.15.78:0/3914233787 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe098003e00 con 0x7fe0a0102ef0 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.484+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a019c540 con 0x7fe0a0102ef0 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.484+0000 7fe09dffb640 1 -- 172.21.15.78:0/3914233787 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098045480 con 0x7fe0a0102ef0 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.484+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a01a3e80 con 0x7fe0a0102ef0 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.485+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a01021b0 con 0x7fe0a0102ef0 2024-09-16T00:10:59.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.486+0000 7fe09dffb640 1 -- 172.21.15.78:0/3914233787 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe098037020 con 0x7fe0a0102ef0 2024-09-16T00:10:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.486+0000 7fe09dffb640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe08c0a3240 0x7fe08c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.487+0000 7fe09ffff640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe08c0a3240 0x7fe08c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.487+0000 7fe09dffb640 1 -- 172.21.15.78:0/3914233787 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe0980f0630 con 0x7fe0a0102ef0 2024-09-16T00:10:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.487+0000 7fe09ffff640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe08c0a3240 0x7fe08c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe09000baa0 tx=0x7fe090005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:59.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.490+0000 7fe09dffb640 1 -- 172.21.15.78:0/3914233787 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0980bd880 con 0x7fe0a0102ef0 2024-09-16T00:10:59.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.657+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe0a01a4050 con 0x7fe0a0102ef0 2024-09-16T00:10:59.658 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:10:59 smithi078 ceph-mon[31157]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:59.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.657+0000 7fe09dffb640 1 -- 172.21.15.78:0/3914233787 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7fe0980bd180 con 0x7fe0a0102ef0 2024-09-16T00:10:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:10:59.659 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-16T00:10:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:10:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-16T00:10:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-16T00:10:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:10:59.660 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe08c0a3240 msgr2=0x7fe08c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe08c0a3240 0x7fe08c0a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe09000baa0 tx=0x7fe090005d90 comp rx=0 tx=0).stop 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 msgr2=0x7fe0a019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 0x7fe0a019bbd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe098006010 tx=0x7fe09802f6f0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 shutdown_connections 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe08c0a3240 0x7fe08c0a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0a0102ef0 0x7fe0a019bbd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe0a0102570 0x7fe0a0197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 --2- 172.21.15.78:0/3914233787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a0100e20 0x7fe0a0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 >> 172.21.15.78:0/3914233787 conn(0x7fe0a00fc7d0 msgr2=0x7fe0a01067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:59.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 shutdown_connections 2024-09-16T00:10:59.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:10:59.660+0000 7fe0a66a5640 1 -- 172.21.15.78:0/3914233787 wait complete. 2024-09-16T00:10:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:10:59 smithi193 ceph-mon[35484]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:10:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:10:59 smithi148 ceph-mon[35599]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:00.340 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T00:11:00.340 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T00:11:00.588 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:00.862 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:00 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3914233787' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:11:00.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:00 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3914233787' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:11:00.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:00 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3914233787' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:11:01.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 -- 172.21.15.78:0/1747865681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2410c1a0 msgr2=0x7fed2410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 --2- 172.21.15.78:0/1747865681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2410c1a0 0x7fed2410e610 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009ca0 tx=0x7fed1c02a950 comp rx=0 tx=0).stop 2024-09-16T00:11:01.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 -- 172.21.15.78:0/1747865681 shutdown_connections 2024-09-16T00:11:01.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 --2- 172.21.15.78:0/1747865681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed2410c1a0 0x7fed2410e610 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 --2- 172.21.15.78:0/1747865681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed241036e0 0x7fed2410bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 --2- 172.21.15.78:0/1747865681 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed24102dc0 0x7fed241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 -- 172.21.15.78:0/1747865681 >> 172.21.15.78:0/1747865681 conn(0x7fed240fc7a0 msgr2=0x7fed240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 -- 172.21.15.78:0/1747865681 shutdown_connections 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 -- 172.21.15.78:0/1747865681 wait complete. 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 Processor -- start 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.088+0000 7fed2aefb640 1 -- start start 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed2aefb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed24102dc0 0x7fed24100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed2aefb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed241036e0 0x7fed24100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed2aefb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2410c1a0 0x7fed241014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed2aefb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed24111fb0 con 0x7fed24102dc0 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed2aefb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed24111e30 con 0x7fed2410c1a0 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed2aefb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fed24112130 con 0x7fed241036e0 2024-09-16T00:11:01.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed24102dc0 0x7fed24100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed24102dc0 0x7fed24100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52622/0 (socket says 172.21.15.78:52622) 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 -- 172.21.15.78:0/3723710074 learned_addr learned my addr 172.21.15.78:0/3723710074 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed23fff640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed241036e0 0x7fed24100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed29498640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2410c1a0 0x7fed241014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 -- 172.21.15.78:0/3723710074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed241036e0 msgr2=0x7fed24100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed241036e0 0x7fed24100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 -- 172.21.15.78:0/3723710074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2410c1a0 msgr2=0x7fed241014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2410c1a0 0x7fed241014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 -- 172.21.15.78:0/3723710074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed241a68f0 con 0x7fed24102dc0 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed28c97640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed24102dc0 0x7fed24100a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed1800d950 tx=0x7fed1800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:01.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed21ffb640 1 -- 172.21.15.78:0/3723710074 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18017070 con 0x7fed24102dc0 2024-09-16T00:11:01.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed241a6be0 con 0x7fed24102dc0 2024-09-16T00:11:01.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed21ffb640 1 -- 172.21.15.78:0/3723710074 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed1800de80 con 0x7fed24102dc0 2024-09-16T00:11:01.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.089+0000 7fed21ffb640 1 -- 172.21.15.78:0/3723710074 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1800f620 con 0x7fed24102dc0 2024-09-16T00:11:01.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.090+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed241a7120 con 0x7fed24102dc0 2024-09-16T00:11:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.091+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed241031a0 con 0x7fed24102dc0 2024-09-16T00:11:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.092+0000 7fed21ffb640 1 -- 172.21.15.78:0/3723710074 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fed18009c30 con 0x7fed24102dc0 2024-09-16T00:11:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.092+0000 7fed21ffb640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed140a3240 0x7fed140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.092+0000 7fed23fff640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed140a3240 0x7fed140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.092+0000 7fed21ffb640 1 -- 172.21.15.78:0/3723710074 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fed180cf030 con 0x7fed24102dc0 2024-09-16T00:11:01.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.093+0000 7fed23fff640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed140a3240 0x7fed140a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed241024b0 tx=0x7fed10009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:01.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.096+0000 7fed21ffb640 1 -- 172.21.15.78:0/3723710074 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed1809c1f0 con 0x7fed24102dc0 2024-09-16T00:11:01.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.264+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fed24106c90 con 0x7fed24102dc0 2024-09-16T00:11:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.601+0000 7fed21ffb640 1 -- 172.21.15.78:0/3723710074 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7fed1809baf0 con 0x7fed24102dc0 2024-09-16T00:11:01.603 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed140a3240 msgr2=0x7fed140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed140a3240 0x7fed140a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fed241024b0 tx=0x7fed10009110 comp rx=0 tx=0).stop 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed24102dc0 msgr2=0x7fed24100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed24102dc0 0x7fed24100a20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed1800d950 tx=0x7fed1800dd50 comp rx=0 tx=0).stop 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 shutdown_connections 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed140a3240 0x7fed140a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed2410c1a0 0x7fed241014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed241036e0 0x7fed24100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 --2- 172.21.15.78:0/3723710074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed24102dc0 0x7fed24100a20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 >> 172.21.15.78:0/3723710074 conn(0x7fed240fc7a0 msgr2=0x7fed240fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:01.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 shutdown_connections 2024-09-16T00:11:01.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:01.604+0000 7fed2aefb640 1 -- 172.21.15.78:0/3723710074 wait complete. 2024-09-16T00:11:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:01 smithi193 ceph-mon[35484]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:01 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3723710074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:11:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:01 smithi148 ceph-mon[35599]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:01 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3723710074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:11:01.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:01 smithi078 ceph-mon[31157]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:01.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:01 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3723710074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:11:02.332 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-16T00:11:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:02.872 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:02 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3723710074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:11:02.872 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:02 smithi078 ceph-mon[31157]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:11:02.872 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:02 smithi078 ceph-mon[31157]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:11:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:02 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3723710074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:11:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:02 smithi193 ceph-mon[35484]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:11:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:02 smithi193 ceph-mon[35484]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:11:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:02 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3723710074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:11:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:02 smithi148 ceph-mon[35599]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:11:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:02 smithi148 ceph-mon[35599]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:11:03.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 -- 172.21.15.78:0/2684984382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888106f10 msgr2=0x7fa88810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 --2- 172.21.15.78:0/2684984382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888106f10 0x7fa88810ea90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa880009ca0 tx=0x7fa88002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 -- 172.21.15.78:0/2684984382 shutdown_connections 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 --2- 172.21.15.78:0/2684984382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888106f10 0x7fa88810ea90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 --2- 172.21.15.78:0/2684984382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888102790 0x7fa8881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 --2- 172.21.15.78:0/2684984382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa888101de0 0x7fa8881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 -- 172.21.15.78:0/2684984382 >> 172.21.15.78:0/2684984382 conn(0x7fa8880fd8c0 msgr2=0x7fa8880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 -- 172.21.15.78:0/2684984382 shutdown_connections 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 -- 172.21.15.78:0/2684984382 wait complete. 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 Processor -- start 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.073+0000 7fa890394640 1 -- start start 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa890394640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa888101de0 0x7fa888198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa890394640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102790 0x7fa888198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa890394640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888106f10 0x7fa88819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa890394640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa888112010 con 0x7fa888102790 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa890394640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa888111e90 con 0x7fa888106f10 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa890394640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa888112190 con 0x7fa888101de0 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102790 0x7fa888198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102790 0x7fa888198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52646/0 (socket says 172.21.15.78:52646) 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 -- 172.21.15.78:0/3780655524 learned_addr learned my addr 172.21.15.78:0/3780655524 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88e931640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888106f10 0x7fa88819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88e130640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa888101de0 0x7fa888198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 -- 172.21.15.78:0/3780655524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa888101de0 msgr2=0x7fa888198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa888101de0 0x7fa888198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 -- 172.21.15.78:0/3780655524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888106f10 msgr2=0x7fa88819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888106f10 0x7fa88819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 -- 172.21.15.78:0/3780655524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88819d510 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88e130640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa888101de0 0x7fa888198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa88d92f640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102790 0x7fa888198aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa87800b4a0 tx=0x7fa87800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa87f7fe640 1 -- 172.21.15.78:0/3780655524 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878014070 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88819d800 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa87f7fe640 1 -- 172.21.15.78:0/3780655524 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa878010af0 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa87f7fe640 1 -- 172.21.15.78:0/3780655524 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878017d10 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.074+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8881a50e0 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.075+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa888102240 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.077+0000 7fa87f7fe640 1 -- 172.21.15.78:0/3780655524 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa878017070 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.077+0000 7fa87f7fe640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8740a3240 0x7fa8740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.077+0000 7fa88e130640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8740a3240 0x7fa8740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.077+0000 7fa87f7fe640 1 -- 172.21.15.78:0/3780655524 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7fa8780ced40 con 0x7fa888102790 2024-09-16T00:11:03.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.078+0000 7fa88e130640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8740a3240 0x7fa8740a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa884009930 tx=0x7fa884008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:03.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.081+0000 7fa87f7fe640 1 -- 172.21.15.78:0/3780655524 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa87809bd90 con 0x7fa888102790 2024-09-16T00:11:03.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.248+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa888109b60 con 0x7fa888102790 2024-09-16T00:11:03.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.616+0000 7fa87f7fe640 1 -- 172.21.15.78:0/3780655524 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v78) ==== 171+0+0 (secure 0 0 0) 0x7fa87809b690 con 0x7fa888102790 2024-09-16T00:11:03.619 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-16T00:11:03.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8740a3240 msgr2=0x7fa8740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8740a3240 0x7fa8740a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa884009930 tx=0x7fa884008040 comp rx=0 tx=0).stop 2024-09-16T00:11:03.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102790 msgr2=0x7fa888198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102790 0x7fa888198aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa87800b4a0 tx=0x7fa87800b960 comp rx=0 tx=0).stop 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 shutdown_connections 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8740a3240 0x7fa8740a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa888106f10 0x7fa88819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa888102790 0x7fa888198aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 --2- 172.21.15.78:0/3780655524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa888101de0 0x7fa888198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 >> 172.21.15.78:0/3780655524 conn(0x7fa8880fd8c0 msgr2=0x7fa8880fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 shutdown_connections 2024-09-16T00:11:03.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:03.619+0000 7fa890394640 1 -- 172.21.15.78:0/3780655524 wait complete. 2024-09-16T00:11:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:03 smithi193 ceph-mon[35484]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:03 smithi193 ceph-mon[35484]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:11:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:03 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3780655524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:11:03.934 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:03 smithi078 ceph-mon[31157]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:03.934 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:03 smithi078 ceph-mon[31157]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:11:03.934 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:03 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3780655524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:11:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:03 smithi148 ceph-mon[35599]: pgmap v276: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:03 smithi148 ceph-mon[35599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:11:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:03 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3780655524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:11:04.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T00:11:04.536 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:04.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:04 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3780655524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:11:04.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:04 smithi193 ceph-mon[35484]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:11:04.940 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:04 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3780655524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:11:04.940 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:04 smithi078 ceph-mon[31157]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:11:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:04 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3780655524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:11:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:04 smithi148 ceph-mon[35599]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:11:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/2378566936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0101420 msgr2=0x7f9ea010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/2378566936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0101420 0x7f9ea010d950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009cd0 tx=0x7f9e9802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/2378566936 shutdown_connections 2024-09-16T00:11:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/2378566936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0101420 0x7f9ea010d950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/2378566936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ea0100a80 0x7f9ea0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/2378566936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea0106a80 0x7f9ea0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/2378566936 >> 172.21.15.78:0/2378566936 conn(0x7f9ea00fc7a0 msgr2=0x7f9ea00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/2378566936 shutdown_connections 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.021+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/2378566936 wait complete. 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9ea4fa6640 1 Processor -- start 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9ea4fa6640 1 -- start start 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9ea4fa6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea0100a80 0x7f9ea01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9ea4fa6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ea0101420 0x7f9ea0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9ea4fa6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0106a80 0x7f9ea019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9ea4fa6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0110f20 con 0x7f9ea0106a80 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9ea4fa6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea0110da0 con 0x7f9ea0100a80 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9ea4fa6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea01110a0 con 0x7f9ea0101420 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9e9ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0106a80 0x7f9ea019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9e9ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0106a80 0x7f9ea019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52676/0 (socket says 172.21.15.78:52676) 2024-09-16T00:11:05.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.022+0000 7f9e9ed9d640 1 -- 172.21.15.78:0/3757013466 learned_addr learned my addr 172.21.15.78:0/3757013466 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e9e59c640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea0100a80 0x7f9ea01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e97fff640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ea0101420 0x7f9ea0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e9ed9d640 1 -- 172.21.15.78:0/3757013466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ea0101420 msgr2=0x7f9ea0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e9ed9d640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ea0101420 0x7f9ea0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e9ed9d640 1 -- 172.21.15.78:0/3757013466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea0100a80 msgr2=0x7f9ea01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e9ed9d640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea0100a80 0x7f9ea01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e9ed9d640 1 -- 172.21.15.78:0/3757013466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea019d550 con 0x7f9ea0106a80 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e9e59c640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea0100a80 0x7f9ea01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e9ed9d640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0106a80 0x7f9ea019ce70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e98006010 tx=0x7f9e98003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e977fe640 1 -- 172.21.15.78:0/3757013466 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98004450 con 0x7f9ea0106a80 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea019d7e0 con 0x7f9ea0106a80 2024-09-16T00:11:05.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e977fe640 1 -- 172.21.15.78:0/3757013466 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e98003e10 con 0x7f9ea0106a80 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9e977fe640 1 -- 172.21.15.78:0/3757013466 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980466e0 con 0x7f9ea0106a80 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.023+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea01a5120 con 0x7f9ea0106a80 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.024+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea0100f60 con 0x7f9ea0106a80 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.025+0000 7f9e977fe640 1 -- 172.21.15.78:0/3757013466 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9e980305a0 con 0x7f9ea0106a80 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.026+0000 7f9e977fe640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e8c0a3240 0x7f9e8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.026+0000 7f9e9e59c640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e8c0a3240 0x7f9e8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.026+0000 7f9e977fe640 1 -- 172.21.15.78:0/3757013466 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f9e9802e030 con 0x7f9ea0106a80 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.026+0000 7f9e9e59c640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e8c0a3240 0x7f9e8c0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009930 tx=0x7f9e90008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:05.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.029+0000 7f9e977fe640 1 -- 172.21.15.78:0/3757013466 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e9804b060 con 0x7f9ea0106a80 2024-09-16T00:11:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.232+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9ea0199900 con 0x7f9ea0106a80 2024-09-16T00:11:05.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.623+0000 7f9e977fe640 1 -- 172.21.15.78:0/3757013466 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f9ea0199900 con 0x7f9ea0106a80 2024-09-16T00:11:05.626 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T00:11:05.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e8c0a3240 msgr2=0x7f9e8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e8c0a3240 0x7f9e8c0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009930 tx=0x7f9e90008040 comp rx=0 tx=0).stop 2024-09-16T00:11:05.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0106a80 msgr2=0x7f9ea019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0106a80 0x7f9ea019ce70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9e98006010 tx=0x7f9e98003c20 comp rx=0 tx=0).stop 2024-09-16T00:11:05.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 shutdown_connections 2024-09-16T00:11:05.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e8c0a3240 0x7f9e8c0a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ea0106a80 0x7f9ea019ce70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ea0101420 0x7f9ea0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 --2- 172.21.15.78:0/3757013466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ea0100a80 0x7f9ea01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 >> 172.21.15.78:0/3757013466 conn(0x7f9ea00fc7a0 msgr2=0x7f9ea006aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:05.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 shutdown_connections 2024-09-16T00:11:05.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:05.626+0000 7f9ea4fa6640 1 -- 172.21.15.78:0/3757013466 wait complete. 2024-09-16T00:11:05.910 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:05 smithi148 ceph-mon[35599]: pgmap v279: 97 pgs: 1 creating+activating, 9 active+clean, 87 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:05.911 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:05 smithi148 ceph-mon[35599]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:11:05.911 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:05 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3757013466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:11:05.911 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:05 smithi148 ceph-mon[35599]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:11:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:05 smithi193 ceph-mon[35484]: pgmap v279: 97 pgs: 1 creating+activating, 9 active+clean, 87 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:05 smithi193 ceph-mon[35484]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:11:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:05 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3757013466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:11:05.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:05 smithi193 ceph-mon[35484]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:11:06.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:05 smithi078 ceph-mon[31157]: pgmap v279: 97 pgs: 1 creating+activating, 9 active+clean, 87 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:06.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:05 smithi078 ceph-mon[31157]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:11:06.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:05 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3757013466' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:11:06.002 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:05 smithi078 ceph-mon[31157]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:11:06.348 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-16T00:11:06.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-16T00:11:06.596 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:06 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3757013466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:11:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:06 smithi193 ceph-mon[35484]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:11:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:06 smithi193 ceph-mon[35484]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:11:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:06 smithi193 ceph-mon[35484]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:11:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:06 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] up:active 2024-09-16T00:11:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:06 smithi193 ceph-mon[35484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:06.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:06 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3757013466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:11:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:06 smithi148 ceph-mon[35599]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:11:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:06 smithi148 ceph-mon[35599]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:11:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:06 smithi148 ceph-mon[35599]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:11:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:06 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] up:active 2024-09-16T00:11:06.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:06 smithi148 ceph-mon[35599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:06.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:06 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3757013466' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:11:06.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:06 smithi078 ceph-mon[31157]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:11:06.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:06 smithi078 ceph-mon[31157]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:11:06.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:06 smithi078 ceph-mon[31157]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:11:06.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:06 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] up:active 2024-09-16T00:11:06.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:06 smithi078 ceph-mon[31157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:07.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 -- 172.21.15.78:0/1542921047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 msgr2=0x7ff274100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 --2- 172.21.15.78:0/1542921047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 0x7ff274100ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff260009930 tx=0x7ff26002a420 comp rx=0 tx=0).stop 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 -- 172.21.15.78:0/1542921047 shutdown_connections 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 --2- 172.21.15.78:0/1542921047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff274101420 0x7ff27410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 --2- 172.21.15.78:0/1542921047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 0x7ff274100ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 --2- 172.21.15.78:0/1542921047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff274106a80 0x7ff274106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 -- 172.21.15.78:0/1542921047 >> 172.21.15.78:0/1542921047 conn(0x7ff2740fc7a0 msgr2=0x7ff2740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 -- 172.21.15.78:0/1542921047 shutdown_connections 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 -- 172.21.15.78:0/1542921047 wait complete. 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 Processor -- start 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.038+0000 7ff27adc4640 1 -- start start 2024-09-16T00:11:07.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff27adc4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 0x7ff2741974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff27adc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff274101420 0x7ff274197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff27adc4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff274106a80 0x7ff27419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff27adc4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff274110f20 con 0x7ff274100a80 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff27adc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff274110da0 con 0x7ff274101420 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff27adc4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2741110a0 con 0x7ff274106a80 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 0x7ff2741974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 0x7ff2741974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58504/0 (socket says 172.21.15.78:58504) 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 -- 172.21.15.78:0/36644273 learned_addr learned my addr 172.21.15.78:0/36644273 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff279361640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff274106a80 0x7ff27419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff273fff640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff274101420 0x7ff274197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 -- 172.21.15.78:0/36644273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff274106a80 msgr2=0x7ff27419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff274106a80 0x7ff27419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 -- 172.21.15.78:0/36644273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff274101420 msgr2=0x7ff274197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff274101420 0x7ff274197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 -- 172.21.15.78:0/36644273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27419c470 con 0x7ff274100a80 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff273fff640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff274101420 0x7ff274197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff279361640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff274106a80 0x7ff27419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff278b60640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 0x7ff2741974c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b4a0 tx=0x7ff26800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff271ffb640 1 -- 172.21.15.78:0/36644273 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268014070 con 0x7ff274100a80 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff271ffb640 1 -- 172.21.15.78:0/36644273 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff268010af0 con 0x7ff274100a80 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27419c760 con 0x7ff274100a80 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.039+0000 7ff271ffb640 1 -- 172.21.15.78:0/36644273 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268017d10 con 0x7ff274100a80 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.040+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2741a4040 con 0x7ff274100a80 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.041+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff274100f60 con 0x7ff274100a80 2024-09-16T00:11:07.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.042+0000 7ff271ffb640 1 -- 172.21.15.78:0/36644273 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff268017070 con 0x7ff274100a80 2024-09-16T00:11:07.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.042+0000 7ff271ffb640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2640a3240 0x7ff2640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.042+0000 7ff273fff640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2640a3240 0x7ff2640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.042+0000 7ff271ffb640 1 -- 172.21.15.78:0/36644273 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff2680cec40 con 0x7ff274100a80 2024-09-16T00:11:07.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.043+0000 7ff273fff640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2640a3240 0x7ff2640a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff260000c00 tx=0x7ff260032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:07.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.046+0000 7ff271ffb640 1 -- 172.21.15.78:0/36644273 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff26809bab0 con 0x7ff274100a80 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.220+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff274109070 con 0x7ff274100a80 2024-09-16T00:11:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.642+0000 7ff271ffb640 1 -- 172.21.15.78:0/36644273 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7ff26809b3b0 con 0x7ff274100a80 2024-09-16T00:11:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2640a3240 msgr2=0x7ff2640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2640a3240 0x7ff2640a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff260000c00 tx=0x7ff260032d50 comp rx=0 tx=0).stop 2024-09-16T00:11:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 msgr2=0x7ff2741974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 0x7ff2741974c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b4a0 tx=0x7ff26800b960 comp rx=0 tx=0).stop 2024-09-16T00:11:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 shutdown_connections 2024-09-16T00:11:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2640a3240 0x7ff2640a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff274106a80 0x7ff27419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff274101420 0x7ff274197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 --2- 172.21.15.78:0/36644273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff274100a80 0x7ff2741974c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 >> 172.21.15.78:0/36644273 conn(0x7ff2740fc7a0 msgr2=0x7ff2740fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 shutdown_connections 2024-09-16T00:11:07.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:07.645+0000 7ff27adc4640 1 -- 172.21.15.78:0/36644273 wait complete. 2024-09-16T00:11:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:07 smithi193 ceph-mon[35484]: pgmap v282: 97 pgs: 4 creating+peering, 1 creating+activating, 49 active+clean, 43 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:07 smithi193 ceph-mon[35484]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:11:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:07 smithi193 ceph-mon[35484]: Cluster is now healthy 2024-09-16T00:11:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:07 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/36644273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:11:07.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:07 smithi078 ceph-mon[31157]: pgmap v282: 97 pgs: 4 creating+peering, 1 creating+activating, 49 active+clean, 43 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:07.954 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:07 smithi078 ceph-mon[31157]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:11:07.954 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:07 smithi078 ceph-mon[31157]: Cluster is now healthy 2024-09-16T00:11:07.954 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:07 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/36644273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:11:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:07 smithi148 ceph-mon[35599]: pgmap v282: 97 pgs: 4 creating+peering, 1 creating+activating, 49 active+clean, 43 unknown; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:07 smithi148 ceph-mon[35599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:11:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:07 smithi148 ceph-mon[35599]: Cluster is now healthy 2024-09-16T00:11:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:07 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/36644273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:11:08.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T00:11:08.569 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:08.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:08 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/36644273' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:11:08.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:08 smithi193 ceph-mon[35484]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:11:08.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:08 smithi193 ceph-mon[35484]: pgmap v284: 97 pgs: 4 creating+peering, 85 active+clean, 8 unknown; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 428 B/s wr, 2 op/s 2024-09-16T00:11:08.949 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:08 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/36644273' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:11:08.950 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:08 smithi078 ceph-mon[31157]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:11:08.950 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:08 smithi078 ceph-mon[31157]: pgmap v284: 97 pgs: 4 creating+peering, 85 active+clean, 8 unknown; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 428 B/s wr, 2 op/s 2024-09-16T00:11:08.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:08 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/36644273' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:11:08.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:08 smithi148 ceph-mon[35599]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:11:08.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:08 smithi148 ceph-mon[35599]: pgmap v284: 97 pgs: 4 creating+peering, 85 active+clean, 8 unknown; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 428 B/s wr, 2 op/s 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 -- 172.21.15.78:0/1008604222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102570 msgr2=0x7fcec81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/1008604222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102570 0x7fcec81029b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc02a480 comp rx=0 tx=0).stop 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 -- 172.21.15.78:0/1008604222 shutdown_connections 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/1008604222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec8102ef0 0x7fcec810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/1008604222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102570 0x7fcec81029b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/1008604222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcec8100e20 0x7fcec8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 -- 172.21.15.78:0/1008604222 >> 172.21.15.78:0/1008604222 conn(0x7fcec80fc7d0 msgr2=0x7fcec80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 -- 172.21.15.78:0/1008604222 shutdown_connections 2024-09-16T00:11:09.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.035+0000 7fcecd7d5640 1 -- 172.21.15.78:0/1008604222 wait complete. 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcecd7d5640 1 Processor -- start 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcecd7d5640 1 -- start start 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcecd7d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec8100e20 0x7fcec80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcecd7d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcec8102570 0x7fcec80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcecd7d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102ef0 0x7fcec80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcecd7d5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8110ff0 con 0x7fcec8102ef0 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcecd7d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8110e70 con 0x7fcec8102570 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcecd7d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcec8111170 con 0x7fcec8100e20 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcec77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102ef0 0x7fcec80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcec77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102ef0 0x7fcec80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58516/0 (socket says 172.21.15.78:58516) 2024-09-16T00:11:09.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcec77fe640 1 -- 172.21.15.78:0/3479718466 learned_addr learned my addr 172.21.15.78:0/3479718466 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcec6ffd640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec8100e20 0x7fcec80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.036+0000 7fcec77fe640 1 -- 172.21.15.78:0/3479718466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec8100e20 msgr2=0x7fcec80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fcec77fe640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec8100e20 0x7fcec80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fcec77fe640 1 -- 172.21.15.78:0/3479718466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcec8102570 msgr2=0x7fcec80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fcec77fe640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcec8102570 0x7fcec80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fcec77fe640 1 -- 172.21.15.78:0/3479718466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec81a6a60 con 0x7fcec8102ef0 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fcec6ffd640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec8100e20 0x7fcec80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fcec77fe640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102ef0 0x7fcec80fd700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcec000eaa0 tx=0x7fcec000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fceaffff640 1 -- 172.21.15.78:0/3479718466 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec000cdb0 con 0x7fcec8102ef0 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec81a6d50 con 0x7fcec8102ef0 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fceaffff640 1 -- 172.21.15.78:0/3479718466 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec0005020 con 0x7fcec8102ef0 2024-09-16T00:11:09.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fceaffff640 1 -- 172.21.15.78:0/3479718466 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec00116b0 con 0x7fcec8102ef0 2024-09-16T00:11:09.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.037+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec81a7290 con 0x7fcec8102ef0 2024-09-16T00:11:09.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.038+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec8101200 con 0x7fcec8102ef0 2024-09-16T00:11:09.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.039+0000 7fceaffff640 1 -- 172.21.15.78:0/3479718466 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fcec0005560 con 0x7fcec8102ef0 2024-09-16T00:11:09.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.040+0000 7fceaffff640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fceb00a3240 0x7fceb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.040+0000 7fcec6ffd640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fceb00a3240 0x7fceb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.040+0000 7fceaffff640 1 -- 172.21.15.78:0/3479718466 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8277+0+0 (secure 0 0 0) 0x7fcec00cffe0 con 0x7fcec8102ef0 2024-09-16T00:11:09.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.040+0000 7fcec6ffd640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fceb00a3240 0x7fceb00a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009930 tx=0x7fceb8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:09.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.043+0000 7fceaffff640 1 -- 172.21.15.78:0/3479718466 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcec0016030 con 0x7fcec8102ef0 2024-09-16T00:11:09.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:09.209+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcec810a0a0 con 0x7fcec8102ef0 2024-09-16T00:11:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:09 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3479718466' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:11:09.954 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:11:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: 2024-09-16T00:11:09.628+0000 7f91dde52640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T00:11:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:09 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3479718466' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:11:09.963 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:09 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3479718466' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:11:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.660+0000 7fceaffff640 1 -- 172.21.15.78:0/3479718466 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7fcec009ccd0 con 0x7fcec8102ef0 2024-09-16T00:11:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_ec' created 2024-09-16T00:11:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fceb00a3240 msgr2=0x7fceb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:10.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fceb00a3240 0x7fceb00a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009930 tx=0x7fceb8008040 comp rx=0 tx=0).stop 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102ef0 msgr2=0x7fcec80fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102ef0 0x7fcec80fd700 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcec000eaa0 tx=0x7fcec000ef60 comp rx=0 tx=0).stop 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 shutdown_connections 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fceb00a3240 0x7fceb00a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcec8102ef0 0x7fcec80fd700 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcec8102570 0x7fcec80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 --2- 172.21.15.78:0/3479718466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcec8100e20 0x7fcec80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 >> 172.21.15.78:0/3479718466 conn(0x7fcec80fc7d0 msgr2=0x7fcec806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 shutdown_connections 2024-09-16T00:11:10.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:10.663+0000 7fcecd7d5640 1 -- 172.21.15.78:0/3479718466 wait complete. 2024-09-16T00:11:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:10 smithi193 ceph-mon[35484]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:11:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:10 smithi193 ceph-mon[35484]: pgmap v286: 97 pgs: 97 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 532 B/s wr, 5 op/s 2024-09-16T00:11:10.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:10 smithi148 ceph-mon[35599]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:11:10.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:10 smithi148 ceph-mon[35599]: pgmap v286: 97 pgs: 97 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 532 B/s wr, 5 op/s 2024-09-16T00:11:10.987 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:10 smithi078 ceph-mon[31157]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:11:10.987 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:10 smithi078 ceph-mon[31157]: pgmap v286: 97 pgs: 97 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 532 B/s wr, 5 op/s 2024-09-16T00:11:11.338 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-16T00:11:11.587 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:11.675 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:11 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3479718466' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:11:11.675 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:11 smithi078 ceph-mon[31157]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:11:11.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:11 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3479718466' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:11:11.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:11 smithi193 ceph-mon[35484]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:11:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:11 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3479718466' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:11:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:11 smithi148 ceph-mon[35599]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 -- 172.21.15.78:0/786018863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 msgr2=0x7f6270106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 --2- 172.21.15.78:0/786018863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 0x7f6270106e70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6260009cd0 tx=0x7f626002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 -- 172.21.15.78:0/786018863 shutdown_connections 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 --2- 172.21.15.78:0/786018863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6270101430 0x7f627010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 --2- 172.21.15.78:0/786018863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6270100a90 0x7f6270100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 --2- 172.21.15.78:0/786018863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 0x7f6270106e70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 -- 172.21.15.78:0/786018863 >> 172.21.15.78:0/786018863 conn(0x7f62700fc7d0 msgr2=0x7f62700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 -- 172.21.15.78:0/786018863 shutdown_connections 2024-09-16T00:11:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.093+0000 7f6274f69640 1 -- 172.21.15.78:0/786018863 wait complete. 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f6274f69640 1 Processor -- start 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f6274f69640 1 -- start start 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f6274f69640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6270100a90 0x7f6270198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f6274f69640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6270101430 0x7f62701989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f6274f69640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 0x7f627019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f6274f69640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6270110e90 con 0x7f6270106a90 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f6274f69640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6270110d10 con 0x7f6270100a90 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f6274f69640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6270111010 con 0x7f6270101430 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 0x7f627019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 0x7f627019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58542/0 (socket says 172.21.15.78:58542) 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626ed9d640 1 -- 172.21.15.78:0/3018544882 learned_addr learned my addr 172.21.15.78:0/3018544882 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626e59c640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6270100a90 0x7f6270198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626dd9b640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6270101430 0x7f62701989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626ed9d640 1 -- 172.21.15.78:0/3018544882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6270101430 msgr2=0x7f62701989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626ed9d640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6270101430 0x7f62701989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626ed9d640 1 -- 172.21.15.78:0/3018544882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6270100a90 msgr2=0x7f6270198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626ed9d640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6270100a90 0x7f6270198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626ed9d640 1 -- 172.21.15.78:0/3018544882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627019d430 con 0x7f6270106a90 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.094+0000 7f626dd9b640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6270101430 0x7f62701989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.095+0000 7f626ed9d640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 0x7f627019cd50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f626800c970 tx=0x7f626800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.095+0000 7f62677fe640 1 -- 172.21.15.78:0/3018544882 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268007bb0 con 0x7f6270106a90 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.095+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627019d720 con 0x7f6270106a90 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.095+0000 7f62677fe640 1 -- 172.21.15.78:0/3018544882 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6268007d50 con 0x7f6270106a90 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.095+0000 7f62677fe640 1 -- 172.21.15.78:0/3018544882 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268018800 con 0x7f6270106a90 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.095+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62701a5000 con 0x7f6270106a90 2024-09-16T00:11:12.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.096+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6270100ef0 con 0x7f6270106a90 2024-09-16T00:11:12.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.097+0000 7f62677fe640 1 -- 172.21.15.78:0/3018544882 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f626800fa50 con 0x7f6270106a90 2024-09-16T00:11:12.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.098+0000 7f62677fe640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f625c0a3240 0x7f625c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.098+0000 7f626e59c640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f625c0a3240 0x7f625c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.098+0000 7f62677fe640 1 -- 172.21.15.78:0/3018544882 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f62680cfd60 con 0x7f6270106a90 2024-09-16T00:11:12.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.098+0000 7f626e59c640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f625c0a3240 0x7f625c0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6260006010 tx=0x7f6260009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:12.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.101+0000 7f62677fe640 1 -- 172.21.15.78:0/3018544882 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6268014030 con 0x7f6270106a90 2024-09-16T00:11:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.267+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f62701997e0 con 0x7f6270106a90 2024-09-16T00:11:12.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.679+0000 7f62677fe640 1 -- 172.21.15.78:0/3018544882 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f626809c860 con 0x7f6270106a90 2024-09-16T00:11:12.681 INFO:teuthology.orchestra.run.smithi078.stderr:set pool 4 allow_ec_overwrites to true 2024-09-16T00:11:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f625c0a3240 msgr2=0x7f625c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f625c0a3240 0x7f625c0a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6260006010 tx=0x7f6260009360 comp rx=0 tx=0).stop 2024-09-16T00:11:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 msgr2=0x7f627019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 0x7f627019cd50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f626800c970 tx=0x7f626800ce30 comp rx=0 tx=0).stop 2024-09-16T00:11:12.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 shutdown_connections 2024-09-16T00:11:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f625c0a3240 0x7f625c0a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6270106a90 0x7f627019cd50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6270101430 0x7f62701989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 --2- 172.21.15.78:0/3018544882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6270100a90 0x7f6270198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 >> 172.21.15.78:0/3018544882 conn(0x7f62700fc7d0 msgr2=0x7f627006a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 shutdown_connections 2024-09-16T00:11:12.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:12.681+0000 7f6274f69640 1 -- 172.21.15.78:0/3018544882 wait complete. 2024-09-16T00:11:12.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:12 smithi148 ceph-mon[35599]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:11:12.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:12 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3018544882' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:11:12.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:12 smithi148 ceph-mon[35599]: pgmap v289: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-16T00:11:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:12 smithi078 ceph-mon[31157]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:11:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:12 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3018544882' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:11:13.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:12 smithi078 ceph-mon[31157]: pgmap v289: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-16T00:11:13.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:12 smithi193 ceph-mon[35484]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:11:13.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:12 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3018544882' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:11:13.107 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:12 smithi193 ceph-mon[35484]: pgmap v289: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-09-16T00:11:13.457 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-16T00:11:13.704 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:13 smithi148 ceph-mon[35599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:11:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:13 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3018544882' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:11:13.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:13 smithi148 ceph-mon[35599]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:11:13.977 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:13 smithi078 ceph-mon[31157]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:11:13.977 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:13 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3018544882' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:11:13.977 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:13 smithi078 ceph-mon[31157]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:11:14.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:13 smithi193 ceph-mon[35484]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:11:14.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:13 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3018544882' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:11:14.050 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:13 smithi193 ceph-mon[35484]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:11:14.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 -- 172.21.15.78:0/2627530011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 msgr2=0x7f55dc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 --2- 172.21.15.78:0/2627530011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 0x7f55dc10e600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009ca0 tx=0x7f55d402a950 comp rx=0 tx=0).stop 2024-09-16T00:11:14.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 -- 172.21.15.78:0/2627530011 shutdown_connections 2024-09-16T00:11:14.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 --2- 172.21.15.78:0/2627530011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 0x7f55dc10e600 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 --2- 172.21.15.78:0/2627530011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55dc1036c0 0x7f55dc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 --2- 172.21.15.78:0/2627530011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55dc102da0 0x7f55dc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 -- 172.21.15.78:0/2627530011 >> 172.21.15.78:0/2627530011 conn(0x7f55dc0fc7a0 msgr2=0x7f55dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 -- 172.21.15.78:0/2627530011 shutdown_connections 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.165+0000 7f55e3319640 1 -- 172.21.15.78:0/2627530011 wait complete. 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e3319640 1 Processor -- start 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e3319640 1 -- start start 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e3319640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55dc102da0 0x7f55dc100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e3319640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55dc1036c0 0x7f55dc100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e3319640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 0x7f55dc101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e3319640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc111f60 con 0x7f55dc10c180 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e3319640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc111de0 con 0x7f55dc1036c0 2024-09-16T00:11:14.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e3319640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1120e0 con 0x7f55dc102da0 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e18b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 0x7f55dc101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e18b6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 0x7f55dc101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58558/0 (socket says 172.21.15.78:58558) 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e18b6640 1 -- 172.21.15.78:0/3111895037 learned_addr learned my addr 172.21.15.78:0/3111895037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e10b5640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55dc102da0 0x7f55dc100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e18b6640 1 -- 172.21.15.78:0/3111895037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55dc102da0 msgr2=0x7f55dc100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e18b6640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55dc102da0 0x7f55dc100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e18b6640 1 -- 172.21.15.78:0/3111895037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55dc1036c0 msgr2=0x7f55dc100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e18b6640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55dc1036c0 0x7f55dc100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.166+0000 7f55e18b6640 1 -- 172.21.15.78:0/3111895037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc1a68a0 con 0x7f55dc10c180 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.167+0000 7f55e18b6640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 0x7f55dc101480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55d402ae70 tx=0x7f55d402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.167+0000 7f55d27fc640 1 -- 172.21.15.78:0/3111895037 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4034040 con 0x7f55dc10c180 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.167+0000 7f55d27fc640 1 -- 172.21.15.78:0/3111895037 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d4030920 con 0x7f55dc10c180 2024-09-16T00:11:14.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.167+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc1a6b30 con 0x7f55dc10c180 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.167+0000 7f55d27fc640 1 -- 172.21.15.78:0/3111895037 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4045480 con 0x7f55dc10c180 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.167+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1a7010 con 0x7f55dc10c180 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.168+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc103180 con 0x7f55dc10c180 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.169+0000 7f55d27fc640 1 -- 172.21.15.78:0/3111895037 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f55d4037020 con 0x7f55dc10c180 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.169+0000 7f55d27fc640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55c80a3240 0x7f55c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.170+0000 7f55e10b5640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55c80a3240 0x7f55c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.170+0000 7f55d27fc640 1 -- 172.21.15.78:0/3111895037 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f55d40f0a60 con 0x7f55dc10c180 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.170+0000 7f55e10b5640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55c80a3240 0x7f55c80a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55d8005fd0 tx=0x7f55d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.173+0000 7f55d27fc640 1 -- 172.21.15.78:0/3111895037 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55d40bd560 con 0x7f55dc10c180 2024-09-16T00:11:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.376+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f55dc106c90 con 0x7f55dc10c180 2024-09-16T00:11:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.686+0000 7f55d27fc640 1 -- 172.21.15.78:0/3111895037 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f55d40bce60 con 0x7f55dc10c180 2024-09-16T00:11:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:added data pool 4 to fsmap 2024-09-16T00:11:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55c80a3240 msgr2=0x7f55c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55c80a3240 0x7f55c80a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55d8005fd0 tx=0x7f55d800b390 comp rx=0 tx=0).stop 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 msgr2=0x7f55dc101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 0x7f55dc101480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f55d402ae70 tx=0x7f55d402f5b0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 shutdown_connections 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55c80a3240 0x7f55c80a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55dc10c180 0x7f55dc101480 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55dc1036c0 0x7f55dc100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 --2- 172.21.15.78:0/3111895037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55dc102da0 0x7f55dc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 >> 172.21.15.78:0/3111895037 conn(0x7f55dc0fc7a0 msgr2=0x7f55dc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.688+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 shutdown_connections 2024-09-16T00:11:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:14.689+0000 7f55e3319640 1 -- 172.21.15.78:0/3111895037 wait complete. 2024-09-16T00:11:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:14 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3111895037' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:11:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:14 smithi148 ceph-mon[35599]: pgmap v291: 129 pgs: 9 creating+peering, 12 unknown, 108 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-09-16T00:11:14.981 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:14 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3111895037' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:11:14.981 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:14 smithi078 ceph-mon[31157]: pgmap v291: 129 pgs: 9 creating+peering, 12 unknown, 108 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-09-16T00:11:15.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:14 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3111895037' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:11:15.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:14 smithi193 ceph-mon[35484]: pgmap v291: 129 pgs: 9 creating+peering, 12 unknown, 108 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 2 op/s 2024-09-16T00:11:15.285 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:11:15.544 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:15 smithi148 ceph-mon[35599]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:11:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:15 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3111895037' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:11:15.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:15 smithi148 ceph-mon[35599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:15.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:15 smithi078 ceph-mon[31157]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:11:15.969 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:15 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3111895037' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:11:15.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:15 smithi078 ceph-mon[31157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:16.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 -- 172.21.15.78:0/1861325803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168100a80 msgr2=0x7fe168100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/1861325803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168100a80 0x7fe168100ee0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe150009930 tx=0x7fe15002a420 comp rx=0 tx=0).stop 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 -- 172.21.15.78:0/1861325803 shutdown_connections 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/1861325803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe168101420 0x7fe16810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/1861325803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168100a80 0x7fe168100ee0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/1861325803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe168106a80 0x7fe168106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 -- 172.21.15.78:0/1861325803 >> 172.21.15.78:0/1861325803 conn(0x7fe1680fc7a0 msgr2=0x7fe1680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 -- 172.21.15.78:0/1861325803 shutdown_connections 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.046+0000 7fe16d3a1640 1 -- 172.21.15.78:0/1861325803 wait complete. 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe16d3a1640 1 Processor -- start 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe16d3a1640 1 -- start start 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe16d3a1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe168100a80 0x7fe1680fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe16d3a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168101420 0x7fe1680fd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe16d3a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe168106a80 0x7fe1680fd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe16d3a1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe168110ed0 con 0x7fe168101420 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe16d3a1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe168110d50 con 0x7fe168100a80 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe16d3a1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe168111050 con 0x7fe168106a80 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe15e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168101420 0x7fe1680fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe15e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168101420 0x7fe1680fd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58580/0 (socket says 172.21.15.78:58580) 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.047+0000 7fe15e7fc640 1 -- 172.21.15.78:0/4129102204 learned_addr learned my addr 172.21.15.78:0/4129102204 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe1677fe640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe168106a80 0x7fe1680fd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe15e7fc640 1 -- 172.21.15.78:0/4129102204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe168106a80 msgr2=0x7fe1680fd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe15e7fc640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe168106a80 0x7fe1680fd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe15e7fc640 1 -- 172.21.15.78:0/4129102204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe168100a80 msgr2=0x7fe1680fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe166ffd640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe168100a80 0x7fe1680fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe15e7fc640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe168100a80 0x7fe1680fcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe15e7fc640 1 -- 172.21.15.78:0/4129102204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681a68b0 con 0x7fe168101420 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe15e7fc640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168101420 0x7fe1680fd240 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe15002a940 tx=0x7fe150003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe164ff9640 1 -- 172.21.15.78:0/4129102204 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15003f070 con 0x7fe168101420 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe164ff9640 1 -- 172.21.15.78:0/4129102204 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe150031c90 con 0x7fe168101420 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe164ff9640 1 -- 172.21.15.78:0/4129102204 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150035030 con 0x7fe168101420 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681a6b40 con 0x7fe168101420 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.048+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681a7020 con 0x7fe168101420 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.049+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe168100ee0 con 0x7fe168101420 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.050+0000 7fe164ff9640 1 -- 172.21.15.78:0/4129102204 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe150042070 con 0x7fe168101420 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.051+0000 7fe164ff9640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1540a3240 0x7fe1540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.051+0000 7fe166ffd640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1540a3240 0x7fe1540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.051+0000 7fe164ff9640 1 -- 172.21.15.78:0/4129102204 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fe1500f08a0 con 0x7fe168101420 2024-09-16T00:11:16.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.051+0000 7fe166ffd640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1540a3240 0x7fe1540a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe15800b9c0 tx=0x7fe158005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.054+0000 7fe164ff9640 1 -- 172.21.15.78:0/4129102204 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1500bd380 con 0x7fe168101420 2024-09-16T00:11:16.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:15 smithi193 ceph-mon[35484]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:11:16.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:15 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3111895037' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:11:16.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:15 smithi193 ceph-mon[35484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:16.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.251+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1680fe230 con 0x7fe168101420 2024-09-16T00:11:16.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.252+0000 7fe164ff9640 1 -- 172.21.15.78:0/4129102204 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11851 (secure 0 0 0) 0x7fe1680fe230 con 0x7fe168101420 2024-09-16T00:11:16.254 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:16.254 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"btime":"2024-09-16T00:11:14:682391+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:11:14.379203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24470,"qdb_cluster":[24470]},"id":1}]} 2024-09-16T00:11:16.254 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 16 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1540a3240 msgr2=0x7fe1540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1540a3240 0x7fe1540a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe15800b9c0 tx=0x7fe158005c50 comp rx=0 tx=0).stop 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168101420 msgr2=0x7fe1680fd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168101420 0x7fe1680fd240 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe15002a940 tx=0x7fe150003b90 comp rx=0 tx=0).stop 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 shutdown_connections 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1540a3240 0x7fe1540a5700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe168106a80 0x7fe1680fd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe168101420 0x7fe1680fd240 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 --2- 172.21.15.78:0/4129102204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe168100a80 0x7fe1680fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 >> 172.21.15.78:0/4129102204 conn(0x7fe1680fc7a0 msgr2=0x7fe16806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 shutdown_connections 2024-09-16T00:11:16.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:16.254+0000 7fe16d3a1640 1 -- 172.21.15.78:0/4129102204 wait complete. 2024-09-16T00:11:16.942 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:16 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4129102204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:16.942 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:16 smithi078 ceph-mon[31157]: pgmap v293: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:16.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:16 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4129102204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:16.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:16 smithi148 ceph-mon[35599]: pgmap v293: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:16.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:11:17.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:16 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4129102204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:17.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:16 smithi193 ceph-mon[35484]: pgmap v293: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:17.213 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 -- 172.21.15.78:0/446325017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc106a80 msgr2=0x7fd2fc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 --2- 172.21.15.78:0/446325017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc106a80 0x7fd2fc106e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009930 tx=0x7fd2f002a420 comp rx=0 tx=0).stop 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 -- 172.21.15.78:0/446325017 shutdown_connections 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 --2- 172.21.15.78:0/446325017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2fc101420 0x7fd2fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 --2- 172.21.15.78:0/446325017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2fc100a80 0x7fd2fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 --2- 172.21.15.78:0/446325017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc106a80 0x7fd2fc106e60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 -- 172.21.15.78:0/446325017 >> 172.21.15.78:0/446325017 conn(0x7fd2fc0fc7a0 msgr2=0x7fd2fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 -- 172.21.15.78:0/446325017 shutdown_connections 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 -- 172.21.15.78:0/446325017 wait complete. 2024-09-16T00:11:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 Processor -- start 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.718+0000 7fd3023c8640 1 -- start start 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd3023c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2fc100a80 0x7fd2fc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd3023c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc101420 0x7fd2fc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd3023c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd3023c8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc110ea0 con 0x7fd2fc101420 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd3023c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc110d20 con 0x7fd2fc100a80 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd3023c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc111020 con 0x7fd2fc106a80 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc101420 0x7fd2fc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc101420 0x7fd2fc198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35858/0 (socket says 172.21.15.78:35858) 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/3116255395 learned_addr learned my addr 172.21.15.78:0/3116255395 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fbfff640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2fc100a80 0x7fd2fc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd300965640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/3116255395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2fc106a80 msgr2=0x7fd2fc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/3116255395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2fc100a80 msgr2=0x7fd2fc198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2fc100a80 0x7fd2fc198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 -- 172.21.15.78:0/3116255395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc19d3d0 con 0x7fd2fc101420 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd300965640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2fb7fe640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc101420 0x7fd2fc198960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b4a0 tx=0x7fd2ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.719+0000 7fd2f97fa640 1 -- 172.21.15.78:0/3116255395 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec014070 con 0x7fd2fc101420 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.720+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc19d6c0 con 0x7fd2fc101420 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.720+0000 7fd2f97fa640 1 -- 172.21.15.78:0/3116255395 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ec010af0 con 0x7fd2fc101420 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.720+0000 7fd2f97fa640 1 -- 172.21.15.78:0/3116255395 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec017d10 con 0x7fd2fc101420 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.720+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc1a4fa0 con 0x7fd2fc101420 2024-09-16T00:11:17.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.721+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc100ee0 con 0x7fd2fc101420 2024-09-16T00:11:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.722+0000 7fd2f97fa640 1 -- 172.21.15.78:0/3116255395 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd2ec017070 con 0x7fd2fc101420 2024-09-16T00:11:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.722+0000 7fd2f97fa640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2e40a3240 0x7fd2e40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.722+0000 7fd2fbfff640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2e40a3240 0x7fd2e40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.722+0000 7fd2f97fa640 1 -- 172.21.15.78:0/3116255395 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd2ec0cef20 con 0x7fd2fc101420 2024-09-16T00:11:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.723+0000 7fd2fbfff640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2e40a3240 0x7fd2e40a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009900 tx=0x7fd2f00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.726+0000 7fd2f97fa640 1 -- 172.21.15.78:0/3116255395 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2ec09ba00 con 0x7fd2fc101420 2024-09-16T00:11:17.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.891+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2fc109a70 con 0x7fd2fc101420 2024-09-16T00:11:17.893 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:17 smithi078 ceph-mon[31157]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:11:17.893 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:17 smithi078 ceph-mon[31157]: Cluster is now healthy 2024-09-16T00:11:17.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.893+0000 7fd2f97fa640 1 -- 172.21.15.78:0/3116255395 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24213 (secure 0 0 0) 0x7fd2ec09b300 con 0x7fd2fc101420 2024-09-16T00:11:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:17.894 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":86,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:11:14.379068+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:17.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2e40a3240 msgr2=0x7fd2e40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2e40a3240 0x7fd2e40a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009900 tx=0x7fd2f00058d0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc101420 msgr2=0x7fd2fc198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc101420 0x7fd2fc198960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00b4a0 tx=0x7fd2ec00b960 comp rx=0 tx=0).stop 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 shutdown_connections 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2e40a3240 0x7fd2e40a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2fc106a80 0x7fd2fc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2fc101420 0x7fd2fc198960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 --2- 172.21.15.78:0/3116255395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2fc100a80 0x7fd2fc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 >> 172.21.15.78:0/3116255395 conn(0x7fd2fc0fc7a0 msgr2=0x7fd2fc06aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.895+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 shutdown_connections 2024-09-16T00:11:17.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:17.896+0000 7fd3023c8640 1 -- 172.21.15.78:0/3116255395 wait complete. 2024-09-16T00:11:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:17 smithi148 ceph-mon[35599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:11:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:17 smithi148 ceph-mon[35599]: Cluster is now healthy 2024-09-16T00:11:18.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:17 smithi193 ceph-mon[35484]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:11:18.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:17 smithi193 ceph-mon[35484]: Cluster is now healthy 2024-09-16T00:11:18.641 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:11:18.886 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:18.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:18 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3116255395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:18.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:18 smithi148 ceph-mon[35599]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:19.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:18 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3116255395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:19.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:18 smithi078 ceph-mon[31157]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:19.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:18 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3116255395' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:19.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:18 smithi193 ceph-mon[35484]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 -- 172.21.15.78:0/1792784033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 msgr2=0x7ff73010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 --2- 172.21.15.78:0/1792784033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 0x7ff73010da40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff728009ca0 tx=0x7ff72802a950 comp rx=0 tx=0).stop 2024-09-16T00:11:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 -- 172.21.15.78:0/1792784033 shutdown_connections 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 --2- 172.21.15.78:0/1792784033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 0x7ff73010da40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 --2- 172.21.15.78:0/1792784033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff730102570 0x7ff7301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 --2- 172.21.15.78:0/1792784033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff730100e20 0x7ff730101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 -- 172.21.15.78:0/1792784033 >> 172.21.15.78:0/1792784033 conn(0x7ff7300fc7d0 msgr2=0x7ff7300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 -- 172.21.15.78:0/1792784033 shutdown_connections 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.385+0000 7ff734f3a640 1 -- 172.21.15.78:0/1792784033 wait complete. 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff734f3a640 1 Processor -- start 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff734f3a640 1 -- start start 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff734f3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff730100e20 0x7ff7301984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff734f3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff730102570 0x7ff730198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff734f3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 0x7ff73019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff734f3a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff730110fa0 con 0x7ff730102ef0 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff734f3a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff730110e20 con 0x7ff730102570 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff734f3a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff730111120 con 0x7ff730100e20 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff72ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 0x7ff73019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff72ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 0x7ff73019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35876/0 (socket says 172.21.15.78:35876) 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff72ed9d640 1 -- 172.21.15.78:0/3370298507 learned_addr learned my addr 172.21.15.78:0/3370298507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff72ed9d640 1 -- 172.21.15.78:0/3370298507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff730100e20 msgr2=0x7ff7301984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff72ed9d640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff730100e20 0x7ff7301984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff72ed9d640 1 -- 172.21.15.78:0/3370298507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff730102570 msgr2=0x7ff730198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff727fff640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff730102570 0x7ff730198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff72ed9d640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff730102570 0x7ff730198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff72ed9d640 1 -- 172.21.15.78:0/3370298507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73019d480 con 0x7ff730102ef0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.386+0000 7ff727fff640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff730102570 0x7ff730198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.387+0000 7ff72ed9d640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 0x7ff73019cda0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff728006010 tx=0x7ff72802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.387+0000 7ff7277fe640 1 -- 172.21.15.78:0/3370298507 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728034040 con 0x7ff730102ef0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.387+0000 7ff7277fe640 1 -- 172.21.15.78:0/3370298507 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff728003e00 con 0x7ff730102ef0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.387+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73019d710 con 0x7ff730102ef0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.387+0000 7ff7277fe640 1 -- 172.21.15.78:0/3370298507 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728045480 con 0x7ff730102ef0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.387+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7301a5050 con 0x7ff730102ef0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.388+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff730101200 con 0x7ff730102ef0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.389+0000 7ff7277fe640 1 -- 172.21.15.78:0/3370298507 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff728037020 con 0x7ff730102ef0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.389+0000 7ff7277fe640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff71c0a3240 0x7ff71c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.390+0000 7ff72e59c640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff71c0a3240 0x7ff71c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.390+0000 7ff7277fe640 1 -- 172.21.15.78:0/3370298507 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7ff7280f0d80 con 0x7ff730102ef0 2024-09-16T00:11:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.390+0000 7ff72e59c640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff71c0a3240 0x7ff71c0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff720009710 tx=0x7ff720006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:19.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.394+0000 7ff7277fe640 1 -- 172.21.15.78:0/3370298507 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7280bd860 con 0x7ff730102ef0 2024-09-16T00:11:19.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.560+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff73010a0a0 con 0x7ff730102ef0 2024-09-16T00:11:19.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.561+0000 7ff7277fe640 1 -- 172.21.15.78:0/3370298507 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24213 (secure 0 0 0) 0x7ff7280bd160 con 0x7ff730102ef0 2024-09-16T00:11:19.562 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:19.563 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":86,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:11:14.379068+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:19.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff71c0a3240 msgr2=0x7ff71c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff71c0a3240 0x7ff71c0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff720009710 tx=0x7ff720006c90 comp rx=0 tx=0).stop 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 msgr2=0x7ff73019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 0x7ff73019cda0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff728006010 tx=0x7ff72802f6f0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 shutdown_connections 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff71c0a3240 0x7ff71c0a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff730102ef0 0x7ff73019cda0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff730102570 0x7ff730198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 --2- 172.21.15.78:0/3370298507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff730100e20 0x7ff7301984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 >> 172.21.15.78:0/3370298507 conn(0x7ff7300fc7d0 msgr2=0x7ff73006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 shutdown_connections 2024-09-16T00:11:19.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:19.564+0000 7ff734f3a640 1 -- 172.21.15.78:0/3370298507 wait complete. 2024-09-16T00:11:19.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:19 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3370298507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:19 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3370298507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:19 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3370298507' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:20.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:11:20.652 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:21.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:20 smithi078 ceph-mon[31157]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:21.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 -- 172.21.15.78:0/3866117458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e0103700 msgr2=0x7f73e010bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 --2- 172.21.15.78:0/3866117458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e0103700 0x7f73e010bbf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009900 tx=0x7f73c802a480 comp rx=0 tx=0).stop 2024-09-16T00:11:21.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 -- 172.21.15.78:0/3866117458 shutdown_connections 2024-09-16T00:11:21.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 --2- 172.21.15.78:0/3866117458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e010c180 0x7f73e010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 --2- 172.21.15.78:0/3866117458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e0103700 0x7f73e010bbf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 --2- 172.21.15.78:0/3866117458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0102de0 0x7f73e01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 -- 172.21.15.78:0/3866117458 >> 172.21.15.78:0/3866117458 conn(0x7f73e00fc7a0 msgr2=0x7f73e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 -- 172.21.15.78:0/3866117458 shutdown_connections 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.078+0000 7f73e5856640 1 -- 172.21.15.78:0/3866117458 wait complete. 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73e5856640 1 Processor -- start 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73e5856640 1 -- start start 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73e5856640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0102de0 0x7f73e0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73e5856640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e0103700 0x7f73e01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73e5856640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e010c180 0x7f73e019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73e5856640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0111fb0 con 0x7f73e010c180 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73e5856640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0111e30 con 0x7f73e0103700 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73e5856640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73e0112130 con 0x7f73e0102de0 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e010c180 0x7f73e019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73df7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e010c180 0x7f73e019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35900/0 (socket says 172.21.15.78:35900) 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73df7fe640 1 -- 172.21.15.78:0/3116192625 learned_addr learned my addr 172.21.15.78:0/3116192625 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73d7fff640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e0103700 0x7f73e01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73deffd640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0102de0 0x7f73e0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73df7fe640 1 -- 172.21.15.78:0/3116192625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0102de0 msgr2=0x7f73e0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73df7fe640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0102de0 0x7f73e0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73df7fe640 1 -- 172.21.15.78:0/3116192625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e0103700 msgr2=0x7f73e01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73df7fe640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e0103700 0x7f73e01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73df7fe640 1 -- 172.21.15.78:0/3116192625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e019d420 con 0x7f73e010c180 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.079+0000 7f73deffd640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0102de0 0x7f73e0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.080+0000 7f73df7fe640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e010c180 0x7f73e019cd40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f73d800cce0 tx=0x7f73d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.080+0000 7f73e4854640 1 -- 172.21.15.78:0/3116192625 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d800f450 con 0x7f73e010c180 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.080+0000 7f73e4854640 1 -- 172.21.15.78:0/3116192625 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d800fad0 con 0x7f73e010c180 2024-09-16T00:11:21.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.080+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e019d710 con 0x7f73e010c180 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.080+0000 7f73e4854640 1 -- 172.21.15.78:0/3116192625 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8018c10 con 0x7f73e010c180 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.080+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e01a4ff0 con 0x7f73e010c180 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.081+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e01031c0 con 0x7f73e010c180 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.082+0000 7f73e4854640 1 -- 172.21.15.78:0/3116192625 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f73d800f5f0 con 0x7f73e010c180 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.083+0000 7f73e4854640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f73cc0a3240 0x7f73cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.083+0000 7f73deffd640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f73cc0a3240 0x7f73cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.083+0000 7f73e4854640 1 -- 172.21.15.78:0/3116192625 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f73d80d0240 con 0x7f73e010c180 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.084+0000 7f73deffd640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f73cc0a3240 0x7f73cc0a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006fd0 tx=0x7f73d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.087+0000 7f73e4854640 1 -- 172.21.15.78:0/3116192625 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73d8014030 con 0x7f73e010c180 2024-09-16T00:11:21.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:20 smithi193 ceph-mon[35484]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:21.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:20 smithi148 ceph-mon[35599]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:21.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.288+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73e0106c70 con 0x7f73e010c180 2024-09-16T00:11:21.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.289+0000 7f73e4854640 1 -- 172.21.15.78:0/3116192625 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11851 (secure 0 0 0) 0x7f73d809cd20 con 0x7f73e010c180 2024-09-16T00:11:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:21.291 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"btime":"2024-09-16T00:11:14:682391+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24427,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:11:14.379203+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24470},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24470,"qdb_cluster":[24470]},"id":1}]} 2024-09-16T00:11:21.291 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 16 2024-09-16T00:11:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f73cc0a3240 msgr2=0x7f73cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f73cc0a3240 0x7f73cc0a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006fd0 tx=0x7f73d0008040 comp rx=0 tx=0).stop 2024-09-16T00:11:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e010c180 msgr2=0x7f73e019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e010c180 0x7f73e019cd40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f73d800cce0 tx=0x7f73d800a620 comp rx=0 tx=0).stop 2024-09-16T00:11:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 shutdown_connections 2024-09-16T00:11:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f73cc0a3240 0x7f73cc0a5700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73e010c180 0x7f73e019cd40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f73e0103700 0x7f73e01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 --2- 172.21.15.78:0/3116192625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0102de0 0x7f73e0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.291+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 >> 172.21.15.78:0/3116192625 conn(0x7f73e00fc7a0 msgr2=0x7f73e00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.292+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 shutdown_connections 2024-09-16T00:11:21.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:21.292+0000 7f73e5856640 1 -- 172.21.15.78:0/3116192625 wait complete. 2024-09-16T00:11:21.939 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:21 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3116192625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:21.968 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:11:22.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:21 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3116192625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:21 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3116192625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:22.212 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:22.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 -- 172.21.15.78:0/3999306155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102570 msgr2=0x7f99681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 --2- 172.21.15.78:0/3999306155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102570 0x7f99681029b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f995c009900 tx=0x7f995c02a480 comp rx=0 tx=0).stop 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 -- 172.21.15.78:0/3999306155 shutdown_connections 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 --2- 172.21.15.78:0/3999306155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9968102ef0 0x7f996810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 --2- 172.21.15.78:0/3999306155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102570 0x7f99681029b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 --2- 172.21.15.78:0/3999306155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9968100e20 0x7f9968101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 -- 172.21.15.78:0/3999306155 >> 172.21.15.78:0/3999306155 conn(0x7f99680fc7d0 msgr2=0x7f99680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 -- 172.21.15.78:0/3999306155 shutdown_connections 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.689+0000 7f996d1f4640 1 -- 172.21.15.78:0/3999306155 wait complete. 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996d1f4640 1 Processor -- start 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996d1f4640 1 -- start start 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996d1f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9968100e20 0x7f9968113380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996d1f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9968102570 0x7f99681116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996d1f4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102ef0 0x7f9968111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996d1f4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9968110f80 con 0x7f9968102ef0 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996d1f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9968110e00 con 0x7f9968100e20 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996d1f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9968111100 con 0x7f9968102570 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102ef0 0x7f9968111cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102ef0 0x7f9968111cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35924/0 (socket says 172.21.15.78:35924) 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996759e640 1 -- 172.21.15.78:0/1108353348 learned_addr learned my addr 172.21.15.78:0/1108353348 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f9966d9d640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9968100e20 0x7f9968113380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.690+0000 7f996659c640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9968102570 0x7f99681116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f996759e640 1 -- 172.21.15.78:0/1108353348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9968102570 msgr2=0x7f99681116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f996759e640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9968102570 0x7f99681116e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f996759e640 1 -- 172.21.15.78:0/1108353348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9968100e20 msgr2=0x7f9968113380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f996759e640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9968100e20 0x7f9968113380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f996759e640 1 -- 172.21.15.78:0/1108353348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99681a6a80 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f996759e640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102ef0 0x7f9968111cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9960009fd0 tx=0x7f996000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f994ffff640 1 -- 172.21.15.78:0/1108353348 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960017070 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f994ffff640 1 -- 172.21.15.78:0/1108353348 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9960003e50 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99681a6c50 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f994ffff640 1 -- 172.21.15.78:0/1108353348 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99600106c0 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.691+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99681a7110 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.692+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9968101200 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.693+0000 7f994ffff640 1 -- 172.21.15.78:0/1108353348 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9960003970 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.694+0000 7f994ffff640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f99500a3240 0x7f99500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.694+0000 7f9966d9d640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f99500a3240 0x7f99500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.694+0000 7f994ffff640 1 -- 172.21.15.78:0/1108353348 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f99600cf830 con 0x7f9968102ef0 2024-09-16T00:11:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.694+0000 7f9966d9d640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f99500a3240 0x7f99500a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9968105790 tx=0x7f995800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.697+0000 7f994ffff640 1 -- 172.21.15.78:0/1108353348 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f996009bcc0 con 0x7f9968102ef0 2024-09-16T00:11:22.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.867+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99681a7310 con 0x7f9968102ef0 2024-09-16T00:11:22.869 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:22 smithi078 ceph-mon[31157]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:22.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.869+0000 7f994ffff640 1 -- 172.21.15.78:0/1108353348 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24213 (secure 0 0 0) 0x7f996009ffc0 con 0x7f9968102ef0 2024-09-16T00:11:22.870 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:22.873 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":86,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:11:14.379068+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f99500a3240 msgr2=0x7f99500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f99500a3240 0x7f99500a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9968105790 tx=0x7f995800b390 comp rx=0 tx=0).stop 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102ef0 msgr2=0x7f9968111cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102ef0 0x7f9968111cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9960009fd0 tx=0x7f996000b260 comp rx=0 tx=0).stop 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 shutdown_connections 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f99500a3240 0x7f99500a5700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9968102ef0 0x7f9968111cf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9968102570 0x7f99681116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 --2- 172.21.15.78:0/1108353348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9968100e20 0x7f9968113380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 >> 172.21.15.78:0/1108353348 conn(0x7f99680fc7d0 msgr2=0x7f99681038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 shutdown_connections 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:22.871+0000 7f996d1f4640 1 -- 172.21.15.78:0/1108353348 wait complete. 2024-09-16T00:11:22.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:22 smithi148 ceph-mon[35599]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:22 smithi193 ceph-mon[35484]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:23.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T00:11:05.234467+0000', 'modified': '2024-09-16T00:11:14.379203+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24470}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.148:6839/831570873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 831570873}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 831570873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24470, 'qdb_cluster': [24470]} 2024-09-16T00:11:23.786 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T00:11:23.787 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:11:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:23 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1108353348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:24.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:23 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1108353348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:23 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1108353348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 -- 172.21.15.78:0/3585153706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c100a80 msgr2=0x7fc22c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 --2- 172.21.15.78:0/3585153706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c100a80 0x7fc22c100ee0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc218009930 tx=0x7fc21802a420 comp rx=0 tx=0).stop 2024-09-16T00:11:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 -- 172.21.15.78:0/3585153706 shutdown_connections 2024-09-16T00:11:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 --2- 172.21.15.78:0/3585153706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc22c101420 0x7fc22c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 --2- 172.21.15.78:0/3585153706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c100a80 0x7fc22c100ee0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 --2- 172.21.15.78:0/3585153706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc22c106a80 0x7fc22c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 -- 172.21.15.78:0/3585153706 >> 172.21.15.78:0/3585153706 conn(0x7fc22c0fc780 msgr2=0x7fc22c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 -- 172.21.15.78:0/3585153706 shutdown_connections 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.473+0000 7fc232d02640 1 -- 172.21.15.78:0/3585153706 wait complete. 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc232d02640 1 Processor -- start 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc232d02640 1 -- start start 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc232d02640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc22c100a80 0x7fc22c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc232d02640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc22c101420 0x7fc22c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc232d02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c106a80 0x7fc22c1a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc232d02640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c110f20 con 0x7fc22c106a80 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc232d02640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c110da0 con 0x7fc22c101420 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc232d02640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1110a0 con 0x7fc22c100a80 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc23129f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c106a80 0x7fc22c1a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc23129f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c106a80 0x7fc22c1a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35942/0 (socket says 172.21.15.78:35942) 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc23129f640 1 -- 172.21.15.78:0/1828818933 learned_addr learned my addr 172.21.15.78:0/1828818933 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:24.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc230a9e640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc22c100a80 0x7fc22c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc22bfff640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc22c101420 0x7fc22c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc23129f640 1 -- 172.21.15.78:0/1828818933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc22c100a80 msgr2=0x7fc22c1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc23129f640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc22c100a80 0x7fc22c1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc23129f640 1 -- 172.21.15.78:0/1828818933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc22c101420 msgr2=0x7fc22c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc23129f640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc22c101420 0x7fc22c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.474+0000 7fc23129f640 1 -- 172.21.15.78:0/1828818933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c1a4a70 con 0x7fc22c106a80 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.475+0000 7fc23129f640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c106a80 0x7fc22c1a4530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc22400b970 tx=0x7fc22400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.475+0000 7fc229ffb640 1 -- 172.21.15.78:0/1828818933 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224019070 con 0x7fc22c106a80 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.475+0000 7fc229ffb640 1 -- 172.21.15.78:0/1828818933 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc22400cb40 con 0x7fc22c106a80 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.475+0000 7fc229ffb640 1 -- 172.21.15.78:0/1828818933 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224007650 con 0x7fc22c106a80 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.475+0000 7fc232d02640 1 -- 172.21.15.78:0/1828818933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c1a4c40 con 0x7fc22c106a80 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.475+0000 7fc232d02640 1 -- 172.21.15.78:0/1828818933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c1a5100 con 0x7fc22c106a80 2024-09-16T00:11:24.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.477+0000 7fc229ffb640 1 -- 172.21.15.78:0/1828818933 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc22401c070 con 0x7fc22c106a80 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc2350db740 1 Processor -- start 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc2350db740 1 -- start start 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc229ffb640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc21c0a3240 0x7fc21c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc2350db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78a9d0 0x56133b78adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc2350db740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78b2f0 0x56133b795670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc2350db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b797fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc2350db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56133b6f1090 con 0x56133b795bb0 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc2350db740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56133b698a70 con 0x56133b78b2f0 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc2350db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56133b6ed050 con 0x56133b78a9d0 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc22bfff640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc21c0a3240 0x7fc21c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc229ffb640 1 -- 172.21.15.78:0/1828818933 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fc2240cf1e0 con 0x7fc22c106a80 2024-09-16T00:11:24.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc23129f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b797fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc23129f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b797fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35944/0 (socket says 172.21.15.78:35944) 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc23129f640 1 -- 172.21.15.78:0/1966804497 learned_addr learned my addr 172.21.15.78:0/1966804497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc230a9e640 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78a9d0 0x56133b78adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc22bfff640 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78b2f0 0x56133b795670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc23129f640 1 -- 172.21.15.78:0/1966804497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78a9d0 msgr2=0x56133b78adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc23129f640 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78a9d0 0x56133b78adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc23129f640 1 -- 172.21.15.78:0/1966804497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78b2f0 msgr2=0x56133b795670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc23129f640 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78b2f0 0x56133b795670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.478+0000 7fc23129f640 1 -- 172.21.15.78:0/1966804497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133b6a5130 con 0x56133b795bb0 2024-09-16T00:11:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.479+0000 7fc22bfff640 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78b2f0 0x56133b795670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.479+0000 7fc22bfff640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc21c0a3240 0x7fc21c0a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc2180316b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.479+0000 7fc23129f640 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b797fa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2240a3d00 tx=0x7fc224004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294321114753ef89 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.479+0000 7fc216ffd640 1 -- 172.21.15.78:0/1966804497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22409f970 con 0x56133b795bb0 2024-09-16T00:11:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.479+0000 7fc216ffd640 1 -- 172.21.15.78:0/1966804497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2240a4230 con 0x56133b795bb0 2024-09-16T00:11:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.479+0000 7fc216ffd640 1 -- 172.21.15.78:0/1966804497 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240a4000 con 0x56133b795bb0 2024-09-16T00:11:24.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.479+0000 7fc2350db740 1 -- 172.21.15.78:0/1966804497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 msgr2=0x56133b797fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.479+0000 7fc2350db740 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b797fa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc2240a3d00 tx=0x7fc224004b50 comp rx=0 tx=0).stop 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 -- 172.21.15.78:0/1966804497 shutdown_connections 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b797fa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78b2f0 0x56133b795670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 --2- 172.21.15.78:0/1966804497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78a9d0 0x56133b78adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 -- 172.21.15.78:0/1966804497 >> 172.21.15.78:0/1966804497 conn(0x56133b786f60 msgr2=0x56133b787340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 -- 172.21.15.78:0/1966804497 shutdown_connections 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 -- 172.21.15.78:0/1966804497 wait complete. 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 -- start start 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.480+0000 7fc2350db740 1 Processor -- start 2024-09-16T00:11:24.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b795400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b792750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 0x56133b792c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56133b798780 con 0x56133b78b2f0 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56133b6abf20 con 0x56133b78a9d0 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56133b798900 con 0x56133b795bb0 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 10 client.0 _ll_register_callbacks cb 0x56133b675030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b792750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b792750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35954/0 (socket says 172.21.15.78:35954) 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 -- 172.21.15.78:0/823579942 learned_addr learned my addr 172.21.15.78:0/823579942 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc23129f640 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 0x56133b792c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc230a9e640 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b795400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 -- 172.21.15.78:0/823579942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 msgr2=0x56133b792c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 0x56133b792c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 -- 172.21.15.78:0/823579942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 msgr2=0x56133b795400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b795400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 -- 172.21.15.78:0/823579942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133b4622b0 con 0x56133b78b2f0 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc22bfff640 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b792750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc218033000 tx=0x7fc21802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c808b55e477a486 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc20bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc20bfff640 1 -- 172.21.15.78:0/823579942 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21804e020 con 0x56133b78b2f0 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc20bfff640 1 -- 172.21.15.78:0/823579942 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fc218003c80 con 0x56133b78b2f0 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 1 -- 172.21.15.78:0/823579942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2240a4230 con 0x56133b78b2f0 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc20bfff640 1 -- 172.21.15.78:0/823579942 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc218059050 con 0x56133b78b2f0 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 10 client.14691 fetch_fsmap learned FSMap version 16 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.481+0000 7fc2350db740 1 -- 172.21.15.78:0/823579942 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x56133b7a3ff0 con 0x56133b78b2f0 2024-09-16T00:11:24.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc20bfff640 1 -- 172.21.15.78:0/823579942 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2240a4230 con 0x56133b78b2f0 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc20bfff640 1 -- 172.21.15.78:0/823579942 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7fc21803d4e0 con 0x56133b78b2f0 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc2350db740 10 client.14691 fetch_fsmap finished waiting for FSMap version 16 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc2350db740 10 client.14691 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc2350db740 20 client.14691 populate_metadata read hostname 'smithi078' 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc2350db740 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56133b7a66b0 0x56133b7a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc2350db740 4 client.14691 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc2350db740 1 -- 172.21.15.78:0/823579942 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56133b6fe600 con 0x56133b7a66b0 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.482+0000 7fc230a9e640 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56133b7a66b0 0x56133b7a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.483+0000 7fc230a9e640 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56133b7a66b0 0x56133b7a7b20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9787fb68632bdd0 server_cookie=9c7161e7bde27114 in_seq=0 out_seq=0 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.483+0000 7fc20bfff640 10 client.14691 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.483+0000 7fc20bfff640 1 -- 172.21.15.78:0/823579942 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56133b6fe600 con 0x56133b7a66b0 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.483+0000 7fc20bfff640 10 client.14691 handle_command_reply: tid=0 2024-09-16T00:11:24.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.483+0000 7fc2350db740 1 client.14691 shutdown 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 -- 172.21.15.78:0/823579942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 msgr2=0x56133b792750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b792750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc218033000 tx=0x7fc21802a420 comp rx=0 tx=0).stop 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 -- 172.21.15.78:0/823579942 shutdown_connections 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56133b7a66b0 0x56133b7a7b20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 0x56133b792c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc20bfff640 0 client.14691 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b792750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 --2- 172.21.15.78:0/823579942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b795400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 -- 172.21.15.78:0/823579942 >> 172.21.15.78:0/823579942 conn(0x56133b786f60 msgr2=0x56133b78cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 -- 172.21.15.78:0/823579942 shutdown_connections 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 1 -- 172.21.15.78:0/823579942 wait complete. 2024-09-16T00:11:24.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.484+0000 7fc2350db740 20 client.14691 trim_cache size 0 max 16384 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc2350db740 1 Processor -- start 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc2350db740 1 -- start start 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc2350db740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b7a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc2350db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78b2f0 0x56133b792010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc2350db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b791020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc2350db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56133b7a5e60 con 0x56133b795bb0 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc2350db740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56133b7a6530 con 0x56133b78a9d0 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc2350db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56133b7a6c00 con 0x56133b78b2f0 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc23129f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b791020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc23129f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b791020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35960/0 (socket says 172.21.15.78:35960) 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc23129f640 1 -- 172.21.15.78:0/1444550962 learned_addr learned my addr 172.21.15.78:0/1444550962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:24.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc230a9e640 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b7a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc22bfff640 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78b2f0 0x56133b792010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc23129f640 1 -- 172.21.15.78:0/1444550962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78b2f0 msgr2=0x56133b792010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.502+0000 7fc23129f640 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78b2f0 0x56133b792010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.503+0000 7fc23129f640 1 -- 172.21.15.78:0/1444550962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 msgr2=0x56133b7a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.503+0000 7fc23129f640 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b7a7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.503+0000 7fc23129f640 1 -- 172.21.15.78:0/1444550962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133b7880f0 con 0x56133b795bb0 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.503+0000 7fc23129f640 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b791020 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc224097a20 tx=0x7fc224097860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc1749a249732b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.503+0000 7fc2167fc640 1 -- 172.21.15.78:0/1444550962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240b4070 con 0x56133b795bb0 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.503+0000 7fc2167fc640 1 -- 172.21.15.78:0/1444550962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2240a4230 con 0x56133b795bb0 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.503+0000 7fc2167fc640 1 -- 172.21.15.78:0/1444550962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240a3dc0 con 0x56133b795bb0 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 -- 172.21.15.78:0/1444550962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 msgr2=0x56133b791020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b791020 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc224097a20 tx=0x7fc224097860 comp rx=0 tx=0).stop 2024-09-16T00:11:24.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 -- 172.21.15.78:0/1444550962 shutdown_connections 2024-09-16T00:11:24.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b795bb0 0x56133b791020 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b78b2f0 0x56133b792010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 --2- 172.21.15.78:0/1444550962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b7a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 -- 172.21.15.78:0/1444550962 >> 172.21.15.78:0/1444550962 conn(0x56133b786f60 msgr2=0x56133b792710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 -- 172.21.15.78:0/1444550962 shutdown_connections 2024-09-16T00:11:24.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 -- 172.21.15.78:0/1444550962 wait complete. 2024-09-16T00:11:24.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 -- start start 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.504+0000 7fc2350db740 1 Processor -- start 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc2350db740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b7a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc2350db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b7a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc2350db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 0x56133b80edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc2350db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56133b7a79a0 con 0x56133b78b2f0 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc2350db740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56133b7a72d0 con 0x56133b78a9d0 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc2350db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56133b7a8070 con 0x56133b795bb0 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc2350db740 10 client.0 _ll_register_callbacks cb 0x7fc218041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc22bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b7a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc22bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b7a97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35968/0 (socket says 172.21.15.78:35968) 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc22bfff640 1 -- 172.21.15.78:0/3774195343 learned_addr learned my addr 172.21.15.78:0/3774195343 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc23129f640 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 0x56133b80edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc22bfff640 1 -- 172.21.15.78:0/3774195343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 msgr2=0x56133b80edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc230a9e640 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b7a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc22bfff640 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 0x56133b80edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc22bfff640 1 -- 172.21.15.78:0/3774195343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 msgr2=0x56133b7a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc22bfff640 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b7a92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.505+0000 7fc22bfff640 1 -- 172.21.15.78:0/3774195343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133b80f2f0 con 0x56133b78b2f0 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc22bfff640 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b7a97e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc218031740 tx=0x7fc21803dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e1aed89bb8ad7f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc20bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc20bfff640 1 -- 172.21.15.78:0/3774195343 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218061020 con 0x56133b78b2f0 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc20bfff640 1 -- 172.21.15.78:0/3774195343 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fc218003a80 con 0x56133b78b2f0 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 1 -- 172.21.15.78:0/3774195343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2240a4230 con 0x56133b78b2f0 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc20bfff640 1 -- 172.21.15.78:0/3774195343 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc21806c050 con 0x56133b78b2f0 2024-09-16T00:11:24.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 10 client.14697 fetch_fsmap learned FSMap version 16 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 1 -- 172.21.15.78:0/3774195343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x56133b810530 con 0x56133b78b2f0 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc20bfff640 1 -- 172.21.15.78:0/3774195343 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2240a4230 con 0x56133b78b2f0 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc20bfff640 1 -- 172.21.15.78:0/3774195343 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7fc21804aa80 con 0x56133b78b2f0 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 10 client.14697 fetch_fsmap finished waiting for FSMap version 16 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 10 client.14697 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 20 client.14697 populate_metadata read hostname 'smithi078' 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56133b8151c0 0x56133b813ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 4 client.14697 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.506+0000 7fc2350db740 1 -- 172.21.15.78:0/3774195343 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "status"}) -- 0x56133b6f8b10 con 0x56133b8151c0 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.507+0000 7fc230a9e640 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56133b8151c0 0x56133b813ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.507+0000 7fc230a9e640 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56133b8151c0 0x56133b813ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90512fa9c2de24b8 server_cookie=8f744167f5157ec1 in_seq=0 out_seq=0 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.507+0000 7fc20bfff640 10 client.14697 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:11:24.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.507+0000 7fc20bfff640 1 -- 172.21.15.78:0/3774195343 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56133b6f8b10 con 0x56133b8151c0 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.507+0000 7fc20bfff640 10 client.14697 handle_command_reply: tid=0 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.507+0000 7fc2350db740 1 client.14697 shutdown 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.507+0000 7fc2350db740 1 -- 172.21.15.78:0/3774195343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 msgr2=0x56133b7a97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.507+0000 7fc2350db740 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b7a97e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc218031740 tx=0x7fc21803dd70 comp rx=0 tx=0).stop 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 1 -- 172.21.15.78:0/3774195343 shutdown_connections 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56133b8151c0 0x56133b813ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc20bfff640 0 client.14697 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56133b795bb0 0x56133b80edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56133b78b2f0 0x56133b7a97e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 1 --2- 172.21.15.78:0/3774195343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56133b78a9d0 0x56133b7a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 1 -- 172.21.15.78:0/3774195343 >> 172.21.15.78:0/3774195343 conn(0x56133b786f60 msgr2=0x56133b790fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 1 -- 172.21.15.78:0/3774195343 shutdown_connections 2024-09-16T00:11:24.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 1 -- 172.21.15.78:0/3774195343 wait complete. 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc2350db740 20 client.14697 trim_cache size 0 max 16384 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24470, 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 18.881051295999999, 2024-09-16T00:11:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 16, 2024-09-16T00:11:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 86, 2024-09-16T00:11:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:11:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 30.674296106 2024-09-16T00:11:24.528 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:11:24.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc232d02640 1 -- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc21c0a3240 msgr2=0x7fc21c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.508+0000 7fc232d02640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc21c0a3240 0x7fc21c0a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc2180316b0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 -- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c106a80 msgr2=0x7fc22c1a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c106a80 0x7fc22c1a4530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc22400b970 tx=0x7fc22400be30 comp rx=0 tx=0).stop 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 -- 172.21.15.78:0/1828818933 shutdown_connections 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc21c0a3240 0x7fc21c0a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc22c106a80 0x7fc22c1a4530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc22c101420 0x7fc22c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 --2- 172.21.15.78:0/1828818933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc22c100a80 0x7fc22c1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 -- 172.21.15.78:0/1828818933 >> 172.21.15.78:0/1828818933 conn(0x7fc22c0fc780 msgr2=0x7fc22c0fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 -- 172.21.15.78:0/1828818933 shutdown_connections 2024-09-16T00:11:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:24.509+0000 7fc232d02640 1 -- 172.21.15.78:0/1828818933 wait complete. 2024-09-16T00:11:24.751 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:24 smithi078 ceph-mon[31157]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:25.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:24 smithi193 ceph-mon[35484]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:25.193 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:11:25.193 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:11:25.193 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:11:25.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:24 smithi148 ceph-mon[35599]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:25.437 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 -- 172.21.15.78:0/1725877030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d01036e0 msgr2=0x7f70d0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 --2- 172.21.15.78:0/1725877030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d01036e0 0x7f70d0107b90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009930 tx=0x7f70b802a420 comp rx=0 tx=0).stop 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 -- 172.21.15.78:0/1725877030 shutdown_connections 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 --2- 172.21.15.78:0/1725877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d01082a0 0x7f70d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 --2- 172.21.15.78:0/1725877030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d01036e0 0x7f70d0107b90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 --2- 172.21.15.78:0/1725877030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70d0102d30 0x7f70d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 -- 172.21.15.78:0/1725877030 >> 172.21.15.78:0/1725877030 conn(0x7f70d00fc7d0 msgr2=0x7f70d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 -- 172.21.15.78:0/1725877030 shutdown_connections 2024-09-16T00:11:25.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.932+0000 7f70d52dd640 1 -- 172.21.15.78:0/1725877030 wait complete. 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70d52dd640 1 Processor -- start 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70d52dd640 1 -- start start 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70d52dd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d0102d30 0x7f70d0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70d52dd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d01036e0 0x7f70d0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70d52dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70d01082a0 0x7f70d0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70d52dd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01120a0 con 0x7f70d0102d30 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70d52dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0111f20 con 0x7f70d01082a0 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70d52dd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0112220 con 0x7f70d01036e0 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d0102d30 0x7f70d0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70ceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d0102d30 0x7f70d0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35988/0 (socket says 172.21.15.78:35988) 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70ceffd640 1 -- 172.21.15.78:0/2121879618 learned_addr learned my addr 172.21.15.78:0/2121879618 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.933+0000 7f70ce7fc640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d01036e0 0x7f70d0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70ceffd640 1 -- 172.21.15.78:0/2121879618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d01036e0 msgr2=0x7f70d0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70ceffd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d01036e0 0x7f70d0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70ceffd640 1 -- 172.21.15.78:0/2121879618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70d01082a0 msgr2=0x7f70d0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70ceffd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70d01082a0 0x7f70d0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70ceffd640 1 -- 172.21.15.78:0/2121879618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d01a69e0 con 0x7f70d0102d30 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70ceffd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d0102d30 0x7f70d0100990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c920 tx=0x7f70c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70c7fff640 1 -- 172.21.15.78:0/2121879618 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8007c70 con 0x7f70d0102d30 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d01a6cd0 con 0x7f70d0102d30 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70c7fff640 1 -- 172.21.15.78:0/2121879618 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c8007e10 con 0x7f70d0102d30 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70c7fff640 1 -- 172.21.15.78:0/2121879618 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8018890 con 0x7f70d0102d30 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.934+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d01a71d0 con 0x7f70d0102d30 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.935+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d0103110 con 0x7f70d0102d30 2024-09-16T00:11:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.936+0000 7f70c7fff640 1 -- 172.21.15.78:0/2121879618 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f70c8018a50 con 0x7f70d0102d30 2024-09-16T00:11:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.937+0000 7f70c7fff640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70bc0a3240 0x7f70bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.937+0000 7f70ce7fc640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70bc0a3240 0x7f70bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.937+0000 7f70c7fff640 1 -- 172.21.15.78:0/2121879618 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f70c80cff80 con 0x7f70d0102d30 2024-09-16T00:11:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.937+0000 7f70ce7fc640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70bc0a3240 0x7f70bc0a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009900 tx=0x7f70b8032d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:25.940+0000 7f70c7fff640 1 -- 172.21.15.78:0/2121879618 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70c8014030 con 0x7f70d0102d30 2024-09-16T00:11:26.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.138+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70d0101ec0 con 0x7f70d0102d30 2024-09-16T00:11:26.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.139+0000 7f70c7fff640 1 -- 172.21.15.78:0/2121879618 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f70d0101ec0 con 0x7f70d0102d30 2024-09-16T00:11:26.141 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70bc0a3240 msgr2=0x7f70bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70bc0a3240 0x7f70bc0a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009900 tx=0x7f70b8032d10 comp rx=0 tx=0).stop 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d0102d30 msgr2=0x7f70d0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d0102d30 0x7f70d0100990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c920 tx=0x7f70c800cde0 comp rx=0 tx=0).stop 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 shutdown_connections 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70bc0a3240 0x7f70bc0a5700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70d01082a0 0x7f70d0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d01036e0 0x7f70d0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 --2- 172.21.15.78:0/2121879618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70d0102d30 0x7f70d0100990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 >> 172.21.15.78:0/2121879618 conn(0x7f70d00fc7d0 msgr2=0x7f70d0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 shutdown_connections 2024-09-16T00:11:26.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26.141+0000 7f70d52dd640 1 -- 172.21.15.78:0/2121879618 wait complete. 2024-09-16T00:11:26.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:26 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2121879618' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:11:26.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:26 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2121879618' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:11:26.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:26 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2121879618' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:11:26.887 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T00:11:26.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:11:26.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-16T00:11:26.887 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-16T00:11:26.887 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:11:26.887 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T00:11:26.887 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:11:26.887 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:11:26.888 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T00:11:26.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:26.888 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-16T00:11:26.904 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:11:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:11:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:11:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:11:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-16T00:11:26.904 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42295sec preferred_lft 42295sec 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link noprefixroute 2024-09-16T00:11:26.905 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:11:26.906 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:11:26.906 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:11:26.906 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:11:26.906 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:11:26.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-09-16T00:11:26.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-09-16T00:11:26.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:11:26.907 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:11:26.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:11:27.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:11:27.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:27.107 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:11:27.131 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-16T00:11:27.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:27.133 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-16T00:11:27.216 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-16T00:11:27.216 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:11:27.216 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T00:11:27.216 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T00:11:27.216 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:11:27.217 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:11:27.217 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:11:27.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T00:11:27.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T00:11:27.217 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T00:11:27.217 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:11:27.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:11:27.407 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:27 smithi078 ceph-mon[31157]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:27.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:27 smithi148 ceph-mon[35599]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:11:27.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:27.522 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-16T00:11:27.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:27.538 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-09-16T00:11:27.593 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:11:27.593 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:11:27.593 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:11:27.593 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-16T00:11:27.593 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:11:27.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:27 smithi193 ceph-mon[35484]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:27.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:11:27.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:11:27.713 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-16T00:11:27.713 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:11:27.713 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:11:27.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-16T00:11:27.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:11:27.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-16T00:11:27.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-16T00:11:27.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-16T00:11:27.713 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:11:27.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:11:27.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:11:27.876 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:11:27.876 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:11:27.876 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-16T00:11:27.876 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:11:27.876 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:11:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:11:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:11:27.953 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-16T00:11:27.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:11:27.953 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:11:27.969 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T00:11:27.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:11:27.970 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:11:28.025 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2024-09-16T00:11:28.090 DEBUG:teuthology.orchestra.run.smithi078:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:11:28.147 INFO:teuthology.orchestra.run.smithi078.stdout:/proc 2024-09-16T00:11:28.147 INFO:teuthology.orchestra.run.smithi078.stdout:/sys 2024-09-16T00:11:28.147 INFO:teuthology.orchestra.run.smithi078.stdout:/dev 2024-09-16T00:11:28.147 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/security 2024-09-16T00:11:28.147 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/shm 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/pts 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/run 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/cgroup 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/pstore 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/bpf 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/ 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/selinux 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/mqueue 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/hugepages 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/debug 2024-09-16T00:11:28.148 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/tracing 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/fuse/connections 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/config 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/1000 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/ceph 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/ab44076036a51f4c7dca14e871a0450c4401165d0e8ac25d0ecf8a986ae6fc48/merged 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/774b073df6f19cfa1dde267476e725a81eb01b1d4f9d7ba4c7fa3dbf286fd7ef/merged 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/0 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:11:28.149 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/d6985cd114d47083b4dbbb8f1370bb10490387f3b350fa2e96cbf09e99f2c9f0/merged 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/2c2fa1d74224a21f9e9cf6b64311491126954905ebfd4837ee54ffad930e8092/merged 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/0f7c771a9254baa8b9cf0eb188a8fe2e9415d3d9f772761515671975aa42c782/merged 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/dfa38b03ccb3db1919823c9538d06044f9984ea5aba5c77332465384985dbc80/merged 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/b2a794c24bd73f3394551e05ec175f6948ad7b7497e1096c1ef252a85e5f2c78/merged 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/8da4177bfa280c18cdb27634f1a2add4a10b3ee625467c84c585abedc76aae04/merged 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/2bc24a4c1b3b1fd005eeb07d584a527e62f86844ffd92b556f2b7129dde6e0e1/merged 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/73195b574f0d9c5a101a5fec0c3bd32c5fb76949b04f4c95f99502d7516bd580/merged 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:11:28.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:28.151 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2024-09-16T00:11:28.203 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T00:11:28.203 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-16T00:11:28.246 DEBUG:teuthology.orchestra.run.smithi078:> sudo modprobe fuse 2024-09-16T00:11:28.271 DEBUG:teuthology.orchestra.run.smithi078:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:11:28.328 INFO:teuthology.orchestra.run.smithi078.stdout:/proc 2024-09-16T00:11:28.328 INFO:teuthology.orchestra.run.smithi078.stdout:/sys 2024-09-16T00:11:28.328 INFO:teuthology.orchestra.run.smithi078.stdout:/dev 2024-09-16T00:11:28.328 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/security 2024-09-16T00:11:28.328 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/shm 2024-09-16T00:11:28.328 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/pts 2024-09-16T00:11:28.328 INFO:teuthology.orchestra.run.smithi078.stdout:/run 2024-09-16T00:11:28.328 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/cgroup 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/pstore 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/bpf 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/ 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/selinux 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/mqueue 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/hugepages 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/debug 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/tracing 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/fs/fuse/connections 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/sys/kernel/config 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:11:28.329 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/1000 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/ceph 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/ab44076036a51f4c7dca14e871a0450c4401165d0e8ac25d0ecf8a986ae6fc48/merged 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/774b073df6f19cfa1dde267476e725a81eb01b1d4f9d7ba4c7fa3dbf286fd7ef/merged 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/run/user/0 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/d6985cd114d47083b4dbbb8f1370bb10490387f3b350fa2e96cbf09e99f2c9f0/merged 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/2c2fa1d74224a21f9e9cf6b64311491126954905ebfd4837ee54ffad930e8092/merged 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/0f7c771a9254baa8b9cf0eb188a8fe2e9415d3d9f772761515671975aa42c782/merged 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/dfa38b03ccb3db1919823c9538d06044f9984ea5aba5c77332465384985dbc80/merged 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/b2a794c24bd73f3394551e05ec175f6948ad7b7497e1096c1ef252a85e5f2c78/merged 2024-09-16T00:11:28.330 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/8da4177bfa280c18cdb27634f1a2add4a10b3ee625467c84c585abedc76aae04/merged 2024-09-16T00:11:28.331 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/2bc24a4c1b3b1fd005eeb07d584a527e62f86844ffd92b556f2b7129dde6e0e1/merged 2024-09-16T00:11:28.331 INFO:teuthology.orchestra.run.smithi078.stdout:/var/lib/containers/storage/overlay/73195b574f0d9c5a101a5fec0c3bd32c5fb76949b04f4c95f99502d7516bd580/merged 2024-09-16T00:11:28.331 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns 2024-09-16T00:11:28.331 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:11:28.331 INFO:teuthology.orchestra.run.smithi078.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:11:28.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:28.331 DEBUG:teuthology.orchestra.run.smithi078:> ls /sys/fs/fuse/connections 2024-09-16T00:11:28.362 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:2024-09-16T00:11:28.361+0000 7fdb290ef480 -1 init, newargv = 0x55982304f9b0 newargc=15 2024-09-16T00:11:28.362 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[75773]: starting ceph client 2024-09-16T00:11:28.368 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[75773]: starting fuse 2024-09-16T00:11:28.377 INFO:teuthology.orchestra.run.smithi078.stdout:96 2024-09-16T00:11:28.378 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [96] 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> sudo stdin-killer -- python3 -c ' 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> import glob 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> import re 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> import os 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> import subprocess 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> def _find_admin_socket(client_name): 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> files = glob.glob(asok_path) 2024-09-16T00:11:28.378 DEBUG:teuthology.orchestra.run.smithi078:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> # Given a non-glob path, it better be there 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> if "*" not in asok_path: 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> assert(len(files) == 1) 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> return files[0] 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> for f in files: 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> contents = proc_f.read() 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> if mountpoint in contents: 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> return f 2024-09-16T00:11:28.379 DEBUG:teuthology.orchestra.run.smithi078:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T00:11:28.380 DEBUG:teuthology.orchestra.run.smithi078:> 2024-09-16T00:11:28.380 DEBUG:teuthology.orchestra.run.smithi078:> print(_find_admin_socket("client.admin")) 2024-09-16T00:11:28.380 DEBUG:teuthology.orchestra.run.smithi078:> ' 2024-09-16T00:11:28.495 INFO:teuthology.orchestra.run.smithi078.stdout:/var/run/ceph/ceph-client.admin.75773.asok 2024-09-16T00:11:28.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:11:28.505 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75773.asok 2024-09-16T00:11:28.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:28.506 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75773.asok status 2024-09-16T00:11:28.642 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:11:28.642 INFO:teuthology.orchestra.run.smithi078.stdout: "metadata": { 2024-09-16T00:11:28.642 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-16T00:11:28.642 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "entity_id": "admin", 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": "smithi078", 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "pid": "75773", 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "root": "/" 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_count": 0, 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "dentry_pinned_count": 0, 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 14709, 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "inst": { 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": { 2024-09-16T00:11:28.643 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "client", 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "num": 14709 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1148806047 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": { 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:11:28.644 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1148806047 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: "inst_str": "client.14709 192.168.0.1:0/1148806047", 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: "addr_str": "192.168.0.1:0/1148806047", 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: "inode_count": 1, 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_epoch": 16, 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch": 86, 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_epoch_barrier": 0, 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: "blocklisted": false, 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs" 2024-09-16T00:11:28.645 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:11:28.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:28.652 DEBUG:teuthology.orchestra.run.smithi078:> stat --file-system '--printf=%T 2024-09-16T00:11:28.652 DEBUG:teuthology.orchestra.run.smithi078:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:11:28.668 INFO:teuthology.orchestra.run.smithi078.stdout:fuseblk 2024-09-16T00:11:28.668 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:11:28.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:28.668 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:11:28.734 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-16T00:11:28.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:11:28.861 INFO:teuthology.orchestra.run.smithi078.stdout:# file: . 2024-09-16T00:11:28.861 INFO:teuthology.orchestra.run.smithi078.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-16T00:11:28.861 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:28.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:11:28.867 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:11:28.867 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:11:28.884 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:11:29.172 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:29.602 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:29 smithi078 ceph-mon[31157]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:29.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:29 smithi148 ceph-mon[35599]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.684+0000 7fb57843d640 1 -- 172.21.15.78:0/4258727163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb57010c180 msgr2=0x7fb57010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.684+0000 7fb57843d640 1 --2- 172.21.15.78:0/4258727163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb57010c180 0x7fb57010e600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb568009ca0 tx=0x7fb56802a950 comp rx=0 tx=0).stop 2024-09-16T00:11:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.684+0000 7fb57843d640 1 -- 172.21.15.78:0/4258727163 shutdown_connections 2024-09-16T00:11:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.684+0000 7fb57843d640 1 --2- 172.21.15.78:0/4258727163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb57010c180 0x7fb57010e600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.684+0000 7fb57843d640 1 --2- 172.21.15.78:0/4258727163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5701036c0 0x7fb57010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.684+0000 7fb57843d640 1 --2- 172.21.15.78:0/4258727163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb570102da0 0x7fb570103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.684+0000 7fb57843d640 1 -- 172.21.15.78:0/4258727163 >> 172.21.15.78:0/4258727163 conn(0x7fb5700fc7a0 msgr2=0x7fb5700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.684+0000 7fb57843d640 1 -- 172.21.15.78:0/4258727163 shutdown_connections 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.685+0000 7fb57843d640 1 -- 172.21.15.78:0/4258727163 wait complete. 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.685+0000 7fb57843d640 1 Processor -- start 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.685+0000 7fb57843d640 1 -- start start 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb57843d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb570102da0 0x7fb570198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb57843d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5701036c0 0x7fb570198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb57843d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57010c180 0x7fb57019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb57843d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb570112020 con 0x7fb5701036c0 2024-09-16T00:11:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb57843d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb570111ea0 con 0x7fb570102da0 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb57843d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701121a0 con 0x7fb57010c180 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5701036c0 0x7fb570198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5701036c0 0x7fb570198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55776/0 (socket says 172.21.15.78:55776) 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 -- 172.21.15.78:0/2350447077 learned_addr learned my addr 172.21.15.78:0/2350447077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5769da640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57010c180 0x7fb57019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5761d9640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb570102da0 0x7fb570198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 -- 172.21.15.78:0/2350447077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57010c180 msgr2=0x7fb57019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57010c180 0x7fb57019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 -- 172.21.15.78:0/2350447077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb570102da0 msgr2=0x7fb570198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb570102da0 0x7fb570198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 -- 172.21.15.78:0/2350447077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57019d4d0 con 0x7fb5701036c0 2024-09-16T00:11:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5759d8640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5701036c0 0x7fb570198a60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb56000b4a0 tx=0x7fb56000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5677fe640 1 -- 172.21.15.78:0/2350447077 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560015020 con 0x7fb5701036c0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5677fe640 1 -- 172.21.15.78:0/2350447077 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb560011d10 con 0x7fb5701036c0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb5677fe640 1 -- 172.21.15.78:0/2350447077 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56000f2e0 con 0x7fb5701036c0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.686+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57019d7c0 con 0x7fb5701036c0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.687+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701a50a0 con 0x7fb5701036c0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.687+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb570103200 con 0x7fb5701036c0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.689+0000 7fb5677fe640 1 -- 172.21.15.78:0/2350447077 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb560018020 con 0x7fb5701036c0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.689+0000 7fb5677fe640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb55c0a3240 0x7fb55c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.689+0000 7fb5761d9640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb55c0a3240 0x7fb55c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.689+0000 7fb5677fe640 1 -- 172.21.15.78:0/2350447077 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fb5600cf980 con 0x7fb5701036c0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.690+0000 7fb5761d9640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb55c0a3240 0x7fb55c0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009990 tx=0x7fb56c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.692+0000 7fb5677fe640 1 -- 172.21.15.78:0/2350447077 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb56001f020 con 0x7fb5701036c0 2024-09-16T00:11:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.871+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb570199880 con 0x7fb5701036c0 2024-09-16T00:11:29.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.872+0000 7fb5677fe640 1 -- 172.21.15.78:0/2350447077 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+591 (secure 0 0 0) 0x7fb570199880 con 0x7fb5701036c0 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1052494441 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3861325161 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1335742179 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3957981561 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2949703267 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1052494441 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1515675480 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/486633606 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1597437728 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:29.874 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1515675480 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-09-16T00:11:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb55c0a3240 msgr2=0x7fb55c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb55c0a3240 0x7fb55c0a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009990 tx=0x7fb56c009110 comp rx=0 tx=0).stop 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5701036c0 msgr2=0x7fb570198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5701036c0 0x7fb570198a60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb56000b4a0 tx=0x7fb56000b8a0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 shutdown_connections 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb55c0a3240 0x7fb55c0a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57010c180 0x7fb57019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5701036c0 0x7fb570198a60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 --2- 172.21.15.78:0/2350447077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb570102da0 0x7fb570198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 >> 172.21.15.78:0/2350447077 conn(0x7fb5700fc7a0 msgr2=0x7fb5700fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 shutdown_connections 2024-09-16T00:11:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:29.874+0000 7fb57843d640 1 -- 172.21.15.78:0/2350447077 wait complete. 2024-09-16T00:11:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:29 smithi193 ceph-mon[35484]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:30.633 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:30 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2350447077' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:11:30.662 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:11:30.663 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:11:30.680 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:11:30.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:30 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2350447077' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:11:30.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:30 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2350447077' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:11:30.970 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:31.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 -- 172.21.15.78:0/751004687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 msgr2=0x7f2ee41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 --2- 172.21.15.78:0/751004687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 0x7f2ee41067e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8009900 tx=0x7f2ed802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 -- 172.21.15.78:0/751004687 shutdown_connections 2024-09-16T00:11:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 --2- 172.21.15.78:0/751004687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ee4106f10 0x7f2ee410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 --2- 172.21.15.78:0/751004687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 0x7f2ee41067e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 --2- 172.21.15.78:0/751004687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4101de0 0x7f2ee41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 -- 172.21.15.78:0/751004687 >> 172.21.15.78:0/751004687 conn(0x7f2ee40fd8c0 msgr2=0x7f2ee40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:31.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 -- 172.21.15.78:0/751004687 shutdown_connections 2024-09-16T00:11:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.455+0000 7f2ee9686640 1 -- 172.21.15.78:0/751004687 wait complete. 2024-09-16T00:11:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee9686640 1 Processor -- start 2024-09-16T00:11:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee9686640 1 -- start start 2024-09-16T00:11:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee9686640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ee4101de0 0x7f2ee4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee9686640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 0x7f2ee41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:31.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee9686640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106f10 0x7f2ee419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee9686640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111f90 con 0x7f2ee4102790 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee9686640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111e10 con 0x7f2ee4106f10 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee9686640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4112110 con 0x7f2ee4101de0 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 0x7f2ee41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 0x7f2ee41989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55806/0 (socket says 172.21.15.78:55806) 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee27fc640 1 -- 172.21.15.78:0/3581457045 learned_addr learned my addr 172.21.15.78:0/3581457045 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee2ffd640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ee4101de0 0x7f2ee4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.456+0000 7f2ee37fe640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106f10 0x7f2ee419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ee27fc640 1 -- 172.21.15.78:0/3581457045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ee4101de0 msgr2=0x7f2ee4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ee27fc640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ee4101de0 0x7f2ee4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ee27fc640 1 -- 172.21.15.78:0/3581457045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106f10 msgr2=0x7f2ee419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:31.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ee27fc640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106f10 0x7f2ee419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ee27fc640 1 -- 172.21.15.78:0/3581457045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee419d440 con 0x7f2ee4102790 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ee27fc640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 0x7f2ee41989d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006010 tx=0x7f2ed80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ecbfff640 1 -- 172.21.15.78:0/3581457045 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed803f030 con 0x7f2ee4102790 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ecbfff640 1 -- 172.21.15.78:0/3581457045 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8003ce0 con 0x7f2ee4102790 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee419d6d0 con 0x7f2ee4102790 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ecbfff640 1 -- 172.21.15.78:0/3581457045 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8035070 con 0x7f2ee4102790 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.457+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41a5010 con 0x7f2ee4102790 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.458+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee41021c0 con 0x7f2ee4102790 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.462+0000 7f2ecbfff640 1 -- 172.21.15.78:0/3581457045 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2ed8042030 con 0x7f2ee4102790 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.463+0000 7f2ecbfff640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2ecc0a32c0 0x7f2ecc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.463+0000 7f2ee2ffd640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2ecc0a32c0 0x7f2ecc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:31.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.463+0000 7f2ecbfff640 1 -- 172.21.15.78:0/3581457045 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f2ed80f0370 con 0x7f2ee4102790 2024-09-16T00:11:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.463+0000 7f2ee2ffd640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2ecc0a32c0 0x7f2ecc0a5780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400bbe0 tx=0x7f2ed400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:31.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.465+0000 7f2ecbfff640 1 -- 172.21.15.78:0/3581457045 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ed80bc8c0 con 0x7f2ee4102790 2024-09-16T00:11:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.645+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ee41997f0 con 0x7f2ee4102790 2024-09-16T00:11:31.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:31 smithi078 ceph-mon[31157]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:31.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.645+0000 7f2ecbfff640 1 -- 172.21.15.78:0/3581457045 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+591 (secure 0 0 0) 0x7f2ee41997f0 con 0x7f2ee4102790 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1052494441 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3861325161 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1335742179 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3957981561 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2949703267 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1052494441 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1515675480 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/486633606 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1597437728 2024-09-17T00:04:02.295421+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1515675480 2024-09-17T00:03:37.955119+0000 2024-09-16T00:11:31.647 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-09-16T00:11:31.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2ecc0a32c0 msgr2=0x7f2ecc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2ecc0a32c0 0x7f2ecc0a5780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ed400bbe0 tx=0x7f2ed400b390 comp rx=0 tx=0).stop 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 msgr2=0x7f2ee41989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 0x7f2ee41989d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8006010 tx=0x7f2ed80042f0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 shutdown_connections 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2ecc0a32c0 0x7f2ecc0a5780 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106f10 0x7f2ee419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ee4102790 0x7f2ee41989d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 --2- 172.21.15.78:0/3581457045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ee4101de0 0x7f2ee4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:31.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.647+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 >> 172.21.15.78:0/3581457045 conn(0x7f2ee40fd8c0 msgr2=0x7f2ee40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.648+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 shutdown_connections 2024-09-16T00:11:31.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:31.648+0000 7f2ee9686640 1 -- 172.21.15.78:0/3581457045 wait complete. 2024-09-16T00:11:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:31 smithi193 ceph-mon[35484]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:31 smithi148 ceph-mon[35599]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:11:32.458 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi078.front.sepia.ceph.com... 2024-09-16T00:11:32.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:32.458 DEBUG:teuthology.orchestra.run.smithi078:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:11:32.488 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi078.stderr:ceph-fuse[75773]: fuse finished with error 0 and tester_r 0 2024-09-16T00:11:32.489 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:32.541 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-16T00:11:32.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:32.541 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-16T00:11:32.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:11:32.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:32 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3581457045' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:11:32.868 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:32.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:32 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3581457045' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:11:32.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:32 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3581457045' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:11:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/3791536359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 msgr2=0x7f6f98106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/3791536359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 0x7f6f98106e60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009930 tx=0x7f6f8802a420 comp rx=0 tx=0).stop 2024-09-16T00:11:33.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/3791536359 shutdown_connections 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/3791536359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101420 0x7f6f9810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/3791536359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98100a80 0x7f6f98100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/3791536359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 0x7f6f98106e60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/3791536359 >> 172.21.15.78:0/3791536359 conn(0x7f6f980fc7a0 msgr2=0x7f6f980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/3791536359 shutdown_connections 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.372+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/3791536359 wait complete. 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9d1fc640 1 Processor -- start 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9d1fc640 1 -- start start 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9d1fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98100a80 0x7f6f981984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9d1fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101420 0x7f6f98198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9d1fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 0x7f6f9819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:33.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9d1fc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98110ea0 con 0x7f6f98106a80 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9d1fc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98110d20 con 0x7f6f98101420 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9d1fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98111020 con 0x7f6f98100a80 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 0x7f6f9819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 0x7f6f9819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55818/0 (socket says 172.21.15.78:55818) 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9759e640 1 -- 172.21.15.78:0/298538498 learned_addr learned my addr 172.21.15.78:0/298538498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f96d9d640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98100a80 0x7f6f981984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9759e640 1 -- 172.21.15.78:0/298538498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98100a80 msgr2=0x7f6f981984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9759e640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98100a80 0x7f6f981984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9759e640 1 -- 172.21.15.78:0/298538498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101420 msgr2=0x7f6f98198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f8ffff640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101420 0x7f6f98198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9759e640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101420 0x7f6f98198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f9759e640 1 -- 172.21.15.78:0/298538498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9819d470 con 0x7f6f98106a80 2024-09-16T00:11:33.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.373+0000 7f6f96d9d640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98100a80 0x7f6f981984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.374+0000 7f6f9759e640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 0x7f6f9819cd90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000cad0 tx=0x7f6f9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.374+0000 7f6f94d99640 1 -- 172.21.15.78:0/298538498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000ae20 con 0x7f6f98106a80 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.374+0000 7f6f94d99640 1 -- 172.21.15.78:0/298538498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f90005020 con 0x7f6f98106a80 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.374+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9819d760 con 0x7f6f98106a80 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.374+0000 7f6f94d99640 1 -- 172.21.15.78:0/298538498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000f6f0 con 0x7f6f98106a80 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.374+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f981a5040 con 0x7f6f98106a80 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.375+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f98100ee0 con 0x7f6f98106a80 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.376+0000 7f6f94d99640 1 -- 172.21.15.78:0/298538498 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6f900051d0 con 0x7f6f98106a80 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.376+0000 7f6f94d99640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f840a3240 0x7f6f840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.376+0000 7f6f96d9d640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f840a3240 0x7f6f840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.377+0000 7f6f94d99640 1 -- 172.21.15.78:0/298538498 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f6f900cfca0 con 0x7f6f98106a80 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.377+0000 7f6f96d9d640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f840a3240 0x7f6f840a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6f88000c00 tx=0x7f6f88032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:33.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.380+0000 7f6f94d99640 1 -- 172.21.15.78:0/298538498 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f90014030 con 0x7f6f98106a80 2024-09-16T00:11:33.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.551+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f981a5270 con 0x7f6f98106a80 2024-09-16T00:11:33.553 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:33 smithi078 ceph-mon[31157]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:11:33.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.553+0000 7f6f94d99640 1 -- 172.21.15.78:0/298538498 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24213 (secure 0 0 0) 0x7f6f9009c780 con 0x7f6f98106a80 2024-09-16T00:11:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:33.555 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":86,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:11:14.379068+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f840a3240 msgr2=0x7f6f840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f840a3240 0x7f6f840a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6f88000c00 tx=0x7f6f88032d50 comp rx=0 tx=0).stop 2024-09-16T00:11:33.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 msgr2=0x7f6f9819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 0x7f6f9819cd90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000cad0 tx=0x7f6f9000cf90 comp rx=0 tx=0).stop 2024-09-16T00:11:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 shutdown_connections 2024-09-16T00:11:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f840a3240 0x7f6f840a5700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98106a80 0x7f6f9819cd90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98101420 0x7f6f98198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 --2- 172.21.15.78:0/298538498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98100a80 0x7f6f981984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:33.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 >> 172.21.15.78:0/298538498 conn(0x7f6f980fc7a0 msgr2=0x7f6f981048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 shutdown_connections 2024-09-16T00:11:33.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:33.556+0000 7f6f9d1fc640 1 -- 172.21.15.78:0/298538498 wait complete. 2024-09-16T00:11:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:33 smithi193 ceph-mon[35484]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:11:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:33 smithi148 ceph-mon[35599]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:11:34.239 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:11:34.490 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:34.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:34 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/298538498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:34.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:34 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/298538498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:34 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/298538498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 -- 172.21.15.78:0/2667427497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10106f10 msgr2=0x7f0f1010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 --2- 172.21.15.78:0/2667427497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10106f10 0x7f0f1010ea90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009ca0 tx=0x7f0f0802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:11:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 -- 172.21.15.78:0/2667427497 shutdown_connections 2024-09-16T00:11:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 --2- 172.21.15.78:0/2667427497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10106f10 0x7f0f1010ea90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 --2- 172.21.15.78:0/2667427497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f10102790 0x7f0f101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 --2- 172.21.15.78:0/2667427497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10101de0 0x7f0f101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 -- 172.21.15.78:0/2667427497 >> 172.21.15.78:0/2667427497 conn(0x7f0f100fd8c0 msgr2=0x7f0f100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:34.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 -- 172.21.15.78:0/2667427497 shutdown_connections 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.965+0000 7f0f178ed640 1 -- 172.21.15.78:0/2667427497 wait complete. 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f178ed640 1 Processor -- start 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f178ed640 1 -- start start 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f178ed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f10101de0 0x7f0f100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f178ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10102790 0x7f0f10074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f178ed640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10106f10 0x7f0f10074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f178ed640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10111f70 con 0x7f0f10102790 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f178ed640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10111df0 con 0x7f0f10106f10 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f178ed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101120f0 con 0x7f0f10101de0 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f14e88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10102790 0x7f0f10074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f14e88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10102790 0x7f0f10074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55836/0 (socket says 172.21.15.78:55836) 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f14e88640 1 -- 172.21.15.78:0/76794147 learned_addr learned my addr 172.21.15.78:0/76794147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:34.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f15689640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f10101de0 0x7f0f100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f14e88640 1 -- 172.21.15.78:0/76794147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f10101de0 msgr2=0x7f0f100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f15e8a640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10106f10 0x7f0f10074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f14e88640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f10101de0 0x7f0f100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f14e88640 1 -- 172.21.15.78:0/76794147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10106f10 msgr2=0x7f0f10074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f14e88640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10106f10 0x7f0f10074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.966+0000 7f0f14e88640 1 -- 172.21.15.78:0/76794147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f101a68b0 con 0x7f0f10102790 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.967+0000 7f0f14e88640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10102790 0x7f0f10074a30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00d770 tx=0x7f0f0c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.967+0000 7f0f067fc640 1 -- 172.21.15.78:0/76794147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c017070 con 0x7f0f10102790 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.967+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f101a6ba0 con 0x7f0f10102790 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.967+0000 7f0f067fc640 1 -- 172.21.15.78:0/76794147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f0c005500 con 0x7f0f10102790 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.967+0000 7f0f067fc640 1 -- 172.21.15.78:0/76794147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c004e00 con 0x7f0f10102790 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.967+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101a7090 con 0x7f0f10102790 2024-09-16T00:11:34.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.968+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f101021c0 con 0x7f0f10102790 2024-09-16T00:11:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.969+0000 7f0f067fc640 1 -- 172.21.15.78:0/76794147 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0f0c004fa0 con 0x7f0f10102790 2024-09-16T00:11:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.970+0000 7f0f067fc640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0efc0a3240 0x7f0efc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.970+0000 7f0f15689640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0efc0a3240 0x7f0efc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.970+0000 7f0f067fc640 1 -- 172.21.15.78:0/76794147 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f0f0c0cf990 con 0x7f0f10102790 2024-09-16T00:11:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.970+0000 7f0f15689640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0efc0a3240 0x7f0efc0a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005fd0 tx=0x7f0f00005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:34.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:34.973+0000 7f0f067fc640 1 -- 172.21.15.78:0/76794147 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f0c09c470 con 0x7f0f10102790 2024-09-16T00:11:35.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.141+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f10109b60 con 0x7f0f10102790 2024-09-16T00:11:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.142+0000 7f0f067fc640 1 -- 172.21.15.78:0/76794147 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24213 (secure 0 0 0) 0x7f0f0c09bd70 con 0x7f0f10102790 2024-09-16T00:11:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:35.144 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":86,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:11:14.379068+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.144+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0efc0a3240 msgr2=0x7f0efc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.144+0000 7f0f178ed640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0efc0a3240 0x7f0efc0a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005fd0 tx=0x7f0f00005850 comp rx=0 tx=0).stop 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.144+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10102790 msgr2=0x7f0f10074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.144+0000 7f0f178ed640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10102790 0x7f0f10074a30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00d770 tx=0x7f0f0c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.145+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 shutdown_connections 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.145+0000 7f0f178ed640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0efc0a3240 0x7f0efc0a5700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.145+0000 7f0f178ed640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0f10106f10 0x7f0f10074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.145+0000 7f0f178ed640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f10102790 0x7f0f10074a30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.145+0000 7f0f178ed640 1 --2- 172.21.15.78:0/76794147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f10101de0 0x7f0f100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.145+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 >> 172.21.15.78:0/76794147 conn(0x7f0f100fd8c0 msgr2=0x7f0f100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.145+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 shutdown_connections 2024-09-16T00:11:35.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:35.145+0000 7f0f178ed640 1 -- 172.21.15.78:0/76794147 wait complete. 2024-09-16T00:11:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:35 smithi078 ceph-mon[31157]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:11:35.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:35 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/76794147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:35 smithi193 ceph-mon[35484]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:11:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:35 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/76794147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:35.955 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-16T00:11:35.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:35 smithi148 ceph-mon[35599]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:11:35.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:35 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/76794147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:36.199 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/2586273773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64105f40 msgr2=0x7f9d641063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/2586273773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64105f40 0x7f9d641063d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009ca0 tx=0x7f9d5c02a940 comp rx=0 tx=0).stop 2024-09-16T00:11:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/2586273773 shutdown_connections 2024-09-16T00:11:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/2586273773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64105f40 0x7f9d641063d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/2586273773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d64101450 0x7f9d64105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/2586273773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d64100b30 0x7f9d64100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/2586273773 >> 172.21.15.78:0/2586273773 conn(0x7f9d640fc7a0 msgr2=0x7f9d640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:36.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/2586273773 shutdown_connections 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.680+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/2586273773 wait complete. 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d6b4e8640 1 Processor -- start 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d6b4e8640 1 -- start start 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d6b4e8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64100b30 0x7f9d641a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d6b4e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d64101450 0x7f9d641a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d6b4e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d64105f40 0x7f9d6419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d6b4e8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64118560 con 0x7f9d64100b30 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d6b4e8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641183e0 con 0x7f9d64101450 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d6b4e8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641186e0 con 0x7f9d64105f40 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.681+0000 7f9d69284640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64100b30 0x7f9d641a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d69284640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64100b30 0x7f9d641a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56136/0 (socket says 172.21.15.78:56136) 2024-09-16T00:11:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d69284640 1 -- 172.21.15.78:0/3145989164 learned_addr learned my addr 172.21.15.78:0/3145989164 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d68a83640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d64101450 0x7f9d641a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d69284640 1 -- 172.21.15.78:0/3145989164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d64105f40 msgr2=0x7f9d6419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d69284640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d64105f40 0x7f9d6419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d69284640 1 -- 172.21.15.78:0/3145989164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d64101450 msgr2=0x7f9d641a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d69284640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d64101450 0x7f9d641a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d69284640 1 -- 172.21.15.78:0/3145989164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6419eff0 con 0x7f9d64100b30 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d69284640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64100b30 0x7f9d641a46d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c890 tx=0x7f9d6000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d5a7fc640 1 -- 172.21.15.78:0/3145989164 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60004220 con 0x7f9d64100b30 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6419f2e0 con 0x7f9d64100b30 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d5a7fc640 1 -- 172.21.15.78:0/3145989164 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d600043c0 con 0x7f9d64100b30 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d5a7fc640 1 -- 172.21.15.78:0/3145989164 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d600049e0 con 0x7f9d64100b30 2024-09-16T00:11:36.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.682+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d641ab530 con 0x7f9d64100b30 2024-09-16T00:11:36.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.683+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d6406b070 con 0x7f9d64100b30 2024-09-16T00:11:36.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.684+0000 7f9d5a7fc640 1 -- 172.21.15.78:0/3145989164 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9d600079b0 con 0x7f9d64100b30 2024-09-16T00:11:36.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.685+0000 7f9d5a7fc640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d500a3240 0x7f9d500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:36.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.685+0000 7f9d68a83640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d500a3240 0x7f9d500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:36.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.685+0000 7f9d5a7fc640 1 -- 172.21.15.78:0/3145989164 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f9d600cfc90 con 0x7f9d64100b30 2024-09-16T00:11:36.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.685+0000 7f9d68a83640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d500a3240 0x7f9d500a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d540058c0 tx=0x7f9d54005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:36.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.688+0000 7f9d5a7fc640 1 -- 172.21.15.78:0/3145989164 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d60014030 con 0x7f9d64100b30 2024-09-16T00:11:36.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:36.888+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9d64105920 con 0x7f9d64100b30 2024-09-16T00:11:37.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.474+0000 7f9d5a7fc640 1 -- 172.21.15.78:0/3145989164 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f9d6009c770 con 0x7f9d64100b30 2024-09-16T00:11:37.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d500a3240 msgr2=0x7f9d500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:37.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d500a3240 0x7f9d500a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9d540058c0 tx=0x7f9d54005850 comp rx=0 tx=0).stop 2024-09-16T00:11:37.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64100b30 msgr2=0x7f9d641a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:37.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64100b30 0x7f9d641a46d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000c890 tx=0x7f9d6000cd50 comp rx=0 tx=0).stop 2024-09-16T00:11:37.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 shutdown_connections 2024-09-16T00:11:37.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d500a3240 0x7f9d500a5700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:37.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d64105f40 0x7f9d6419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:37.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d64101450 0x7f9d641a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:37.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 --2- 172.21.15.78:0/3145989164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d64100b30 0x7f9d641a46d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:37.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 >> 172.21.15.78:0/3145989164 conn(0x7f9d640fc7a0 msgr2=0x7f9d641038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:37.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 shutdown_connections 2024-09-16T00:11:37.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:37.477+0000 7f9d6b4e8640 1 -- 172.21.15.78:0/3145989164 wait complete. 2024-09-16T00:11:37.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:37 smithi078 ceph-mon[31157]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-16T00:11:37.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:37 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3145989164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:11:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:37 smithi193 ceph-mon[35484]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-16T00:11:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:37 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3145989164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:11:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:37 smithi148 ceph-mon[35599]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-16T00:11:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:37 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3145989164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:11:38.297 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-16T00:11:38.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-16T00:11:38.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:38 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3145989164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:11:38.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:38 smithi078 ceph-mon[31157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:38.549 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:38.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:38 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3145989164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:11:38.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:38 smithi193 ceph-mon[35484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:38.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:38 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3145989164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:11:38.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:38 smithi148 ceph-mon[35599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:39.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.015+0000 7fea2d543640 1 -- 172.21.15.78:0/2940691108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 msgr2=0x7fea281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:39.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.015+0000 7fea2d543640 1 --2- 172.21.15.78:0/2940691108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 0x7fea281021c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea18009900 tx=0x7fea1802a410 comp rx=0 tx=0).stop 2024-09-16T00:11:39.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 -- 172.21.15.78:0/2940691108 shutdown_connections 2024-09-16T00:11:39.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 --2- 172.21.15.78:0/2940691108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea28106f10 0x7fea2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 --2- 172.21.15.78:0/2940691108 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea28102790 0x7fea281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 --2- 172.21.15.78:0/2940691108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 0x7fea281021c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 -- 172.21.15.78:0/2940691108 >> 172.21.15.78:0/2940691108 conn(0x7fea280fd8c0 msgr2=0x7fea280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 -- 172.21.15.78:0/2940691108 shutdown_connections 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 -- 172.21.15.78:0/2940691108 wait complete. 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 Processor -- start 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.016+0000 7fea2d543640 1 -- start start 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea2d543640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 0x7fea28198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea2d543640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea28102790 0x7fea28198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea2d543640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea28106f10 0x7fea2819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea2d543640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea28111fe0 con 0x7fea28101de0 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea2d543640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea28111e60 con 0x7fea28106f10 2024-09-16T00:11:39.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea2d543640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea28112160 con 0x7fea28102790 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 0x7fea28198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 0x7fea28198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56146/0 (socket says 172.21.15.78:56146) 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 -- 172.21.15.78:0/244787299 learned_addr learned my addr 172.21.15.78:0/244787299 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea277fe640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea28106f10 0x7fea2819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea267fc640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea28102790 0x7fea28198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 -- 172.21.15.78:0/244787299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea28102790 msgr2=0x7fea28198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea28102790 0x7fea28198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 -- 172.21.15.78:0/244787299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea28106f10 msgr2=0x7fea2819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea28106f10 0x7fea2819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 -- 172.21.15.78:0/244787299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2819d3d0 con 0x7fea28101de0 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea267fc640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea28102790 0x7fea28198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea26ffd640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 0x7fea28198420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea18006010 tx=0x7fea18003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:39.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea0ffff640 1 -- 172.21.15.78:0/244787299 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1803f070 con 0x7fea28101de0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2819d660 con 0x7fea28101de0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea0ffff640 1 -- 172.21.15.78:0/244787299 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea18031c90 con 0x7fea28101de0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea0ffff640 1 -- 172.21.15.78:0/244787299 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18035030 con 0x7fea28101de0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.017+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea281a4fa0 con 0x7fea28101de0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.018+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea281021c0 con 0x7fea28101de0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.019+0000 7fea0ffff640 1 -- 172.21.15.78:0/244787299 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fea18042070 con 0x7fea28101de0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.020+0000 7fea0ffff640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea100a3240 0x7fea100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.020+0000 7fea267fc640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea100a3240 0x7fea100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.020+0000 7fea0ffff640 1 -- 172.21.15.78:0/244787299 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fea180f0a40 con 0x7fea28101de0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.021+0000 7fea267fc640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea100a3240 0x7fea100a5700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea1c006fd0 tx=0x7fea1c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:39.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.023+0000 7fea0ffff640 1 -- 172.21.15.78:0/244787299 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea180bd4c0 con 0x7fea28101de0 2024-09-16T00:11:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.225+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fea2810a0f0 con 0x7fea28101de0 2024-09-16T00:11:39.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.492+0000 7fea0ffff640 1 -- 172.21.15.78:0/244787299 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fea180bcdc0 con 0x7fea28101de0 2024-09-16T00:11:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea100a3240 msgr2=0x7fea100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea100a3240 0x7fea100a5700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea1c006fd0 tx=0x7fea1c008040 comp rx=0 tx=0).stop 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 msgr2=0x7fea28198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 0x7fea28198420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea18006010 tx=0x7fea18003ba0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 shutdown_connections 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea100a3240 0x7fea100a5700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea28106f10 0x7fea2819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea28102790 0x7fea28198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 --2- 172.21.15.78:0/244787299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea28101de0 0x7fea28198420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.494+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 >> 172.21.15.78:0/244787299 conn(0x7fea280fd8c0 msgr2=0x7fea280fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.495+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 shutdown_connections 2024-09-16T00:11:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:39.495+0000 7fea2d543640 1 -- 172.21.15.78:0/244787299 wait complete. 2024-09-16T00:11:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:39 smithi078 ceph-mon[31157]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-16T00:11:39.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:39 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/244787299' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T00:11:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:39 smithi193 ceph-mon[35484]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-16T00:11:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:39 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/244787299' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T00:11:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:39 smithi148 ceph-mon[35599]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-16T00:11:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:39 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/244787299' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-16T00:11:40.183 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-16T00:11:40.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:40 smithi148 ceph-mon[35599]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:11:40.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:40 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/244787299' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T00:11:40.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:40 smithi148 ceph-mon[35599]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:11:40.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:40 smithi148 ceph-mon[35599]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:40.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:40 smithi148 ceph-mon[35599]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T00:11:40.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:40 smithi148 ceph-mon[35599]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T00:11:40.957 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:40 smithi193 ceph-mon[35484]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:11:40.957 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:40 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/244787299' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T00:11:40.957 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:40 smithi193 ceph-mon[35484]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:11:40.958 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:40 smithi193 ceph-mon[35484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:40.958 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:40 smithi193 ceph-mon[35484]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T00:11:40.958 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:40 smithi193 ceph-mon[35484]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T00:11:40.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:40 smithi078 ceph-mon[31157]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:11:40.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:40 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/244787299' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-16T00:11:40.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:40 smithi078 ceph-mon[31157]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:11:40.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:40 smithi078 ceph-mon[31157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:11:40.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:40 smithi078 ceph-mon[31157]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-16T00:11:40.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:40 smithi078 ceph-mon[31157]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-16T00:11:41.207 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:41.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:41 smithi078 ceph-mon[31157]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-09-16T00:11:41.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:41 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] up:active 2024-09-16T00:11:41.635 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:41 smithi078 ceph-mon[31157]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:11:41.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:41 smithi078 ceph-mon[31157]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T00:11:41.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:41 smithi078 ceph-mon[31157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T00:11:41.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:41 smithi078 ceph-mon[31157]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 -- 172.21.15.78:0/107464333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 msgr2=0x7f47f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 --2- 172.21.15.78:0/107464333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 0x7f47f4101200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009900 tx=0x7f47e402a480 comp rx=0 tx=0).stop 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 -- 172.21.15.78:0/107464333 shutdown_connections 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 --2- 172.21.15.78:0/107464333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f4102ef0 0x7f47f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 --2- 172.21.15.78:0/107464333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f4102570 0x7f47f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 --2- 172.21.15.78:0/107464333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 0x7f47f4101200 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 -- 172.21.15.78:0/107464333 >> 172.21.15.78:0/107464333 conn(0x7f47f40fc7d0 msgr2=0x7f47f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 -- 172.21.15.78:0/107464333 shutdown_connections 2024-09-16T00:11:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 -- 172.21.15.78:0/107464333 wait complete. 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.719+0000 7f47f8d31640 1 Processor -- start 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f8d31640 1 -- start start 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f8d31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 0x7f47f40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f8d31640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f4102570 0x7f47f40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f8d31640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f4102ef0 0x7f47f40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f8d31640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4110f50 con 0x7f47f4100e20 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f8d31640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47f4110dd0 con 0x7f47f4102ef0 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f8d31640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47f41110d0 con 0x7f47f4102570 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 0x7f47f40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 0x7f47f40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56164/0 (socket says 172.21.15.78:56164) 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f259c640 1 -- 172.21.15.78:0/3065119329 learned_addr learned my addr 172.21.15.78:0/3065119329 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47e9d9b640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f4102570 0x7f47f40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f2d9d640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f4102ef0 0x7f47f40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f259c640 1 -- 172.21.15.78:0/3065119329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f4102570 msgr2=0x7f47f40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f259c640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f4102570 0x7f47f40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f259c640 1 -- 172.21.15.78:0/3065119329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f4102ef0 msgr2=0x7f47f40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f259c640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f4102ef0 0x7f47f40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f259c640 1 -- 172.21.15.78:0/3065119329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f41a69a0 con 0x7f47f4100e20 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.720+0000 7f47f2d9d640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f4102ef0 0x7f47f40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.721+0000 7f47f259c640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 0x7f47f40fcc80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47e4006010 tx=0x7f47e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.721+0000 7f47ebfff640 1 -- 172.21.15.78:0/3065119329 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e403f070 con 0x7f47f4100e20 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.721+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f41a6c30 con 0x7f47f4100e20 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.721+0000 7f47ebfff640 1 -- 172.21.15.78:0/3065119329 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e4031c30 con 0x7f47f4100e20 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.721+0000 7f47ebfff640 1 -- 172.21.15.78:0/3065119329 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e4035030 con 0x7f47f4100e20 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.721+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f41a7110 con 0x7f47f4100e20 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.722+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f4101200 con 0x7f47f4100e20 2024-09-16T00:11:41.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.723+0000 7f47ebfff640 1 -- 172.21.15.78:0/3065119329 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f47e4042070 con 0x7f47f4100e20 2024-09-16T00:11:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.724+0000 7f47ebfff640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f47e00a3240 0x7f47e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.724+0000 7f47e9d9b640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f47e00a3240 0x7f47e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.724+0000 7f47ebfff640 1 -- 172.21.15.78:0/3065119329 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f47e40f0780 con 0x7f47f4100e20 2024-09-16T00:11:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.724+0000 7f47e9d9b640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f47e00a3240 0x7f47e00a5700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f47f40fe710 tx=0x7f47dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.727+0000 7f47ebfff640 1 -- 172.21.15.78:0/3065119329 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47e40bd200 con 0x7f47f4100e20 2024-09-16T00:11:41.831 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:41 smithi193 ceph-mon[35484]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-09-16T00:11:41.831 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:41 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] up:active 2024-09-16T00:11:41.831 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:41 smithi193 ceph-mon[35484]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:11:41.831 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:41 smithi193 ceph-mon[35484]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T00:11:41.832 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:41 smithi193 ceph-mon[35484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T00:11:41.832 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:41 smithi193 ceph-mon[35484]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T00:11:41.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:41 smithi148 ceph-mon[35599]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-09-16T00:11:41.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:41 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] up:active 2024-09-16T00:11:41.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:41 smithi148 ceph-mon[35599]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:11:41.856 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:41 smithi148 ceph-mon[35599]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-16T00:11:41.857 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:41 smithi148 ceph-mon[35599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-16T00:11:41.857 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:41 smithi148 ceph-mon[35599]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-16T00:11:41.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:41.928+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f47f410a0a0 con 0x7f47f4100e20 2024-09-16T00:11:42.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.528+0000 7f47ebfff640 1 -- 172.21.15.78:0/3065119329 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f47e4031de0 con 0x7f47f4100e20 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f47e00a3240 msgr2=0x7f47e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f47e00a3240 0x7f47e00a5700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f47f40fe710 tx=0x7f47dc00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 msgr2=0x7f47f40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 0x7f47f40fcc80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47e4006010 tx=0x7f47e4003c10 comp rx=0 tx=0).stop 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 shutdown_connections 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f47e00a3240 0x7f47e00a5700 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47f4102ef0 0x7f47f40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47f4102570 0x7f47f40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 --2- 172.21.15.78:0/3065119329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47f4100e20 0x7f47f40fcc80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 >> 172.21.15.78:0/3065119329 conn(0x7f47f40fc7d0 msgr2=0x7f47f406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:42.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 shutdown_connections 2024-09-16T00:11:42.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:42.531+0000 7f47f8d31640 1 -- 172.21.15.78:0/3065119329 wait complete. 2024-09-16T00:11:42.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:42 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] up:active 2024-09-16T00:11:42.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:42 smithi078 ceph-mon[31157]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T00:11:42.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:42 smithi078 ceph-mon[31157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T00:11:42.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:42 smithi078 ceph-mon[31157]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T00:11:42.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:42 smithi078 ceph-mon[31157]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T00:11:42.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:42 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3065119329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:11:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:42 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] up:active 2024-09-16T00:11:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:42 smithi193 ceph-mon[35484]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T00:11:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:42 smithi193 ceph-mon[35484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T00:11:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:42 smithi193 ceph-mon[35484]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T00:11:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:42 smithi193 ceph-mon[35484]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T00:11:42.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:42 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3065119329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:11:42.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:42 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] up:active 2024-09-16T00:11:42.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:42 smithi148 ceph-mon[35599]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-16T00:11:42.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:42 smithi148 ceph-mon[35599]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-16T00:11:42.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:42 smithi148 ceph-mon[35599]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-16T00:11:42.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:42 smithi148 ceph-mon[35599]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-16T00:11:42.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:42 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3065119329' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:11:43.312 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-16T00:11:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:43.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-16T00:11:43.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3065119329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:11:43.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] up:active 2024-09-16T00:11:43.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T00:11:43.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:11:43.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: Cluster is now healthy 2024-09-16T00:11:43.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T00:11:43.833 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T00:11:43.834 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:43 smithi078 ceph-mon[31157]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3065119329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] up:active 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: Cluster is now healthy 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T00:11:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:43 smithi193 ceph-mon[35484]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T00:11:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-16T00:11:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3065119329' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:11:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] up:active 2024-09-16T00:11:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-16T00:11:43.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:11:43.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: Cluster is now healthy 2024-09-16T00:11:43.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-16T00:11:43.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-16T00:11:43.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:43 smithi148 ceph-mon[35599]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-16T00:11:44.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 -- 172.21.15.78:0/2615726443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a4106bb0 msgr2=0x7f06a4106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:44.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 --2- 172.21.15.78:0/2615726443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a4106bb0 0x7f06a4106f90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009900 tx=0x7f06a002a410 comp rx=0 tx=0).stop 2024-09-16T00:11:44.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 -- 172.21.15.78:0/2615726443 shutdown_connections 2024-09-16T00:11:44.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 --2- 172.21.15.78:0/2615726443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4100970 0x7f06a410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 --2- 172.21.15.78:0/2615726443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06a40fffd0 0x7f06a4100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 --2- 172.21.15.78:0/2615726443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a4106bb0 0x7f06a4106f90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 -- 172.21.15.78:0/2615726443 >> 172.21.15.78:0/2615726443 conn(0x7f06a40fbee0 msgr2=0x7f06a40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:44.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 -- 172.21.15.78:0/2615726443 shutdown_connections 2024-09-16T00:11:44.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.054+0000 7f06ac223640 1 -- 172.21.15.78:0/2615726443 wait complete. 2024-09-16T00:11:44.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06ac223640 1 Processor -- start 2024-09-16T00:11:44.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06ac223640 1 -- start start 2024-09-16T00:11:44.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06ac223640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a40fffd0 0x7f06a4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:44.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06ac223640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06a4100970 0x7f06a4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:44.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06ac223640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4106bb0 0x7f06a419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:44.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06ac223640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41120c0 con 0x7f06a40fffd0 2024-09-16T00:11:44.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06ac223640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4111f40 con 0x7f06a4100970 2024-09-16T00:11:44.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06ac223640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4112240 con 0x7f06a4106bb0 2024-09-16T00:11:44.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06a9fbf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a40fffd0 0x7f06a4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:44.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06a9fbf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a40fffd0 0x7f06a4198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56182/0 (socket says 172.21.15.78:56182) 2024-09-16T00:11:44.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.055+0000 7f06a9fbf640 1 -- 172.21.15.78:0/1009891232 learned_addr learned my addr 172.21.15.78:0/1009891232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:44.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06a97be640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06a4100970 0x7f06a4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:44.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06aa7c0640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4106bb0 0x7f06a419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:44.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06a9fbf640 1 -- 172.21.15.78:0/1009891232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4106bb0 msgr2=0x7f06a419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:44.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06a9fbf640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4106bb0 0x7f06a419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06a9fbf640 1 -- 172.21.15.78:0/1009891232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06a4100970 msgr2=0x7f06a4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:44.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06a9fbf640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06a4100970 0x7f06a4198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06a9fbf640 1 -- 172.21.15.78:0/1009891232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a419d4d0 con 0x7f06a40fffd0 2024-09-16T00:11:44.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06aa7c0640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4106bb0 0x7f06a419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:44.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06a9fbf640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a40fffd0 0x7f06a4198520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06a0006010 tx=0x7f06a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:44.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f069affd640 1 -- 172.21.15.78:0/1009891232 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a003f030 con 0x7f06a40fffd0 2024-09-16T00:11:44.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f069affd640 1 -- 172.21.15.78:0/1009891232 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06a0003ce0 con 0x7f06a40fffd0 2024-09-16T00:11:44.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f069affd640 1 -- 172.21.15.78:0/1009891232 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0035070 con 0x7f06a40fffd0 2024-09-16T00:11:44.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a419d760 con 0x7f06a40fffd0 2024-09-16T00:11:44.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.056+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a41a50a0 con 0x7f06a40fffd0 2024-09-16T00:11:44.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.057+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a41032f0 con 0x7f06a40fffd0 2024-09-16T00:11:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.058+0000 7f069affd640 1 -- 172.21.15.78:0/1009891232 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f06a0042030 con 0x7f06a40fffd0 2024-09-16T00:11:44.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.059+0000 7f069affd640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06900a3240 0x7f06900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:44.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.059+0000 7f069affd640 1 -- 172.21.15.78:0/1009891232 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f06a00efd30 con 0x7f06a40fffd0 2024-09-16T00:11:44.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.059+0000 7f06a97be640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06900a3240 0x7f06900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:44.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.059+0000 7f06a97be640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06900a3240 0x7f06900a5700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f06a40fe410 tx=0x7f069400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:44.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.063+0000 7f069affd640 1 -- 172.21.15.78:0/1009891232 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f06a00bc8d0 con 0x7f06a40fffd0 2024-09-16T00:11:44.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.261+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f06a41a5270 con 0x7f06a40fffd0 2024-09-16T00:11:44.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.550+0000 7f069affd640 1 -- 172.21.15.78:0/1009891232 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f06a00bc1d0 con 0x7f06a40fffd0 2024-09-16T00:11:44.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06900a3240 msgr2=0x7f06900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:44.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06900a3240 0x7f06900a5700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f06a40fe410 tx=0x7f069400b390 comp rx=0 tx=0).stop 2024-09-16T00:11:44.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a40fffd0 msgr2=0x7f06a4198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:44.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a40fffd0 0x7f06a4198520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06a0006010 tx=0x7f06a00042f0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 shutdown_connections 2024-09-16T00:11:44.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06900a3240 0x7f06900a5700 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06a4106bb0 0x7f06a419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f06a4100970 0x7f06a4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 --2- 172.21.15.78:0/1009891232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06a40fffd0 0x7f06a4198520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 >> 172.21.15.78:0/1009891232 conn(0x7f06a40fbee0 msgr2=0x7f06a40fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 shutdown_connections 2024-09-16T00:11:44.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:44.553+0000 7f06ac223640 1 -- 172.21.15.78:0/1009891232 wait complete. 2024-09-16T00:11:44.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:44 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] up:active 2024-09-16T00:11:44.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:44 smithi078 ceph-mon[31157]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T00:11:44.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:44 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:11:44.901 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:44 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1009891232' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:11:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:44 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] up:active 2024-09-16T00:11:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:44 smithi193 ceph-mon[35484]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T00:11:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:44 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:11:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:44 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1009891232' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:11:44.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:44 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] up:active 2024-09-16T00:11:44.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:44 smithi148 ceph-mon[35599]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-16T00:11:44.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:44 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:11:44.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:44 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1009891232' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:11:44.956 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:11:44 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: 2024-09-16T00:11:44.630+0000 7f91dde52640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-16T00:11:44.956 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:11:44 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: 2024-09-16T00:11:44.630+0000 7f91dde52640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-16T00:11:44.957 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:11:44 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: 2024-09-16T00:11:44.630+0000 7f91dde52640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-16T00:11:45.331 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-16T00:11:45.331 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-16T00:11:45.578 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:45.699 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:45 smithi078 ceph-mon[31157]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-16T00:11:45.699 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:45 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1009891232' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:11:45.699 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:45 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:11:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:45 smithi193 ceph-mon[35484]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-16T00:11:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:45 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1009891232' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:11:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:45 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:11:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:45 smithi148 ceph-mon[35599]: pgmap v307: 129 pgs: 129 active+clean; 584 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-16T00:11:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:45 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1009891232' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:11:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:45 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 -- 172.21.15.78:0/2662821666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8100e20 msgr2=0x7f12c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/2662821666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8100e20 0x7f12c8101200 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009900 tx=0x7f12b802a480 comp rx=0 tx=0).stop 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 -- 172.21.15.78:0/2662821666 shutdown_connections 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/2662821666 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c8102ef0 0x7f12c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/2662821666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c8102570 0x7f12c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/2662821666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8100e20 0x7f12c8101200 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 -- 172.21.15.78:0/2662821666 >> 172.21.15.78:0/2662821666 conn(0x7f12c80fc7d0 msgr2=0x7f12c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 -- 172.21.15.78:0/2662821666 shutdown_connections 2024-09-16T00:11:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.089+0000 7f12ce1b5640 1 -- 172.21.15.78:0/2662821666 wait complete. 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12ce1b5640 1 Processor -- start 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12ce1b5640 1 -- start start 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12ce1b5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c8100e20 0x7f12c81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12ce1b5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c8102570 0x7f12c81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12ce1b5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8102ef0 0x7f12c819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12ce1b5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8110f80 con 0x7f12c8102ef0 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12ce1b5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8110e00 con 0x7f12c8100e20 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12ce1b5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12c8111100 con 0x7f12c8102570 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8102ef0 0x7f12c819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8102ef0 0x7f12c819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56198/0 (socket says 172.21.15.78:56198) 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c7fff640 1 -- 172.21.15.78:0/3380540016 learned_addr learned my addr 172.21.15.78:0/3380540016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c7fff640 1 -- 172.21.15.78:0/3380540016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c8102570 msgr2=0x7f12c81989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12bffff640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c8102570 0x7f12c81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c77fe640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c8100e20 0x7f12c81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c7fff640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c8102570 0x7f12c81989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c7fff640 1 -- 172.21.15.78:0/3380540016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c8100e20 msgr2=0x7f12c81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c7fff640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c8100e20 0x7f12c81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c7fff640 1 -- 172.21.15.78:0/3380540016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c819d460 con 0x7f12c8102ef0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12bffff640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c8102570 0x7f12c81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.090+0000 7f12c77fe640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c8100e20 0x7f12c81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.091+0000 7f12c7fff640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8102ef0 0x7f12c819cd80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12c000cce0 tx=0x7f12c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.091+0000 7f12c57fa640 1 -- 172.21.15.78:0/3380540016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c000f450 con 0x7f12c8102ef0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.091+0000 7f12c57fa640 1 -- 172.21.15.78:0/3380540016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12c000fad0 con 0x7f12c8102ef0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.091+0000 7f12c57fa640 1 -- 172.21.15.78:0/3380540016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c0018c10 con 0x7f12c8102ef0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.091+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c819d750 con 0x7f12c8102ef0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.091+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c81a5030 con 0x7f12c8102ef0 2024-09-16T00:11:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.092+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c8101200 con 0x7f12c8102ef0 2024-09-16T00:11:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.093+0000 7f12c57fa640 1 -- 172.21.15.78:0/3380540016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f12c000f5f0 con 0x7f12c8102ef0 2024-09-16T00:11:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.093+0000 7f12c57fa640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12b40a32c0 0x7f12b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.094+0000 7f12c77fe640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12b40a32c0 0x7f12b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.094+0000 7f12c57fa640 1 -- 172.21.15.78:0/3380540016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f12c00d02e0 con 0x7f12c8102ef0 2024-09-16T00:11:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.094+0000 7f12c77fe640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12b40a32c0 0x7f12b40a5780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12b8000c00 tx=0x7f12b80041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.097+0000 7f12c57fa640 1 -- 172.21.15.78:0/3380540016 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12c0014030 con 0x7f12c8102ef0 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.286+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f12c81068a0 con 0x7f12b40a32c0 2024-09-16T00:11:46.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.314+0000 7f12c57fa640 1 -- 172.21.15.78:0/3380540016 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12c81068a0 con 0x7f12b40a32c0 2024-09-16T00:11:46.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12b40a32c0 msgr2=0x7f12b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12b40a32c0 0x7f12b40a5780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12b8000c00 tx=0x7f12b80041e0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8102ef0 msgr2=0x7f12c819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8102ef0 0x7f12c819cd80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12c000cce0 tx=0x7f12c000a620 comp rx=0 tx=0).stop 2024-09-16T00:11:46.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 shutdown_connections 2024-09-16T00:11:46.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12b40a32c0 0x7f12b40a5780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c8102ef0 0x7f12c819cd80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c8102570 0x7f12c81989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 --2- 172.21.15.78:0/3380540016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12c8100e20 0x7f12c81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 >> 172.21.15.78:0/3380540016 conn(0x7f12c80fc7d0 msgr2=0x7f12c806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:46.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 shutdown_connections 2024-09-16T00:11:46.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:46.316+0000 7f12ce1b5640 1 -- 172.21.15.78:0/3380540016 wait complete. 2024-09-16T00:11:46.629 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:11:46 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:11:46.291+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:11:46.630 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:11:46 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:11:46.291+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:11:46.630 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:11:46 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:11:46.291+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:11:46.630 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:11:46 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:11:46.291+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:11:46.630 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:11:46 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:11:46.291+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:11:46.974 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-16T00:11:47.221 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:47.618 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:47 smithi078 ceph-mon[31157]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:47.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:47 smithi078 ceph-mon[31157]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.4 KiB/s wr, 15 op/s 2024-09-16T00:11:47.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:47 smithi078 ceph-mon[31157]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:11:47.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 -- 172.21.15.78:0/4153216653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 msgr2=0x7f409c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 --2- 172.21.15.78:0/4153216653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 0x7f409c1067e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f408c009900 tx=0x7f408c02a410 comp rx=0 tx=0).stop 2024-09-16T00:11:47.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 -- 172.21.15.78:0/4153216653 shutdown_connections 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 --2- 172.21.15.78:0/4153216653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c106f10 0x7f409c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 --2- 172.21.15.78:0/4153216653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 0x7f409c1067e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 --2- 172.21.15.78:0/4153216653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f409c101de0 0x7f409c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 -- 172.21.15.78:0/4153216653 >> 172.21.15.78:0/4153216653 conn(0x7f409c0fd8c0 msgr2=0x7f409c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 -- 172.21.15.78:0/4153216653 shutdown_connections 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.704+0000 7f40a3656640 1 -- 172.21.15.78:0/4153216653 wait complete. 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a3656640 1 Processor -- start 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a3656640 1 -- start start 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a3656640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c101de0 0x7f409c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a3656640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 0x7f409c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a3656640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f409c106f10 0x7f409c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a3656640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f409c111f70 con 0x7f409c102790 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a3656640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f409c111df0 con 0x7f409c101de0 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a3656640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1120f0 con 0x7f409c106f10 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a0bf1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 0x7f409c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a0bf1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 0x7f409c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52276/0 (socket says 172.21.15.78:52276) 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a0bf1640 1 -- 172.21.15.78:0/147639574 learned_addr learned my addr 172.21.15.78:0/147639574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a1bf3640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f409c106f10 0x7f409c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a13f2640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c101de0 0x7f409c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a0bf1640 1 -- 172.21.15.78:0/147639574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f409c106f10 msgr2=0x7f409c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a0bf1640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f409c106f10 0x7f409c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a0bf1640 1 -- 172.21.15.78:0/147639574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c101de0 msgr2=0x7f409c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a0bf1640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c101de0 0x7f409c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a0bf1640 1 -- 172.21.15.78:0/147639574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c19d460 con 0x7f409c102790 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.705+0000 7f40a13f2640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c101de0 0x7f409c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.706+0000 7f40a0bf1640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 0x7f409c1989f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f408c006010 tx=0x7f408c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.706+0000 7f40927fc640 1 -- 172.21.15.78:0/147639574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c03f030 con 0x7f409c102790 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.706+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c19d6f0 con 0x7f409c102790 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.706+0000 7f40927fc640 1 -- 172.21.15.78:0/147639574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f408c003ce0 con 0x7f409c102790 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.706+0000 7f40927fc640 1 -- 172.21.15.78:0/147639574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c035070 con 0x7f409c102790 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.706+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a5030 con 0x7f409c102790 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.707+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c1021c0 con 0x7f409c102790 2024-09-16T00:11:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.708+0000 7f40927fc640 1 -- 172.21.15.78:0/147639574 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f408c042030 con 0x7f409c102790 2024-09-16T00:11:47.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.709+0000 7f40927fc640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40880a32d0 0x7f40880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.709+0000 7f40a13f2640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40880a32d0 0x7f40880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.709+0000 7f40927fc640 1 -- 172.21.15.78:0/147639574 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f408c0f06b0 con 0x7f409c102790 2024-09-16T00:11:47.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.709+0000 7f40a13f2640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40880a32d0 0x7f40880a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f40980058c0 tx=0x7f4098005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:47.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.712+0000 7f40927fc640 1 -- 172.21.15.78:0/147639574 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f408c0bd0f0 con 0x7f409c102790 2024-09-16T00:11:47.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.902+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f409c062630 con 0x7f40880a32d0 2024-09-16T00:11:47.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.927+0000 7f40927fc640 1 -- 172.21.15.78:0/147639574 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f409c062630 con 0x7f40880a32d0 2024-09-16T00:11:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:47 smithi193 ceph-mon[35484]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:47 smithi193 ceph-mon[35484]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.4 KiB/s wr, 15 op/s 2024-09-16T00:11:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:47 smithi193 ceph-mon[35484]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:11:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40880a32d0 msgr2=0x7f40880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40880a32d0 0x7f40880a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f40980058c0 tx=0x7f4098005d30 comp rx=0 tx=0).stop 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 msgr2=0x7f409c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 0x7f409c1989f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f408c006010 tx=0x7f408c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 shutdown_connections 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40880a32d0 0x7f40880a5790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f409c106f10 0x7f409c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f409c102790 0x7f409c1989f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 --2- 172.21.15.78:0/147639574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f409c101de0 0x7f409c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 >> 172.21.15.78:0/147639574 conn(0x7f409c0fd8c0 msgr2=0x7f409c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 shutdown_connections 2024-09-16T00:11:47.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:47.929+0000 7f40a3656640 1 -- 172.21.15.78:0/147639574 wait complete. 2024-09-16T00:11:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:47 smithi148 ceph-mon[35599]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:47 smithi148 ceph-mon[35599]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.4 KiB/s wr, 15 op/s 2024-09-16T00:11:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:47 smithi148 ceph-mon[35599]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:11:48.653 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-16T00:11:48.900 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/3231713431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 msgr2=0x7fa1b410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/3231713431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 0x7fa1b410d130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009ca0 tx=0x7fa1ac02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/3231713431 shutdown_connections 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/3231713431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 0x7fa1b410d130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/3231713431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b4100e30 0x7fa1b4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/3231713431 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1b4100480 0x7fa1b4100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/3231713431 >> 172.21.15.78:0/3231713431 conn(0x7fa1b4077210 msgr2=0x7fa1b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/3231713431 shutdown_connections 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.387+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/3231713431 wait complete. 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1bb0a4640 1 Processor -- start 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1bb0a4640 1 -- start start 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1bb0a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1b4100480 0x7fa1b41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1bb0a4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b4100e30 0x7fa1b4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1bb0a4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 0x7fa1b41a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1bb0a4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4110630 con 0x7fa1b41055b0 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1bb0a4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41104b0 con 0x7fa1b4100480 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1bb0a4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41107b0 con 0x7fa1b4100e30 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1b9641640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 0x7fa1b41a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1b9641640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 0x7fa1b41a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52286/0 (socket says 172.21.15.78:52286) 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1b9641640 1 -- 172.21.15.78:0/489473180 learned_addr learned my addr 172.21.15.78:0/489473180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1b8e40640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1b4100480 0x7fa1b41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1b3fff640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b4100e30 0x7fa1b4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1b9641640 1 -- 172.21.15.78:0/489473180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b4100e30 msgr2=0x7fa1b4199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.388+0000 7fa1b9641640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b4100e30 0x7fa1b4199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1b9641640 1 -- 172.21.15.78:0/489473180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1b4100480 msgr2=0x7fa1b41992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1b9641640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1b4100480 0x7fa1b41992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1b9641640 1 -- 172.21.15.78:0/489473180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b41a4890 con 0x7fa1b41055b0 2024-09-16T00:11:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1b9641640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 0x7fa1b41a4350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac031a20 tx=0x7fa1ac031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1b1ffb640 1 -- 172.21.15.78:0/489473180 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac003c30 con 0x7fa1b41055b0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b41a4a60 con 0x7fa1b41055b0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1b1ffb640 1 -- 172.21.15.78:0/489473180 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac003dd0 con 0x7fa1b41055b0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1b1ffb640 1 -- 172.21.15.78:0/489473180 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac02fcf0 con 0x7fa1b41055b0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.389+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b41a4ef0 con 0x7fa1b41055b0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.390+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b4101960 con 0x7fa1b41055b0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.391+0000 7fa1b1ffb640 1 -- 172.21.15.78:0/489473180 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa1ac004050 con 0x7fa1b41055b0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.392+0000 7fa1b1ffb640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa1a40a32d0 0x7fa1a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.392+0000 7fa1b1ffb640 1 -- 172.21.15.78:0/489473180 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fa1ac0f0f20 con 0x7fa1b41055b0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.392+0000 7fa1b8e40640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa1a40a32d0 0x7fa1a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.392+0000 7fa1b8e40640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa1a40a32d0 0x7fa1a40a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800bbe0 tx=0x7fa1a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.396+0000 7fa1b1ffb640 1 -- 172.21.15.78:0/489473180 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1ac0bdb60 con 0x7fa1b41055b0 2024-09-16T00:11:49.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.589+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fa1b40fece0 con 0x7fa1a40a32d0 2024-09-16T00:11:49.594 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_0/611ba463-a319-4e78-9cac-7545b9ff14d6 2024-09-16T00:11:49.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.594+0000 7fa1b1ffb640 1 -- 172.21.15.78:0/489473180 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa1b40fece0 con 0x7fa1a40a32d0 2024-09-16T00:11:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa1a40a32d0 msgr2=0x7fa1a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa1a40a32d0 0x7fa1a40a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800bbe0 tx=0x7fa1a800b390 comp rx=0 tx=0).stop 2024-09-16T00:11:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 msgr2=0x7fa1b41a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 0x7fa1b41a4350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac031a20 tx=0x7fa1ac031a50 comp rx=0 tx=0).stop 2024-09-16T00:11:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 shutdown_connections 2024-09-16T00:11:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa1a40a32d0 0x7fa1a40a5790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b41055b0 0x7fa1b41a4350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1b4100e30 0x7fa1b4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 --2- 172.21.15.78:0/489473180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1b4100480 0x7fa1b41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 >> 172.21.15.78:0/489473180 conn(0x7fa1b4077210 msgr2=0x7fa1b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:49.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 shutdown_connections 2024-09-16T00:11:49.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:49.596+0000 7fa1bb0a4640 1 -- 172.21.15.78:0/489473180 wait complete. 2024-09-16T00:11:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:49 smithi193 ceph-mon[35484]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:49 smithi193 ceph-mon[35484]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-16T00:11:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:49 smithi148 ceph-mon[35599]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:49 smithi148 ceph-mon[35599]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-16T00:11:49.967 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:49 smithi078 ceph-mon[31157]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:49.967 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:49 smithi078 ceph-mon[31157]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-16T00:11:50.355 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-16T00:11:50.601 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:50.723 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:50 smithi078 ceph-mon[31157]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:50.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:50 smithi193 ceph-mon[35484]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:50.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:50 smithi148 ceph-mon[35599]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:51.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 -- 172.21.15.78:0/1860811357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 msgr2=0x7f94b4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:51.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 --2- 172.21.15.78:0/1860811357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 0x7f94b4100ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f949c009930 tx=0x7f949c02a420 comp rx=0 tx=0).stop 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 -- 172.21.15.78:0/1860811357 shutdown_connections 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 --2- 172.21.15.78:0/1860811357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b4101420 0x7f94b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 --2- 172.21.15.78:0/1860811357 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 0x7f94b4100ee0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 --2- 172.21.15.78:0/1860811357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94b4106a80 0x7f94b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 -- 172.21.15.78:0/1860811357 >> 172.21.15.78:0/1860811357 conn(0x7f94b40fc7a0 msgr2=0x7f94b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 -- 172.21.15.78:0/1860811357 shutdown_connections 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 -- 172.21.15.78:0/1860811357 wait complete. 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 Processor -- start 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.098+0000 7f94ba60a640 1 -- start start 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94ba60a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 0x7f94b4197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94ba60a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b4101420 0x7f94b41977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:51.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94ba60a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94b4106a80 0x7f94b419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:51.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94ba60a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4110e80 con 0x7f94b4100a80 2024-09-16T00:11:51.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94ba60a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4110d00 con 0x7f94b4106a80 2024-09-16T00:11:51.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94ba60a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4111000 con 0x7f94b4101420 2024-09-16T00:11:51.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 0x7f94b4197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:51.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 0x7f94b4197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52298/0 (socket says 172.21.15.78:52298) 2024-09-16T00:11:51.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b3fff640 1 -- 172.21.15.78:0/894491505 learned_addr learned my addr 172.21.15.78:0/894491505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:51.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b8ba7640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94b4106a80 0x7f94b419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b37fe640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b4101420 0x7f94b41977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b3fff640 1 -- 172.21.15.78:0/894491505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b4101420 msgr2=0x7f94b41977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b3fff640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b4101420 0x7f94b41977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b3fff640 1 -- 172.21.15.78:0/894491505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94b4106a80 msgr2=0x7f94b419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b3fff640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94b4106a80 0x7f94b419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b3fff640 1 -- 172.21.15.78:0/894491505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b419c220 con 0x7f94b4100a80 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b8ba7640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94b4106a80 0x7f94b419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.099+0000 7f94b37fe640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b4101420 0x7f94b41977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.100+0000 7f94b3fff640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 0x7f94b4197270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f94a400c920 tx=0x7f94a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.100+0000 7f94b17fa640 1 -- 172.21.15.78:0/894491505 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4007c70 con 0x7f94b4100a80 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.100+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b419c510 con 0x7f94b4100a80 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.100+0000 7f94b17fa640 1 -- 172.21.15.78:0/894491505 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a4007e10 con 0x7f94b4100a80 2024-09-16T00:11:51.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.100+0000 7f94b17fa640 1 -- 172.21.15.78:0/894491505 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4018890 con 0x7f94b4100a80 2024-09-16T00:11:51.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.100+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b41a3df0 con 0x7f94b4100a80 2024-09-16T00:11:51.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.101+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b4100ee0 con 0x7f94b4100a80 2024-09-16T00:11:51.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.102+0000 7f94b17fa640 1 -- 172.21.15.78:0/894491505 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f94a4018a70 con 0x7f94b4100a80 2024-09-16T00:11:51.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.103+0000 7f94b17fa640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94a00a32d0 0x7f94a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:51.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.103+0000 7f94b37fe640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94a00a32d0 0x7f94a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:51.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.103+0000 7f94b17fa640 1 -- 172.21.15.78:0/894491505 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f94a40cfe60 con 0x7f94b4100a80 2024-09-16T00:11:51.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.103+0000 7f94b37fe640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94a00a32d0 0x7f94a00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f949c000c00 tx=0x7f949c032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:51.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.107+0000 7f94b17fa640 1 -- 172.21.15.78:0/894491505 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94a4014030 con 0x7f94b4100a80 2024-09-16T00:11:51.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.298+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f94b40fd580 con 0x7f94a00a32d0 2024-09-16T00:11:51.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.323+0000 7f94b17fa640 1 -- 172.21.15.78:0/894491505 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f94b40fd580 con 0x7f94a00a32d0 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94a00a32d0 msgr2=0x7f94a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94a00a32d0 0x7f94a00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f949c000c00 tx=0x7f949c032d50 comp rx=0 tx=0).stop 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 msgr2=0x7f94b4197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 0x7f94b4197270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f94a400c920 tx=0x7f94a400cde0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 shutdown_connections 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94a00a32d0 0x7f94a00a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94b4106a80 0x7f94b419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94b4101420 0x7f94b41977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 --2- 172.21.15.78:0/894491505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94b4100a80 0x7f94b4197270 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 >> 172.21.15.78:0/894491505 conn(0x7f94b40fc7a0 msgr2=0x7f94b40fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 shutdown_connections 2024-09-16T00:11:51.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:51.325+0000 7f94ba60a640 1 -- 172.21.15.78:0/894491505 wait complete. 2024-09-16T00:11:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:51 smithi193 ceph-mon[35484]: pgmap v310: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 19 op/s 2024-09-16T00:11:51.937 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:51 smithi078 ceph-mon[31157]: pgmap v310: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 19 op/s 2024-09-16T00:11:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:51 smithi148 ceph-mon[35599]: pgmap v310: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.2 KiB/s wr, 19 op/s 2024-09-16T00:11:51.965 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-16T00:11:52.209 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:52.617 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:52 smithi078 ceph-mon[31157]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:52.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 -- 172.21.15.78:0/373921971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4101de0 msgr2=0x7f34f41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:52.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 --2- 172.21.15.78:0/373921971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4101de0 0x7f34f41021c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009ca0 tx=0x7f34e402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 -- 172.21.15.78:0/373921971 shutdown_connections 2024-09-16T00:11:52.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 --2- 172.21.15.78:0/373921971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f4106f10 0x7f34f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 --2- 172.21.15.78:0/373921971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34f4102790 0x7f34f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 --2- 172.21.15.78:0/373921971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4101de0 0x7f34f41021c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 -- 172.21.15.78:0/373921971 >> 172.21.15.78:0/373921971 conn(0x7f34f40fd8c0 msgr2=0x7f34f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 -- 172.21.15.78:0/373921971 shutdown_connections 2024-09-16T00:11:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 -- 172.21.15.78:0/373921971 wait complete. 2024-09-16T00:11:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.703+0000 7f34f9bad640 1 Processor -- start 2024-09-16T00:11:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f9bad640 1 -- start start 2024-09-16T00:11:52.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f9bad640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34f4101de0 0x7f34f41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f9bad640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f4102790 0x7f34f41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f9bad640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4106f10 0x7f34f419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f9bad640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34f4111fc0 con 0x7f34f4106f10 2024-09-16T00:11:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f9bad640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34f4111e40 con 0x7f34f4102790 2024-09-16T00:11:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f9bad640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f34f4112140 con 0x7f34f4101de0 2024-09-16T00:11:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4106f10 0x7f34f419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4106f10 0x7f34f419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52328/0 (socket says 172.21.15.78:52328) 2024-09-16T00:11:52.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f3fff640 1 -- 172.21.15.78:0/3656043920 learned_addr learned my addr 172.21.15.78:0/3656043920 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f2ffd640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f4102790 0x7f34f41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f3fff640 1 -- 172.21.15.78:0/3656043920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34f4101de0 msgr2=0x7f34f41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f3fff640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34f4101de0 0x7f34f41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f3fff640 1 -- 172.21.15.78:0/3656043920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f4102790 msgr2=0x7f34f41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f3fff640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f4102790 0x7f34f41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f3fff640 1 -- 172.21.15.78:0/3656043920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f419d450 con 0x7f34f4106f10 2024-09-16T00:11:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.704+0000 7f34f2ffd640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f4102790 0x7f34f41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:52.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.705+0000 7f34f3fff640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4106f10 0x7f34f419cd70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00d750 tx=0x7f34ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.705+0000 7f34f0ff9640 1 -- 172.21.15.78:0/3656043920 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec017070 con 0x7f34f4106f10 2024-09-16T00:11:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.705+0000 7f34f0ff9640 1 -- 172.21.15.78:0/3656043920 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ec005500 con 0x7f34f4106f10 2024-09-16T00:11:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.705+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f419d740 con 0x7f34f4106f10 2024-09-16T00:11:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.705+0000 7f34f0ff9640 1 -- 172.21.15.78:0/3656043920 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec004d90 con 0x7f34f4106f10 2024-09-16T00:11:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.705+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f41a5020 con 0x7f34f4106f10 2024-09-16T00:11:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.706+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f41021c0 con 0x7f34f4106f10 2024-09-16T00:11:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.707+0000 7f34f0ff9640 1 -- 172.21.15.78:0/3656043920 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f34ec004f30 con 0x7f34f4106f10 2024-09-16T00:11:52.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.707+0000 7f34f0ff9640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34dc0a32d0 0x7f34dc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:52.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.708+0000 7f34f37fe640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34dc0a32d0 0x7f34dc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:52.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.708+0000 7f34f0ff9640 1 -- 172.21.15.78:0/3656043920 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f34ec0cf760 con 0x7f34f4106f10 2024-09-16T00:11:52.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.708+0000 7f34f37fe640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34dc0a32d0 0x7f34dc0a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34e4006010 tx=0x7f34e4005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:52.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.712+0000 7f34f0ff9640 1 -- 172.21.15.78:0/3656043920 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34ec09c080 con 0x7f34f4106f10 2024-09-16T00:11:52.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.904+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f34f410a160 con 0x7f34dc0a32d0 2024-09-16T00:11:52.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.908+0000 7f34f0ff9640 1 -- 172.21.15.78:0/3656043920 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f34e4036050 con 0x7f34dc0a32d0 2024-09-16T00:11:52.909 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/qa/sv_1/21dc2d71-c2a3-4518-91bc-47ab979ecd32 2024-09-16T00:11:52.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34dc0a32d0 msgr2=0x7f34dc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:52.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34dc0a32d0 0x7f34dc0a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34e4006010 tx=0x7f34e4005f80 comp rx=0 tx=0).stop 2024-09-16T00:11:52.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4106f10 msgr2=0x7f34f419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:52.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4106f10 0x7f34f419cd70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00d750 tx=0x7f34ec00dc10 comp rx=0 tx=0).stop 2024-09-16T00:11:52.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 shutdown_connections 2024-09-16T00:11:52.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34dc0a32d0 0x7f34dc0a5790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34f4106f10 0x7f34f419cd70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34f4102790 0x7f34f41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 --2- 172.21.15.78:0/3656043920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34f4101de0 0x7f34f41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:52.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 >> 172.21.15.78:0/3656043920 conn(0x7f34f40fd8c0 msgr2=0x7f34f40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:52.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 shutdown_connections 2024-09-16T00:11:52.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:52.910+0000 7f34f9bad640 1 -- 172.21.15.78:0/3656043920 wait complete. 2024-09-16T00:11:52.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:52 smithi193 ceph-mon[35484]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:52.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:52 smithi148 ceph-mon[35599]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:53.593 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:11:53.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:53 smithi078 ceph-mon[31157]: pgmap v311: 129 pgs: 129 active+clean; 612 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-09-16T00:11:53.838 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:53 smithi193 ceph-mon[35484]: pgmap v311: 129 pgs: 129 active+clean; 612 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-09-16T00:11:53.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:53 smithi148 ceph-mon[35599]: pgmap v311: 129 pgs: 129 active+clean; 612 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-09-16T00:11:54.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1184080057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 msgr2=0x7f659c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:54.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1184080057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 0x7f659c106e60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f658c009930 tx=0x7f658c02a420 comp rx=0 tx=0).stop 2024-09-16T00:11:54.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1184080057 shutdown_connections 2024-09-16T00:11:54.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1184080057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f659c101420 0x7f659c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1184080057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659c100a80 0x7f659c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1184080057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 0x7f659c106e60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1184080057 >> 172.21.15.78:0/1184080057 conn(0x7f659c0fc7a0 msgr2=0x7f659c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:54.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1184080057 shutdown_connections 2024-09-16T00:11:54.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.421+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1184080057 wait complete. 2024-09-16T00:11:54.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f65a0b8f640 1 Processor -- start 2024-09-16T00:11:54.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f65a0b8f640 1 -- start start 2024-09-16T00:11:54.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f65a0b8f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659c100a80 0x7f659c0fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:54.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f65a0b8f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f659c101420 0x7f659c0fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:54.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f65a0b8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 0x7f659c0fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:54.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f65a0b8f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f659c110f20 con 0x7f659c106a80 2024-09-16T00:11:54.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f65a0b8f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f659c110da0 con 0x7f659c100a80 2024-09-16T00:11:54.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f65a0b8f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1110a0 con 0x7f659c101420 2024-09-16T00:11:54.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.422+0000 7f659ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 0x7f659c0fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:54.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 0x7f659c0fd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52344/0 (socket says 172.21.15.78:52344) 2024-09-16T00:11:54.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659ad9d640 1 -- 172.21.15.78:0/1470942662 learned_addr learned my addr 172.21.15.78:0/1470942662 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:54.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f6593fff640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f659c101420 0x7f659c0fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659a59c640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659c100a80 0x7f659c0fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659ad9d640 1 -- 172.21.15.78:0/1470942662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f659c101420 msgr2=0x7f659c0fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659ad9d640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f659c101420 0x7f659c0fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659ad9d640 1 -- 172.21.15.78:0/1470942662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659c100a80 msgr2=0x7f659c0fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:54.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659ad9d640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659c100a80 0x7f659c0fce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659ad9d640 1 -- 172.21.15.78:0/1470942662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c1a6ad0 con 0x7f659c106a80 2024-09-16T00:11:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f659ad9d640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 0x7f659c0fd8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f659400b970 tx=0x7f659400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f65937fe640 1 -- 172.21.15.78:0/1470942662 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594019070 con 0x7f659c106a80 2024-09-16T00:11:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c1a6dc0 con 0x7f659c106a80 2024-09-16T00:11:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f65937fe640 1 -- 172.21.15.78:0/1470942662 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659400cb40 con 0x7f659c106a80 2024-09-16T00:11:54.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f65937fe640 1 -- 172.21.15.78:0/1470942662 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594007610 con 0x7f659c106a80 2024-09-16T00:11:54.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.423+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1a7320 con 0x7f659c106a80 2024-09-16T00:11:54.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.424+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c100f60 con 0x7f659c106a80 2024-09-16T00:11:54.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.425+0000 7f65937fe640 1 -- 172.21.15.78:0/1470942662 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f659401c070 con 0x7f659c106a80 2024-09-16T00:11:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.426+0000 7f65937fe640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65880a32d0 0x7f65880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.426+0000 7f659a59c640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65880a32d0 0x7f65880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.426+0000 7f65937fe640 1 -- 172.21.15.78:0/1470942662 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f65940cf1f0 con 0x7f659c106a80 2024-09-16T00:11:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.426+0000 7f659a59c640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65880a32d0 0x7f65880a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f658c0096c0 tx=0x7f658c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.430+0000 7f65937fe640 1 -- 172.21.15.78:0/1470942662 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f659409bc80 con 0x7f659c106a80 2024-09-16T00:11:54.619 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:54 smithi078 ceph-mon[31157]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.640+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f659c10a000 con 0x7f659c106a80 2024-09-16T00:11:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.640+0000 7f65937fe640 1 -- 172.21.15.78:0/1470942662 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11986 (secure 0 0 0) 0x7f659409b580 con 0x7f659c106a80 2024-09-16T00:11:54.642 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:54.642 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-16T00:11:44:544077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:11:44.262714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:11:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-16T00:11:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65880a32d0 msgr2=0x7f65880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65880a32d0 0x7f65880a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f658c0096c0 tx=0x7f658c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 msgr2=0x7f659c0fd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 0x7f659c0fd8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f659400b970 tx=0x7f659400be30 comp rx=0 tx=0).stop 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 shutdown_connections 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65880a32d0 0x7f65880a5790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f659c106a80 0x7f659c0fd8d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f659c101420 0x7f659c0fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 --2- 172.21.15.78:0/1470942662 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f659c100a80 0x7f659c0fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 >> 172.21.15.78:0/1470942662 conn(0x7f659c0fc7a0 msgr2=0x7f659c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 shutdown_connections 2024-09-16T00:11:54.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:54.643+0000 7f65a0b8f640 1 -- 172.21.15.78:0/1470942662 wait complete. 2024-09-16T00:11:54.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:54 smithi193 ceph-mon[35484]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:54.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:54 smithi148 ceph-mon[35599]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:11:55.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:11:55.569 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:55.839 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:55 smithi078 ceph-mon[31157]: pgmap v312: 129 pgs: 129 active+clean; 612 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-16T00:11:55.839 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:55 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1470942662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:55 smithi193 ceph-mon[35484]: pgmap v312: 129 pgs: 129 active+clean; 612 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-16T00:11:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:55 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1470942662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:55 smithi148 ceph-mon[35599]: pgmap v312: 129 pgs: 129 active+clean; 612 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-16T00:11:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:55 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1470942662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:11:56.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.071+0000 7fd285532640 1 -- 172.21.15.78:0/126560073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2801036e0 msgr2=0x7fd280107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.071+0000 7fd285532640 1 --2- 172.21.15.78:0/126560073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2801036e0 0x7fd280107b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd274009930 tx=0x7fd27402a420 comp rx=0 tx=0).stop 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 -- 172.21.15.78:0/126560073 shutdown_connections 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 --2- 172.21.15.78:0/126560073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2801082a0 0x7fd28010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 --2- 172.21.15.78:0/126560073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2801036e0 0x7fd280107b90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 --2- 172.21.15.78:0/126560073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd280102d30 0x7fd280103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 -- 172.21.15.78:0/126560073 >> 172.21.15.78:0/126560073 conn(0x7fd2800fc7d0 msgr2=0x7fd2800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 -- 172.21.15.78:0/126560073 shutdown_connections 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 -- 172.21.15.78:0/126560073 wait complete. 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 Processor -- start 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.072+0000 7fd285532640 1 -- start start 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd285532640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd280102d30 0x7fd280198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd285532640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2801036e0 0x7fd280198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd285532640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2801082a0 0x7fd28019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd285532640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801120a0 con 0x7fd280102d30 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd285532640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd280111f20 con 0x7fd2801082a0 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd285532640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd280112220 con 0x7fd2801036e0 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd280102d30 0x7fd280198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd280102d30 0x7fd280198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52372/0 (socket says 172.21.15.78:52372) 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 -- 172.21.15.78:0/1961345644 learned_addr learned my addr 172.21.15.78:0/1961345644 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27f7fe640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2801082a0 0x7fd28019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27e7fc640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2801036e0 0x7fd280198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 -- 172.21.15.78:0/1961345644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2801036e0 msgr2=0x7fd280198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2801036e0 0x7fd280198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 -- 172.21.15.78:0/1961345644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2801082a0 msgr2=0x7fd28019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2801082a0 0x7fd28019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 -- 172.21.15.78:0/1961345644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28019d5b0 con 0x7fd280102d30 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd27effd640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd280102d30 0x7fd280198600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd27000d770 tx=0x7fd27000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd267fff640 1 -- 172.21.15.78:0/1961345644 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270017070 con 0x7fd280102d30 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd267fff640 1 -- 172.21.15.78:0/1961345644 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd270005500 con 0x7fd280102d30 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28019d8a0 con 0x7fd280102d30 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd267fff640 1 -- 172.21.15.78:0/1961345644 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270004e00 con 0x7fd280102d30 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.073+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801a5180 con 0x7fd280102d30 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.074+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd280103110 con 0x7fd280102d30 2024-09-16T00:11:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.075+0000 7fd267fff640 1 -- 172.21.15.78:0/1961345644 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd270004fa0 con 0x7fd280102d30 2024-09-16T00:11:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.076+0000 7fd267fff640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2680a32d0 0x7fd2680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.076+0000 7fd27e7fc640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2680a32d0 0x7fd2680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.076+0000 7fd267fff640 1 -- 172.21.15.78:0/1961345644 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd2700cfbf0 con 0x7fd280102d30 2024-09-16T00:11:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.077+0000 7fd27e7fc640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2680a32d0 0x7fd2680a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd274009230 tx=0x7fd274000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.079+0000 7fd267fff640 1 -- 172.21.15.78:0/1961345644 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd27009c570 con 0x7fd280102d30 2024-09-16T00:11:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.246+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd28010a2b0 con 0x7fd280102d30 2024-09-16T00:11:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.247+0000 7fd267fff640 1 -- 172.21.15.78:0/1961345644 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24213 (secure 0 0 0) 0x7fd27009be70 con 0x7fd280102d30 2024-09-16T00:11:56.249 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:56.249 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":86,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:11:14.379068+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2680a32d0 msgr2=0x7fd2680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2680a32d0 0x7fd2680a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd274009230 tx=0x7fd274000bc0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd280102d30 msgr2=0x7fd280198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd280102d30 0x7fd280198600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd27000d770 tx=0x7fd27000dc30 comp rx=0 tx=0).stop 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 shutdown_connections 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2680a32d0 0x7fd2680a5790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2801082a0 0x7fd28019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2801036e0 0x7fd280198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 --2- 172.21.15.78:0/1961345644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd280102d30 0x7fd280198600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 >> 172.21.15.78:0/1961345644 conn(0x7fd2800fc7d0 msgr2=0x7fd280105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 shutdown_connections 2024-09-16T00:11:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:56.250+0000 7fd285532640 1 -- 172.21.15.78:0/1961345644 wait complete. 2024-09-16T00:11:56.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:56 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1961345644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:56.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:56 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1961345644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:56.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:56 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1961345644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:11:56.961 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:11:56.962 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:11:57.206 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:57.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 -- 172.21.15.78:0/2788935752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60102ef0 msgr2=0x7f0e6010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:57.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 --2- 172.21.15.78:0/2788935752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60102ef0 0x7f0e6010da40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009ca0 tx=0x7f0e5802a950 comp rx=0 tx=0).stop 2024-09-16T00:11:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 -- 172.21.15.78:0/2788935752 shutdown_connections 2024-09-16T00:11:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 --2- 172.21.15.78:0/2788935752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60102ef0 0x7f0e6010da40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 --2- 172.21.15.78:0/2788935752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60102570 0x7f0e601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 --2- 172.21.15.78:0/2788935752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60100e20 0x7f0e60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 -- 172.21.15.78:0/2788935752 >> 172.21.15.78:0/2788935752 conn(0x7f0e600fc7d0 msgr2=0x7f0e600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 -- 172.21.15.78:0/2788935752 shutdown_connections 2024-09-16T00:11:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.707+0000 7f0e672f6640 1 -- 172.21.15.78:0/2788935752 wait complete. 2024-09-16T00:11:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 Processor -- start 2024-09-16T00:11:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 -- start start 2024-09-16T00:11:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60100e20 0x7f0e60198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60102570 0x7f0e60198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60102ef0 0x7f0e6019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60110f80 con 0x7f0e60100e20 2024-09-16T00:11:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60110e00 con 0x7f0e60102570 2024-09-16T00:11:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60111100 con 0x7f0e60102ef0 2024-09-16T00:11:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60100e20 0x7f0e60198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60100e20 0x7f0e60198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47058/0 (socket says 172.21.15.78:47058) 2024-09-16T00:11:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 -- 172.21.15.78:0/3498794385 learned_addr learned my addr 172.21.15.78:0/3498794385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:11:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e64891640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60102570 0x7f0e60198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 -- 172.21.15.78:0/3498794385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60102ef0 msgr2=0x7f0e6019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60102ef0 0x7f0e6019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 -- 172.21.15.78:0/3498794385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60102570 msgr2=0x7f0e60198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60102570 0x7f0e60198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 -- 172.21.15.78:0/3498794385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6019d4e0 con 0x7f0e60100e20 2024-09-16T00:11:57.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e65092640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60100e20 0x7f0e60198530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d770 tx=0x7f0e5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:57.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e567fc640 1 -- 172.21.15.78:0/3498794385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c017070 con 0x7f0e60100e20 2024-09-16T00:11:57.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.708+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6019d7d0 con 0x7f0e60100e20 2024-09-16T00:11:57.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.709+0000 7f0e567fc640 1 -- 172.21.15.78:0/3498794385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e5c005500 con 0x7f0e60100e20 2024-09-16T00:11:57.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.709+0000 7f0e567fc640 1 -- 172.21.15.78:0/3498794385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c004e00 con 0x7f0e60100e20 2024-09-16T00:11:57.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.709+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e601a50b0 con 0x7f0e60100e20 2024-09-16T00:11:57.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.710+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e60101200 con 0x7f0e60100e20 2024-09-16T00:11:57.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.711+0000 7f0e567fc640 1 -- 172.21.15.78:0/3498794385 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0e5c004fa0 con 0x7f0e60100e20 2024-09-16T00:11:57.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.712+0000 7f0e567fc640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a32d0 0x7f0e4c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:57.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.712+0000 7f0e64891640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a32d0 0x7f0e4c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:57.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.712+0000 7f0e567fc640 1 -- 172.21.15.78:0/3498794385 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f0e5c0cf810 con 0x7f0e60100e20 2024-09-16T00:11:57.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.712+0000 7f0e64891640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a32d0 0x7f0e4c0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005fd0 tx=0x7f0e50005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:57.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.716+0000 7f0e567fc640 1 -- 172.21.15.78:0/3498794385 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e5c09c190 con 0x7f0e60100e20 2024-09-16T00:11:57.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.880+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e6010a0d0 con 0x7f0e4c0a32d0 2024-09-16T00:11:57.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:57 smithi078 ceph-mon[31157]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-16T00:11:57.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.888+0000 7f0e567fc640 1 -- 172.21.15.78:0/3498794385 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340380 (secure 0 0 0) 0x7f0e6010a0d0 con 0x7f0e4c0a32d0 2024-09-16T00:11:57.893 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:11:57.895 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:11:57.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.891+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a32d0 msgr2=0x7f0e4c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.891+0000 7f0e672f6640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a32d0 0x7f0e4c0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0e50005fd0 tx=0x7f0e50005850 comp rx=0 tx=0).stop 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60100e20 msgr2=0x7f0e60198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60100e20 0x7f0e60198530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d770 tx=0x7f0e5c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 shutdown_connections 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a32d0 0x7f0e4c0a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60102ef0 0x7f0e6019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60102570 0x7f0e60198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 --2- 172.21.15.78:0/3498794385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60100e20 0x7f0e60198530 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 >> 172.21.15.78:0/3498794385 conn(0x7f0e600fc7d0 msgr2=0x7f0e6006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 shutdown_connections 2024-09-16T00:11:57.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:11:57.892+0000 7f0e672f6640 1 -- 172.21.15.78:0/3498794385 wait complete. 2024-09-16T00:11:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:57 smithi193 ceph-mon[35484]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-16T00:11:57.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:57 smithi148 ceph-mon[35599]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 KiB/s wr, 21 op/s 2024-09-16T00:11:58.649 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-16T00:11:56.406155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650223,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":275,"num_read_kb":253,"num_write":428,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358620,"kb_used_data":13532,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583204,"statfs":{"total":1151940427776,"available":1151573200896,"internally_reserved":0,"allocated":13856768,"data_stored":7645883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52429,"num_objects":41,"num_object_clones":0,"num_object_copies":125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":149,"num_read_kb":144,"num_write":114,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003281"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:53.564307+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:11:53.564307+0000","last_peered":"2024-09-16T00:11:53.564307+0000","last_clean":"2024-09-16T00:11:53.564307+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:11:53.564307+0000","last_undegraded":"2024-09-16T00:11:53.564307+0000","last_fullsized":"2024-09-16T00:11:53.564307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685467+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:11:14.685467+0000","last_peered":"2024-09-16T00:11:14.685467+0000","last_clean":"2024-09-16T00:11:14.685467+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:11:14.685467+0000","last_undegraded":"2024-09-16T00:11:14.685467+0000","last_fullsized":"2024-09-16T00:11:14.685467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318059+0000","last_change":"2024-09-16T00:11:11.688171+0000","last_active":"2024-09-16T00:11:15.318059+0000","last_peered":"2024-09-16T00:11:15.318059+0000","last_clean":"2024-09-16T00:11:15.318059+0000","last_became_active":"2024-09-16T00:11:11.687750+0000","last_became_peered":"2024-09-16T00:11:11.687750+0000","last_unstale":"2024-09-16T00:11:15.318059+0000","last_undegraded":"2024-09-16T00:11:15.318059+0000","last_fullsized":"2024-09-16T00:11:15.318059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:32:06.177592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685585+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:11:14.685585+0000","last_peered":"2024-09-16T00:11:14.685585+0000","last_clean":"2024-09-16T00:11:14.685585+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:11:14.685585+0000","last_undegraded":"2024-09-16T00:11:14.685585+0000","last_fullsized":"2024-09-16T00:11:14.685585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377631+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:11:15.377631+0000","last_peered":"2024-09-16T00:11:15.377631+0000","last_clean":"2024-09-16T00:11:15.377631+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:11:15.377631+0000","last_undegraded":"2024-09-16T00:11:15.377631+0000","last_fullsized":"2024-09-16T00:11:15.377631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685764+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:11:14.685764+0000","last_peered":"2024-09-16T00:11:14.685764+0000","last_clean":"2024-09-16T00:11:14.685764+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:11:14.685764+0000","last_undegraded":"2024-09-16T00:11:14.685764+0000","last_fullsized":"2024-09-16T00:11:14.685764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:43.549348+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:11:43.549348+0000","last_peered":"2024-09-16T00:11:43.549348+0000","last_clean":"2024-09-16T00:11:43.549348+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:11:43.549348+0000","last_undegraded":"2024-09-16T00:11:43.549348+0000","last_fullsized":"2024-09-16T00:11:43.549348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376188+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:11:15.376188+0000","last_peered":"2024-09-16T00:11:15.376188+0000","last_clean":"2024-09-16T00:11:15.376188+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:11:15.376188+0000","last_undegraded":"2024-09-16T00:11:15.376188+0000","last_fullsized":"2024-09-16T00:11:15.376188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686058+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:11:14.686058+0000","last_peered":"2024-09-16T00:11:14.686058+0000","last_clean":"2024-09-16T00:11:14.686058+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:11:14.686058+0000","last_undegraded":"2024-09-16T00:11:14.686058+0000","last_fullsized":"2024-09-16T00:11:14.686058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.544736+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:11:42.544736+0000","last_peered":"2024-09-16T00:11:42.544736+0000","last_clean":"2024-09-16T00:11:42.544736+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:11:42.544736+0000","last_undegraded":"2024-09-16T00:11:42.544736+0000","last_fullsized":"2024-09-16T00:11:42.544736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685925+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:11:14.685925+0000","last_peered":"2024-09-16T00:11:14.685925+0000","last_clean":"2024-09-16T00:11:14.685925+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:11:14.685925+0000","last_undegraded":"2024-09-16T00:11:14.685925+0000","last_fullsized":"2024-09-16T00:11:14.685925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685874+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:11:14.685874+0000","last_peered":"2024-09-16T00:11:14.685874+0000","last_clean":"2024-09-16T00:11:14.685874+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:11:14.685874+0000","last_undegraded":"2024-09-16T00:11:14.685874+0000","last_fullsized":"2024-09-16T00:11:14.685874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377210+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:11:15.377210+0000","last_peered":"2024-09-16T00:11:15.377210+0000","last_clean":"2024-09-16T00:11:15.377210+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:11:15.377210+0000","last_undegraded":"2024-09-16T00:11:15.377210+0000","last_fullsized":"2024-09-16T00:11:15.377210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:39.508906+0000","last_change":"2024-09-16T00:11:02.617063+0000","last_active":"2024-09-16T00:11:39.508906+0000","last_peered":"2024-09-16T00:11:39.508906+0000","last_clean":"2024-09-16T00:11:39.508906+0000","last_became_active":"2024-09-16T00:11:02.616550+0000","last_became_peered":"2024-09-16T00:11:02.616550+0000","last_unstale":"2024-09-16T00:11:39.508906+0000","last_undegraded":"2024-09-16T00:11:39.508906+0000","last_fullsized":"2024-09-16T00:11:39.508906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:32.325698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686298+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:11:14.686298+0000","last_peered":"2024-09-16T00:11:14.686298+0000","last_clean":"2024-09-16T00:11:14.686298+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:11:14.686298+0000","last_undegraded":"2024-09-16T00:11:14.686298+0000","last_fullsized":"2024-09-16T00:11:14.686298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685102+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:11:14.685102+0000","last_peered":"2024-09-16T00:11:14.685102+0000","last_clean":"2024-09-16T00:11:14.685102+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:11:14.685102+0000","last_undegraded":"2024-09-16T00:11:14.685102+0000","last_fullsized":"2024-09-16T00:11:14.685102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:41.531417+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:11:41.531417+0000","last_peered":"2024-09-16T00:11:41.531417+0000","last_clean":"2024-09-16T00:11:41.531417+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:11:41.531417+0000","last_undegraded":"2024-09-16T00:11:41.531417+0000","last_fullsized":"2024-09-16T00:11:41.531417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686231+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:11:14.686231+0000","last_peered":"2024-09-16T00:11:14.686231+0000","last_clean":"2024-09-16T00:11:14.686231+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:11:14.686231+0000","last_undegraded":"2024-09-16T00:11:14.686231+0000","last_fullsized":"2024-09-16T00:11:14.686231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764483+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:11:16.764483+0000","last_peered":"2024-09-16T00:11:16.764483+0000","last_clean":"2024-09-16T00:11:16.764483+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:11:16.764483+0000","last_undegraded":"2024-09-16T00:11:16.764483+0000","last_fullsized":"2024-09-16T00:11:16.764483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543231+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:11:42.543231+0000","last_peered":"2024-09-16T00:11:42.543231+0000","last_clean":"2024-09-16T00:11:42.543231+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:11:42.543231+0000","last_undegraded":"2024-09-16T00:11:42.543231+0000","last_fullsized":"2024-09-16T00:11:42.543231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376607+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:11:15.376607+0000","last_peered":"2024-09-16T00:11:15.376607+0000","last_clean":"2024-09-16T00:11:15.376607+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:11:15.376607+0000","last_undegraded":"2024-09-16T00:11:15.376607+0000","last_fullsized":"2024-09-16T00:11:15.376607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375598+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:11:15.375598+0000","last_peered":"2024-09-16T00:11:15.375598+0000","last_clean":"2024-09-16T00:11:15.375598+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:11:15.375598+0000","last_undegraded":"2024-09-16T00:11:15.375598+0000","last_fullsized":"2024-09-16T00:11:15.375598+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:39.507685+0000","last_change":"2024-09-16T00:11:02.809403+0000","last_active":"2024-09-16T00:11:39.507685+0000","last_peered":"2024-09-16T00:11:39.507685+0000","last_clean":"2024-09-16T00:11:39.507685+0000","last_became_active":"2024-09-16T00:11:02.809066+0000","last_became_peered":"2024-09-16T00:11:02.809066+0000","last_unstale":"2024-09-16T00:11:39.507685+0000","last_undegraded":"2024-09-16T00:11:39.507685+0000","last_fullsized":"2024-09-16T00:11:39.507685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:58.544890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686467+0000","last_change":"2024-09-16T00:11:11.682995+0000","last_active":"2024-09-16T00:11:14.686467+0000","last_peered":"2024-09-16T00:11:14.686467+0000","last_clean":"2024-09-16T00:11:14.686467+0000","last_became_active":"2024-09-16T00:11:11.682592+0000","last_became_peered":"2024-09-16T00:11:11.682592+0000","last_unstale":"2024-09-16T00:11:14.686467+0000","last_undegraded":"2024-09-16T00:11:14.686467+0000","last_fullsized":"2024-09-16T00:11:14.686467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.159350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375997+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:11:15.375997+0000","last_peered":"2024-09-16T00:11:15.375997+0000","last_clean":"2024-09-16T00:11:15.375997+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:11:15.375997+0000","last_undegraded":"2024-09-16T00:11:15.375997+0000","last_fullsized":"2024-09-16T00:11:15.375997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543401+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:11:42.543401+0000","last_peered":"2024-09-16T00:11:42.543401+0000","last_clean":"2024-09-16T00:11:42.543401+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:11:42.543401+0000","last_undegraded":"2024-09-16T00:11:42.543401+0000","last_fullsized":"2024-09-16T00:11:42.543401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318059+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:11:15.318059+0000","last_peered":"2024-09-16T00:11:15.318059+0000","last_clean":"2024-09-16T00:11:15.318059+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:11:15.318059+0000","last_undegraded":"2024-09-16T00:11:15.318059+0000","last_fullsized":"2024-09-16T00:11:15.318059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685434+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:11:14.685434+0000","last_peered":"2024-09-16T00:11:14.685434+0000","last_clean":"2024-09-16T00:11:14.685434+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:11:14.685434+0000","last_undegraded":"2024-09-16T00:11:14.685434+0000","last_fullsized":"2024-09-16T00:11:14.685434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'15","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:53.640204+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:11:53.640204+0000","last_peered":"2024-09-16T00:11:53.640204+0000","last_clean":"2024-09-16T00:11:53.640204+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:11:53.640204+0000","last_undegraded":"2024-09-16T00:11:53.640204+0000","last_fullsized":"2024-09-16T00:11:53.640204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.684901+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:11:14.684901+0000","last_peered":"2024-09-16T00:11:14.684901+0000","last_clean":"2024-09-16T00:11:14.684901+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:11:14.684901+0000","last_undegraded":"2024-09-16T00:11:14.684901+0000","last_fullsized":"2024-09-16T00:11:14.684901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376421+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:11:15.376421+0000","last_peered":"2024-09-16T00:11:15.376421+0000","last_clean":"2024-09-16T00:11:15.376421+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:11:15.376421+0000","last_undegraded":"2024-09-16T00:11:15.376421+0000","last_fullsized":"2024-09-16T00:11:15.376421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.544554+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:11:42.544554+0000","last_peered":"2024-09-16T00:11:42.544554+0000","last_clean":"2024-09-16T00:11:42.544554+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:11:42.544554+0000","last_undegraded":"2024-09-16T00:11:42.544554+0000","last_fullsized":"2024-09-16T00:11:42.544554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764536+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:11:16.764536+0000","last_peered":"2024-09-16T00:11:16.764536+0000","last_clean":"2024-09-16T00:11:16.764536+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:11:16.764536+0000","last_undegraded":"2024-09-16T00:11:16.764536+0000","last_fullsized":"2024-09-16T00:11:16.764536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375889+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:11:15.375889+0000","last_peered":"2024-09-16T00:11:15.375889+0000","last_clean":"2024-09-16T00:11:15.375889+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:11:15.375889+0000","last_undegraded":"2024-09-16T00:11:15.375889+0000","last_fullsized":"2024-09-16T00:11:15.375889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376869+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:11:15.376869+0000","last_peered":"2024-09-16T00:11:15.376869+0000","last_clean":"2024-09-16T00:11:15.376869+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:11:15.376869+0000","last_undegraded":"2024-09-16T00:11:15.376869+0000","last_fullsized":"2024-09-16T00:11:15.376869+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375705+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:11:15.375705+0000","last_peered":"2024-09-16T00:11:15.375705+0000","last_clean":"2024-09-16T00:11:15.375705+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:11:15.375705+0000","last_undegraded":"2024-09-16T00:11:15.375705+0000","last_fullsized":"2024-09-16T00:11:15.375705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376002+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:11:15.376002+0000","last_peered":"2024-09-16T00:11:15.376002+0000","last_clean":"2024-09-16T00:11:15.376002+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:11:15.376002+0000","last_undegraded":"2024-09-16T00:11:15.376002+0000","last_fullsized":"2024-09-16T00:11:15.376002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:51.562695+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:11:51.562695+0000","last_peered":"2024-09-16T00:11:51.562695+0000","last_clean":"2024-09-16T00:11:51.562695+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:11:51.562695+0000","last_undegraded":"2024-09-16T00:11:51.562695+0000","last_fullsized":"2024-09-16T00:11:51.562695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376075+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:11:15.376075+0000","last_peered":"2024-09-16T00:11:15.376075+0000","last_clean":"2024-09-16T00:11:15.376075+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:11:15.376075+0000","last_undegraded":"2024-09-16T00:11:15.376075+0000","last_fullsized":"2024-09-16T00:11:15.376075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685530+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:11:14.685530+0000","last_peered":"2024-09-16T00:11:14.685530+0000","last_clean":"2024-09-16T00:11:14.685530+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:11:14.685530+0000","last_undegraded":"2024-09-16T00:11:14.685530+0000","last_fullsized":"2024-09-16T00:11:14.685530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:51.562966+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:11:51.562966+0000","last_peered":"2024-09-16T00:11:51.562966+0000","last_clean":"2024-09-16T00:11:51.562966+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:11:51.562966+0000","last_undegraded":"2024-09-16T00:11:51.562966+0000","last_fullsized":"2024-09-16T00:11:51.562966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376211+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:11:15.376211+0000","last_peered":"2024-09-16T00:11:15.376211+0000","last_clean":"2024-09-16T00:11:15.376211+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:11:15.376211+0000","last_undegraded":"2024-09-16T00:11:15.376211+0000","last_fullsized":"2024-09-16T00:11:15.376211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375278+0000","last_change":"2024-09-16T00:11:04.641679+0000","last_active":"2024-09-16T00:11:15.375278+0000","last_peered":"2024-09-16T00:11:15.375278+0000","last_clean":"2024-09-16T00:11:15.375278+0000","last_became_active":"2024-09-16T00:11:04.641333+0000","last_became_peered":"2024-09-16T00:11:04.641333+0000","last_unstale":"2024-09-16T00:11:15.375278+0000","last_undegraded":"2024-09-16T00:11:15.375278+0000","last_fullsized":"2024-09-16T00:11:15.375278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:52.103553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:40.518041+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:11:40.518041+0000","last_peered":"2024-09-16T00:11:40.518041+0000","last_clean":"2024-09-16T00:11:40.518041+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:11:40.518041+0000","last_undegraded":"2024-09-16T00:11:40.518041+0000","last_fullsized":"2024-09-16T00:11:40.518041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686102+0000","last_change":"2024-09-16T00:11:11.677781+0000","last_active":"2024-09-16T00:11:14.686102+0000","last_peered":"2024-09-16T00:11:14.686102+0000","last_clean":"2024-09-16T00:11:14.686102+0000","last_became_active":"2024-09-16T00:11:11.677032+0000","last_became_peered":"2024-09-16T00:11:11.677032+0000","last_unstale":"2024-09-16T00:11:14.686102+0000","last_undegraded":"2024-09-16T00:11:14.686102+0000","last_fullsized":"2024-09-16T00:11:14.686102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:57.523334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685061+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:11:14.685061+0000","last_peered":"2024-09-16T00:11:14.685061+0000","last_clean":"2024-09-16T00:11:14.685061+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:11:14.685061+0000","last_undegraded":"2024-09-16T00:11:14.685061+0000","last_fullsized":"2024-09-16T00:11:14.685061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:43.548706+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:11:43.548706+0000","last_peered":"2024-09-16T00:11:43.548706+0000","last_clean":"2024-09-16T00:11:43.548706+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:11:43.548706+0000","last_undegraded":"2024-09-16T00:11:43.548706+0000","last_fullsized":"2024-09-16T00:11:43.548706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376286+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:11:15.376286+0000","last_peered":"2024-09-16T00:11:15.376286+0000","last_clean":"2024-09-16T00:11:15.376286+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:11:15.376286+0000","last_undegraded":"2024-09-16T00:11:15.376286+0000","last_fullsized":"2024-09-16T00:11:15.376286+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377101+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:11:15.377101+0000","last_peered":"2024-09-16T00:11:15.377101+0000","last_clean":"2024-09-16T00:11:15.377101+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:11:15.377101+0000","last_undegraded":"2024-09-16T00:11:15.377101+0000","last_fullsized":"2024-09-16T00:11:15.377101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377354+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:11:15.377354+0000","last_peered":"2024-09-16T00:11:15.377354+0000","last_clean":"2024-09-16T00:11:15.377354+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:11:15.377354+0000","last_undegraded":"2024-09-16T00:11:15.377354+0000","last_fullsized":"2024-09-16T00:11:15.377354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:53.565579+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:11:53.565579+0000","last_peered":"2024-09-16T00:11:53.565579+0000","last_clean":"2024-09-16T00:11:53.565579+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:11:53.565579+0000","last_undegraded":"2024-09-16T00:11:53.565579+0000","last_fullsized":"2024-09-16T00:11:53.565579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376721+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:11:15.376721+0000","last_peered":"2024-09-16T00:11:15.376721+0000","last_clean":"2024-09-16T00:11:15.376721+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:11:15.376721+0000","last_undegraded":"2024-09-16T00:11:15.376721+0000","last_fullsized":"2024-09-16T00:11:15.376721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376453+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:11:15.376453+0000","last_peered":"2024-09-16T00:11:15.376453+0000","last_clean":"2024-09-16T00:11:15.376453+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:11:15.376453+0000","last_undegraded":"2024-09-16T00:11:15.376453+0000","last_fullsized":"2024-09-16T00:11:15.376453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543483+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:11:42.543483+0000","last_peered":"2024-09-16T00:11:42.543483+0000","last_clean":"2024-09-16T00:11:42.543483+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:11:42.543483+0000","last_undegraded":"2024-09-16T00:11:42.543483+0000","last_fullsized":"2024-09-16T00:11:42.543483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376386+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:11:15.376386+0000","last_peered":"2024-09-16T00:11:15.376386+0000","last_clean":"2024-09-16T00:11:15.376386+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:11:15.376386+0000","last_undegraded":"2024-09-16T00:11:15.376386+0000","last_fullsized":"2024-09-16T00:11:15.376386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376338+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:11:15.376338+0000","last_peered":"2024-09-16T00:11:15.376338+0000","last_clean":"2024-09-16T00:11:15.376338+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:11:15.376338+0000","last_undegraded":"2024-09-16T00:11:15.376338+0000","last_fullsized":"2024-09-16T00:11:15.376338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543297+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:11:42.543297+0000","last_peered":"2024-09-16T00:11:42.543297+0000","last_clean":"2024-09-16T00:11:42.543297+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:11:42.543297+0000","last_undegraded":"2024-09-16T00:11:42.543297+0000","last_fullsized":"2024-09-16T00:11:42.543297+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685402+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:11:14.685402+0000","last_peered":"2024-09-16T00:11:14.685402+0000","last_clean":"2024-09-16T00:11:14.685402+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:11:14.685402+0000","last_undegraded":"2024-09-16T00:11:14.685402+0000","last_fullsized":"2024-09-16T00:11:14.685402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764778+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:11:16.764778+0000","last_peered":"2024-09-16T00:11:16.764778+0000","last_clean":"2024-09-16T00:11:16.764778+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:11:16.764778+0000","last_undegraded":"2024-09-16T00:11:16.764778+0000","last_fullsized":"2024-09-16T00:11:16.764778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.513120+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:11:42.513120+0000","last_peered":"2024-09-16T00:11:42.513120+0000","last_clean":"2024-09-16T00:11:42.513120+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:11:42.513120+0000","last_undegraded":"2024-09-16T00:11:42.513120+0000","last_fullsized":"2024-09-16T00:11:42.513120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685512+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:11:14.685512+0000","last_peered":"2024-09-16T00:11:14.685512+0000","last_clean":"2024-09-16T00:11:14.685512+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:11:14.685512+0000","last_undegraded":"2024-09-16T00:11:14.685512+0000","last_fullsized":"2024-09-16T00:11:14.685512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685716+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:11:14.685716+0000","last_peered":"2024-09-16T00:11:14.685716+0000","last_clean":"2024-09-16T00:11:14.685716+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:11:14.685716+0000","last_undegraded":"2024-09-16T00:11:14.685716+0000","last_fullsized":"2024-09-16T00:11:14.685716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543932+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:11:42.543932+0000","last_peered":"2024-09-16T00:11:42.543932+0000","last_clean":"2024-09-16T00:11:42.543932+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:11:42.543932+0000","last_undegraded":"2024-09-16T00:11:42.543932+0000","last_fullsized":"2024-09-16T00:11:42.543932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376353+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:11:15.376353+0000","last_peered":"2024-09-16T00:11:15.376353+0000","last_clean":"2024-09-16T00:11:15.376353+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:11:15.376353+0000","last_undegraded":"2024-09-16T00:11:15.376353+0000","last_fullsized":"2024-09-16T00:11:15.376353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375923+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:11:15.375923+0000","last_peered":"2024-09-16T00:11:15.375923+0000","last_clean":"2024-09-16T00:11:15.375923+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:11:15.375923+0000","last_undegraded":"2024-09-16T00:11:15.375923+0000","last_fullsized":"2024-09-16T00:11:15.375923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:51.563365+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:11:51.563365+0000","last_peered":"2024-09-16T00:11:51.563365+0000","last_clean":"2024-09-16T00:11:51.563365+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:11:51.563365+0000","last_undegraded":"2024-09-16T00:11:51.563365+0000","last_fullsized":"2024-09-16T00:11:51.563365+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376726+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:11:15.376726+0000","last_peered":"2024-09-16T00:11:15.376726+0000","last_clean":"2024-09-16T00:11:15.376726+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:11:15.376726+0000","last_undegraded":"2024-09-16T00:11:15.376726+0000","last_fullsized":"2024-09-16T00:11:15.376726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377353+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:11:15.377353+0000","last_peered":"2024-09-16T00:11:15.377353+0000","last_clean":"2024-09-16T00:11:15.377353+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:11:15.377353+0000","last_undegraded":"2024-09-16T00:11:15.377353+0000","last_fullsized":"2024-09-16T00:11:15.377353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543903+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:11:42.543903+0000","last_peered":"2024-09-16T00:11:42.543903+0000","last_clean":"2024-09-16T00:11:42.543903+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:11:42.543903+0000","last_undegraded":"2024-09-16T00:11:42.543903+0000","last_fullsized":"2024-09-16T00:11:42.543903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764742+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:11:16.764742+0000","last_peered":"2024-09-16T00:11:16.764742+0000","last_clean":"2024-09-16T00:11:16.764742+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:11:16.764742+0000","last_undegraded":"2024-09-16T00:11:16.764742+0000","last_fullsized":"2024-09-16T00:11:16.764742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375612+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:11:15.375612+0000","last_peered":"2024-09-16T00:11:15.375612+0000","last_clean":"2024-09-16T00:11:15.375612+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:11:15.375612+0000","last_undegraded":"2024-09-16T00:11:15.375612+0000","last_fullsized":"2024-09-16T00:11:15.375612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:43.548732+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:11:43.548732+0000","last_peered":"2024-09-16T00:11:43.548732+0000","last_clean":"2024-09-16T00:11:43.548732+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:11:43.548732+0000","last_undegraded":"2024-09-16T00:11:43.548732+0000","last_fullsized":"2024-09-16T00:11:43.548732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686090+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:11:14.686090+0000","last_peered":"2024-09-16T00:11:14.686090+0000","last_clean":"2024-09-16T00:11:14.686090+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:11:14.686090+0000","last_undegraded":"2024-09-16T00:11:14.686090+0000","last_fullsized":"2024-09-16T00:11:14.686090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375927+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:11:15.375927+0000","last_peered":"2024-09-16T00:11:15.375927+0000","last_clean":"2024-09-16T00:11:15.375927+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:11:15.375927+0000","last_undegraded":"2024-09-16T00:11:15.375927+0000","last_fullsized":"2024-09-16T00:11:15.375927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'32","reported_seq":93,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:54.568755+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:11:54.568755+0000","last_peered":"2024-09-16T00:11:54.568755+0000","last_clean":"2024-09-16T00:11:54.568755+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:11:54.568755+0000","last_undegraded":"2024-09-16T00:11:54.568755+0000","last_fullsized":"2024-09-16T00:11:54.568755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52413,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":66,"num_write":34,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377704+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:11:15.377704+0000","last_peered":"2024-09-16T00:11:15.377704+0000","last_clean":"2024-09-16T00:11:15.377704+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:11:15.377704+0000","last_undegraded":"2024-09-16T00:11:15.377704+0000","last_fullsized":"2024-09-16T00:11:15.377704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376300+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:11:15.376300+0000","last_peered":"2024-09-16T00:11:15.376300+0000","last_clean":"2024-09-16T00:11:15.376300+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:11:15.376300+0000","last_undegraded":"2024-09-16T00:11:15.376300+0000","last_fullsized":"2024-09-16T00:11:15.376300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:41.530675+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:11:41.530675+0000","last_peered":"2024-09-16T00:11:41.530675+0000","last_clean":"2024-09-16T00:11:41.530675+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:11:41.530675+0000","last_undegraded":"2024-09-16T00:11:41.530675+0000","last_fullsized":"2024-09-16T00:11:41.530675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686161+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:11:14.686161+0000","last_peered":"2024-09-16T00:11:14.686161+0000","last_clean":"2024-09-16T00:11:14.686161+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:11:14.686161+0000","last_undegraded":"2024-09-16T00:11:14.686161+0000","last_fullsized":"2024-09-16T00:11:14.686161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377685+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:11:15.377685+0000","last_peered":"2024-09-16T00:11:15.377685+0000","last_clean":"2024-09-16T00:11:15.377685+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:11:15.377685+0000","last_undegraded":"2024-09-16T00:11:15.377685+0000","last_fullsized":"2024-09-16T00:11:15.377685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.546096+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:11:42.546096+0000","last_peered":"2024-09-16T00:11:42.546096+0000","last_clean":"2024-09-16T00:11:42.546096+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:11:42.546096+0000","last_undegraded":"2024-09-16T00:11:42.546096+0000","last_fullsized":"2024-09-16T00:11:42.546096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377451+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:11:15.377451+0000","last_peered":"2024-09-16T00:11:15.377451+0000","last_clean":"2024-09-16T00:11:15.377451+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:11:15.377451+0000","last_undegraded":"2024-09-16T00:11:15.377451+0000","last_fullsized":"2024-09-16T00:11:15.377451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376585+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:11:15.376585+0000","last_peered":"2024-09-16T00:11:15.376585+0000","last_clean":"2024-09-16T00:11:15.376585+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:11:15.376585+0000","last_undegraded":"2024-09-16T00:11:15.376585+0000","last_fullsized":"2024-09-16T00:11:15.376585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":58,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:54.566275+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:11:54.566275+0000","last_peered":"2024-09-16T00:11:54.566275+0000","last_clean":"2024-09-16T00:11:54.566275+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:11:54.566275+0000","last_undegraded":"2024-09-16T00:11:54.566275+0000","last_fullsized":"2024-09-16T00:11:54.566275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685044+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:11:14.685044+0000","last_peered":"2024-09-16T00:11:14.685044+0000","last_clean":"2024-09-16T00:11:14.685044+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:11:14.685044+0000","last_undegraded":"2024-09-16T00:11:14.685044+0000","last_fullsized":"2024-09-16T00:11:14.685044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375852+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:11:15.375852+0000","last_peered":"2024-09-16T00:11:15.375852+0000","last_clean":"2024-09-16T00:11:15.375852+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:11:15.375852+0000","last_undegraded":"2024-09-16T00:11:15.375852+0000","last_fullsized":"2024-09-16T00:11:15.375852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376620+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:11:15.376620+0000","last_peered":"2024-09-16T00:11:15.376620+0000","last_clean":"2024-09-16T00:11:15.376620+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:11:15.376620+0000","last_undegraded":"2024-09-16T00:11:15.376620+0000","last_fullsized":"2024-09-16T00:11:15.376620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.544126+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:11:42.544126+0000","last_peered":"2024-09-16T00:11:42.544126+0000","last_clean":"2024-09-16T00:11:42.544126+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:11:42.544126+0000","last_undegraded":"2024-09-16T00:11:42.544126+0000","last_fullsized":"2024-09-16T00:11:42.544126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764814+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:11:16.764814+0000","last_peered":"2024-09-16T00:11:16.764814+0000","last_clean":"2024-09-16T00:11:16.764814+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:11:16.764814+0000","last_undegraded":"2024-09-16T00:11:16.764814+0000","last_fullsized":"2024-09-16T00:11:16.764814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685199+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:11:14.685199+0000","last_peered":"2024-09-16T00:11:14.685199+0000","last_clean":"2024-09-16T00:11:14.685199+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:11:14.685199+0000","last_undegraded":"2024-09-16T00:11:14.685199+0000","last_fullsized":"2024-09-16T00:11:14.685199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:40.517394+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:11:40.517394+0000","last_peered":"2024-09-16T00:11:40.517394+0000","last_clean":"2024-09-16T00:11:40.517394+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:11:40.517394+0000","last_undegraded":"2024-09-16T00:11:40.517394+0000","last_fullsized":"2024-09-16T00:11:40.517394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:41.531601+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:11:41.531601+0000","last_peered":"2024-09-16T00:11:41.531601+0000","last_clean":"2024-09-16T00:11:41.531601+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:11:41.531601+0000","last_undegraded":"2024-09-16T00:11:41.531601+0000","last_fullsized":"2024-09-16T00:11:41.531601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685847+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:11:14.685847+0000","last_peered":"2024-09-16T00:11:14.685847+0000","last_clean":"2024-09-16T00:11:14.685847+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:11:14.685847+0000","last_undegraded":"2024-09-16T00:11:14.685847+0000","last_fullsized":"2024-09-16T00:11:14.685847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318337+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:11:15.318337+0000","last_peered":"2024-09-16T00:11:15.318337+0000","last_clean":"2024-09-16T00:11:15.318337+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:11:15.318337+0000","last_undegraded":"2024-09-16T00:11:15.318337+0000","last_fullsized":"2024-09-16T00:11:15.318337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:49.560139+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:11:49.560139+0000","last_peered":"2024-09-16T00:11:49.560139+0000","last_clean":"2024-09-16T00:11:49.560139+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:11:49.560139+0000","last_undegraded":"2024-09-16T00:11:49.560139+0000","last_fullsized":"2024-09-16T00:11:49.560139+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375752+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:11:15.375752+0000","last_peered":"2024-09-16T00:11:15.375752+0000","last_clean":"2024-09-16T00:11:15.375752+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:11:15.375752+0000","last_undegraded":"2024-09-16T00:11:15.375752+0000","last_fullsized":"2024-09-16T00:11:15.375752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318426+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:11:15.318426+0000","last_peered":"2024-09-16T00:11:15.318426+0000","last_clean":"2024-09-16T00:11:15.318426+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:11:15.318426+0000","last_undegraded":"2024-09-16T00:11:15.318426+0000","last_fullsized":"2024-09-16T00:11:15.318426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764381+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:11:16.764381+0000","last_peered":"2024-09-16T00:11:16.764381+0000","last_clean":"2024-09-16T00:11:16.764381+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:11:16.764381+0000","last_undegraded":"2024-09-16T00:11:16.764381+0000","last_fullsized":"2024-09-16T00:11:16.764381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318388+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:11:15.318388+0000","last_peered":"2024-09-16T00:11:15.318388+0000","last_clean":"2024-09-16T00:11:15.318388+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:11:15.318388+0000","last_undegraded":"2024-09-16T00:11:15.318388+0000","last_fullsized":"2024-09-16T00:11:15.318388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.317945+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:11:15.317945+0000","last_peered":"2024-09-16T00:11:15.317945+0000","last_clean":"2024-09-16T00:11:15.317945+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:11:15.317945+0000","last_undegraded":"2024-09-16T00:11:15.317945+0000","last_fullsized":"2024-09-16T00:11:15.317945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376436+0000","last_change":"2024-09-16T00:11:04.641885+0000","last_active":"2024-09-16T00:11:15.376436+0000","last_peered":"2024-09-16T00:11:15.376436+0000","last_clean":"2024-09-16T00:11:15.376436+0000","last_became_active":"2024-09-16T00:11:04.641496+0000","last_became_peered":"2024-09-16T00:11:04.641496+0000","last_unstale":"2024-09-16T00:11:15.376436+0000","last_undegraded":"2024-09-16T00:11:15.376436+0000","last_fullsized":"2024-09-16T00:11:15.376436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:40.585243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376212+0000","last_change":"2024-09-16T00:11:04.649287+0000","last_active":"2024-09-16T00:11:15.376212+0000","last_peered":"2024-09-16T00:11:15.376212+0000","last_clean":"2024-09-16T00:11:15.376212+0000","last_became_active":"2024-09-16T00:11:04.648952+0000","last_became_peered":"2024-09-16T00:11:04.648952+0000","last_unstale":"2024-09-16T00:11:15.376212+0000","last_undegraded":"2024-09-16T00:11:15.376212+0000","last_fullsized":"2024-09-16T00:11:15.376212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:03:57.189388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764770+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:11:16.764770+0000","last_peered":"2024-09-16T00:11:16.764770+0000","last_clean":"2024-09-16T00:11:16.764770+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:11:16.764770+0000","last_undegraded":"2024-09-16T00:11:16.764770+0000","last_fullsized":"2024-09-16T00:11:16.764770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318531+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:11:15.318531+0000","last_peered":"2024-09-16T00:11:15.318531+0000","last_clean":"2024-09-16T00:11:15.318531+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:11:15.318531+0000","last_undegraded":"2024-09-16T00:11:15.318531+0000","last_fullsized":"2024-09-16T00:11:15.318531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685394+0000","last_change":"2024-09-16T00:11:04.649155+0000","last_active":"2024-09-16T00:11:14.685394+0000","last_peered":"2024-09-16T00:11:14.685394+0000","last_clean":"2024-09-16T00:11:14.685394+0000","last_became_active":"2024-09-16T00:11:04.648844+0000","last_became_peered":"2024-09-16T00:11:04.648844+0000","last_unstale":"2024-09-16T00:11:14.685394+0000","last_undegraded":"2024-09-16T00:11:14.685394+0000","last_fullsized":"2024-09-16T00:11:14.685394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:20.205113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685482+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:11:14.685482+0000","last_peered":"2024-09-16T00:11:14.685482+0000","last_clean":"2024-09-16T00:11:14.685482+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:11:14.685482+0000","last_undegraded":"2024-09-16T00:11:14.685482+0000","last_fullsized":"2024-09-16T00:11:14.685482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685108+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:11:14.685108+0000","last_peered":"2024-09-16T00:11:14.685108+0000","last_clean":"2024-09-16T00:11:14.685108+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:11:14.685108+0000","last_undegraded":"2024-09-16T00:11:14.685108+0000","last_fullsized":"2024-09-16T00:11:14.685108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376984+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:11:15.376984+0000","last_peered":"2024-09-16T00:11:15.376984+0000","last_clean":"2024-09-16T00:11:15.376984+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:11:15.376984+0000","last_undegraded":"2024-09-16T00:11:15.376984+0000","last_fullsized":"2024-09-16T00:11:15.376984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375777+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:11:15.375777+0000","last_peered":"2024-09-16T00:11:15.375777+0000","last_clean":"2024-09-16T00:11:15.375777+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:11:15.375777+0000","last_undegraded":"2024-09-16T00:11:15.375777+0000","last_fullsized":"2024-09-16T00:11:15.375777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764663+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:11:16.764663+0000","last_peered":"2024-09-16T00:11:16.764663+0000","last_clean":"2024-09-16T00:11:16.764663+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:11:16.764663+0000","last_undegraded":"2024-09-16T00:11:16.764663+0000","last_fullsized":"2024-09-16T00:11:16.764663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318511+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:11:15.318511+0000","last_peered":"2024-09-16T00:11:15.318511+0000","last_clean":"2024-09-16T00:11:15.318511+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:11:15.318511+0000","last_undegraded":"2024-09-16T00:11:15.318511+0000","last_fullsized":"2024-09-16T00:11:15.318511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376301+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:11:15.376301+0000","last_peered":"2024-09-16T00:11:15.376301+0000","last_clean":"2024-09-16T00:11:15.376301+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:11:15.376301+0000","last_undegraded":"2024-09-16T00:11:15.376301+0000","last_fullsized":"2024-09-16T00:11:15.376301+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764943+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:11:16.764943+0000","last_peered":"2024-09-16T00:11:16.764943+0000","last_clean":"2024-09-16T00:11:16.764943+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:11:16.764943+0000","last_undegraded":"2024-09-16T00:11:16.764943+0000","last_fullsized":"2024-09-16T00:11:16.764943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376118+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:11:15.376118+0000","last_peered":"2024-09-16T00:11:15.376118+0000","last_clean":"2024-09-16T00:11:15.376118+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:11:15.376118+0000","last_undegraded":"2024-09-16T00:11:15.376118+0000","last_fullsized":"2024-09-16T00:11:15.376118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375790+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:11:15.375790+0000","last_peered":"2024-09-16T00:11:15.375790+0000","last_clean":"2024-09-16T00:11:15.375790+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:11:15.375790+0000","last_undegraded":"2024-09-16T00:11:15.375790+0000","last_fullsized":"2024-09-16T00:11:15.375790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376012+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:11:15.376012+0000","last_peered":"2024-09-16T00:11:15.376012+0000","last_clean":"2024-09-16T00:11:15.376012+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:11:15.376012+0000","last_undegraded":"2024-09-16T00:11:15.376012+0000","last_fullsized":"2024-09-16T00:11:15.376012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375697+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:11:15.375697+0000","last_peered":"2024-09-16T00:11:15.375697+0000","last_clean":"2024-09-16T00:11:15.375697+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:11:15.375697+0000","last_undegraded":"2024-09-16T00:11:15.375697+0000","last_fullsized":"2024-09-16T00:11:15.375697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686296+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:11:14.686296+0000","last_peered":"2024-09-16T00:11:14.686296+0000","last_clean":"2024-09-16T00:11:14.686296+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:11:14.686296+0000","last_undegraded":"2024-09-16T00:11:14.686296+0000","last_fullsized":"2024-09-16T00:11:14.686296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377246+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:11:15.377246+0000","last_peered":"2024-09-16T00:11:15.377246+0000","last_clean":"2024-09-16T00:11:15.377246+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:11:15.377246+0000","last_undegraded":"2024-09-16T00:11:15.377246+0000","last_fullsized":"2024-09-16T00:11:15.377246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376735+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:11:15.376735+0000","last_peered":"2024-09-16T00:11:15.376735+0000","last_clean":"2024-09-16T00:11:15.376735+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:11:15.376735+0000","last_undegraded":"2024-09-16T00:11:15.376735+0000","last_fullsized":"2024-09-16T00:11:15.376735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377385+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:11:15.377385+0000","last_peered":"2024-09-16T00:11:15.377385+0000","last_clean":"2024-09-16T00:11:15.377385+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:11:15.377385+0000","last_undegraded":"2024-09-16T00:11:15.377385+0000","last_fullsized":"2024-09-16T00:11:15.377385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685023+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:11:14.685023+0000","last_peered":"2024-09-16T00:11:14.685023+0000","last_clean":"2024-09-16T00:11:14.685023+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:11:14.685023+0000","last_undegraded":"2024-09-16T00:11:14.685023+0000","last_fullsized":"2024-09-16T00:11:14.685023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376994+0000","last_change":"2024-09-16T00:11:04.644081+0000","last_active":"2024-09-16T00:11:15.376994+0000","last_peered":"2024-09-16T00:11:15.376994+0000","last_clean":"2024-09-16T00:11:15.376994+0000","last_became_active":"2024-09-16T00:11:04.643779+0000","last_became_peered":"2024-09-16T00:11:04.643779+0000","last_unstale":"2024-09-16T00:11:15.376994+0000","last_undegraded":"2024-09-16T00:11:15.376994+0000","last_fullsized":"2024-09-16T00:11:15.376994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:56:13.966468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376155+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:11:15.376155+0000","last_peered":"2024-09-16T00:11:15.376155+0000","last_clean":"2024-09-16T00:11:15.376155+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:11:15.376155+0000","last_undegraded":"2024-09-16T00:11:15.376155+0000","last_fullsized":"2024-09-16T00:11:15.376155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375686+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:11:15.375686+0000","last_peered":"2024-09-16T00:11:15.375686+0000","last_clean":"2024-09-16T00:11:15.375686+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:11:15.375686+0000","last_undegraded":"2024-09-16T00:11:15.375686+0000","last_fullsized":"2024-09-16T00:11:15.375686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686266+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:11:14.686266+0000","last_peered":"2024-09-16T00:11:14.686266+0000","last_clean":"2024-09-16T00:11:14.686266+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:11:14.686266+0000","last_undegraded":"2024-09-16T00:11:14.686266+0000","last_fullsized":"2024-09-16T00:11:14.686266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376859+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:11:15.376859+0000","last_peered":"2024-09-16T00:11:15.376859+0000","last_clean":"2024-09-16T00:11:15.376859+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:11:15.376859+0000","last_undegraded":"2024-09-16T00:11:15.376859+0000","last_fullsized":"2024-09-16T00:11:15.376859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375909+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:11:15.375909+0000","last_peered":"2024-09-16T00:11:15.375909+0000","last_clean":"2024-09-16T00:11:15.375909+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:11:15.375909+0000","last_undegraded":"2024-09-16T00:11:15.375909+0000","last_fullsized":"2024-09-16T00:11:15.375909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764909+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:11:16.764909+0000","last_peered":"2024-09-16T00:11:16.764909+0000","last_clean":"2024-09-16T00:11:16.764909+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:11:16.764909+0000","last_undegraded":"2024-09-16T00:11:16.764909+0000","last_fullsized":"2024-09-16T00:11:16.764909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59621,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":149,"num_read_kb":144,"num_write":245,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":178863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579927,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":477610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1116839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":481243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201319,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":477525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430251,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1074024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593776,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":472651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822709,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708244,"statfs":{"total":95995035648,"available":95957241856,"internally_reserved":0,"allocated":1601536,"data_stored":1069213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":477118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":472359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":526707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":526094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":474500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:11:58.656 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:11:58.901 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:58 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:11:58.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:58 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:11:59.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:58 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:11:59.559 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:11:59.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:59 smithi078 ceph-mon[31157]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:11:59.858 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:11:59 smithi078 ceph-mon[31157]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:11:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:59 smithi193 ceph-mon[35484]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:11:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:11:59 smithi193 ceph-mon[35484]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:11:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:59 smithi148 ceph-mon[35599]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:11:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:11:59 smithi148 ceph-mon[35599]: pgmap v314: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 14 op/s 2024-09-16T00:12:00.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 -- 172.21.15.78:0/3605888516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100e10 msgr2=0x7fcc28104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 --2- 172.21.15.78:0/3605888516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100e10 0x7fcc28104e60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009900 tx=0x7fcc1c02b520 comp rx=0 tx=0).stop 2024-09-16T00:12:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 -- 172.21.15.78:0/3605888516 shutdown_connections 2024-09-16T00:12:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 --2- 172.21.15.78:0/3605888516 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc28105590 0x7fcc2810d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 --2- 172.21.15.78:0/3605888516 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100e10 0x7fcc28104e60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 --2- 172.21.15.78:0/3605888516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc28100460 0x7fcc28100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 -- 172.21.15.78:0/3605888516 >> 172.21.15.78:0/3605888516 conn(0x7fcc280fbee0 msgr2=0x7fcc280fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 -- 172.21.15.78:0/3605888516 shutdown_connections 2024-09-16T00:12:00.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.056+0000 7fcc2d743640 1 -- 172.21.15.78:0/3605888516 wait complete. 2024-09-16T00:12:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.057+0000 7fcc2d743640 1 Processor -- start 2024-09-16T00:12:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.057+0000 7fcc2d743640 1 -- start start 2024-09-16T00:12:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc2d743640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100460 0x7fcc28196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc2d743640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc28100e10 0x7fcc281967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc2d743640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc28105590 0x7fcc2819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc2d743640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810fdb0 con 0x7fcc28100460 2024-09-16T00:12:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc2d743640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810fc30 con 0x7fcc28100e10 2024-09-16T00:12:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc2d743640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2810ff30 con 0x7fcc28105590 2024-09-16T00:12:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100460 0x7fcc28196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100460 0x7fcc28196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47082/0 (socket says 172.21.15.78:47082) 2024-09-16T00:12:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 -- 172.21.15.78:0/4099823176 learned_addr learned my addr 172.21.15.78:0/4099823176 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc277fe640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc28105590 0x7fcc2819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc267fc640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc28100e10 0x7fcc281967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 -- 172.21.15.78:0/4099823176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc28105590 msgr2=0x7fcc2819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc28105590 0x7fcc2819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 -- 172.21.15.78:0/4099823176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc28100e10 msgr2=0x7fcc281967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc28100e10 0x7fcc281967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 -- 172.21.15.78:0/4099823176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2819b240 con 0x7fcc28100460 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc26ffd640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100460 0x7fcc28196290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800d770 tx=0x7fcc1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc0ffff640 1 -- 172.21.15.78:0/4099823176 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18017070 con 0x7fcc28100460 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc0ffff640 1 -- 172.21.15.78:0/4099823176 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc18005500 con 0x7fcc28100460 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2819b530 con 0x7fcc28100460 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc0ffff640 1 -- 172.21.15.78:0/4099823176 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18004e00 con 0x7fcc28100460 2024-09-16T00:12:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.058+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc281a2e10 con 0x7fcc28100460 2024-09-16T00:12:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.059+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc28100840 con 0x7fcc28100460 2024-09-16T00:12:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.061+0000 7fcc0ffff640 1 -- 172.21.15.78:0/4099823176 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcc18004fa0 con 0x7fcc28100460 2024-09-16T00:12:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.061+0000 7fcc0ffff640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc100a32d0 0x7fcc100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.061+0000 7fcc267fc640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc100a32d0 0x7fcc100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.061+0000 7fcc0ffff640 1 -- 172.21.15.78:0/4099823176 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fcc180cfb90 con 0x7fcc28100460 2024-09-16T00:12:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.062+0000 7fcc267fc640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc100a32d0 0x7fcc100a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcc280fc300 tx=0x7fcc1c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.064+0000 7fcc0ffff640 1 -- 172.21.15.78:0/4099823176 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc1809c510 con 0x7fcc28100460 2024-09-16T00:12:00.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.255+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc28062630 con 0x7fcc100a32d0 2024-09-16T00:12:00.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.263+0000 7fcc0ffff640 1 -- 172.21.15.78:0/4099823176 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340375 (secure 0 0 0) 0x7fcc28062630 con 0x7fcc100a32d0 2024-09-16T00:12:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:00.269 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:12:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc100a32d0 msgr2=0x7fcc100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc100a32d0 0x7fcc100a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcc280fc300 tx=0x7fcc1c005a20 comp rx=0 tx=0).stop 2024-09-16T00:12:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100460 msgr2=0x7fcc28196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100460 0x7fcc28196290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800d770 tx=0x7fcc1800dc30 comp rx=0 tx=0).stop 2024-09-16T00:12:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 shutdown_connections 2024-09-16T00:12:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcc100a32d0 0x7fcc100a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc28105590 0x7fcc2819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc28100e10 0x7fcc281967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 --2- 172.21.15.78:0/4099823176 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc28100460 0x7fcc28196290 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 >> 172.21.15.78:0/4099823176 conn(0x7fcc280fbee0 msgr2=0x7fcc280fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:00.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 shutdown_connections 2024-09-16T00:12:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:00.267+0000 7fcc2d743640 1 -- 172.21.15.78:0/4099823176 wait complete. 2024-09-16T00:12:00.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:00 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:00 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.958 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.959 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:00 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:00.987 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-16T00:11:58.406679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650223,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":290,"num_read_kb":258,"num_write":428,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358620,"kb_used_data":13532,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583204,"statfs":{"total":1151940427776,"available":1151573200896,"internally_reserved":0,"allocated":13856768,"data_stored":7645883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50261,"num_objects":9,"num_object_clones":0,"num_object_copies":29,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":133,"num_read_kb":120,"num_write":45,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003319"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:53.564307+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:11:53.564307+0000","last_peered":"2024-09-16T00:11:53.564307+0000","last_clean":"2024-09-16T00:11:53.564307+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:11:53.564307+0000","last_undegraded":"2024-09-16T00:11:53.564307+0000","last_fullsized":"2024-09-16T00:11:53.564307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685467+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:11:14.685467+0000","last_peered":"2024-09-16T00:11:14.685467+0000","last_clean":"2024-09-16T00:11:14.685467+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:11:14.685467+0000","last_undegraded":"2024-09-16T00:11:14.685467+0000","last_fullsized":"2024-09-16T00:11:14.685467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318059+0000","last_change":"2024-09-16T00:11:11.688171+0000","last_active":"2024-09-16T00:11:15.318059+0000","last_peered":"2024-09-16T00:11:15.318059+0000","last_clean":"2024-09-16T00:11:15.318059+0000","last_became_active":"2024-09-16T00:11:11.687750+0000","last_became_peered":"2024-09-16T00:11:11.687750+0000","last_unstale":"2024-09-16T00:11:15.318059+0000","last_undegraded":"2024-09-16T00:11:15.318059+0000","last_fullsized":"2024-09-16T00:11:15.318059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:32:06.177592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685585+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:11:14.685585+0000","last_peered":"2024-09-16T00:11:14.685585+0000","last_clean":"2024-09-16T00:11:14.685585+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:11:14.685585+0000","last_undegraded":"2024-09-16T00:11:14.685585+0000","last_fullsized":"2024-09-16T00:11:14.685585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377631+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:11:15.377631+0000","last_peered":"2024-09-16T00:11:15.377631+0000","last_clean":"2024-09-16T00:11:15.377631+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:11:15.377631+0000","last_undegraded":"2024-09-16T00:11:15.377631+0000","last_fullsized":"2024-09-16T00:11:15.377631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685764+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:11:14.685764+0000","last_peered":"2024-09-16T00:11:14.685764+0000","last_clean":"2024-09-16T00:11:14.685764+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:11:14.685764+0000","last_undegraded":"2024-09-16T00:11:14.685764+0000","last_fullsized":"2024-09-16T00:11:14.685764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:43.549348+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:11:43.549348+0000","last_peered":"2024-09-16T00:11:43.549348+0000","last_clean":"2024-09-16T00:11:43.549348+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:11:43.549348+0000","last_undegraded":"2024-09-16T00:11:43.549348+0000","last_fullsized":"2024-09-16T00:11:43.549348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376188+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:11:15.376188+0000","last_peered":"2024-09-16T00:11:15.376188+0000","last_clean":"2024-09-16T00:11:15.376188+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:11:15.376188+0000","last_undegraded":"2024-09-16T00:11:15.376188+0000","last_fullsized":"2024-09-16T00:11:15.376188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686058+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:11:14.686058+0000","last_peered":"2024-09-16T00:11:14.686058+0000","last_clean":"2024-09-16T00:11:14.686058+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:11:14.686058+0000","last_undegraded":"2024-09-16T00:11:14.686058+0000","last_fullsized":"2024-09-16T00:11:14.686058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.544736+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:11:42.544736+0000","last_peered":"2024-09-16T00:11:42.544736+0000","last_clean":"2024-09-16T00:11:42.544736+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:11:42.544736+0000","last_undegraded":"2024-09-16T00:11:42.544736+0000","last_fullsized":"2024-09-16T00:11:42.544736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685925+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:11:14.685925+0000","last_peered":"2024-09-16T00:11:14.685925+0000","last_clean":"2024-09-16T00:11:14.685925+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:11:14.685925+0000","last_undegraded":"2024-09-16T00:11:14.685925+0000","last_fullsized":"2024-09-16T00:11:14.685925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685874+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:11:14.685874+0000","last_peered":"2024-09-16T00:11:14.685874+0000","last_clean":"2024-09-16T00:11:14.685874+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:11:14.685874+0000","last_undegraded":"2024-09-16T00:11:14.685874+0000","last_fullsized":"2024-09-16T00:11:14.685874+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377210+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:11:15.377210+0000","last_peered":"2024-09-16T00:11:15.377210+0000","last_clean":"2024-09-16T00:11:15.377210+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:11:15.377210+0000","last_undegraded":"2024-09-16T00:11:15.377210+0000","last_fullsized":"2024-09-16T00:11:15.377210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:39.508906+0000","last_change":"2024-09-16T00:11:02.617063+0000","last_active":"2024-09-16T00:11:39.508906+0000","last_peered":"2024-09-16T00:11:39.508906+0000","last_clean":"2024-09-16T00:11:39.508906+0000","last_became_active":"2024-09-16T00:11:02.616550+0000","last_became_peered":"2024-09-16T00:11:02.616550+0000","last_unstale":"2024-09-16T00:11:39.508906+0000","last_undegraded":"2024-09-16T00:11:39.508906+0000","last_fullsized":"2024-09-16T00:11:39.508906+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:32.325698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686298+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:11:14.686298+0000","last_peered":"2024-09-16T00:11:14.686298+0000","last_clean":"2024-09-16T00:11:14.686298+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:11:14.686298+0000","last_undegraded":"2024-09-16T00:11:14.686298+0000","last_fullsized":"2024-09-16T00:11:14.686298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685102+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:11:14.685102+0000","last_peered":"2024-09-16T00:11:14.685102+0000","last_clean":"2024-09-16T00:11:14.685102+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:11:14.685102+0000","last_undegraded":"2024-09-16T00:11:14.685102+0000","last_fullsized":"2024-09-16T00:11:14.685102+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:41.531417+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:11:41.531417+0000","last_peered":"2024-09-16T00:11:41.531417+0000","last_clean":"2024-09-16T00:11:41.531417+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:11:41.531417+0000","last_undegraded":"2024-09-16T00:11:41.531417+0000","last_fullsized":"2024-09-16T00:11:41.531417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686231+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:11:14.686231+0000","last_peered":"2024-09-16T00:11:14.686231+0000","last_clean":"2024-09-16T00:11:14.686231+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:11:14.686231+0000","last_undegraded":"2024-09-16T00:11:14.686231+0000","last_fullsized":"2024-09-16T00:11:14.686231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764483+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:11:16.764483+0000","last_peered":"2024-09-16T00:11:16.764483+0000","last_clean":"2024-09-16T00:11:16.764483+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:11:16.764483+0000","last_undegraded":"2024-09-16T00:11:16.764483+0000","last_fullsized":"2024-09-16T00:11:16.764483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543231+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:11:42.543231+0000","last_peered":"2024-09-16T00:11:42.543231+0000","last_clean":"2024-09-16T00:11:42.543231+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:11:42.543231+0000","last_undegraded":"2024-09-16T00:11:42.543231+0000","last_fullsized":"2024-09-16T00:11:42.543231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376607+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:11:15.376607+0000","last_peered":"2024-09-16T00:11:15.376607+0000","last_clean":"2024-09-16T00:11:15.376607+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:11:15.376607+0000","last_undegraded":"2024-09-16T00:11:15.376607+0000","last_fullsized":"2024-09-16T00:11:15.376607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375598+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:11:15.375598+0000","last_peered":"2024-09-16T00:11:15.375598+0000","last_clean":"2024-09-16T00:11:15.375598+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:11:15.375598+0000","last_undegraded":"2024-09-16T00:11:15.375598+0000","last_fullsized":"2024-09-16T00:11:15.375598+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:39.507685+0000","last_change":"2024-09-16T00:11:02.809403+0000","last_active":"2024-09-16T00:11:39.507685+0000","last_peered":"2024-09-16T00:11:39.507685+0000","last_clean":"2024-09-16T00:11:39.507685+0000","last_became_active":"2024-09-16T00:11:02.809066+0000","last_became_peered":"2024-09-16T00:11:02.809066+0000","last_unstale":"2024-09-16T00:11:39.507685+0000","last_undegraded":"2024-09-16T00:11:39.507685+0000","last_fullsized":"2024-09-16T00:11:39.507685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:58.544890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686467+0000","last_change":"2024-09-16T00:11:11.682995+0000","last_active":"2024-09-16T00:11:14.686467+0000","last_peered":"2024-09-16T00:11:14.686467+0000","last_clean":"2024-09-16T00:11:14.686467+0000","last_became_active":"2024-09-16T00:11:11.682592+0000","last_became_peered":"2024-09-16T00:11:11.682592+0000","last_unstale":"2024-09-16T00:11:14.686467+0000","last_undegraded":"2024-09-16T00:11:14.686467+0000","last_fullsized":"2024-09-16T00:11:14.686467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.159350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375997+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:11:15.375997+0000","last_peered":"2024-09-16T00:11:15.375997+0000","last_clean":"2024-09-16T00:11:15.375997+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:11:15.375997+0000","last_undegraded":"2024-09-16T00:11:15.375997+0000","last_fullsized":"2024-09-16T00:11:15.375997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543401+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:11:42.543401+0000","last_peered":"2024-09-16T00:11:42.543401+0000","last_clean":"2024-09-16T00:11:42.543401+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:11:42.543401+0000","last_undegraded":"2024-09-16T00:11:42.543401+0000","last_fullsized":"2024-09-16T00:11:42.543401+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318059+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:11:15.318059+0000","last_peered":"2024-09-16T00:11:15.318059+0000","last_clean":"2024-09-16T00:11:15.318059+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:11:15.318059+0000","last_undegraded":"2024-09-16T00:11:15.318059+0000","last_fullsized":"2024-09-16T00:11:15.318059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685434+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:11:14.685434+0000","last_peered":"2024-09-16T00:11:14.685434+0000","last_clean":"2024-09-16T00:11:14.685434+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:11:14.685434+0000","last_undegraded":"2024-09-16T00:11:14.685434+0000","last_fullsized":"2024-09-16T00:11:14.685434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'15","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:53.640204+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:11:53.640204+0000","last_peered":"2024-09-16T00:11:53.640204+0000","last_clean":"2024-09-16T00:11:53.640204+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:11:53.640204+0000","last_undegraded":"2024-09-16T00:11:53.640204+0000","last_fullsized":"2024-09-16T00:11:53.640204+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.684901+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:11:14.684901+0000","last_peered":"2024-09-16T00:11:14.684901+0000","last_clean":"2024-09-16T00:11:14.684901+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:11:14.684901+0000","last_undegraded":"2024-09-16T00:11:14.684901+0000","last_fullsized":"2024-09-16T00:11:14.684901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376421+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:11:15.376421+0000","last_peered":"2024-09-16T00:11:15.376421+0000","last_clean":"2024-09-16T00:11:15.376421+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:11:15.376421+0000","last_undegraded":"2024-09-16T00:11:15.376421+0000","last_fullsized":"2024-09-16T00:11:15.376421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.544554+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:11:42.544554+0000","last_peered":"2024-09-16T00:11:42.544554+0000","last_clean":"2024-09-16T00:11:42.544554+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:11:42.544554+0000","last_undegraded":"2024-09-16T00:11:42.544554+0000","last_fullsized":"2024-09-16T00:11:42.544554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764536+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:11:16.764536+0000","last_peered":"2024-09-16T00:11:16.764536+0000","last_clean":"2024-09-16T00:11:16.764536+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:11:16.764536+0000","last_undegraded":"2024-09-16T00:11:16.764536+0000","last_fullsized":"2024-09-16T00:11:16.764536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375889+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:11:15.375889+0000","last_peered":"2024-09-16T00:11:15.375889+0000","last_clean":"2024-09-16T00:11:15.375889+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:11:15.375889+0000","last_undegraded":"2024-09-16T00:11:15.375889+0000","last_fullsized":"2024-09-16T00:11:15.375889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376869+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:11:15.376869+0000","last_peered":"2024-09-16T00:11:15.376869+0000","last_clean":"2024-09-16T00:11:15.376869+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:11:15.376869+0000","last_undegraded":"2024-09-16T00:11:15.376869+0000","last_fullsized":"2024-09-16T00:11:15.376869+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375705+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:11:15.375705+0000","last_peered":"2024-09-16T00:11:15.375705+0000","last_clean":"2024-09-16T00:11:15.375705+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:11:15.375705+0000","last_undegraded":"2024-09-16T00:11:15.375705+0000","last_fullsized":"2024-09-16T00:11:15.375705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376002+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:11:15.376002+0000","last_peered":"2024-09-16T00:11:15.376002+0000","last_clean":"2024-09-16T00:11:15.376002+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:11:15.376002+0000","last_undegraded":"2024-09-16T00:11:15.376002+0000","last_fullsized":"2024-09-16T00:11:15.376002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:51.562695+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:11:51.562695+0000","last_peered":"2024-09-16T00:11:51.562695+0000","last_clean":"2024-09-16T00:11:51.562695+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:11:51.562695+0000","last_undegraded":"2024-09-16T00:11:51.562695+0000","last_fullsized":"2024-09-16T00:11:51.562695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376075+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:11:15.376075+0000","last_peered":"2024-09-16T00:11:15.376075+0000","last_clean":"2024-09-16T00:11:15.376075+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:11:15.376075+0000","last_undegraded":"2024-09-16T00:11:15.376075+0000","last_fullsized":"2024-09-16T00:11:15.376075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685530+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:11:14.685530+0000","last_peered":"2024-09-16T00:11:14.685530+0000","last_clean":"2024-09-16T00:11:14.685530+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:11:14.685530+0000","last_undegraded":"2024-09-16T00:11:14.685530+0000","last_fullsized":"2024-09-16T00:11:14.685530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:56.569331+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:11:56.569331+0000","last_peered":"2024-09-16T00:11:56.569331+0000","last_clean":"2024-09-16T00:11:56.569331+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:11:56.569331+0000","last_undegraded":"2024-09-16T00:11:56.569331+0000","last_fullsized":"2024-09-16T00:11:56.569331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376211+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:11:15.376211+0000","last_peered":"2024-09-16T00:11:15.376211+0000","last_clean":"2024-09-16T00:11:15.376211+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:11:15.376211+0000","last_undegraded":"2024-09-16T00:11:15.376211+0000","last_fullsized":"2024-09-16T00:11:15.376211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375278+0000","last_change":"2024-09-16T00:11:04.641679+0000","last_active":"2024-09-16T00:11:15.375278+0000","last_peered":"2024-09-16T00:11:15.375278+0000","last_clean":"2024-09-16T00:11:15.375278+0000","last_became_active":"2024-09-16T00:11:04.641333+0000","last_became_peered":"2024-09-16T00:11:04.641333+0000","last_unstale":"2024-09-16T00:11:15.375278+0000","last_undegraded":"2024-09-16T00:11:15.375278+0000","last_fullsized":"2024-09-16T00:11:15.375278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:35:52.103553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:40.518041+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:11:40.518041+0000","last_peered":"2024-09-16T00:11:40.518041+0000","last_clean":"2024-09-16T00:11:40.518041+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:11:40.518041+0000","last_undegraded":"2024-09-16T00:11:40.518041+0000","last_fullsized":"2024-09-16T00:11:40.518041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686102+0000","last_change":"2024-09-16T00:11:11.677781+0000","last_active":"2024-09-16T00:11:14.686102+0000","last_peered":"2024-09-16T00:11:14.686102+0000","last_clean":"2024-09-16T00:11:14.686102+0000","last_became_active":"2024-09-16T00:11:11.677032+0000","last_became_peered":"2024-09-16T00:11:11.677032+0000","last_unstale":"2024-09-16T00:11:14.686102+0000","last_undegraded":"2024-09-16T00:11:14.686102+0000","last_fullsized":"2024-09-16T00:11:14.686102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:57.523334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685061+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:11:14.685061+0000","last_peered":"2024-09-16T00:11:14.685061+0000","last_clean":"2024-09-16T00:11:14.685061+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:11:14.685061+0000","last_undegraded":"2024-09-16T00:11:14.685061+0000","last_fullsized":"2024-09-16T00:11:14.685061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:43.548706+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:11:43.548706+0000","last_peered":"2024-09-16T00:11:43.548706+0000","last_clean":"2024-09-16T00:11:43.548706+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:11:43.548706+0000","last_undegraded":"2024-09-16T00:11:43.548706+0000","last_fullsized":"2024-09-16T00:11:43.548706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376286+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:11:15.376286+0000","last_peered":"2024-09-16T00:11:15.376286+0000","last_clean":"2024-09-16T00:11:15.376286+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:11:15.376286+0000","last_undegraded":"2024-09-16T00:11:15.376286+0000","last_fullsized":"2024-09-16T00:11:15.376286+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377101+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:11:15.377101+0000","last_peered":"2024-09-16T00:11:15.377101+0000","last_clean":"2024-09-16T00:11:15.377101+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:11:15.377101+0000","last_undegraded":"2024-09-16T00:11:15.377101+0000","last_fullsized":"2024-09-16T00:11:15.377101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377354+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:11:15.377354+0000","last_peered":"2024-09-16T00:11:15.377354+0000","last_clean":"2024-09-16T00:11:15.377354+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:11:15.377354+0000","last_undegraded":"2024-09-16T00:11:15.377354+0000","last_fullsized":"2024-09-16T00:11:15.377354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:53.565579+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:11:53.565579+0000","last_peered":"2024-09-16T00:11:53.565579+0000","last_clean":"2024-09-16T00:11:53.565579+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:11:53.565579+0000","last_undegraded":"2024-09-16T00:11:53.565579+0000","last_fullsized":"2024-09-16T00:11:53.565579+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376721+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:11:15.376721+0000","last_peered":"2024-09-16T00:11:15.376721+0000","last_clean":"2024-09-16T00:11:15.376721+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:11:15.376721+0000","last_undegraded":"2024-09-16T00:11:15.376721+0000","last_fullsized":"2024-09-16T00:11:15.376721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376453+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:11:15.376453+0000","last_peered":"2024-09-16T00:11:15.376453+0000","last_clean":"2024-09-16T00:11:15.376453+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:11:15.376453+0000","last_undegraded":"2024-09-16T00:11:15.376453+0000","last_fullsized":"2024-09-16T00:11:15.376453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543483+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:11:42.543483+0000","last_peered":"2024-09-16T00:11:42.543483+0000","last_clean":"2024-09-16T00:11:42.543483+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:11:42.543483+0000","last_undegraded":"2024-09-16T00:11:42.543483+0000","last_fullsized":"2024-09-16T00:11:42.543483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376386+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:11:15.376386+0000","last_peered":"2024-09-16T00:11:15.376386+0000","last_clean":"2024-09-16T00:11:15.376386+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:11:15.376386+0000","last_undegraded":"2024-09-16T00:11:15.376386+0000","last_fullsized":"2024-09-16T00:11:15.376386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376338+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:11:15.376338+0000","last_peered":"2024-09-16T00:11:15.376338+0000","last_clean":"2024-09-16T00:11:15.376338+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:11:15.376338+0000","last_undegraded":"2024-09-16T00:11:15.376338+0000","last_fullsized":"2024-09-16T00:11:15.376338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543297+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:11:42.543297+0000","last_peered":"2024-09-16T00:11:42.543297+0000","last_clean":"2024-09-16T00:11:42.543297+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:11:42.543297+0000","last_undegraded":"2024-09-16T00:11:42.543297+0000","last_fullsized":"2024-09-16T00:11:42.543297+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685402+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:11:14.685402+0000","last_peered":"2024-09-16T00:11:14.685402+0000","last_clean":"2024-09-16T00:11:14.685402+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:11:14.685402+0000","last_undegraded":"2024-09-16T00:11:14.685402+0000","last_fullsized":"2024-09-16T00:11:14.685402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764778+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:11:16.764778+0000","last_peered":"2024-09-16T00:11:16.764778+0000","last_clean":"2024-09-16T00:11:16.764778+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:11:16.764778+0000","last_undegraded":"2024-09-16T00:11:16.764778+0000","last_fullsized":"2024-09-16T00:11:16.764778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.513120+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:11:42.513120+0000","last_peered":"2024-09-16T00:11:42.513120+0000","last_clean":"2024-09-16T00:11:42.513120+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:11:42.513120+0000","last_undegraded":"2024-09-16T00:11:42.513120+0000","last_fullsized":"2024-09-16T00:11:42.513120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685512+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:11:14.685512+0000","last_peered":"2024-09-16T00:11:14.685512+0000","last_clean":"2024-09-16T00:11:14.685512+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:11:14.685512+0000","last_undegraded":"2024-09-16T00:11:14.685512+0000","last_fullsized":"2024-09-16T00:11:14.685512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685716+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:11:14.685716+0000","last_peered":"2024-09-16T00:11:14.685716+0000","last_clean":"2024-09-16T00:11:14.685716+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:11:14.685716+0000","last_undegraded":"2024-09-16T00:11:14.685716+0000","last_fullsized":"2024-09-16T00:11:14.685716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543932+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:11:42.543932+0000","last_peered":"2024-09-16T00:11:42.543932+0000","last_clean":"2024-09-16T00:11:42.543932+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:11:42.543932+0000","last_undegraded":"2024-09-16T00:11:42.543932+0000","last_fullsized":"2024-09-16T00:11:42.543932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376353+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:11:15.376353+0000","last_peered":"2024-09-16T00:11:15.376353+0000","last_clean":"2024-09-16T00:11:15.376353+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:11:15.376353+0000","last_undegraded":"2024-09-16T00:11:15.376353+0000","last_fullsized":"2024-09-16T00:11:15.376353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375923+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:11:15.375923+0000","last_peered":"2024-09-16T00:11:15.375923+0000","last_clean":"2024-09-16T00:11:15.375923+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:11:15.375923+0000","last_undegraded":"2024-09-16T00:11:15.375923+0000","last_fullsized":"2024-09-16T00:11:15.375923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":54,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:56.569923+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:11:56.569923+0000","last_peered":"2024-09-16T00:11:56.569923+0000","last_clean":"2024-09-16T00:11:56.569923+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:11:56.569923+0000","last_undegraded":"2024-09-16T00:11:56.569923+0000","last_fullsized":"2024-09-16T00:11:56.569923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376726+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:11:15.376726+0000","last_peered":"2024-09-16T00:11:15.376726+0000","last_clean":"2024-09-16T00:11:15.376726+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:11:15.376726+0000","last_undegraded":"2024-09-16T00:11:15.376726+0000","last_fullsized":"2024-09-16T00:11:15.376726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377353+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:11:15.377353+0000","last_peered":"2024-09-16T00:11:15.377353+0000","last_clean":"2024-09-16T00:11:15.377353+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:11:15.377353+0000","last_undegraded":"2024-09-16T00:11:15.377353+0000","last_fullsized":"2024-09-16T00:11:15.377353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.543903+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:11:42.543903+0000","last_peered":"2024-09-16T00:11:42.543903+0000","last_clean":"2024-09-16T00:11:42.543903+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:11:42.543903+0000","last_undegraded":"2024-09-16T00:11:42.543903+0000","last_fullsized":"2024-09-16T00:11:42.543903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764742+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:11:16.764742+0000","last_peered":"2024-09-16T00:11:16.764742+0000","last_clean":"2024-09-16T00:11:16.764742+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:11:16.764742+0000","last_undegraded":"2024-09-16T00:11:16.764742+0000","last_fullsized":"2024-09-16T00:11:16.764742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375612+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:11:15.375612+0000","last_peered":"2024-09-16T00:11:15.375612+0000","last_clean":"2024-09-16T00:11:15.375612+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:11:15.375612+0000","last_undegraded":"2024-09-16T00:11:15.375612+0000","last_fullsized":"2024-09-16T00:11:15.375612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:43.548732+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:11:43.548732+0000","last_peered":"2024-09-16T00:11:43.548732+0000","last_clean":"2024-09-16T00:11:43.548732+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:11:43.548732+0000","last_undegraded":"2024-09-16T00:11:43.548732+0000","last_fullsized":"2024-09-16T00:11:43.548732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686090+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:11:14.686090+0000","last_peered":"2024-09-16T00:11:14.686090+0000","last_clean":"2024-09-16T00:11:14.686090+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:11:14.686090+0000","last_undegraded":"2024-09-16T00:11:14.686090+0000","last_fullsized":"2024-09-16T00:11:14.686090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375927+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:11:15.375927+0000","last_peered":"2024-09-16T00:11:15.375927+0000","last_clean":"2024-09-16T00:11:15.375927+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:11:15.375927+0000","last_undegraded":"2024-09-16T00:11:15.375927+0000","last_fullsized":"2024-09-16T00:11:15.375927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'32","reported_seq":93,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:54.568755+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:11:54.568755+0000","last_peered":"2024-09-16T00:11:54.568755+0000","last_clean":"2024-09-16T00:11:54.568755+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:11:54.568755+0000","last_undegraded":"2024-09-16T00:11:54.568755+0000","last_fullsized":"2024-09-16T00:11:54.568755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52413,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":31,"num_read_kb":66,"num_write":34,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377704+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:11:15.377704+0000","last_peered":"2024-09-16T00:11:15.377704+0000","last_clean":"2024-09-16T00:11:15.377704+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:11:15.377704+0000","last_undegraded":"2024-09-16T00:11:15.377704+0000","last_fullsized":"2024-09-16T00:11:15.377704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376300+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:11:15.376300+0000","last_peered":"2024-09-16T00:11:15.376300+0000","last_clean":"2024-09-16T00:11:15.376300+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:11:15.376300+0000","last_undegraded":"2024-09-16T00:11:15.376300+0000","last_fullsized":"2024-09-16T00:11:15.376300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:41.530675+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:11:41.530675+0000","last_peered":"2024-09-16T00:11:41.530675+0000","last_clean":"2024-09-16T00:11:41.530675+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:11:41.530675+0000","last_undegraded":"2024-09-16T00:11:41.530675+0000","last_fullsized":"2024-09-16T00:11:41.530675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686161+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:11:14.686161+0000","last_peered":"2024-09-16T00:11:14.686161+0000","last_clean":"2024-09-16T00:11:14.686161+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:11:14.686161+0000","last_undegraded":"2024-09-16T00:11:14.686161+0000","last_fullsized":"2024-09-16T00:11:14.686161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377685+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:11:15.377685+0000","last_peered":"2024-09-16T00:11:15.377685+0000","last_clean":"2024-09-16T00:11:15.377685+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:11:15.377685+0000","last_undegraded":"2024-09-16T00:11:15.377685+0000","last_fullsized":"2024-09-16T00:11:15.377685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.546096+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:11:42.546096+0000","last_peered":"2024-09-16T00:11:42.546096+0000","last_clean":"2024-09-16T00:11:42.546096+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:11:42.546096+0000","last_undegraded":"2024-09-16T00:11:42.546096+0000","last_fullsized":"2024-09-16T00:11:42.546096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377451+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:11:15.377451+0000","last_peered":"2024-09-16T00:11:15.377451+0000","last_clean":"2024-09-16T00:11:15.377451+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:11:15.377451+0000","last_undegraded":"2024-09-16T00:11:15.377451+0000","last_fullsized":"2024-09-16T00:11:15.377451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376585+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:11:15.376585+0000","last_peered":"2024-09-16T00:11:15.376585+0000","last_clean":"2024-09-16T00:11:15.376585+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:11:15.376585+0000","last_undegraded":"2024-09-16T00:11:15.376585+0000","last_fullsized":"2024-09-16T00:11:15.376585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":58,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:54.566275+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:11:54.566275+0000","last_peered":"2024-09-16T00:11:54.566275+0000","last_clean":"2024-09-16T00:11:54.566275+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:11:54.566275+0000","last_undegraded":"2024-09-16T00:11:54.566275+0000","last_fullsized":"2024-09-16T00:11:54.566275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685044+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:11:14.685044+0000","last_peered":"2024-09-16T00:11:14.685044+0000","last_clean":"2024-09-16T00:11:14.685044+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:11:14.685044+0000","last_undegraded":"2024-09-16T00:11:14.685044+0000","last_fullsized":"2024-09-16T00:11:14.685044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375852+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:11:15.375852+0000","last_peered":"2024-09-16T00:11:15.375852+0000","last_clean":"2024-09-16T00:11:15.375852+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:11:15.375852+0000","last_undegraded":"2024-09-16T00:11:15.375852+0000","last_fullsized":"2024-09-16T00:11:15.375852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376620+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:11:15.376620+0000","last_peered":"2024-09-16T00:11:15.376620+0000","last_clean":"2024-09-16T00:11:15.376620+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:11:15.376620+0000","last_undegraded":"2024-09-16T00:11:15.376620+0000","last_fullsized":"2024-09-16T00:11:15.376620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.544126+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:11:42.544126+0000","last_peered":"2024-09-16T00:11:42.544126+0000","last_clean":"2024-09-16T00:11:42.544126+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:11:42.544126+0000","last_undegraded":"2024-09-16T00:11:42.544126+0000","last_fullsized":"2024-09-16T00:11:42.544126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764814+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:11:16.764814+0000","last_peered":"2024-09-16T00:11:16.764814+0000","last_clean":"2024-09-16T00:11:16.764814+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:11:16.764814+0000","last_undegraded":"2024-09-16T00:11:16.764814+0000","last_fullsized":"2024-09-16T00:11:16.764814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685199+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:11:14.685199+0000","last_peered":"2024-09-16T00:11:14.685199+0000","last_clean":"2024-09-16T00:11:14.685199+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:11:14.685199+0000","last_undegraded":"2024-09-16T00:11:14.685199+0000","last_fullsized":"2024-09-16T00:11:14.685199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:40.517394+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:11:40.517394+0000","last_peered":"2024-09-16T00:11:40.517394+0000","last_clean":"2024-09-16T00:11:40.517394+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:11:40.517394+0000","last_undegraded":"2024-09-16T00:11:40.517394+0000","last_fullsized":"2024-09-16T00:11:40.517394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:41.531601+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:11:41.531601+0000","last_peered":"2024-09-16T00:11:41.531601+0000","last_clean":"2024-09-16T00:11:41.531601+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:11:41.531601+0000","last_undegraded":"2024-09-16T00:11:41.531601+0000","last_fullsized":"2024-09-16T00:11:41.531601+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685847+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:11:14.685847+0000","last_peered":"2024-09-16T00:11:14.685847+0000","last_clean":"2024-09-16T00:11:14.685847+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:11:14.685847+0000","last_undegraded":"2024-09-16T00:11:14.685847+0000","last_fullsized":"2024-09-16T00:11:14.685847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318337+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:11:15.318337+0000","last_peered":"2024-09-16T00:11:15.318337+0000","last_clean":"2024-09-16T00:11:15.318337+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:11:15.318337+0000","last_undegraded":"2024-09-16T00:11:15.318337+0000","last_fullsized":"2024-09-16T00:11:15.318337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:54.566917+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:11:54.566917+0000","last_peered":"2024-09-16T00:11:54.566917+0000","last_clean":"2024-09-16T00:11:54.566917+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:11:54.566917+0000","last_undegraded":"2024-09-16T00:11:54.566917+0000","last_fullsized":"2024-09-16T00:11:54.566917+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375752+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:11:15.375752+0000","last_peered":"2024-09-16T00:11:15.375752+0000","last_clean":"2024-09-16T00:11:15.375752+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:11:15.375752+0000","last_undegraded":"2024-09-16T00:11:15.375752+0000","last_fullsized":"2024-09-16T00:11:15.375752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318426+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:11:15.318426+0000","last_peered":"2024-09-16T00:11:15.318426+0000","last_clean":"2024-09-16T00:11:15.318426+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:11:15.318426+0000","last_undegraded":"2024-09-16T00:11:15.318426+0000","last_fullsized":"2024-09-16T00:11:15.318426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764381+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:11:16.764381+0000","last_peered":"2024-09-16T00:11:16.764381+0000","last_clean":"2024-09-16T00:11:16.764381+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:11:16.764381+0000","last_undegraded":"2024-09-16T00:11:16.764381+0000","last_fullsized":"2024-09-16T00:11:16.764381+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318388+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:11:15.318388+0000","last_peered":"2024-09-16T00:11:15.318388+0000","last_clean":"2024-09-16T00:11:15.318388+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:11:15.318388+0000","last_undegraded":"2024-09-16T00:11:15.318388+0000","last_fullsized":"2024-09-16T00:11:15.318388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.317945+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:11:15.317945+0000","last_peered":"2024-09-16T00:11:15.317945+0000","last_clean":"2024-09-16T00:11:15.317945+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:11:15.317945+0000","last_undegraded":"2024-09-16T00:11:15.317945+0000","last_fullsized":"2024-09-16T00:11:15.317945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376436+0000","last_change":"2024-09-16T00:11:04.641885+0000","last_active":"2024-09-16T00:11:15.376436+0000","last_peered":"2024-09-16T00:11:15.376436+0000","last_clean":"2024-09-16T00:11:15.376436+0000","last_became_active":"2024-09-16T00:11:04.641496+0000","last_became_peered":"2024-09-16T00:11:04.641496+0000","last_unstale":"2024-09-16T00:11:15.376436+0000","last_undegraded":"2024-09-16T00:11:15.376436+0000","last_fullsized":"2024-09-16T00:11:15.376436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:40.585243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376212+0000","last_change":"2024-09-16T00:11:04.649287+0000","last_active":"2024-09-16T00:11:15.376212+0000","last_peered":"2024-09-16T00:11:15.376212+0000","last_clean":"2024-09-16T00:11:15.376212+0000","last_became_active":"2024-09-16T00:11:04.648952+0000","last_became_peered":"2024-09-16T00:11:04.648952+0000","last_unstale":"2024-09-16T00:11:15.376212+0000","last_undegraded":"2024-09-16T00:11:15.376212+0000","last_fullsized":"2024-09-16T00:11:15.376212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:03:57.189388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764770+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:11:16.764770+0000","last_peered":"2024-09-16T00:11:16.764770+0000","last_clean":"2024-09-16T00:11:16.764770+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:11:16.764770+0000","last_undegraded":"2024-09-16T00:11:16.764770+0000","last_fullsized":"2024-09-16T00:11:16.764770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318531+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:11:15.318531+0000","last_peered":"2024-09-16T00:11:15.318531+0000","last_clean":"2024-09-16T00:11:15.318531+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:11:15.318531+0000","last_undegraded":"2024-09-16T00:11:15.318531+0000","last_fullsized":"2024-09-16T00:11:15.318531+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685394+0000","last_change":"2024-09-16T00:11:04.649155+0000","last_active":"2024-09-16T00:11:14.685394+0000","last_peered":"2024-09-16T00:11:14.685394+0000","last_clean":"2024-09-16T00:11:14.685394+0000","last_became_active":"2024-09-16T00:11:04.648844+0000","last_became_peered":"2024-09-16T00:11:04.648844+0000","last_unstale":"2024-09-16T00:11:14.685394+0000","last_undegraded":"2024-09-16T00:11:14.685394+0000","last_fullsized":"2024-09-16T00:11:14.685394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:20.205113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685482+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:11:14.685482+0000","last_peered":"2024-09-16T00:11:14.685482+0000","last_clean":"2024-09-16T00:11:14.685482+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:11:14.685482+0000","last_undegraded":"2024-09-16T00:11:14.685482+0000","last_fullsized":"2024-09-16T00:11:14.685482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685108+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:11:14.685108+0000","last_peered":"2024-09-16T00:11:14.685108+0000","last_clean":"2024-09-16T00:11:14.685108+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:11:14.685108+0000","last_undegraded":"2024-09-16T00:11:14.685108+0000","last_fullsized":"2024-09-16T00:11:14.685108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376984+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:11:15.376984+0000","last_peered":"2024-09-16T00:11:15.376984+0000","last_clean":"2024-09-16T00:11:15.376984+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:11:15.376984+0000","last_undegraded":"2024-09-16T00:11:15.376984+0000","last_fullsized":"2024-09-16T00:11:15.376984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375777+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:11:15.375777+0000","last_peered":"2024-09-16T00:11:15.375777+0000","last_clean":"2024-09-16T00:11:15.375777+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:11:15.375777+0000","last_undegraded":"2024-09-16T00:11:15.375777+0000","last_fullsized":"2024-09-16T00:11:15.375777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764663+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:11:16.764663+0000","last_peered":"2024-09-16T00:11:16.764663+0000","last_clean":"2024-09-16T00:11:16.764663+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:11:16.764663+0000","last_undegraded":"2024-09-16T00:11:16.764663+0000","last_fullsized":"2024-09-16T00:11:16.764663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.318511+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:11:15.318511+0000","last_peered":"2024-09-16T00:11:15.318511+0000","last_clean":"2024-09-16T00:11:15.318511+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:11:15.318511+0000","last_undegraded":"2024-09-16T00:11:15.318511+0000","last_fullsized":"2024-09-16T00:11:15.318511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376301+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:11:15.376301+0000","last_peered":"2024-09-16T00:11:15.376301+0000","last_clean":"2024-09-16T00:11:15.376301+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:11:15.376301+0000","last_undegraded":"2024-09-16T00:11:15.376301+0000","last_fullsized":"2024-09-16T00:11:15.376301+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764943+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:11:16.764943+0000","last_peered":"2024-09-16T00:11:16.764943+0000","last_clean":"2024-09-16T00:11:16.764943+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:11:16.764943+0000","last_undegraded":"2024-09-16T00:11:16.764943+0000","last_fullsized":"2024-09-16T00:11:16.764943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376118+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:11:15.376118+0000","last_peered":"2024-09-16T00:11:15.376118+0000","last_clean":"2024-09-16T00:11:15.376118+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:11:15.376118+0000","last_undegraded":"2024-09-16T00:11:15.376118+0000","last_fullsized":"2024-09-16T00:11:15.376118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375790+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:11:15.375790+0000","last_peered":"2024-09-16T00:11:15.375790+0000","last_clean":"2024-09-16T00:11:15.375790+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:11:15.375790+0000","last_undegraded":"2024-09-16T00:11:15.375790+0000","last_fullsized":"2024-09-16T00:11:15.375790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376012+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:11:15.376012+0000","last_peered":"2024-09-16T00:11:15.376012+0000","last_clean":"2024-09-16T00:11:15.376012+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:11:15.376012+0000","last_undegraded":"2024-09-16T00:11:15.376012+0000","last_fullsized":"2024-09-16T00:11:15.376012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375697+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:11:15.375697+0000","last_peered":"2024-09-16T00:11:15.375697+0000","last_clean":"2024-09-16T00:11:15.375697+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:11:15.375697+0000","last_undegraded":"2024-09-16T00:11:15.375697+0000","last_fullsized":"2024-09-16T00:11:15.375697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686296+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:11:14.686296+0000","last_peered":"2024-09-16T00:11:14.686296+0000","last_clean":"2024-09-16T00:11:14.686296+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:11:14.686296+0000","last_undegraded":"2024-09-16T00:11:14.686296+0000","last_fullsized":"2024-09-16T00:11:14.686296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377246+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:11:15.377246+0000","last_peered":"2024-09-16T00:11:15.377246+0000","last_clean":"2024-09-16T00:11:15.377246+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:11:15.377246+0000","last_undegraded":"2024-09-16T00:11:15.377246+0000","last_fullsized":"2024-09-16T00:11:15.377246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376735+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:11:15.376735+0000","last_peered":"2024-09-16T00:11:15.376735+0000","last_clean":"2024-09-16T00:11:15.376735+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:11:15.376735+0000","last_undegraded":"2024-09-16T00:11:15.376735+0000","last_fullsized":"2024-09-16T00:11:15.376735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.377385+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:11:15.377385+0000","last_peered":"2024-09-16T00:11:15.377385+0000","last_clean":"2024-09-16T00:11:15.377385+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:11:15.377385+0000","last_undegraded":"2024-09-16T00:11:15.377385+0000","last_fullsized":"2024-09-16T00:11:15.377385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.685023+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:11:14.685023+0000","last_peered":"2024-09-16T00:11:14.685023+0000","last_clean":"2024-09-16T00:11:14.685023+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:11:14.685023+0000","last_undegraded":"2024-09-16T00:11:14.685023+0000","last_fullsized":"2024-09-16T00:11:14.685023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376994+0000","last_change":"2024-09-16T00:11:04.644081+0000","last_active":"2024-09-16T00:11:15.376994+0000","last_peered":"2024-09-16T00:11:15.376994+0000","last_clean":"2024-09-16T00:11:15.376994+0000","last_became_active":"2024-09-16T00:11:04.643779+0000","last_became_peered":"2024-09-16T00:11:04.643779+0000","last_unstale":"2024-09-16T00:11:15.376994+0000","last_undegraded":"2024-09-16T00:11:15.376994+0000","last_fullsized":"2024-09-16T00:11:15.376994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:56:13.966468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376155+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:11:15.376155+0000","last_peered":"2024-09-16T00:11:15.376155+0000","last_clean":"2024-09-16T00:11:15.376155+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:11:15.376155+0000","last_undegraded":"2024-09-16T00:11:15.376155+0000","last_fullsized":"2024-09-16T00:11:15.376155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375686+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:11:15.375686+0000","last_peered":"2024-09-16T00:11:15.375686+0000","last_clean":"2024-09-16T00:11:15.375686+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:11:15.375686+0000","last_undegraded":"2024-09-16T00:11:15.375686+0000","last_fullsized":"2024-09-16T00:11:15.375686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.686266+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:11:14.686266+0000","last_peered":"2024-09-16T00:11:14.686266+0000","last_clean":"2024-09-16T00:11:14.686266+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:11:14.686266+0000","last_undegraded":"2024-09-16T00:11:14.686266+0000","last_fullsized":"2024-09-16T00:11:14.686266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.376859+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:11:15.376859+0000","last_peered":"2024-09-16T00:11:15.376859+0000","last_clean":"2024-09-16T00:11:15.376859+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:11:15.376859+0000","last_undegraded":"2024-09-16T00:11:15.376859+0000","last_fullsized":"2024-09-16T00:11:15.376859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.375909+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:11:15.375909+0000","last_peered":"2024-09-16T00:11:15.375909+0000","last_clean":"2024-09-16T00:11:15.375909+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:11:15.375909+0000","last_undegraded":"2024-09-16T00:11:15.375909+0000","last_fullsized":"2024-09-16T00:11:15.375909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-16T00:11:16.764909+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:11:16.764909+0000","last_peered":"2024-09-16T00:11:16.764909+0000","last_clean":"2024-09-16T00:11:16.764909+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:11:16.764909+0000","last_undegraded":"2024-09-16T00:11:16.764909+0000","last_fullsized":"2024-09-16T00:11:16.764909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59621,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":164,"num_read_kb":149,"num_write":245,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":178863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":477610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1116839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":481243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201319,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":477525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430251,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1593344,"data_stored":1074024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593777,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":472651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36908,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708244,"statfs":{"total":95995035648,"available":95957241856,"internally_reserved":0,"allocated":1601536,"data_stored":1069213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":477118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":472359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":526707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":526094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":474500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:12:00.993 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:12:00.993 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T00:12:00.994 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T00:12:01.259 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:01.670 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:01 smithi078 ceph-mon[31157]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:01.670 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:01 smithi078 ceph-mon[31157]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-16T00:12:01.671 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:01.671 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:01 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:01.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 -- 172.21.15.78:0/1600008157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 msgr2=0x7f16bc108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 --2- 172.21.15.78:0/1600008157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 0x7f16bc108720 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009ca0 tx=0x7f16b402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 -- 172.21.15.78:0/1600008157 shutdown_connections 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 --2- 172.21.15.78:0/1600008157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 0x7f16bc108720 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 --2- 172.21.15.78:0/1600008157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16bc1014d0 0x7f16bc101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 --2- 172.21.15.78:0/1600008157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16bc1002d0 0x7f16bc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 -- 172.21.15.78:0/1600008157 >> 172.21.15.78:0/1600008157 conn(0x7f16bc0fba80 msgr2=0x7f16bc0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 -- 172.21.15.78:0/1600008157 shutdown_connections 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.767+0000 7f16c47d5640 1 -- 172.21.15.78:0/1600008157 wait complete. 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c47d5640 1 Processor -- start 2024-09-16T00:12:01.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c47d5640 1 -- start start 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c47d5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16bc1002d0 0x7f16bc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c47d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16bc1014d0 0x7f16bc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c47d5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 0x7f16bc19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c47d5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc10bf00 con 0x7f16bc101e90 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c47d5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc10bd80 con 0x7f16bc1002d0 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c47d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc10c080 con 0x7f16bc1014d0 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2d72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 0x7f16bc19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2d72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 0x7f16bc19f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47098/0 (socket says 172.21.15.78:47098) 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2d72640 1 -- 172.21.15.78:0/2822124120 learned_addr learned my addr 172.21.15.78:0/2822124120 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c1d70640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16bc1014d0 0x7f16bc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2571640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16bc1002d0 0x7f16bc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2d72640 1 -- 172.21.15.78:0/2822124120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16bc1014d0 msgr2=0x7f16bc1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:01.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2d72640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16bc1014d0 0x7f16bc1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2d72640 1 -- 172.21.15.78:0/2822124120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16bc1002d0 msgr2=0x7f16bc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2d72640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16bc1002d0 0x7f16bc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.768+0000 7f16c2d72640 1 -- 172.21.15.78:0/2822124120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc19fad0 con 0x7f16bc101e90 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.769+0000 7f16c2d72640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 0x7f16bc19f460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005e30 tx=0x7f16b4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.769+0000 7f16b37fe640 1 -- 172.21.15.78:0/2822124120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4003c30 con 0x7f16bc101e90 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.769+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc19fd60 con 0x7f16bc101e90 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.769+0000 7f16b37fe640 1 -- 172.21.15.78:0/2822124120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b4003dd0 con 0x7f16bc101e90 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.769+0000 7f16b37fe640 1 -- 172.21.15.78:0/2822124120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b402fde0 con 0x7f16bc101e90 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.769+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1a0240 con 0x7f16bc101e90 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.770+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc108020 con 0x7f16bc101e90 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.772+0000 7f16b37fe640 1 -- 172.21.15.78:0/2822124120 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f16b4004050 con 0x7f16bc101e90 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.772+0000 7f16b37fe640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16a80a3300 0x7f16a80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.772+0000 7f16c2571640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16a80a3300 0x7f16a80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:01.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.772+0000 7f16b37fe640 1 -- 172.21.15.78:0/2822124120 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f16b40f0fc0 con 0x7f16bc101e90 2024-09-16T00:12:01.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.773+0000 7f16c2571640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16a80a3300 0x7f16a80a57c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005fd0 tx=0x7f16b800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:01.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:01.776+0000 7f16b37fe640 1 -- 172.21.15.78:0/2822124120 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16b40bda50 con 0x7f16bc101e90 2024-09-16T00:12:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:01 smithi193 ceph-mon[35484]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:01 smithi193 ceph-mon[35484]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-16T00:12:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:01 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:01 smithi148 ceph-mon[35599]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:01 smithi148 ceph-mon[35599]: pgmap v315: 129 pgs: 129 active+clean; 635 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-16T00:12:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:01 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:02.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.009+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f16bc02d7c0 con 0x7f16bc101e90 2024-09-16T00:12:02.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.012+0000 7f16b37fe640 1 -- 172.21.15.78:0/2822124120 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f16b40bd350 con 0x7f16bc101e90 2024-09-16T00:12:02.013 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-16T00:12:02.013 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQDSd+dmypygABAAB+kT3tK86qvYDEmFF+EkCw== 2024-09-16T00:12:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16a80a3300 msgr2=0x7f16a80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16a80a3300 0x7f16a80a57c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f16b8005fd0 tx=0x7f16b800b390 comp rx=0 tx=0).stop 2024-09-16T00:12:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 msgr2=0x7f16bc19f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 0x7f16bc19f460 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f16b4005e30 tx=0x7f16b4031ee0 comp rx=0 tx=0).stop 2024-09-16T00:12:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 shutdown_connections 2024-09-16T00:12:02.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16a80a3300 0x7f16a80a57c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:02.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16bc101e90 0x7f16bc19f460 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:02.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16bc1014d0 0x7f16bc1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:02.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 --2- 172.21.15.78:0/2822124120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16bc1002d0 0x7f16bc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:02.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 >> 172.21.15.78:0/2822124120 conn(0x7f16bc0fba80 msgr2=0x7f16bc10cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:02.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 shutdown_connections 2024-09-16T00:12:02.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:02.014+0000 7f16c47d5640 1 -- 172.21.15.78:0/2822124120 wait complete. 2024-09-16T00:12:02.628 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:12:02.628 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T00:12:02.628 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T00:12:02.664 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T00:12:02.664 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T00:12:02.664 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T00:12:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:12:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:12:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2822124120' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:12:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2822124120' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:12:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T00:12:02.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-16T00:12:02.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-16T00:12:02.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-16T00:12:02.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-16T00:12:02.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-16T00:12:02.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-16T00:12:02.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1, 5, 8]}]: dispatch 2024-09-16T00:12:02.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-16T00:12:02.948 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:12:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2822124120' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2822124120' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1, 5, 8]}]: dispatch 2024-09-16T00:12:02.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-16T00:12:03.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:12:03.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:12:03.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:12:03.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2822124120' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:12:03.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2822124120' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:12:03.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-16T00:12:03.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-16T00:12:03.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-16T00:12:03.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-16T00:12:03.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-16T00:12:03.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-16T00:12:03.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-16T00:12:03.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1, 5, 8]}]: dispatch 2024-09-16T00:12:03.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-09-16T00:12:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- 172.21.15.78:0/3547480565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750108180 msgr2=0x7f675010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 --2- 172.21.15.78:0/3547480565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750108180 0x7f675010eb40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6748009ca0 tx=0x7f674802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- 172.21.15.78:0/3547480565 shutdown_connections 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 --2- 172.21.15.78:0/3547480565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750108180 0x7f675010eb40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 --2- 172.21.15.78:0/3547480565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6750101290 0x7f6750107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 --2- 172.21.15.78:0/3547480565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67501008e0 0x7f6750100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- 172.21.15.78:0/3547480565 >> 172.21.15.78:0/3547480565 conn(0x7f6750077210 msgr2=0x7f6750077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- 172.21.15.78:0/3547480565 shutdown_connections 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- 172.21.15.78:0/3547480565 wait complete. 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 Processor -- start 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- start start 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67501008e0 0x7f67501984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750101290 0x7f6750198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6750108180 0x7f675019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6750112070 con 0x7f6750101290 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6750111ef0 con 0x7f67501008e0 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.424+0000 7f6758091640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67501121f0 con 0x7f6750108180 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750101290 0x7f6750198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750101290 0x7f6750198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47108/0 (socket says 172.21.15.78:47108) 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 -- 172.21.15.78:0/2116957106 learned_addr learned my addr 172.21.15.78:0/2116957106 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f6755e2d640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67501008e0 0x7f67501984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675662e640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6750108180 0x7f675019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 -- 172.21.15.78:0/2116957106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6750108180 msgr2=0x7f675019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6750108180 0x7f675019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 -- 172.21.15.78:0/2116957106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67501008e0 msgr2=0x7f67501984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67501008e0 0x7f67501984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 -- 172.21.15.78:0/2116957106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675019d480 con 0x7f6750101290 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f675562c640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750101290 0x7f6750198a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f674c00d770 tx=0x7f674c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f6742ffd640 1 -- 172.21.15.78:0/2116957106 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c017070 con 0x7f6750101290 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f6742ffd640 1 -- 172.21.15.78:0/2116957106 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f674c009c30 con 0x7f6750101290 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675019d770 con 0x7f6750101290 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f6742ffd640 1 -- 172.21.15.78:0/2116957106 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c00f620 con 0x7f6750101290 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.425+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67501a5050 con 0x7f6750101290 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.426+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6750100cc0 con 0x7f6750101290 2024-09-16T00:12:03.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.427+0000 7f6742ffd640 1 -- 172.21.15.78:0/2116957106 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f674c01a070 con 0x7f6750101290 2024-09-16T00:12:03.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.428+0000 7f6742ffd640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f673c0a32d0 0x7f673c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:03.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.428+0000 7f6755e2d640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f673c0a32d0 0x7f673c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:03.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.428+0000 7f6742ffd640 1 -- 172.21.15.78:0/2116957106 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9071+0+0 (secure 0 0 0) 0x7f674c0cf8f0 con 0x7f6750101290 2024-09-16T00:12:03.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.429+0000 7f6755e2d640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f673c0a32d0 0x7f673c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6744005fd0 tx=0x7f674400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:03.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.432+0000 7f6742ffd640 1 -- 172.21.15.78:0/2116957106 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f674c009de0 con 0x7f6750101290 2024-09-16T00:12:03.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.646+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f67501a5250 con 0x7f6750101290 2024-09-16T00:12:03.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.651+0000 7f6742ffd640 1 -- 172.21.15.78:0/2116957106 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264045 (secure 0 0 0) 0x7f674c09c170 con 0x7f6750101290 2024-09-16T00:12:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:03.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f673c0a32d0 msgr2=0x7f673c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:03.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f673c0a32d0 0x7f673c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6744005fd0 tx=0x7f674400b390 comp rx=0 tx=0).stop 2024-09-16T00:12:03.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750101290 msgr2=0x7f6750198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:03.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750101290 0x7f6750198a10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f674c00d770 tx=0x7f674c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:12:03.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 shutdown_connections 2024-09-16T00:12:03.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f673c0a32d0 0x7f673c0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6750108180 0x7f675019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6750101290 0x7f6750198a10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 --2- 172.21.15.78:0/2116957106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f67501008e0 0x7f67501984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:03.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 >> 172.21.15.78:0/2116957106 conn(0x7f6750077210 msgr2=0x7f67500ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:03.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 shutdown_connections 2024-09-16T00:12:03.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:03.654+0000 7f6758091640 1 -- 172.21.15.78:0/2116957106 wait complete. 2024-09-16T00:12:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-16T00:12:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T00:12:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-16T00:12:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-16T00:12:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-16T00:12:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-16T00:12:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-16T00:12:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-16T00:12:03.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1, 5, 8]}]': finished 2024-09-16T00:12:03.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-16T00:12:03.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:03 smithi193 ceph-mon[35484]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:12:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-16T00:12:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T00:12:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-16T00:12:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-16T00:12:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-16T00:12:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-16T00:12:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-16T00:12:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-16T00:12:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1, 5, 8]}]': finished 2024-09-16T00:12:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-16T00:12:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:03 smithi148 ceph-mon[35599]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: pgmap v316: 129 pgs: 129 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 1, 5, 8]}]': finished 2024-09-16T00:12:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-09-16T00:12:04.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:03 smithi078 ceph-mon[31157]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:12:04.337 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":119552845},{"type":"v1","addr":"172.21.15.78:6801","nonce":119552845}]},"active_addr":"172.21.15.78:6801/119552845","active_change":"2024-09-16T00:04:02.295526+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.78:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":1188369420}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":956594699}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":1075644778}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":118507306}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":1661917902}]}]} 2024-09-16T00:12:04.342 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T00:12:04.342 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T00:12:04.343 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:12:04.591 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:04.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:04 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2116957106' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:12:04.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:04 smithi193 ceph-mon[35484]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:12:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:04 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2116957106' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:12:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:04 smithi148 ceph-mon[35599]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:12:04.989 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:04 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2116957106' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:12:04.989 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:04 smithi078 ceph-mon[31157]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/365115848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 msgr2=0x7f6fc4114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/365115848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 0x7f6fc4114f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009ca0 tx=0x7f6fbc02a950 comp rx=0 tx=0).stop 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/365115848 shutdown_connections 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/365115848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 0x7f6fc4114f80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/365115848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc4074eb0 0x7f6fc4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/365115848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc4074590 0x7f6fc4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/365115848 >> 172.21.15.78:0/365115848 conn(0x7f6fc40fd8c0 msgr2=0x7f6fc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/365115848 shutdown_connections 2024-09-16T00:12:05.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.067+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/365115848 wait complete. 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fcb3cd640 1 Processor -- start 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fcb3cd640 1 -- start start 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fcb3cd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc4074590 0x7f6fc41a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fcb3cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc4074eb0 0x7f6fc41a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fcb3cd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 0x7f6fc419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fcb3cd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4118600 con 0x7f6fc4112b90 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fcb3cd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4118480 con 0x7f6fc4074590 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fcb3cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4118780 con 0x7f6fc4074eb0 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fc996a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 0x7f6fc419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fc996a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 0x7f6fc419e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47130/0 (socket says 172.21.15.78:47130) 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fc996a640 1 -- 172.21.15.78:0/2552340511 learned_addr learned my addr 172.21.15.78:0/2552340511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.068+0000 7f6fc9169640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc4074590 0x7f6fc41a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:05.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fc8968640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc4074eb0 0x7f6fc41a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fc996a640 1 -- 172.21.15.78:0/2552340511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc4074eb0 msgr2=0x7f6fc41a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fc996a640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc4074eb0 0x7f6fc41a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fc996a640 1 -- 172.21.15.78:0/2552340511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc4074590 msgr2=0x7f6fc41a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fc996a640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc4074590 0x7f6fc41a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fc996a640 1 -- 172.21.15.78:0/2552340511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc419f020 con 0x7f6fc4112b90 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fc9169640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc4074590 0x7f6fc41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fc996a640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 0x7f6fc419e8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005e30 tx=0x7f6fbc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fba7fc640 1 -- 172.21.15.78:0/2552340511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc034040 con 0x7f6fc4112b90 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc419f2b0 con 0x7f6fc4112b90 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fba7fc640 1 -- 172.21.15.78:0/2552340511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fbc02fe20 con 0x7f6fc4112b90 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fba7fc640 1 -- 172.21.15.78:0/2552340511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0047c0 con 0x7f6fc4112b90 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.069+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc41ab600 con 0x7f6fc4112b90 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.070+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc4074970 con 0x7f6fc4112b90 2024-09-16T00:12:05.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.071+0000 7f6fba7fc640 1 -- 172.21.15.78:0/2552340511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6fbc037020 con 0x7f6fc4112b90 2024-09-16T00:12:05.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.072+0000 7f6fba7fc640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6fb00a32d0 0x7f6fb00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:05.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.072+0000 7f6fc9169640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6fb00a32d0 0x7f6fb00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:05.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.072+0000 7f6fba7fc640 1 -- 172.21.15.78:0/2552340511 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f6fbc0f0d80 con 0x7f6fc4112b90 2024-09-16T00:12:05.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.072+0000 7f6fc9169640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6fb00a32d0 0x7f6fb00a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0005fd0 tx=0x7f6fc0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:05.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.075+0000 7f6fba7fc640 1 -- 172.21.15.78:0/2552340511 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fbc0bd710 con 0x7f6fc4112b90 2024-09-16T00:12:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.244+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fc41ab830 con 0x7f6fc4112b90 2024-09-16T00:12:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.245+0000 7f6fba7fc640 1 -- 172.21.15.78:0/2552340511 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24710 (secure 0 0 0) 0x7f6fbc0bd010 con 0x7f6fc4112b90 2024-09-16T00:12:05.247 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:05.247 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:12:04.018109+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:12:05.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6fb00a32d0 msgr2=0x7f6fb00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6fb00a32d0 0x7f6fb00a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0005fd0 tx=0x7f6fc0005850 comp rx=0 tx=0).stop 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 msgr2=0x7f6fc419e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 0x7f6fc419e8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005e30 tx=0x7f6fbc003ce0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 shutdown_connections 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6fb00a32d0 0x7f6fb00a5790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fc4112b90 0x7f6fc419e8f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc4074eb0 0x7f6fc41a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 --2- 172.21.15.78:0/2552340511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6fc4074590 0x7f6fc41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 >> 172.21.15.78:0/2552340511 conn(0x7f6fc40fd8c0 msgr2=0x7f6fc40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 shutdown_connections 2024-09-16T00:12:05.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:05.248+0000 7f6fcb3cd640 1 -- 172.21.15.78:0/2552340511 wait complete. 2024-09-16T00:12:05.852 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:05 smithi078 ceph-mon[31157]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-16T00:12:05.852 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:05 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2552340511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:05.882 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T00:12:05.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:12:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:05 smithi193 ceph-mon[35484]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-16T00:12:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:05 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2552340511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:05 smithi148 ceph-mon[35599]: pgmap v319: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-16T00:12:05.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:05 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2552340511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:06.127 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:06.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 -- 172.21.15.78:0/431750170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20105f40 msgr2=0x7fdb201063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:06.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 --2- 172.21.15.78:0/431750170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20105f40 0x7fdb201063d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ca0 tx=0x7fdb1802a940 comp rx=0 tx=0).stop 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 -- 172.21.15.78:0/431750170 shutdown_connections 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 --2- 172.21.15.78:0/431750170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20105f40 0x7fdb201063d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 --2- 172.21.15.78:0/431750170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb20101450 0x7fdb20105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 --2- 172.21.15.78:0/431750170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb20100b30 0x7fdb20100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 -- 172.21.15.78:0/431750170 >> 172.21.15.78:0/431750170 conn(0x7fdb200fc7a0 msgr2=0x7fdb200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 -- 172.21.15.78:0/431750170 shutdown_connections 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.576+0000 7fdb2793a640 1 -- 172.21.15.78:0/431750170 wait complete. 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb2793a640 1 Processor -- start 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb2793a640 1 -- start start 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb2793a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb20100b30 0x7fdb2019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb2793a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20101450 0x7fdb2019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb2793a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb20105f40 0x7fdb201a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:06.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb2793a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb20118640 con 0x7fdb20101450 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb2793a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201184c0 con 0x7fdb20105f40 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb2793a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201187c0 con 0x7fdb20100b30 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb24ed5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20101450 0x7fdb2019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb24ed5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20101450 0x7fdb2019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52304/0 (socket says 172.21.15.78:52304) 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.577+0000 7fdb24ed5640 1 -- 172.21.15.78:0/1143714926 learned_addr learned my addr 172.21.15.78:0/1143714926 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb256d6640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb20100b30 0x7fdb2019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb24ed5640 1 -- 172.21.15.78:0/1143714926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb20100b30 msgr2=0x7fdb2019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb25ed7640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb20105f40 0x7fdb201a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb24ed5640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb20100b30 0x7fdb2019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb24ed5640 1 -- 172.21.15.78:0/1143714926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb20105f40 msgr2=0x7fdb201a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb24ed5640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb20105f40 0x7fdb201a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb24ed5640 1 -- 172.21.15.78:0/1143714926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb201a3c10 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb24ed5640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20101450 0x7fdb2019f120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000b3f0 tx=0x7fdb1000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb167fc640 1 -- 172.21.15.78:0/1143714926 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10015020 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb201a3f00 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb167fc640 1 -- 172.21.15.78:0/1143714926 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb10011d10 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb167fc640 1 -- 172.21.15.78:0/1143714926 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000f2e0 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.578+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb201ab790 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.579+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb201059a0 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.580+0000 7fdb167fc640 1 -- 172.21.15.78:0/1143714926 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdb10018020 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.581+0000 7fdb167fc640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb0c0a32d0 0x7fdb0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.581+0000 7fdb256d6640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb0c0a32d0 0x7fdb0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.581+0000 7fdb167fc640 1 -- 172.21.15.78:0/1143714926 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fdb100cf5e0 con 0x7fdb20101450 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.581+0000 7fdb256d6640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb0c0a32d0 0x7fdb0c0a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009890 tx=0x7fdb1c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:06.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.585+0000 7fdb167fc640 1 -- 172.21.15.78:0/1143714926 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb1001f020 con 0x7fdb20101450 2024-09-16T00:12:06.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.756+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb201ab9c0 con 0x7fdb20101450 2024-09-16T00:12:06.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.757+0000 7fdb167fc640 1 -- 172.21.15.78:0/1143714926 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24710 (secure 0 0 0) 0x7fdb1009bf70 con 0x7fdb20101450 2024-09-16T00:12:06.759 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:06.759 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:12:04.018109+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.759+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb0c0a32d0 msgr2=0x7fdb0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb0c0a32d0 0x7fdb0c0a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c009890 tx=0x7fdb1c008040 comp rx=0 tx=0).stop 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20101450 msgr2=0x7fdb2019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20101450 0x7fdb2019f120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000b3f0 tx=0x7fdb1000b8b0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 shutdown_connections 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb0c0a32d0 0x7fdb0c0a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb20105f40 0x7fdb201a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb20101450 0x7fdb2019f120 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 --2- 172.21.15.78:0/1143714926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb20100b30 0x7fdb2019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 >> 172.21.15.78:0/1143714926 conn(0x7fdb200fc7a0 msgr2=0x7fdb2010f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:06.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 shutdown_connections 2024-09-16T00:12:06.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:06.760+0000 7fdb2793a640 1 -- 172.21.15.78:0/1143714926 wait complete. 2024-09-16T00:12:07.381 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T00:12:07.382 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T00:12:07.382 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T00:12:07.383 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T00:12:07.383 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T00:12:07.384 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T00:12:07.384 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T00:12:07.385 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T00:12:07.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-16T00:12:07.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-16T00:12:07.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-16T00:12:07.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-16T00:12:07.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:07 smithi078 ceph-mon[31157]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:07.637 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:07 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1143714926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:07 smithi193 ceph-mon[35484]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:07 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1143714926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:07 smithi148 ceph-mon[35599]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:07 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1143714926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:08.113 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.120 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.123 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.127 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.129 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.157 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:08.159 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 -- 172.21.15.78:0/684646407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 msgr2=0x7fc2701067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 --2- 172.21.15.78:0/684646407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 0x7fc2701067d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc258009900 tx=0x7fc25802a410 comp rx=0 tx=0).stop 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 -- 172.21.15.78:0/684646407 shutdown_connections 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 --2- 172.21.15.78:0/684646407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc270106f00 0x7fc27010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 --2- 172.21.15.78:0/684646407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 0x7fc2701067d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 --2- 172.21.15.78:0/684646407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc270101dd0 0x7fc2701021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 -- 172.21.15.78:0/684646407 >> 172.21.15.78:0/684646407 conn(0x7fc2700fd8b0 msgr2=0x7fc2700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 -- 172.21.15.78:0/684646407 shutdown_connections 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 -- 172.21.15.78:0/684646407 wait complete. 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 Processor -- start 2024-09-16T00:12:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 -- start start 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.219+0000 7fc2755cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc270101dd0 0x7fc2701983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc2755cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 0x7fc270198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc2755cc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc270106f00 0x7fc27019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc2755cc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc270111f60 con 0x7fc270102780 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc2755cc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc270111de0 con 0x7fc270101dd0 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc2755cc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc2701120e0 con 0x7fc270106f00 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 0x7fc270198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 0x7fc270198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52316/0 (socket says 172.21.15.78:52316) 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 -- 172.21.15.78:0/2947276020 learned_addr learned my addr 172.21.15.78:0/2947276020 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc26effd640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc270101dd0 0x7fc2701983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc26f7fe640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc270106f00 0x7fc27019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:09.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 -- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc270106f00 msgr2=0x7fc27019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc270106f00 0x7fc27019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 -- 172.21.15.78:0/2947276020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc270101dd0 msgr2=0x7fc2701983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc270101dd0 0x7fc2701983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 -- 172.21.15.78:0/2947276020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27019d390 con 0x7fc270102780 2024-09-16T00:12:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc26effd640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc270101dd0 0x7fc2701983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc26f7fe640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc270106f00 0x7fc27019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:09.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc267fff640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 0x7fc270198920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc258006010 tx=0x7fc258004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25803f030 con 0x7fc270102780 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc258004440 con 0x7fc270102780 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27019d620 con 0x7fc270102780 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc258035070 con 0x7fc270102780 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.220+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2701a4f60 con 0x7fc270102780 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.221+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc270062980 con 0x7fc270102780 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.223+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc258042030 con 0x7fc270102780 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.223+0000 7fc26cff9640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc25c0a3350 0x7fc25c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.224+0000 7fc26effd640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc25c0a3350 0x7fc25c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.224+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fc2580f0500 con 0x7fc270102780 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.224+0000 7fc26cff9640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] conn(0x7fc25c0ae4a0 0x7fc25c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.224+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 --> [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc25c0b1010 con 0x7fc25c0ae4a0 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.224+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc2580f0950 con 0x7fc270102780 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.224+0000 7fc26effd640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc25c0a3350 0x7fc25c0a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc260009c20 tx=0x7fc260005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.224+0000 7fc26f7fe640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] conn(0x7fc25c0ae4a0 0x7fc25c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.225+0000 7fc26f7fe640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] conn(0x7fc25c0ae4a0 0x7fc25c0b0940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.225+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 <== osd.2 v2:172.21.15.193:6800/2579590991 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc25c0b1010 con 0x7fc25c0ae4a0 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.239+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 --> [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc270062d80 con 0x7fc25c0ae4a0 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.239+0000 7fc26cff9640 1 -- 172.21.15.78:0/2947276020 <== osd.2 v2:172.21.15.193:6800/2579590991 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc270062d80 con 0x7fc25c0ae4a0 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] conn(0x7fc25c0ae4a0 msgr2=0x7fc25c0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] conn(0x7fc25c0ae4a0 0x7fc25c0b0940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc25c0a3350 msgr2=0x7fc25c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc25c0a3350 0x7fc25c0a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc260009c20 tx=0x7fc260005f90 comp rx=0 tx=0).stop 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 msgr2=0x7fc270198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:09.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 --2- 172.21.15.78:0/2947276020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc270102780 0x7fc270198920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc258006010 tx=0x7fc258004040 comp rx=0 tx=0).stop 2024-09-16T00:12:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc26f7fe640 1 -- 172.21.15.78:0/2947276020 reap_dead start 2024-09-16T00:12:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 shutdown_connections 2024-09-16T00:12:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 >> 172.21.15.78:0/2947276020 conn(0x7fc2700fd8b0 msgr2=0x7fc2700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 shutdown_connections 2024-09-16T00:12:09.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:09.240+0000 7fc2755cc640 1 -- 172.21.15.78:0/2947276020 wait complete. 2024-09-16T00:12:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:09 smithi193 ceph-mon[35484]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:09 smithi148 ceph-mon[35599]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:10.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:09 smithi078 ceph-mon[31157]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:10.143 INFO:teuthology.orchestra.run.smithi078.stdout:77309411401 2024-09-16T00:12:10.143 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T00:12:11.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 -- 172.21.15.78:0/131868763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 msgr2=0x7fd63810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 --2- 172.21.15.78:0/131868763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 0x7fd63810ec40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd630009cd0 tx=0x7fd63002a910 comp rx=0 tx=0).stop 2024-09-16T00:12:11.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 -- 172.21.15.78:0/131868763 shutdown_connections 2024-09-16T00:12:11.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 --2- 172.21.15.78:0/131868763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 0x7fd63810ec40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 --2- 172.21.15.78:0/131868763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6381036e0 0x7fd638107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 --2- 172.21.15.78:0/131868763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd638102d30 0x7fd638103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 -- 172.21.15.78:0/131868763 >> 172.21.15.78:0/131868763 conn(0x7fd6380fc7d0 msgr2=0x7fd6380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 -- 172.21.15.78:0/131868763 shutdown_connections 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.064+0000 7fd63f699640 1 -- 172.21.15.78:0/131868763 wait complete. 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63f699640 1 Processor -- start 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63f699640 1 -- start start 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63f699640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd638102d30 0x7fd6381a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63f699640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6381036e0 0x7fd6381a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63f699640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 0x7fd63819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63f699640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381120e0 con 0x7fd6381082a0 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63f699640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd638111f60 con 0x7fd6381036e0 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63f699640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd638112260 con 0x7fd638102d30 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63dc36640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 0x7fd63819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63dc36640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 0x7fd63819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52346/0 (socket says 172.21.15.78:52346) 2024-09-16T00:12:11.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63dc36640 1 -- 172.21.15.78:0/3383577316 learned_addr learned my addr 172.21.15.78:0/3383577316 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63cc34640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6381036e0 0x7fd6381a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63d435640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd638102d30 0x7fd6381a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63dc36640 1 -- 172.21.15.78:0/3383577316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd638102d30 msgr2=0x7fd6381a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63dc36640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd638102d30 0x7fd6381a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63dc36640 1 -- 172.21.15.78:0/3383577316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6381036e0 msgr2=0x7fd6381a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63dc36640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6381036e0 0x7fd6381a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63dc36640 1 -- 172.21.15.78:0/3383577316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63819ace0 con 0x7fd6381082a0 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.065+0000 7fd63d435640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd638102d30 0x7fd6381a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.066+0000 7fd63dc36640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 0x7fd63819a5b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd630005cc0 tx=0x7fd630031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.066+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd630031850 con 0x7fd6381082a0 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.066+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd630004520 con 0x7fd6381082a0 2024-09-16T00:12:11.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.066+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63819af70 con 0x7fd6381082a0 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.066+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63002fd20 con 0x7fd6381082a0 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.066+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6381a72c0 con 0x7fd6381082a0 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.067+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd638062900 con 0x7fd6381082a0 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.068+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd630004040 con 0x7fd6381082a0 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.068+0000 7fd62e7fc640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6240a32d0 0x7fd6240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.069+0000 7fd63d435640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6240a32d0 0x7fd6240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.069+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fd6300f17d0 con 0x7fd6381082a0 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.069+0000 7fd62e7fc640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] conn(0x7fd6240ae420 0x7fd6240b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.069+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 --> [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd63003fec0 con 0x7fd6240ae420 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.069+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd6300f1c20 con 0x7fd6381082a0 2024-09-16T00:12:11.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.069+0000 7fd63cc34640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] conn(0x7fd6240ae420 0x7fd6240b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.069+0000 7fd63d435640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6240a32d0 0x7fd6240a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd6340058c0 tx=0x7fd634005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.069+0000 7fd63cc34640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] conn(0x7fd6240ae420 0x7fd6240b08c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.070+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 <== osd.10 v2:172.21.15.148:6824/1827220524 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd63003fec0 con 0x7fd6240ae420 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 --> [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd638062d80 con 0x7fd6240ae420 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd62e7fc640 1 -- 172.21.15.78:0/3383577316 <== osd.10 v2:172.21.15.148:6824/1827220524 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd638062d80 con 0x7fd6240ae420 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 >> [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] conn(0x7fd6240ae420 msgr2=0x7fd6240b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd63f699640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] conn(0x7fd6240ae420 0x7fd6240b08c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6240a32d0 msgr2=0x7fd6240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd63f699640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6240a32d0 0x7fd6240a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd6340058c0 tx=0x7fd634005d30 comp rx=0 tx=0).stop 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 msgr2=0x7fd63819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd63f699640 1 --2- 172.21.15.78:0/3383577316 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6381082a0 0x7fd63819a5b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd630005cc0 tx=0x7fd630031460 comp rx=0 tx=0).stop 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.084+0000 7fd63dc36640 1 -- 172.21.15.78:0/3383577316 reap_dead start 2024-09-16T00:12:11.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.085+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 shutdown_connections 2024-09-16T00:12:11.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.085+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 >> 172.21.15.78:0/3383577316 conn(0x7fd6380fc7d0 msgr2=0x7fd638100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:11.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.085+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 shutdown_connections 2024-09-16T00:12:11.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.085+0000 7fd63f699640 1 -- 172.21.15.78:0/3383577316 wait complete. 2024-09-16T00:12:11.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:11 smithi078 ceph-mon[31157]: pgmap v322: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:11.793 INFO:teuthology.orchestra.run.smithi078.stdout:287762808865 2024-09-16T00:12:11.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-16T00:12:11.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:11 smithi193 ceph-mon[35484]: pgmap v322: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:11 smithi148 ceph-mon[35599]: pgmap v322: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.966+0000 7f533158b640 1 -- 172.21.15.78:0/3738561137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c108300 msgr2=0x7f532c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.966+0000 7f533158b640 1 --2- 172.21.15.78:0/3738561137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c108300 0x7f532c10ec70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5324009cd0 tx=0x7f532402a8f0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.967+0000 7f533158b640 1 -- 172.21.15.78:0/3738561137 shutdown_connections 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.967+0000 7f533158b640 1 --2- 172.21.15.78:0/3738561137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c108300 0x7f532c10ec70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.967+0000 7f533158b640 1 --2- 172.21.15.78:0/3738561137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f532c103780 0x7f532c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.967+0000 7f533158b640 1 --2- 172.21.15.78:0/3738561137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f532c102dd0 0x7f532c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.967+0000 7f533158b640 1 -- 172.21.15.78:0/3738561137 >> 172.21.15.78:0/3738561137 conn(0x7f532c0fc7d0 msgr2=0x7f532c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.967+0000 7f533158b640 1 -- 172.21.15.78:0/3738561137 shutdown_connections 2024-09-16T00:12:11.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.967+0000 7f533158b640 1 -- 172.21.15.78:0/3738561137 wait complete. 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.967+0000 7f533158b640 1 Processor -- start 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f533158b640 1 -- start start 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f533158b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c102dd0 0x7f532c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f533158b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f532c103780 0x7f532c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f533158b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f532c108300 0x7f532c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f533158b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f532c112170 con 0x7f532c102dd0 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f533158b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f532c111ff0 con 0x7f532c108300 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f533158b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f532c1122f0 con 0x7f532c103780 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c102dd0 0x7f532c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c102dd0 0x7f532c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52376/0 (socket says 172.21.15.78:52376) 2024-09-16T00:12:11.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532affd640 1 -- 172.21.15.78:0/553377745 learned_addr learned my addr 172.21.15.78:0/553377745 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532b7fe640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f532c108300 0x7f532c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532a7fc640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f532c103780 0x7f532c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532affd640 1 -- 172.21.15.78:0/553377745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f532c103780 msgr2=0x7f532c198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532affd640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f532c103780 0x7f532c198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532affd640 1 -- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f532c108300 msgr2=0x7f532c19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.968+0000 7f532affd640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f532c108300 0x7f532c19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f532affd640 1 -- 172.21.15.78:0/553377745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532c19d680 con 0x7f532c102dd0 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f532b7fe640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f532c108300 0x7f532c19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f532a7fc640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f532c103780 0x7f532c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f532affd640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c102dd0 0x7f532c1986d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f531c00b4a0 tx=0x7f531c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c014070 con 0x7f532c102dd0 2024-09-16T00:12:11.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532c19d970 con 0x7f532c102dd0 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f531c010af0 con 0x7f532c102dd0 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c017d10 con 0x7f532c102dd0 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.969+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532c1a5250 con 0x7f532c102dd0 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.970+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f532c062980 con 0x7f532c102dd0 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.972+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f531c017070 con 0x7f532c102dd0 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.972+0000 7f5313fff640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53140a32d0 0x7f53140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.973+0000 7f532a7fc640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53140a32d0 0x7f53140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.973+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f531c0cf700 con 0x7f532c102dd0 2024-09-16T00:12:11.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.973+0000 7f5313fff640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] conn(0x7f53140ae420 0x7f53140b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.973+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 --> [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f531c00ee50 con 0x7f53140ae420 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.973+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f531c0cfb50 con 0x7f532c102dd0 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.973+0000 7f532a7fc640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53140a32d0 0x7f53140a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5320006fd0 tx=0x7f5320008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.973+0000 7f532b7fe640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] conn(0x7f53140ae420 0x7f53140b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.974+0000 7f532b7fe640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] conn(0x7f53140ae420 0x7f53140b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.974+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 <== osd.1 v2:172.21.15.148:6800/2952692564 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f531c00ee50 con 0x7f53140ae420 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.989+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 --> [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f532c062e00 con 0x7f53140ae420 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.989+0000 7f5313fff640 1 -- 172.21.15.78:0/553377745 <== osd.1 v2:172.21.15.148:6800/2952692564 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f532c062e00 con 0x7f53140ae420 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.989+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] conn(0x7f53140ae420 msgr2=0x7f53140b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.989+0000 7f533158b640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] conn(0x7f53140ae420 0x7f53140b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53140a32d0 msgr2=0x7f53140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f533158b640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53140a32d0 0x7f53140a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5320006fd0 tx=0x7f5320008040 comp rx=0 tx=0).stop 2024-09-16T00:12:11.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c102dd0 msgr2=0x7f532c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f533158b640 1 --2- 172.21.15.78:0/553377745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f532c102dd0 0x7f532c1986d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f531c00b4a0 tx=0x7f531c00b960 comp rx=0 tx=0).stop 2024-09-16T00:12:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f532b7fe640 1 -- 172.21.15.78:0/553377745 reap_dead start 2024-09-16T00:12:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 shutdown_connections 2024-09-16T00:12:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 >> 172.21.15.78:0/553377745 conn(0x7f532c0fc7d0 msgr2=0x7f532c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 shutdown_connections 2024-09-16T00:12:12.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:11.990+0000 7f533158b640 1 -- 172.21.15.78:0/553377745 wait complete. 2024-09-16T00:12:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 -- 172.21.15.78:0/2138953032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac103780 msgr2=0x7f36ac107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 --2- 172.21.15.78:0/2138953032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac103780 0x7f36ac107bc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3694009930 tx=0x7f369402a420 comp rx=0 tx=0).stop 2024-09-16T00:12:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 -- 172.21.15.78:0/2138953032 shutdown_connections 2024-09-16T00:12:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 --2- 172.21.15.78:0/2138953032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac108300 0x7f36ac10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 --2- 172.21.15.78:0/2138953032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac103780 0x7f36ac107bc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 --2- 172.21.15.78:0/2138953032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ac102dd0 0x7f36ac1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 -- 172.21.15.78:0/2138953032 >> 172.21.15.78:0/2138953032 conn(0x7f36ac0fc7d0 msgr2=0x7f36ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 -- 172.21.15.78:0/2138953032 shutdown_connections 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.942+0000 7f36b2620640 1 -- 172.21.15.78:0/2138953032 wait complete. 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b2620640 1 Processor -- start 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b2620640 1 -- start start 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b2620640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ac102dd0 0x7f36ac1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b2620640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac103780 0x7f36ac198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b2620640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac108300 0x7f36ac19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b2620640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac112070 con 0x7f36ac108300 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b2620640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac111ef0 con 0x7f36ac102dd0 2024-09-16T00:12:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b2620640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac1121f0 con 0x7f36ac103780 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b0bbd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac108300 0x7f36ac19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36abfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ac102dd0 0x7f36ac1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b0bbd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac108300 0x7f36ac19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52322/0 (socket says 172.21.15.78:52322) 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36b0bbd640 1 -- 172.21.15.78:0/2895559567 learned_addr learned my addr 172.21.15.78:0/2895559567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.943+0000 7f36a3fff640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac103780 0x7f36ac198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36b0bbd640 1 -- 172.21.15.78:0/2895559567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac103780 msgr2=0x7f36ac198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36b0bbd640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac103780 0x7f36ac198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36b0bbd640 1 -- 172.21.15.78:0/2895559567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ac102dd0 msgr2=0x7f36ac1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36b0bbd640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f36ac102dd0 0x7f36ac1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36b0bbd640 1 -- 172.21.15.78:0/2895559567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac19d5a0 con 0x7f36ac108300 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36b0bbd640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac108300 0x7f36ac19cec0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f36a400eaa0 tx=0x7f36a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a400cdb0 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a4005020 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac19d890 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a40116b0 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.944+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac1a5170 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.945+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36ac062980 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.946+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f36a4005560 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.947+0000 7f36a9ffb640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f36980a3350 0x7f36980a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.947+0000 7f36abfff640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f36980a3350 0x7f36980a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.947+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f36a40d02f0 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.947+0000 7f36a9ffb640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] conn(0x7f36980ae4a0 0x7f36980b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.947+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 --> [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36a4014ec0 con 0x7f36980ae4a0 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.947+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f36a40d0740 con 0x7f36ac108300 2024-09-16T00:12:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.947+0000 7f36abfff640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f36980a3350 0x7f36980a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f369c00bbe0 tx=0x7f369c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.948+0000 7f36a3fff640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] conn(0x7f36980ae4a0 0x7f36980b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.948+0000 7f36a3fff640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] conn(0x7f36980ae4a0 0x7f36980b0940 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.949+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 <== osd.4 v2:172.21.15.148:6808/336734377 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36a4014ec0 con 0x7f36980ae4a0 2024-09-16T00:12:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.962+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 --> [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36ac062d80 con 0x7f36980ae4a0 2024-09-16T00:12:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36a9ffb640 1 -- 172.21.15.78:0/2895559567 <== osd.4 v2:172.21.15.148:6808/336734377 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f36ac062d80 con 0x7f36980ae4a0 2024-09-16T00:12:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 >> [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] conn(0x7f36980ae4a0 msgr2=0x7f36980b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] conn(0x7f36980ae4a0 0x7f36980b0940 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f36980a3350 msgr2=0x7f36980a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f36980a3350 0x7f36980a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f369c00bbe0 tx=0x7f369c00b390 comp rx=0 tx=0).stop 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac108300 msgr2=0x7f36ac19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 --2- 172.21.15.78:0/2895559567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f36ac108300 0x7f36ac19cec0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f36a400eaa0 tx=0x7f36a400ef60 comp rx=0 tx=0).stop 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b0bbd640 1 -- 172.21.15.78:0/2895559567 reap_dead start 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 shutdown_connections 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 >> 172.21.15.78:0/2895559567 conn(0x7f36ac0fc7d0 msgr2=0x7f36ac105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.963+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 shutdown_connections 2024-09-16T00:12:12.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:10.964+0000 7f36b2620640 1 -- 172.21.15.78:0/2895559567 wait complete. 2024-09-16T00:12:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 -- 172.21.15.78:0/4107512807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b60102790 msgr2=0x7f9b601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 --2- 172.21.15.78:0/4107512807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b60102790 0x7f9b601067e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009900 tx=0x7f9b4c02a410 comp rx=0 tx=0).stop 2024-09-16T00:12:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 -- 172.21.15.78:0/4107512807 shutdown_connections 2024-09-16T00:12:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 --2- 172.21.15.78:0/4107512807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b60106f10 0x7f9b6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 --2- 172.21.15.78:0/4107512807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b60102790 0x7f9b601067e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 --2- 172.21.15.78:0/4107512807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b60101de0 0x7f9b601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 -- 172.21.15.78:0/4107512807 >> 172.21.15.78:0/4107512807 conn(0x7f9b600fd8c0 msgr2=0x7f9b600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 -- 172.21.15.78:0/4107512807 shutdown_connections 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.640+0000 7f9b67019640 1 -- 172.21.15.78:0/4107512807 wait complete. 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.641+0000 7f9b67019640 1 Processor -- start 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.641+0000 7f9b67019640 1 -- start start 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.641+0000 7f9b67019640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b60101de0 0x7f9b600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.641+0000 7f9b67019640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b60102790 0x7f9b60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.641+0000 7f9b67019640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b60106f10 0x7f9b60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.641+0000 7f9b67019640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60111f30 con 0x7f9b60106f10 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.641+0000 7f9b67019640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60111db0 con 0x7f9b60101de0 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.641+0000 7f9b67019640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601120b0 con 0x7f9b60102790 2024-09-16T00:12:13.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b64db5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b60101de0 0x7f9b600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b64db5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b60101de0 0x7f9b600744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:38396/0 (socket says 172.21.15.78:38396) 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b64db5640 1 -- 172.21.15.78:0/3573891854 learned_addr learned my addr 172.21.15.78:0/3573891854 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b655b6640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b60106f10 0x7f9b60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b5ffff640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b60102790 0x7f9b60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b655b6640 1 -- 172.21.15.78:0/3573891854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b60102790 msgr2=0x7f9b60074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b655b6640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b60102790 0x7f9b60074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b655b6640 1 -- 172.21.15.78:0/3573891854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b60101de0 msgr2=0x7f9b600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b655b6640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9b60101de0 0x7f9b600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b655b6640 1 -- 172.21.15.78:0/3573891854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b601a68f0 con 0x7f9b60106f10 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.642+0000 7f9b655b6640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b60106f10 0x7f9b60074f70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800e940 tx=0x7f9b5800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.647+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58004d20 con 0x7f9b60106f10 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.647+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b58005020 con 0x7f9b60106f10 2024-09-16T00:12:13.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.647+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b601a6be0 con 0x7f9b60106f10 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.647+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b580116b0 con 0x7f9b60106f10 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.647+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b601a70f0 con 0x7f9b60106f10 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.648+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b600628c0 con 0x7f9b60106f10 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.649+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9b580054d0 con 0x7f9b60106f10 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.650+0000 7f9b5dffb640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b500a3350 0x7f9b500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.650+0000 7f9b64db5640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b500a3350 0x7f9b500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.650+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f9b580d0160 con 0x7f9b60106f10 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.650+0000 7f9b5dffb640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] conn(0x7f9b500ae4a0 0x7f9b500b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.650+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 --> [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b58014ec0 con 0x7f9b500ae4a0 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.650+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f9b580d05b0 con 0x7f9b60106f10 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.650+0000 7f9b64db5640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b500a3350 0x7f9b500a5810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005fd0 tx=0x7f9b5400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.650+0000 7f9b5ffff640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] conn(0x7f9b500ae4a0 0x7f9b500b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.651+0000 7f9b5ffff640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] conn(0x7f9b500ae4a0 0x7f9b500b0940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:13.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.651+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 <== osd.7 v2:172.21.15.148:6816/155815362 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b58014ec0 con 0x7f9b500ae4a0 2024-09-16T00:12:13.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.665+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 --> [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b60062d40 con 0x7f9b500ae4a0 2024-09-16T00:12:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.665+0000 7f9b5dffb640 1 -- 172.21.15.78:0/3573891854 <== osd.7 v2:172.21.15.148:6816/155815362 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b60062d40 con 0x7f9b500ae4a0 2024-09-16T00:12:13.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.665+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 >> [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] conn(0x7f9b500ae4a0 msgr2=0x7f9b500b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.665+0000 7f9b67019640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] conn(0x7f9b500ae4a0 0x7f9b500b0940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.665+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b500a3350 msgr2=0x7f9b500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.665+0000 7f9b67019640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9b500a3350 0x7f9b500a5810 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005fd0 tx=0x7f9b5400b480 comp rx=0 tx=0).stop 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.665+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b60106f10 msgr2=0x7f9b60074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.665+0000 7f9b67019640 1 --2- 172.21.15.78:0/3573891854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b60106f10 0x7f9b60074f70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800e940 tx=0x7f9b5800ee00 comp rx=0 tx=0).stop 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.666+0000 7f9b655b6640 1 -- 172.21.15.78:0/3573891854 reap_dead start 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.666+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 shutdown_connections 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.666+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 >> 172.21.15.78:0/3573891854 conn(0x7f9b600fd8c0 msgr2=0x7f9b600fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.666+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 shutdown_connections 2024-09-16T00:12:13.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.666+0000 7f9b67019640 1 -- 172.21.15.78:0/3573891854 wait complete. 2024-09-16T00:12:13.842 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:13 smithi078 ceph-mon[31157]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 327 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 -- 172.21.15.78:0/1139363555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100e30 msgr2=0x7fc584104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 --2- 172.21.15.78:0/1139363555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100e30 0x7fc584104e80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc578009900 tx=0x7fc57802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 -- 172.21.15.78:0/1139363555 shutdown_connections 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 --2- 172.21.15.78:0/1139363555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5841055b0 0x7fc58410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 --2- 172.21.15.78:0/1139363555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100e30 0x7fc584104e80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 --2- 172.21.15.78:0/1139363555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc584100480 0x7fc584100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 -- 172.21.15.78:0/1139363555 >> 172.21.15.78:0/1139363555 conn(0x7fc584077210 msgr2=0x7fc5840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 -- 172.21.15.78:0/1139363555 shutdown_connections 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.840+0000 7fc58a5c5640 1 -- 172.21.15.78:0/1139363555 wait complete. 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.841+0000 7fc58a5c5640 1 Processor -- start 2024-09-16T00:12:13.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.841+0000 7fc58a5c5640 1 -- start start 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.841+0000 7fc58a5c5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100480 0x7fc584199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.841+0000 7fc58a5c5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc584100e30 0x7fc5841999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.841+0000 7fc58a5c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5841055b0 0x7fc5841a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc58a5c5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5841106b0 con 0x7fc584100480 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc58a5c5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc584110530 con 0x7fc5841055b0 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc58a5c5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc584110830 con 0x7fc584100e30 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100480 0x7fc584199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100480 0x7fc584199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52412/0 (socket says 172.21.15.78:52412) 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 -- 172.21.15.78:0/3782566784 learned_addr learned my addr 172.21.15.78:0/3782566784 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc5837fe640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc584100e30 0x7fc5841999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc588b62640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5841055b0 0x7fc5841a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 -- 172.21.15.78:0/3782566784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc584100e30 msgr2=0x7fc5841999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc584100e30 0x7fc5841999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 -- 172.21.15.78:0/3782566784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5841055b0 msgr2=0x7fc5841a44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5841055b0 0x7fc5841a44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 -- 172.21.15.78:0/3782566784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5841a4a20 con 0x7fc584100480 2024-09-16T00:12:13.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc588b62640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5841055b0 0x7fc5841a44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:13.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc583fff640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100480 0x7fc584199460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc57400b4a0 tx=0x7fc57400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:13.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574014070 con 0x7fc584100480 2024-09-16T00:12:13.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc574010af0 con 0x7fc584100480 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5841a4bf0 con 0x7fc584100480 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574017d10 con 0x7fc584100480 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.842+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841a50b0 con 0x7fc584100480 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.843+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5840fef30 con 0x7fc584100480 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.844+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc574017070 con 0x7fc584100480 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.845+0000 7fc5817fa640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc56c0a32d0 0x7fc56c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.845+0000 7fc5837fe640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc56c0a32d0 0x7fc56c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.845+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fc5740cf430 con 0x7fc584100480 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.845+0000 7fc5817fa640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] conn(0x7fc56c0ae420 0x7fc56c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.845+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 --> [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc57400dec0 con 0x7fc56c0ae420 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.845+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc5740cf880 con 0x7fc584100480 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.846+0000 7fc5837fe640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc56c0a32d0 0x7fc56c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc578000c00 tx=0x7fc5780058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:13.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.846+0000 7fc588b62640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] conn(0x7fc56c0ae420 0x7fc56c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.846+0000 7fc588b62640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] conn(0x7fc56c0ae420 0x7fc56c0b08c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.847+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 <== osd.5 v2:172.21.15.193:6808/2332816249 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc57400dec0 con 0x7fc56c0ae420 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.861+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 --> [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc584101960 con 0x7fc56c0ae420 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.861+0000 7fc5817fa640 1 -- 172.21.15.78:0/3782566784 <== osd.5 v2:172.21.15.193:6808/2332816249 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc584101960 con 0x7fc56c0ae420 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 >> [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] conn(0x7fc56c0ae420 msgr2=0x7fc56c0b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] conn(0x7fc56c0ae420 0x7fc56c0b08c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc56c0a32d0 msgr2=0x7fc56c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc56c0a32d0 0x7fc56c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc578000c00 tx=0x7fc5780058d0 comp rx=0 tx=0).stop 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100480 msgr2=0x7fc584199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 --2- 172.21.15.78:0/3782566784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584100480 0x7fc584199460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc57400b4a0 tx=0x7fc57400b960 comp rx=0 tx=0).stop 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc588b62640 1 -- 172.21.15.78:0/3782566784 reap_dead start 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 shutdown_connections 2024-09-16T00:12:13.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 >> 172.21.15.78:0/3782566784 conn(0x7fc584077210 msgr2=0x7fc5840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 shutdown_connections 2024-09-16T00:12:13.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:13.862+0000 7fc58a5c5640 1 -- 172.21.15.78:0/3782566784 wait complete. 2024-09-16T00:12:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:13 smithi193 ceph-mon[35484]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 327 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:13 smithi148 ceph-mon[35599]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 327 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:14.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 -- 172.21.15.78:0/2849527976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 msgr2=0x7fed2010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 --2- 172.21.15.78:0/2849527976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 0x7fed2010ec40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fed18009930 tx=0x7fed1802a420 comp rx=0 tx=0).stop 2024-09-16T00:12:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 -- 172.21.15.78:0/2849527976 shutdown_connections 2024-09-16T00:12:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 --2- 172.21.15.78:0/2849527976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 0x7fed2010ec40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 --2- 172.21.15.78:0/2849527976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed201036e0 0x7fed20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 --2- 172.21.15.78:0/2849527976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed20102d30 0x7fed20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 -- 172.21.15.78:0/2849527976 >> 172.21.15.78:0/2849527976 conn(0x7fed200fc7d0 msgr2=0x7fed200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 -- 172.21.15.78:0/2849527976 shutdown_connections 2024-09-16T00:12:14.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 -- 172.21.15.78:0/2849527976 wait complete. 2024-09-16T00:12:14.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 Processor -- start 2024-09-16T00:12:14.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.286+0000 7fed253f9640 1 -- start start 2024-09-16T00:12:14.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed253f9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed20102d30 0x7fed20198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:14.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed253f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed201036e0 0x7fed20198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:14.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed253f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 0x7fed2019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:14.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed253f9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed201120a0 con 0x7fed201082a0 2024-09-16T00:12:14.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed253f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed20111f20 con 0x7fed201036e0 2024-09-16T00:12:14.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed253f9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fed20112220 con 0x7fed20102d30 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 0x7fed2019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 0x7fed2019ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52422/0 (socket says 172.21.15.78:52422) 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 -- 172.21.15.78:0/1784057262 learned_addr learned my addr 172.21.15.78:0/1784057262 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1effd640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed20102d30 0x7fed20198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1e7fc640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed201036e0 0x7fed20198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 -- 172.21.15.78:0/1784057262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed20102d30 msgr2=0x7fed20198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed20102d30 0x7fed20198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 -- 172.21.15.78:0/1784057262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed201036e0 msgr2=0x7fed20198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed201036e0 0x7fed20198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:14.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 -- 172.21.15.78:0/1784057262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2019d510 con 0x7fed201082a0 2024-09-16T00:12:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed1f7fe640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 0x7fed2019ce30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fed18005ab0 tx=0x7fed180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1803f030 con 0x7fed201082a0 2024-09-16T00:12:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2019d7a0 con 0x7fed201082a0 2024-09-16T00:12:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed18003ce0 con 0x7fed201082a0 2024-09-16T00:12:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18035070 con 0x7fed201082a0 2024-09-16T00:12:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.287+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed201a50e0 con 0x7fed201082a0 2024-09-16T00:12:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.288+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed20062980 con 0x7fed201082a0 2024-09-16T00:12:14.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.290+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fed18042030 con 0x7fed201082a0 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.290+0000 7fed17fff640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed0c0a32d0 0x7fed0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.290+0000 7fed1effd640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed0c0a32d0 0x7fed0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.290+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fed180f0640 con 0x7fed201082a0 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.291+0000 7fed17fff640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] conn(0x7fed0c0ae420 0x7fed0c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.291+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 --> [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed18003b30 con 0x7fed0c0ae420 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.291+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fed180f0a90 con 0x7fed201082a0 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.291+0000 7fed1e7fc640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] conn(0x7fed0c0ae420 0x7fed0c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.291+0000 7fed1effd640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed0c0a32d0 0x7fed0c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed10005fd0 tx=0x7fed10009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.291+0000 7fed1e7fc640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] conn(0x7fed0c0ae420 0x7fed0c0b08c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.292+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 <== osd.6 v2:172.21.15.78:6818/445240021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed18003b30 con 0x7fed0c0ae420 2024-09-16T00:12:14.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.305+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 --> [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed20062d80 con 0x7fed0c0ae420 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.305+0000 7fed17fff640 1 -- 172.21.15.78:0/1784057262 <== osd.6 v2:172.21.15.78:6818/445240021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fed20062d80 con 0x7fed0c0ae420 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] conn(0x7fed0c0ae420 msgr2=0x7fed0c0b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] conn(0x7fed0c0ae420 0x7fed0c0b08c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed0c0a32d0 msgr2=0x7fed0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed0c0a32d0 0x7fed0c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed10005fd0 tx=0x7fed10009680 comp rx=0 tx=0).stop 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 msgr2=0x7fed2019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 --2- 172.21.15.78:0/1784057262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed201082a0 0x7fed2019ce30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fed18005ab0 tx=0x7fed180042f0 comp rx=0 tx=0).stop 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed1f7fe640 1 -- 172.21.15.78:0/1784057262 reap_dead start 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 shutdown_connections 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 >> 172.21.15.78:0/1784057262 conn(0x7fed200fc7d0 msgr2=0x7fed20105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 shutdown_connections 2024-09-16T00:12:14.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:14.306+0000 7fed253f9640 1 -- 172.21.15.78:0/1784057262 wait complete. 2024-09-16T00:12:15.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 -- 172.21.15.78:0/1924989343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 msgr2=0x7f0db0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:15.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 --2- 172.21.15.78:0/1924989343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 0x7f0db0110810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009ca0 tx=0x7f0da802a950 comp rx=0 tx=0).stop 2024-09-16T00:12:15.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 -- 172.21.15.78:0/1924989343 shutdown_connections 2024-09-16T00:12:15.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 --2- 172.21.15.78:0/1924989343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 0x7f0db0110810 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:15.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 --2- 172.21.15.78:0/1924989343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db0074f40 0x7f0db010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:15.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 --2- 172.21.15.78:0/1924989343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0db0074620 0x7f0db0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:15.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 -- 172.21.15.78:0/1924989343 >> 172.21.15.78:0/1924989343 conn(0x7f0db00fd8c0 msgr2=0x7f0db00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 -- 172.21.15.78:0/1924989343 shutdown_connections 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.016+0000 7f0db535c640 1 -- 172.21.15.78:0/1924989343 wait complete. 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0db535c640 1 Processor -- start 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0db535c640 1 -- start start 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0db535c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0db0074620 0x7f0db019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0db535c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db0074f40 0x7f0db019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0db535c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 0x7f0db019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0db535c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0db01141b0 con 0x7f0db010e3c0 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0db535c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0db0114030 con 0x7f0db0074620 2024-09-16T00:12:15.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0db535c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0db0114330 con 0x7f0db0074f40 2024-09-16T00:12:15.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0daf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 0x7f0db019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:15.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0daf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 0x7f0db019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52442/0 (socket says 172.21.15.78:52442) 2024-09-16T00:12:15.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0daf7fe640 1 -- 172.21.15.78:0/490291317 learned_addr learned my addr 172.21.15.78:0/490291317 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:15.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0daf7fe640 1 -- 172.21.15.78:0/490291317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db0074f40 msgr2=0x7f0db019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:15.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0daf7fe640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db0074f40 0x7f0db019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:15.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0daeffd640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0db0074620 0x7f0db019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0daf7fe640 1 -- 172.21.15.78:0/490291317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0db0074620 msgr2=0x7f0db019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.017+0000 7f0daf7fe640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0db0074620 0x7f0db019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.018+0000 7f0daf7fe640 1 -- 172.21.15.78:0/490291317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db019f620 con 0x7f0db010e3c0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.018+0000 7f0daf7fe640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 0x7f0db019ef40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0da8006010 tx=0x7f0da802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.018+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8034040 con 0x7f0db010e3c0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.018+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db019f8b0 con 0x7f0db010e3c0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.018+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da8003e00 con 0x7f0db010e3c0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.018+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da80456e0 con 0x7f0db010e3c0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.018+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db01a71f0 con 0x7f0db010e3c0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.019+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0db00fff30 con 0x7f0db010e3c0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.020+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0da8037020 con 0x7f0db010e3c0 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.021+0000 7f0d97fff640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0d980a32d0 0x7f0d980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:15.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.021+0000 7f0daeffd640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0d980a32d0 0x7f0d980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.021+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f0da80f1080 con 0x7f0db010e3c0 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.021+0000 7f0d97fff640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] conn(0x7f0d980ae420 0x7f0d980b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.021+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 --> [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0da803ce50 con 0x7f0d980ae420 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.021+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0da80f14d0 con 0x7f0db010e3c0 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.021+0000 7f0dae7fc640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] conn(0x7f0d980ae420 0x7f0d980b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.021+0000 7f0daeffd640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0d980a32d0 0x7f0d980a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0da00058c0 tx=0x7f0da0005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.022+0000 7f0dae7fc640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] conn(0x7f0d980ae420 0x7f0d980b08c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.022+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 <== osd.3 v2:172.21.15.78:6810/2314919115 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0da803ce50 con 0x7f0d980ae420 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.035+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 --> [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0db0075cd0 con 0x7f0d980ae420 2024-09-16T00:12:15.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0d97fff640 1 -- 172.21.15.78:0/490291317 <== osd.3 v2:172.21.15.78:6810/2314919115 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0db0075cd0 con 0x7f0d980ae420 2024-09-16T00:12:15.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] conn(0x7f0d980ae420 msgr2=0x7f0d980b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:15.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] conn(0x7f0d980ae420 0x7f0d980b08c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:15.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0d980a32d0 msgr2=0x7f0d980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:15.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0d980a32d0 0x7f0d980a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0da00058c0 tx=0x7f0da0005c60 comp rx=0 tx=0).stop 2024-09-16T00:12:15.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 msgr2=0x7f0db019ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:15.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 --2- 172.21.15.78:0/490291317 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0db010e3c0 0x7f0db019ef40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0da8006010 tx=0x7f0da802f710 comp rx=0 tx=0).stop 2024-09-16T00:12:15.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0daf7fe640 1 -- 172.21.15.78:0/490291317 reap_dead start 2024-09-16T00:12:15.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 shutdown_connections 2024-09-16T00:12:15.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 >> 172.21.15.78:0/490291317 conn(0x7f0db00fd8c0 msgr2=0x7f0db00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:15.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 shutdown_connections 2024-09-16T00:12:15.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:15.036+0000 7f0db535c640 1 -- 172.21.15.78:0/490291317 wait complete. 2024-09-16T00:12:15.714 INFO:teuthology.orchestra.run.smithi078.stdout:103079215173 2024-09-16T00:12:15.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T00:12:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:15 smithi193 ceph-mon[35484]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 295 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:15 smithi148 ceph-mon[35599]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 295 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:16.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:15 smithi078 ceph-mon[31157]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 295 B/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:16.140 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:16.153 INFO:teuthology.orchestra.run.smithi078.stdout:154618822714 2024-09-16T00:12:16.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T00:12:16.607 INFO:teuthology.orchestra.run.smithi078.stdout:55834574927 2024-09-16T00:12:16.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.877+0000 7f4a43d01640 1 -- 172.21.15.78:0/1399542173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a80 msgr2=0x7f4a3c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.877+0000 7f4a43d01640 1 --2- 172.21.15.78:0/1399542173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a80 0x7f4a3c106e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009930 tx=0x7f4a3802a420 comp rx=0 tx=0).stop 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.877+0000 7f4a43d01640 1 -- 172.21.15.78:0/1399542173 shutdown_connections 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.877+0000 7f4a43d01640 1 --2- 172.21.15.78:0/1399542173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c101420 0x7f4a3c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.877+0000 7f4a43d01640 1 --2- 172.21.15.78:0/1399542173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c100a80 0x7f4a3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.877+0000 7f4a43d01640 1 --2- 172.21.15.78:0/1399542173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a80 0x7f4a3c106e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.877+0000 7f4a43d01640 1 -- 172.21.15.78:0/1399542173 >> 172.21.15.78:0/1399542173 conn(0x7f4a3c0fc7a0 msgr2=0x7f4a3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 -- 172.21.15.78:0/1399542173 shutdown_connections 2024-09-16T00:12:16.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 -- 172.21.15.78:0/1399542173 wait complete. 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 Processor -- start 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 -- start start 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a80 0x7f4a3c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c101420 0x7f4a3c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c106a80 0x7f4a3c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c110ea0 con 0x7f4a3c101420 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c110d20 con 0x7f4a3c106a80 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.878+0000 7f4a43d01640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c111020 con 0x7f4a3c100a80 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c101420 0x7f4a3c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c101420 0x7f4a3c102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53546/0 (socket says 172.21.15.78:53546) 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 -- 172.21.15.78:0/523005453 learned_addr learned my addr 172.21.15.78:0/523005453 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a41a9d640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a80 0x7f4a3c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:16.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4229e640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c106a80 0x7f4a3c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 -- 172.21.15.78:0/523005453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a80 msgr2=0x7f4a3c102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a80 0x7f4a3c102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 -- 172.21.15.78:0/523005453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c106a80 msgr2=0x7f4a3c1030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c106a80 0x7f4a3c1030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 -- 172.21.15.78:0/523005453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c1a6940 con 0x7f4a3c101420 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a4129c640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c101420 0x7f4a3c102b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00b4a0 tx=0x7f4a2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c014070 con 0x7f4a3c101420 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a2c010af0 con 0x7f4a3c101420 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c1a6c30 con 0x7f4a3c101420 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c017d10 con 0x7f4a3c101420 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.879+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c1a71c0 con 0x7f4a3c101420 2024-09-16T00:12:16.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.880+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a3c06aa20 con 0x7f4a3c101420 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.881+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4a2c017070 con 0x7f4a3c101420 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.882+0000 7f4a32ffd640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a280a3350 0x7f4a280a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.882+0000 7f4a41a9d640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a280a3350 0x7f4a280a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.882+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f4a2c0cf320 con 0x7f4a3c101420 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.882+0000 7f4a32ffd640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] conn(0x7f4a280ae4a0 0x7f4a280b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.882+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 --> [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a2c00dec0 con 0x7f4a280ae4a0 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.882+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f4a2c0cf760 con 0x7f4a3c101420 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.883+0000 7f4a41a9d640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a280a3350 0x7f4a280a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009900 tx=0x7f4a380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:16.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.883+0000 7f4a4229e640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] conn(0x7f4a280ae4a0 0x7f4a280b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.883+0000 7f4a4229e640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] conn(0x7f4a280ae4a0 0x7f4a280b0940 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.884+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 <== osd.8 v2:172.21.15.193:6816/3026218583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a2c00dec0 con 0x7f4a280ae4a0 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.897+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 --> [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a3c1021c0 con 0x7f4a280ae4a0 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a32ffd640 1 -- 172.21.15.78:0/523005453 <== osd.8 v2:172.21.15.193:6816/3026218583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a3c1021c0 con 0x7f4a280ae4a0 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 >> [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] conn(0x7f4a280ae4a0 msgr2=0x7f4a280b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a43d01640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] conn(0x7f4a280ae4a0 0x7f4a280b0940 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a280a3350 msgr2=0x7f4a280a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a43d01640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a280a3350 0x7f4a280a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009900 tx=0x7f4a380058d0 comp rx=0 tx=0).stop 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c101420 msgr2=0x7f4a3c102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a43d01640 1 --2- 172.21.15.78:0/523005453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c101420 0x7f4a3c102b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00b4a0 tx=0x7f4a2c00b960 comp rx=0 tx=0).stop 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a4229e640 1 -- 172.21.15.78:0/523005453 reap_dead start 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 shutdown_connections 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.898+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 >> 172.21.15.78:0/523005453 conn(0x7f4a3c0fc7a0 msgr2=0x7f4a3c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:16.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.899+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 shutdown_connections 2024-09-16T00:12:16.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:16.899+0000 7f4a43d01640 1 -- 172.21.15.78:0/523005453 wait complete. 2024-09-16T00:12:17.885 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:17 smithi078 ceph-mon[31157]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:17.913 INFO:teuthology.orchestra.run.smithi078.stdout:206158430256 2024-09-16T00:12:17.914 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T00:12:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:17 smithi193 ceph-mon[35484]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:17 smithi148 ceph-mon[35599]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:18.354 INFO:teuthology.orchestra.run.smithi078.stdout:133143986239 2024-09-16T00:12:18.355 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T00:12:18.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 -- 172.21.15.78:0/3545490669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01082a0 msgr2=0x7f2be010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 --2- 172.21.15.78:0/3545490669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01082a0 0x7f2be010ec40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009cd0 tx=0x7f2bd802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 -- 172.21.15.78:0/3545490669 shutdown_connections 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 --2- 172.21.15.78:0/3545490669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01082a0 0x7f2be010ec40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 --2- 172.21.15.78:0/3545490669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be01036e0 0x7f2be0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 --2- 172.21.15.78:0/3545490669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be0102d30 0x7f2be0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 -- 172.21.15.78:0/3545490669 >> 172.21.15.78:0/3545490669 conn(0x7f2be00fc7d0 msgr2=0x7f2be00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 -- 172.21.15.78:0/3545490669 shutdown_connections 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.594+0000 7f2be7f27640 1 -- 172.21.15.78:0/3545490669 wait complete. 2024-09-16T00:12:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be7f27640 1 Processor -- start 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be7f27640 1 -- start start 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be7f27640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be0102d30 0x7f2be0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be7f27640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01036e0 0x7f2be0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be7f27640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be01082a0 0x7f2be019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be7f27640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0112100 con 0x7f2be01036e0 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be7f27640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0111f80 con 0x7f2be01082a0 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be7f27640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0112280 con 0x7f2be0102d30 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be54c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01036e0 0x7f2be0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be54c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01036e0 0x7f2be0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53580/0 (socket says 172.21.15.78:53580) 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be54c2640 1 -- 172.21.15.78:0/4294949378 learned_addr learned my addr 172.21.15.78:0/4294949378 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be5cc3640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be0102d30 0x7f2be0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.595+0000 7f2be64c4640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be01082a0 0x7f2be019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be54c2640 1 -- 172.21.15.78:0/4294949378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be0102d30 msgr2=0x7f2be0198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be54c2640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be0102d30 0x7f2be0198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be54c2640 1 -- 172.21.15.78:0/4294949378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be01082a0 msgr2=0x7f2be019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be54c2640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be01082a0 0x7f2be019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be54c2640 1 -- 172.21.15.78:0/4294949378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be019d4f0 con 0x7f2be01036e0 2024-09-16T00:12:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be64c4640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2be01082a0 0x7f2be019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be54c2640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01036e0 0x7f2be0198a80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c8c0 tx=0x7f2bd000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0007bb0 con 0x7f2be01036e0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0007d50 con 0x7f2be01036e0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be019d7e0 con 0x7f2be01036e0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0018830 con 0x7f2be01036e0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.596+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be01a50c0 con 0x7f2be01036e0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.597+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2be00fee10 con 0x7f2be01036e0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.598+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2bd00189f0 con 0x7f2be01036e0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2bd6ffd640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2bcc0a3350 0x7f2bcc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2be5cc3640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2bcc0a3350 0x7f2bcc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f2bd00d0370 con 0x7f2be01036e0 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2bd6ffd640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] conn(0x7f2bcc0ae4a0 0x7f2bcc0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 --> [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bcc0b1010 con 0x7f2bcc0ae4a0 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f2bd00d07c0 con 0x7f2be01036e0 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2be64c4640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] conn(0x7f2bcc0ae4a0 0x7f2bcc0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2be5cc3640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2bcc0a3350 0x7f2bcc0a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009890 tx=0x7f2bdc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.599+0000 7f2be64c4640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] conn(0x7f2bcc0ae4a0 0x7f2bcc0b0940 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.600+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 <== osd.0 v2:172.21.15.78:6802/1264645608 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bcc0b1010 con 0x7f2bcc0ae4a0 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.613+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 --> [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2be006a850 con 0x7f2bcc0ae4a0 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2bd6ffd640 1 -- 172.21.15.78:0/4294949378 <== osd.0 v2:172.21.15.78:6802/1264645608 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2be006a850 con 0x7f2bcc0ae4a0 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] conn(0x7f2bcc0ae4a0 msgr2=0x7f2bcc0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] conn(0x7f2bcc0ae4a0 0x7f2bcc0b0940 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2bcc0a3350 msgr2=0x7f2bcc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2bcc0a3350 0x7f2bcc0a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009890 tx=0x7f2bdc008040 comp rx=0 tx=0).stop 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01036e0 msgr2=0x7f2be0198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:18.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 --2- 172.21.15.78:0/4294949378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2be01036e0 0x7f2be0198a80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c8c0 tx=0x7f2bd000cd80 comp rx=0 tx=0).stop 2024-09-16T00:12:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be64c4640 1 -- 172.21.15.78:0/4294949378 reap_dead start 2024-09-16T00:12:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 shutdown_connections 2024-09-16T00:12:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 >> 172.21.15.78:0/4294949378 conn(0x7f2be00fc7d0 msgr2=0x7f2be00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 shutdown_connections 2024-09-16T00:12:18.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:18.614+0000 7f2be7f27640 1 -- 172.21.15.78:0/4294949378 wait complete. 2024-09-16T00:12:18.943 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:18 smithi078 ceph-mon[31157]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:18.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:18 smithi148 ceph-mon[35599]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:19.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.610+0000 7f947fdf5640 1 -- 172.21.15.78:0/2152277658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94781036e0 msgr2=0x7f9478107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:19.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.610+0000 7f947fdf5640 1 --2- 172.21.15.78:0/2152277658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94781036e0 0x7f9478107b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9468009930 tx=0x7f946802a420 comp rx=0 tx=0).stop 2024-09-16T00:12:19.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.610+0000 7f947fdf5640 1 -- 172.21.15.78:0/2152277658 shutdown_connections 2024-09-16T00:12:19.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.610+0000 7f947fdf5640 1 --2- 172.21.15.78:0/2152277658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94781082a0 0x7f947810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 --2- 172.21.15.78:0/2152277658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94781036e0 0x7f9478107b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 --2- 172.21.15.78:0/2152277658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9478102d30 0x7f9478103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 -- 172.21.15.78:0/2152277658 >> 172.21.15.78:0/2152277658 conn(0x7f94780fc7d0 msgr2=0x7f94780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 -- 172.21.15.78:0/2152277658 shutdown_connections 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 -- 172.21.15.78:0/2152277658 wait complete. 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 Processor -- start 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 -- start start 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478102d30 0x7f9478198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94781036e0 0x7f9478198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94781082a0 0x7f947819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94781120a0 con 0x7f9478102d30 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9478111f20 con 0x7f94781082a0 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.611+0000 7f947fdf5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9478112220 con 0x7f94781036e0 2024-09-16T00:12:19.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478102d30 0x7f9478198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:19.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478102d30 0x7f9478198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53568/0 (socket says 172.21.15.78:53568) 2024-09-16T00:12:19.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 -- 172.21.15.78:0/4207877315 learned_addr learned my addr 172.21.15.78:0/4207877315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:19.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947d390640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94781036e0 0x7f9478198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:19.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947e392640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94781082a0 0x7f947819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:19.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 -- 172.21.15.78:0/4207877315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94781036e0 msgr2=0x7f9478198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:19.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94781036e0 0x7f9478198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 -- 172.21.15.78:0/4207877315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94781082a0 msgr2=0x7f947819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94781082a0 0x7f947819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 -- 172.21.15.78:0/4207877315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947819d510 con 0x7f9478102d30 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947e392640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f94781082a0 0x7f947819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947db91640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478102d30 0x7f9478198560 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f947400c920 tx=0x7f947400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474007c70 con 0x7f9478102d30 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9474007e10 con 0x7f9478102d30 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947819d800 con 0x7f9478102d30 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474018890 con 0x7f9478102d30 2024-09-16T00:12:19.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.612+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781a50e0 con 0x7f9478102d30 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.613+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9478062900 con 0x7f9478102d30 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.614+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9474018a30 con 0x7f9478102d30 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.615+0000 7f946effd640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94640a3350 0x7f94640a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.615+0000 7f947d390640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94640a3350 0x7f94640a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.615+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f94740d0370 con 0x7f9478102d30 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.615+0000 7f946effd640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] conn(0x7f94640ae4a0 0x7f94640b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.615+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 --> [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94640b1010 con 0x7f94640ae4a0 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.615+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f94740d07c0 con 0x7f9478102d30 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.615+0000 7f947e392640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] conn(0x7f94640ae4a0 0x7f94640b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:19.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.615+0000 7f947d390640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94640a3350 0x7f94640a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9468009230 tx=0x7f9468000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.616+0000 7f947e392640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] conn(0x7f94640ae4a0 0x7f94640b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.616+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 <== osd.9 v2:172.21.15.78:6826/1947247517 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94640b1010 con 0x7f94640ae4a0 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.630+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 --> [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9478062d00 con 0x7f94640ae4a0 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.630+0000 7f946effd640 1 -- 172.21.15.78:0/4207877315 <== osd.9 v2:172.21.15.78:6826/1947247517 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9478062d00 con 0x7f94640ae4a0 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] conn(0x7f94640ae4a0 msgr2=0x7f94640b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] conn(0x7f94640ae4a0 0x7f94640b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94640a3350 msgr2=0x7f94640a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f94640a3350 0x7f94640a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9468009230 tx=0x7f9468000bc0 comp rx=0 tx=0).stop 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478102d30 msgr2=0x7f9478198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 --2- 172.21.15.78:0/4207877315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9478102d30 0x7f9478198560 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f947400c920 tx=0x7f947400cde0 comp rx=0 tx=0).stop 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947e392640 1 -- 172.21.15.78:0/4207877315 reap_dead start 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 shutdown_connections 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 >> 172.21.15.78:0/4207877315 conn(0x7f94780fc7d0 msgr2=0x7f9478105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:19.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 shutdown_connections 2024-09-16T00:12:19.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:17.631+0000 7f947fdf5640 1 -- 172.21.15.78:0/4207877315 wait complete. 2024-09-16T00:12:19.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:18 smithi193 ceph-mon[35484]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T00:12:19.844 INFO:teuthology.orchestra.run.smithi078.stdout:266287972391 2024-09-16T00:12:19.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 -- 172.21.15.78:0/859081865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 msgr2=0x7f1ac81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 --2- 172.21.15.78:0/859081865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 0x7f1ac81029b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009900 tx=0x7f1ab802a480 comp rx=0 tx=0).stop 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 -- 172.21.15.78:0/859081865 shutdown_connections 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 --2- 172.21.15.78:0/859081865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102ef0 0x7f1ac810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 --2- 172.21.15.78:0/859081865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 0x7f1ac81029b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 --2- 172.21.15.78:0/859081865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8100e20 0x7f1ac8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 -- 172.21.15.78:0/859081865 >> 172.21.15.78:0/859081865 conn(0x7f1ac80fc7d0 msgr2=0x7f1ac80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 -- 172.21.15.78:0/859081865 shutdown_connections 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.000+0000 7f1ad0281640 1 -- 172.21.15.78:0/859081865 wait complete. 2024-09-16T00:12:20.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ad0281640 1 Processor -- start 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ad0281640 1 -- start start 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ad0281640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8100e20 0x7f1ac8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ad0281640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 0x7f1ac8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ad0281640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102ef0 0x7f1ac819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ad0281640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8110f80 con 0x7f1ac8102570 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ad0281640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8110e00 con 0x7f1ac8102ef0 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ad0281640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111100 con 0x7f1ac8100e20 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 0x7f1ac8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 0x7f1ac8198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53594/0 (socket says 172.21.15.78:53594) 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 -- 172.21.15.78:0/1753979880 learned_addr learned my addr 172.21.15.78:0/1753979880 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1ace81e640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102ef0 0x7f1ac819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 -- 172.21.15.78:0/1753979880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8100e20 msgr2=0x7f1ac8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:20.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8100e20 0x7f1ac8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 -- 172.21.15.78:0/1753979880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102ef0 msgr2=0x7f1ac819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102ef0 0x7f1ac819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 -- 172.21.15.78:0/1753979880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac819d540 con 0x7f1ac8102570 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.001+0000 7f1acd81c640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 0x7f1ac8198ad0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.002+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab803f070 con 0x7f1ac8102570 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.002+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab8031c30 con 0x7f1ac8102570 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.002+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac819d7d0 con 0x7f1ac8102570 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.002+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8035030 con 0x7f1ac8102570 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.002+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a5110 con 0x7f1ac8102570 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.003+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1ac8062980 con 0x7f1ac8102570 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.004+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1ab8042070 con 0x7f1ac8102570 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.004+0000 7f1abf7fe640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab40a32d0 0x7f1ab40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.005+0000 7f1ace01d640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab40a32d0 0x7f1ab40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:20.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.005+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f1ab80f0700 con 0x7f1ac8102570 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.005+0000 7f1abf7fe640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] conn(0x7f1ab40ae420 0x7f1ab40b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.005+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 --> [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ab803fe70 con 0x7f1ab40ae420 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.005+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1ab80f7050 con 0x7f1ac8102570 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.005+0000 7f1ace01d640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab40a32d0 0x7f1ab40a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400a8f0 tx=0x7f1ac4005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.005+0000 7f1ace81e640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] conn(0x7f1ab40ae420 0x7f1ab40b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.006+0000 7f1ace81e640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] conn(0x7f1ab40ae420 0x7f1ab40b08c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.006+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 <== osd.11 v2:172.21.15.193:6824/872081348 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ab803fe70 con 0x7f1ab40ae420 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.020+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 --> [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1ac8062e00 con 0x7f1ab40ae420 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1abf7fe640 1 -- 172.21.15.78:0/1753979880 <== osd.11 v2:172.21.15.193:6824/872081348 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1ac8062e00 con 0x7f1ab40ae420 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 >> [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] conn(0x7f1ab40ae420 msgr2=0x7f1ab40b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] conn(0x7f1ab40ae420 0x7f1ab40b08c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab40a32d0 msgr2=0x7f1ab40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:20.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab40a32d0 0x7f1ab40a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400a8f0 tx=0x7f1ac4005bf0 comp rx=0 tx=0).stop 2024-09-16T00:12:20.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 msgr2=0x7f1ac8198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:20.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 --2- 172.21.15.78:0/1753979880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8102570 0x7f1ac8198ad0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006010 tx=0x7f1ab8003c10 comp rx=0 tx=0).stop 2024-09-16T00:12:20.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ace81e640 1 -- 172.21.15.78:0/1753979880 reap_dead start 2024-09-16T00:12:20.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 shutdown_connections 2024-09-16T00:12:20.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 >> 172.21.15.78:0/1753979880 conn(0x7f1ac80fc7d0 msgr2=0x7f1ac806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:20.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 shutdown_connections 2024-09-16T00:12:20.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:20.021+0000 7f1ad0281640 1 -- 172.21.15.78:0/1753979880 wait complete. 2024-09-16T00:12:20.729 INFO:teuthology.orchestra.run.smithi078.stdout:236223201324 2024-09-16T00:12:20.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-16T00:12:21.237 INFO:teuthology.orchestra.run.smithi078.stdout:34359738452 2024-09-16T00:12:21.237 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T00:12:21.717 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:21.757 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:21 smithi078 ceph-mon[31157]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-16T00:12:21.758 INFO:teuthology.orchestra.run.smithi078.stdout:184683593781 2024-09-16T00:12:21.758 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T00:12:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:21 smithi193 ceph-mon[35484]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-16T00:12:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:21 smithi148 ceph-mon[35599]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-16T00:12:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.191+0000 7ff748550640 1 -- 172.21.15.78:0/1912583026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7401036c0 msgr2=0x7ff74010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.191+0000 7ff748550640 1 --2- 172.21.15.78:0/1912583026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7401036c0 0x7ff74010bbb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff730009900 tx=0x7ff73002a480 comp rx=0 tx=0).stop 2024-09-16T00:12:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.191+0000 7ff748550640 1 -- 172.21.15.78:0/1912583026 shutdown_connections 2024-09-16T00:12:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.191+0000 7ff748550640 1 --2- 172.21.15.78:0/1912583026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff74010c180 0x7ff74010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.191+0000 7ff748550640 1 --2- 172.21.15.78:0/1912583026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff7401036c0 0x7ff74010bbb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.191+0000 7ff748550640 1 --2- 172.21.15.78:0/1912583026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff740102da0 0x7ff740103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.191+0000 7ff748550640 1 -- 172.21.15.78:0/1912583026 >> 172.21.15.78:0/1912583026 conn(0x7ff7400fc7a0 msgr2=0x7ff7400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.191+0000 7ff748550640 1 -- 172.21.15.78:0/1912583026 shutdown_connections 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 -- 172.21.15.78:0/1912583026 wait complete. 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 Processor -- start 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 -- start start 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff740102da0 0x7ff7401983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7401036c0 0x7ff740198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff74010c180 0x7ff74019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff740111f60 con 0x7ff74010c180 2024-09-16T00:12:22.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff740111de0 con 0x7ff740102da0 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff748550640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7401120e0 con 0x7ff7401036c0 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff746aed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff74010c180 0x7ff74019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff746aed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff74010c180 0x7ff74019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53610/0 (socket says 172.21.15.78:53610) 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.192+0000 7ff746aed640 1 -- 172.21.15.78:0/2344877030 learned_addr learned my addr 172.21.15.78:0/2344877030 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff745aeb640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7401036c0 0x7ff740198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff7462ec640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff740102da0 0x7ff7401983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff746aed640 1 -- 172.21.15.78:0/2344877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7401036c0 msgr2=0x7ff740198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff746aed640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7401036c0 0x7ff740198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff746aed640 1 -- 172.21.15.78:0/2344877030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff740102da0 msgr2=0x7ff7401983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:22.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff746aed640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff740102da0 0x7ff7401983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff746aed640 1 -- 172.21.15.78:0/2344877030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74019d390 con 0x7ff74010c180 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff746aed640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff74010c180 0x7ff74019ccb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff73800cad0 tx=0x7ff73800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff7377fe640 1 -- 172.21.15.78:0/2344877030 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73800ae20 con 0x7ff74010c180 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74019d680 con 0x7ff74010c180 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff7377fe640 1 -- 172.21.15.78:0/2344877030 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff738005020 con 0x7ff74010c180 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff7377fe640 1 -- 172.21.15.78:0/2344877030 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73800f6f0 con 0x7ff74010c180 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.193+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7401a4f60 con 0x7ff74010c180 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.194+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff740103180 con 0x7ff74010c180 2024-09-16T00:12:22.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.195+0000 7ff7377fe640 1 -- 172.21.15.78:0/2344877030 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff7380051d0 con 0x7ff74010c180 2024-09-16T00:12:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.196+0000 7ff7377fe640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff72c0a32d0 0x7ff72c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.196+0000 7ff7462ec640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff72c0a32d0 0x7ff72c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.196+0000 7ff7377fe640 1 -- 172.21.15.78:0/2344877030 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7ff7380d0220 con 0x7ff74010c180 2024-09-16T00:12:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.196+0000 7ff7462ec640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff72c0a32d0 0x7ff72c0a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff73c005e10 tx=0x7ff73c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:22.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.199+0000 7ff7377fe640 1 -- 172.21.15.78:0/2344877030 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff738014030 con 0x7ff74010c180 2024-09-16T00:12:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:22.221 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:22.223 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:22.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.383+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff740106c30 con 0x7ff74010c180 2024-09-16T00:12:22.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.383+0000 7ff7377fe640 1 -- 172.21.15.78:0/2344877030 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff73809caa0 con 0x7ff74010c180 2024-09-16T00:12:22.384 INFO:teuthology.orchestra.run.smithi078.stdout:287762808866 2024-09-16T00:12:22.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff72c0a32d0 msgr2=0x7ff72c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:22.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff72c0a32d0 0x7ff72c0a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff73c005e10 tx=0x7ff73c005da0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff74010c180 msgr2=0x7ff74019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff74010c180 0x7ff74019ccb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff73800cad0 tx=0x7ff73800cf90 comp rx=0 tx=0).stop 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 shutdown_connections 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff72c0a32d0 0x7ff72c0a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff74010c180 0x7ff74019ccb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7401036c0 0x7ff740198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 --2- 172.21.15.78:0/2344877030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff740102da0 0x7ff7401983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 >> 172.21.15.78:0/2344877030 conn(0x7ff7400fc7a0 msgr2=0x7ff7400fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 shutdown_connections 2024-09-16T00:12:22.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:22.386+0000 7ff748550640 1 -- 172.21.15.78:0/2344877030 wait complete. 2024-09-16T00:12:22.668 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:22 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2344877030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:12:22.701 INFO:teuthology.orchestra.run.smithi078.stdout:317827579933 2024-09-16T00:12:22.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-16T00:12:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:22 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2344877030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:12:22.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:22 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2344877030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- 172.21.15.78:0/3979272577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710106f20 msgr2=0x7f571010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 --2- 172.21.15.78:0/3979272577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710106f20 0x7f571010eaa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5708009ca0 tx=0x7f570802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- 172.21.15.78:0/3979272577 shutdown_connections 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 --2- 172.21.15.78:0/3979272577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710106f20 0x7f571010eaa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 --2- 172.21.15.78:0/3979272577 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57101027a0 0x7f57101067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 --2- 172.21.15.78:0/3979272577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5710101df0 0x7f57101021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- 172.21.15.78:0/3979272577 >> 172.21.15.78:0/3979272577 conn(0x7f57100fd8f0 msgr2=0x7f57100ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- 172.21.15.78:0/3979272577 shutdown_connections 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- 172.21.15.78:0/3979272577 wait complete. 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 Processor -- start 2024-09-16T00:12:23.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- start start 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710101df0 0x7f5710198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57101027a0 0x7f5710198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5710106f20 0x7f571019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5710111f80 con 0x7f5710101df0 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5710111e00 con 0x7f57101027a0 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.330+0000 7f5716d3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5710112100 con 0x7f5710106f20 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710101df0 0x7f5710198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710101df0 0x7f5710198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53620/0 (socket says 172.21.15.78:53620) 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 -- 172.21.15.78:0/2436823832 learned_addr learned my addr 172.21.15.78:0/2436823832 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f57152d8640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5710106f20 0x7f571019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 -- 172.21.15.78:0/2436823832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5710106f20 msgr2=0x7f571019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:23.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5710106f20 0x7f571019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 -- 172.21.15.78:0/2436823832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57101027a0 msgr2=0x7f5710198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f570ffff640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57101027a0 0x7f5710198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57101027a0 0x7f5710198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 -- 172.21.15.78:0/2436823832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571019d2f0 con 0x7f5710101df0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5714ad7640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710101df0 0x7f5710198340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f570400d950 tx=0x7f570400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f570dffb640 1 -- 172.21.15.78:0/2436823832 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704017070 con 0x7f5710101df0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571019d5e0 con 0x7f5710101df0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f570dffb640 1 -- 172.21.15.78:0/2436823832 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5704009c30 con 0x7f5710101df0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.331+0000 7f570dffb640 1 -- 172.21.15.78:0/2436823832 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570400f6b0 con 0x7f5710101df0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.332+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101a4ec0 con 0x7f5710101df0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.332+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57101021d0 con 0x7f5710101df0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.333+0000 7f570dffb640 1 -- 172.21.15.78:0/2436823832 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f57040109d0 con 0x7f5710101df0 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.334+0000 7f570dffb640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57000a32d0 0x7f57000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:23.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.334+0000 7f570ffff640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57000a32d0 0x7f57000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:23.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.335+0000 7f570dffb640 1 -- 172.21.15.78:0/2436823832 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f57040cfb20 con 0x7f5710101df0 2024-09-16T00:12:23.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.335+0000 7f570ffff640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57000a32d0 0x7f57000a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57101088c0 tx=0x7f56fc005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:23.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.339+0000 7f570dffb640 1 -- 172.21.15.78:0/2436823832 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5704009de0 con 0x7f5710101df0 2024-09-16T00:12:23.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.511+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f571010a1c0 con 0x7f5710101df0 2024-09-16T00:12:23.513 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:23 smithi078 ceph-mon[31157]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:23.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.512+0000 7f570dffb640 1 -- 172.21.15.78:0/2436823832 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f570409c3a0 con 0x7f5710101df0 2024-09-16T00:12:23.513 INFO:teuthology.orchestra.run.smithi078.stdout:206158430257 2024-09-16T00:12:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57000a32d0 msgr2=0x7f57000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57000a32d0 0x7f57000a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57101088c0 tx=0x7f56fc005c50 comp rx=0 tx=0).stop 2024-09-16T00:12:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710101df0 msgr2=0x7f5710198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710101df0 0x7f5710198340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f570400d950 tx=0x7f570400de10 comp rx=0 tx=0).stop 2024-09-16T00:12:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 shutdown_connections 2024-09-16T00:12:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57000a32d0 0x7f57000a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5710106f20 0x7f571019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57101027a0 0x7f5710198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 --2- 172.21.15.78:0/2436823832 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5710101df0 0x7f5710198340 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 >> 172.21.15.78:0/2436823832 conn(0x7f57100fd8f0 msgr2=0x7f57100feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 shutdown_connections 2024-09-16T00:12:23.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.514+0000 7f5716d3b640 1 -- 172.21.15.78:0/2436823832 wait complete. 2024-09-16T00:12:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:23 smithi193 ceph-mon[35484]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:23 smithi148 ceph-mon[35599]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 -- 172.21.15.78:0/3449681112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 msgr2=0x7fac701067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 --2- 172.21.15.78:0/3449681112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 0x7fac701067f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fac64009900 tx=0x7fac6402a410 comp rx=0 tx=0).stop 2024-09-16T00:12:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 -- 172.21.15.78:0/3449681112 shutdown_connections 2024-09-16T00:12:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 --2- 172.21.15.78:0/3449681112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac70106f20 0x7fac7010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 --2- 172.21.15.78:0/3449681112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 0x7fac701067f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 --2- 172.21.15.78:0/3449681112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac70101df0 0x7fac701021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 -- 172.21.15.78:0/3449681112 >> 172.21.15.78:0/3449681112 conn(0x7fac700fd8b0 msgr2=0x7fac700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 -- 172.21.15.78:0/3449681112 shutdown_connections 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.995+0000 7fac76a5e640 1 -- 172.21.15.78:0/3449681112 wait complete. 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac76a5e640 1 Processor -- start 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac76a5e640 1 -- start start 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac76a5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac70101df0 0x7fac70198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac76a5e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 0x7fac70198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac76a5e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac70106f20 0x7fac7019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac76a5e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac70111f80 con 0x7fac701027a0 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac76a5e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fac70111e00 con 0x7fac70106f20 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac76a5e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fac70112100 con 0x7fac70101df0 2024-09-16T00:12:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 0x7fac70198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 0x7fac70198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53642/0 (socket says 172.21.15.78:53642) 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.996+0000 7fac6f7fe640 1 -- 172.21.15.78:0/2714009728 learned_addr learned my addr 172.21.15.78:0/2714009728 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6ffff640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac70101df0 0x7fac70198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6f7fe640 1 -- 172.21.15.78:0/2714009728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac70101df0 msgr2=0x7fac70198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6f7fe640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac70101df0 0x7fac70198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6f7fe640 1 -- 172.21.15.78:0/2714009728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac70106f20 msgr2=0x7fac7019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6f7fe640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac70106f20 0x7fac7019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6f7fe640 1 -- 172.21.15.78:0/2714009728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7019d3d0 con 0x7fac701027a0 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6f7fe640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 0x7fac70198960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fac64005ab0 tx=0x7fac640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6d7fa640 1 -- 172.21.15.78:0/2714009728 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6403f030 con 0x7fac701027a0 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7019d660 con 0x7fac701027a0 2024-09-16T00:12:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6d7fa640 1 -- 172.21.15.78:0/2714009728 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac64003ce0 con 0x7fac701027a0 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac6d7fa640 1 -- 172.21.15.78:0/2714009728 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64035070 con 0x7fac701027a0 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.997+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac701a4fa0 con 0x7fac701027a0 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.998+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac701021d0 con 0x7fac701027a0 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:23.999+0000 7fac6d7fa640 1 -- 172.21.15.78:0/2714009728 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fac64042030 con 0x7fac701027a0 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.000+0000 7fac6d7fa640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fac580a32d0 0x7fac580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.000+0000 7fac6ffff640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fac580a32d0 0x7fac580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.000+0000 7fac6d7fa640 1 -- 172.21.15.78:0/2714009728 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fac640f0780 con 0x7fac701027a0 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.000+0000 7fac6ffff640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fac580a32d0 0x7fac580a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fac600058c0 tx=0x7fac6000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.004+0000 7fac6d7fa640 1 -- 172.21.15.78:0/2714009728 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac640bd000 con 0x7fac701027a0 2024-09-16T00:12:24.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.174+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fac70199780 con 0x7fac701027a0 2024-09-16T00:12:24.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.175+0000 7fac6d7fa640 1 -- 172.21.15.78:0/2714009728 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fac70199780 con 0x7fac701027a0 2024-09-16T00:12:24.176 INFO:teuthology.orchestra.run.smithi078.stdout:77309411404 2024-09-16T00:12:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fac580a32d0 msgr2=0x7fac580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fac580a32d0 0x7fac580a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fac600058c0 tx=0x7fac6000a4c0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 msgr2=0x7fac70198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 0x7fac70198960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fac64005ab0 tx=0x7fac640042f0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 shutdown_connections 2024-09-16T00:12:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fac580a32d0 0x7fac580a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fac70106f20 0x7fac7019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac701027a0 0x7fac70198960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 --2- 172.21.15.78:0/2714009728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac70101df0 0x7fac70198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 >> 172.21.15.78:0/2714009728 conn(0x7fac700fd8b0 msgr2=0x7fac700febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:24.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 shutdown_connections 2024-09-16T00:12:24.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.177+0000 7fac76a5e640 1 -- 172.21.15.78:0/2714009728 wait complete. 2024-09-16T00:12:24.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 -- 172.21.15.78:0/3804714588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 msgr2=0x7f890c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 --2- 172.21.15.78:0/3804714588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 0x7f890c103120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009900 tx=0x7f88fc02a480 comp rx=0 tx=0).stop 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 -- 172.21.15.78:0/3804714588 shutdown_connections 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 --2- 172.21.15.78:0/3804714588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f890c10c170 0x7f890c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 --2- 172.21.15.78:0/3804714588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f890c103660 0x7f890c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 --2- 172.21.15.78:0/3804714588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 0x7f890c103120 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 -- 172.21.15.78:0/3804714588 >> 172.21.15.78:0/3804714588 conn(0x7f890c0fc7a0 msgr2=0x7f890c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 -- 172.21.15.78:0/3804714588 shutdown_connections 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.224+0000 7f891212b640 1 -- 172.21.15.78:0/3804714588 wait complete. 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f891212b640 1 Processor -- start 2024-09-16T00:12:24.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f891212b640 1 -- start start 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f891212b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 0x7f890c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f891212b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f890c103660 0x7f890c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f891212b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f890c10c170 0x7f890c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f891212b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f890c111f70 con 0x7f890c102d40 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f891212b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f890c111df0 con 0x7f890c103660 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f891212b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f890c1120f0 con 0x7f890c10c170 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.225+0000 7f890b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 0x7f890c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 0x7f890c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53660/0 (socket says 172.21.15.78:53660) 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890b7fe640 1 -- 172.21.15.78:0/3847868227 learned_addr learned my addr 172.21.15.78:0/3847868227 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890affd640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f890c103660 0x7f890c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890bfff640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f890c10c170 0x7f890c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:24.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890b7fe640 1 -- 172.21.15.78:0/3847868227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f890c10c170 msgr2=0x7f890c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890b7fe640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f890c10c170 0x7f890c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890b7fe640 1 -- 172.21.15.78:0/3847868227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f890c103660 msgr2=0x7f890c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890b7fe640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f890c103660 0x7f890c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890b7fe640 1 -- 172.21.15.78:0/3847868227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890c19d420 con 0x7f890c102d40 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f890b7fe640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 0x7f890c198470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005ab0 tx=0x7f88fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f8908ff9640 1 -- 172.21.15.78:0/3847868227 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc03f030 con 0x7f890c102d40 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f8908ff9640 1 -- 172.21.15.78:0/3847868227 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc003ce0 con 0x7f890c102d40 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890c19d6b0 con 0x7f890c102d40 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f8908ff9640 1 -- 172.21.15.78:0/3847868227 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc035070 con 0x7f890c102d40 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.226+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f890c1a4ff0 con 0x7f890c102d40 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.227+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890c06b070 con 0x7f890c102d40 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.228+0000 7f8908ff9640 1 -- 172.21.15.78:0/3847868227 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f88fc042030 con 0x7f890c102d40 2024-09-16T00:12:24.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.229+0000 7f8908ff9640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f88f40a32d0 0x7f88f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.229+0000 7f890affd640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f88f40a32d0 0x7f88f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.229+0000 7f8908ff9640 1 -- 172.21.15.78:0/3847868227 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f88fc0f0780 con 0x7f890c102d40 2024-09-16T00:12:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.230+0000 7f890affd640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f88f40a32d0 0x7f88f40a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f890c10d6b0 tx=0x7f8900005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:24.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.233+0000 7f8908ff9640 1 -- 172.21.15.78:0/3847868227 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88fc0bd000 con 0x7f890c102d40 2024-09-16T00:12:24.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.421+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f890c1071e0 con 0x7f890c102d40 2024-09-16T00:12:24.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.422+0000 7f8908ff9640 1 -- 172.21.15.78:0/3847868227 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88fc0bc900 con 0x7f890c102d40 2024-09-16T00:12:24.423 INFO:teuthology.orchestra.run.smithi078.stdout:55834574929 2024-09-16T00:12:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f88f40a32d0 msgr2=0x7f88f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f88f40a32d0 0x7f88f40a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f890c10d6b0 tx=0x7f8900005ec0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 msgr2=0x7f890c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 0x7f890c198470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005ab0 tx=0x7f88fc0042f0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 shutdown_connections 2024-09-16T00:12:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f88f40a32d0 0x7f88f40a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f890c10c170 0x7f890c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f890c103660 0x7f890c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 --2- 172.21.15.78:0/3847868227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f890c102d40 0x7f890c198470 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 >> 172.21.15.78:0/3847868227 conn(0x7f890c0fc7a0 msgr2=0x7f890c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 shutdown_connections 2024-09-16T00:12:24.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:24.424+0000 7f891212b640 1 -- 172.21.15.78:0/3847868227 wait complete. 2024-09-16T00:12:24.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:24 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2436823832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:12:24.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:24 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2714009728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:12:24.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:24 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3847868227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:12:24.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:24 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2436823832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:12:24.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:24 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2714009728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:12:24.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:24 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3847868227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:12:24.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:24 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2436823832' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:12:24.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:24 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2714009728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:12:24.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:24 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3847868227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:12:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:25.184 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411404 for osd.2 2024-09-16T00:12:25.184 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:25.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:25 smithi078 ceph-mon[31157]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:12:25.867 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574929 for osd.1 2024-09-16T00:12:25.867 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:25.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:25 smithi193 ceph-mon[35484]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:12:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:25 smithi148 ceph-mon[35599]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-16T00:12:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 -- 172.21.15.78:0/3166635922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830101de0 msgr2=0x7fd8301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 --2- 172.21.15.78:0/3166635922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830101de0 0x7fd8301021c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009900 tx=0x7fd82c02a410 comp rx=0 tx=0).stop 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 -- 172.21.15.78:0/3166635922 shutdown_connections 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 --2- 172.21.15.78:0/3166635922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd830106f10 0x7fd83010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 --2- 172.21.15.78:0/3166635922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd830102790 0x7fd8301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 --2- 172.21.15.78:0/3166635922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830101de0 0x7fd8301021c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 -- 172.21.15.78:0/3166635922 >> 172.21.15.78:0/3166635922 conn(0x7fd8300fd8c0 msgr2=0x7fd8300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 -- 172.21.15.78:0/3166635922 shutdown_connections 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.824+0000 7fd836c90640 1 -- 172.21.15.78:0/3166635922 wait complete. 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd836c90640 1 Processor -- start 2024-09-16T00:12:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd836c90640 1 -- start start 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd836c90640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd830101de0 0x7fd8301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd836c90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd830102790 0x7fd8301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd836c90640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830106f10 0x7fd83019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd836c90640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd830111f90 con 0x7fd830106f10 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd836c90640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd830111e10 con 0x7fd830102790 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd836c90640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd830112110 con 0x7fd830101de0 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd83522d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830106f10 0x7fd83019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd83522d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830106f10 0x7fd83019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35416/0 (socket says 172.21.15.78:35416) 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.825+0000 7fd83522d640 1 -- 172.21.15.78:0/415833933 learned_addr learned my addr 172.21.15.78:0/415833933 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd834a2c640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd830101de0 0x7fd8301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd827fff640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd830102790 0x7fd8301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd83522d640 1 -- 172.21.15.78:0/415833933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd830101de0 msgr2=0x7fd8301984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd83522d640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd830101de0 0x7fd8301984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd83522d640 1 -- 172.21.15.78:0/415833933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd830102790 msgr2=0x7fd8301989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd83522d640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd830102790 0x7fd8301989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd83522d640 1 -- 172.21.15.78:0/415833933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83019d460 con 0x7fd830106f10 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd827fff640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd830102790 0x7fd8301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd83522d640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830106f10 0x7fd83019cd80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd82800b970 tx=0x7fd82800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd825ffb640 1 -- 172.21.15.78:0/415833933 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828019070 con 0x7fd830106f10 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd825ffb640 1 -- 172.21.15.78:0/415833933 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82800cb40 con 0x7fd830106f10 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83019d750 con 0x7fd830106f10 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd825ffb640 1 -- 172.21.15.78:0/415833933 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828007650 con 0x7fd830106f10 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.826+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a5030 con 0x7fd830106f10 2024-09-16T00:12:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.827+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8301021c0 con 0x7fd830106f10 2024-09-16T00:12:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.828+0000 7fd825ffb640 1 -- 172.21.15.78:0/415833933 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd82801c070 con 0x7fd830106f10 2024-09-16T00:12:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.829+0000 7fd825ffb640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8180a32d0 0x7fd8180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.829+0000 7fd834a2c640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8180a32d0 0x7fd8180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.829+0000 7fd825ffb640 1 -- 172.21.15.78:0/415833933 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fd8280cf370 con 0x7fd830106f10 2024-09-16T00:12:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.829+0000 7fd834a2c640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8180a32d0 0x7fd8180a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0096c0 tx=0x7fd82c031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:26.833+0000 7fd825ffb640 1 -- 172.21.15.78:0/415833933 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd82809bd00 con 0x7fd830106f10 2024-09-16T00:12:27.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.003+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd83010a150 con 0x7fd830106f10 2024-09-16T00:12:27.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.003+0000 7fd825ffb640 1 -- 172.21.15.78:0/415833933 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd82809b600 con 0x7fd830106f10 2024-09-16T00:12:27.005 INFO:teuthology.orchestra.run.smithi078.stdout:266287972393 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8180a32d0 msgr2=0x7fd8180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8180a32d0 0x7fd8180a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0096c0 tx=0x7fd82c031490 comp rx=0 tx=0).stop 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830106f10 msgr2=0x7fd83019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830106f10 0x7fd83019cd80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd82800b970 tx=0x7fd82800be30 comp rx=0 tx=0).stop 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 shutdown_connections 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8180a32d0 0x7fd8180a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd830106f10 0x7fd83019cd80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd830102790 0x7fd8301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 --2- 172.21.15.78:0/415833933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd830101de0 0x7fd8301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.005+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 >> 172.21.15.78:0/415833933 conn(0x7fd8300fd8c0 msgr2=0x7fd8300fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.006+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 shutdown_connections 2024-09-16T00:12:27.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.006+0000 7fd836c90640 1 -- 172.21.15.78:0/415833933 wait complete. 2024-09-16T00:12:27.688 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:27 smithi078 ceph-mon[31157]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-16T00:12:27.688 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:27 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/415833933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:12:27.714 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972391 got 266287972393 for osd.9 2024-09-16T00:12:27.714 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:27 smithi193 ceph-mon[35484]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-16T00:12:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:27 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/415833933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:12:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 -- 172.21.15.78:0/3197655245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590102790 msgr2=0x7ff5901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 --2- 172.21.15.78:0/3197655245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590102790 0x7ff5901067e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009900 tx=0x7ff58c02a410 comp rx=0 tx=0).stop 2024-09-16T00:12:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 -- 172.21.15.78:0/3197655245 shutdown_connections 2024-09-16T00:12:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 --2- 172.21.15.78:0/3197655245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff590106f10 0x7ff59010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 --2- 172.21.15.78:0/3197655245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590102790 0x7ff5901067e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 --2- 172.21.15.78:0/3197655245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff590101de0 0x7ff5901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 -- 172.21.15.78:0/3197655245 >> 172.21.15.78:0/3197655245 conn(0x7ff5900fd8c0 msgr2=0x7ff5900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 -- 172.21.15.78:0/3197655245 shutdown_connections 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.943+0000 7ff59813d640 1 -- 172.21.15.78:0/3197655245 wait complete. 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff59813d640 1 Processor -- start 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff59813d640 1 -- start start 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff59813d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590101de0 0x7ff5900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff59813d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff590102790 0x7ff590074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff59813d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff590106f10 0x7ff590074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff59813d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff590111f70 con 0x7ff590101de0 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff59813d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff590111df0 con 0x7ff590106f10 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff59813d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901120f0 con 0x7ff590102790 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff595ed9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590101de0 0x7ff5900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff595ed9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590101de0 0x7ff5900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35430/0 (socket says 172.21.15.78:35430) 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.944+0000 7ff595ed9640 1 -- 172.21.15.78:0/1383927852 learned_addr learned my addr 172.21.15.78:0/1383927852 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff595ed9640 1 -- 172.21.15.78:0/1383927852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff590102790 msgr2=0x7ff590074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff595ed9640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff590102790 0x7ff590074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff595ed9640 1 -- 172.21.15.78:0/1383927852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff590106f10 msgr2=0x7ff590074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff595ed9640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff590106f10 0x7ff590074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff595ed9640 1 -- 172.21.15.78:0/1383927852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5901a68b0 con 0x7ff590101de0 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff595ed9640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590101de0 0x7ff5900744f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c890 tx=0x7ff58000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff586ffd640 1 -- 172.21.15.78:0/1383927852 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580004220 con 0x7ff590101de0 2024-09-16T00:12:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff586ffd640 1 -- 172.21.15.78:0/1383927852 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5800043c0 con 0x7ff590101de0 2024-09-16T00:12:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5901a6ba0 con 0x7ff590101de0 2024-09-16T00:12:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff586ffd640 1 -- 172.21.15.78:0/1383927852 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5800049e0 con 0x7ff590101de0 2024-09-16T00:12:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.945+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5901a7130 con 0x7ff590101de0 2024-09-16T00:12:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.946+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5901021c0 con 0x7ff590101de0 2024-09-16T00:12:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.947+0000 7ff586ffd640 1 -- 172.21.15.78:0/1383927852 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff5800079b0 con 0x7ff590101de0 2024-09-16T00:12:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.948+0000 7ff586ffd640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff57c0a3350 0x7ff57c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.948+0000 7ff5956d8640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff57c0a3350 0x7ff57c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.948+0000 7ff586ffd640 1 -- 172.21.15.78:0/1383927852 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7ff5800cfe50 con 0x7ff590101de0 2024-09-16T00:12:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.948+0000 7ff5956d8640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff57c0a3350 0x7ff57c0a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff58c02a930 tx=0x7ff58c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:27.951+0000 7ff586ffd640 1 -- 172.21.15.78:0/1383927852 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff580014030 con 0x7ff590101de0 2024-09-16T00:12:27.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:27 smithi148 ceph-mon[35599]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-09-16T00:12:27.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:27 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/415833933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:12:28.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.123+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff590075a20 con 0x7ff590101de0 2024-09-16T00:12:28.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.124+0000 7ff586ffd640 1 -- 172.21.15.78:0/1383927852 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff58009c750 con 0x7ff590101de0 2024-09-16T00:12:28.125 INFO:teuthology.orchestra.run.smithi078.stdout:103079215175 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff57c0a3350 msgr2=0x7ff57c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff57c0a3350 0x7ff57c0a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff58c02a930 tx=0x7ff58c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590101de0 msgr2=0x7ff5900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590101de0 0x7ff5900744f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c890 tx=0x7ff58000cd50 comp rx=0 tx=0).stop 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 shutdown_connections 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff57c0a3350 0x7ff57c0a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff590106f10 0x7ff590074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff590102790 0x7ff590074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 --2- 172.21.15.78:0/1383927852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff590101de0 0x7ff5900744f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 >> 172.21.15.78:0/1383927852 conn(0x7ff5900fd8c0 msgr2=0x7ff5900feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 shutdown_connections 2024-09-16T00:12:28.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.126+0000 7ff59813d640 1 -- 172.21.15.78:0/1383927852 wait complete. 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 -- 172.21.15.78:0/960767321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880102d90 msgr2=0x7f6880103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 --2- 172.21.15.78:0/960767321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880102d90 0x7f6880103170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6874009930 tx=0x7f687402a420 comp rx=0 tx=0).stop 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 -- 172.21.15.78:0/960767321 shutdown_connections 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 --2- 172.21.15.78:0/960767321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6880108290 0x7f688010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 --2- 172.21.15.78:0/960767321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6880103740 0x7f6880107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 --2- 172.21.15.78:0/960767321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880102d90 0x7f6880103170 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 -- 172.21.15.78:0/960767321 >> 172.21.15.78:0/960767321 conn(0x7f68800fc7d0 msgr2=0x7f68800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 -- 172.21.15.78:0/960767321 shutdown_connections 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 -- 172.21.15.78:0/960767321 wait complete. 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 Processor -- start 2024-09-16T00:12:28.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.130+0000 7f6886c08640 1 -- start start 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f6886c08640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6880102d90 0x7f6880198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f6886c08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880103740 0x7f6880198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f6886c08640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6880108290 0x7f688019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f6886c08640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6880112090 con 0x7f6880103740 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f6886c08640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6880111f10 con 0x7f6880108290 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f6886c08640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6880112210 con 0x7f6880102d90 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880103740 0x7f6880198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880103740 0x7f6880198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35452/0 (socket says 172.21.15.78:35452) 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 -- 172.21.15.78:0/678164012 learned_addr learned my addr 172.21.15.78:0/678164012 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f68849a4640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6880102d90 0x7f6880198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f68851a5640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6880108290 0x7f688019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:28.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 -- 172.21.15.78:0/678164012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6880102d90 msgr2=0x7f6880198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6880102d90 0x7f6880198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 -- 172.21.15.78:0/678164012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6880108290 msgr2=0x7f688019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6880108290 0x7f688019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 -- 172.21.15.78:0/678164012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688019d500 con 0x7f6880103740 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f68851a5640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6880108290 0x7f688019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.131+0000 7f687ffff640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880103740 0x7f6880198a90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f686c00d770 tx=0x7f686c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.132+0000 7f687dffb640 1 -- 172.21.15.78:0/678164012 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c017070 con 0x7f6880103740 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.132+0000 7f687dffb640 1 -- 172.21.15.78:0/678164012 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686c009c30 con 0x7f6880103740 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.132+0000 7f687dffb640 1 -- 172.21.15.78:0/678164012 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c00f620 con 0x7f6880103740 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.132+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688019d7f0 con 0x7f6880103740 2024-09-16T00:12:28.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.132+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68801a50d0 con 0x7f6880103740 2024-09-16T00:12:28.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.133+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f688006a7f0 con 0x7f6880103740 2024-09-16T00:12:28.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.135+0000 7f687dffb640 1 -- 172.21.15.78:0/678164012 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f686c01a070 con 0x7f6880103740 2024-09-16T00:12:28.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.136+0000 7f687dffb640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f68700a3350 0x7f68700a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:28.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.136+0000 7f687dffb640 1 -- 172.21.15.78:0/678164012 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f686c0cf880 con 0x7f6880103740 2024-09-16T00:12:28.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.136+0000 7f68849a4640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f68700a3350 0x7f68700a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:28.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.136+0000 7f68849a4640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f68700a3350 0x7f68700a5810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6874009900 tx=0x7f68740058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:28.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.139+0000 7f687dffb640 1 -- 172.21.15.78:0/678164012 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f686c009de0 con 0x7f6880103740 2024-09-16T00:12:28.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.319+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f688010a2a0 con 0x7f6880103740 2024-09-16T00:12:28.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.319+0000 7f687dffb640 1 -- 172.21.15.78:0/678164012 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f686c09c180 con 0x7f6880103740 2024-09-16T00:12:28.321 INFO:teuthology.orchestra.run.smithi078.stdout:154618822716 2024-09-16T00:12:28.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f68700a3350 msgr2=0x7f68700a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:28.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f68700a3350 0x7f68700a5810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6874009900 tx=0x7f68740058d0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880103740 msgr2=0x7f6880198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:28.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880103740 0x7f6880198a90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f686c00d770 tx=0x7f686c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:12:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 shutdown_connections 2024-09-16T00:12:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f68700a3350 0x7f68700a5810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6880108290 0x7f688019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6880103740 0x7f6880198a90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 --2- 172.21.15.78:0/678164012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6880102d90 0x7f6880198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 >> 172.21.15.78:0/678164012 conn(0x7f68800fc7d0 msgr2=0x7f6880105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 shutdown_connections 2024-09-16T00:12:28.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:28.321+0000 7f6886c08640 1 -- 172.21.15.78:0/678164012 wait complete. 2024-09-16T00:12:28.663 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:28 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1383927852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:12:28.663 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:28 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/678164012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:12:28.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:28 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1383927852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:12:28.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:28 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/678164012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:12:28.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:28 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1383927852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:12:28.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:28 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/678164012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:12:29.008 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808866 for osd.10 2024-09-16T00:12:29.009 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:29.550 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822714 got 154618822716 for osd.5 2024-09-16T00:12:29.551 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:29.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:29 smithi078 ceph-mon[31157]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-16T00:12:29.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:29 smithi193 ceph-mon[35484]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-16T00:12:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:29 smithi148 ceph-mon[35599]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-16T00:12:30.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215173 got 103079215175 for osd.3 2024-09-16T00:12:30.030 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:30.537 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:30.537 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:30.538 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:30.539 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:30.587 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430256 got 206158430257 for osd.7 2024-09-16T00:12:30.587 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:30.595 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 -- 172.21.15.78:0/3444593731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 msgr2=0x7fde9810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/3444593731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 0x7fde9810e600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fde80009900 tx=0x7fde8002a480 comp rx=0 tx=0).stop 2024-09-16T00:12:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 -- 172.21.15.78:0/3444593731 shutdown_connections 2024-09-16T00:12:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/3444593731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 0x7fde9810e600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/3444593731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde981036c0 0x7fde9810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/3444593731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde98102da0 0x7fde98103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 -- 172.21.15.78:0/3444593731 >> 172.21.15.78:0/3444593731 conn(0x7fde980fc7a0 msgr2=0x7fde980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:31.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 -- 172.21.15.78:0/3444593731 shutdown_connections 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.269+0000 7fde9ef5c640 1 -- 172.21.15.78:0/3444593731 wait complete. 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ef5c640 1 Processor -- start 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ef5c640 1 -- start start 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ef5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde98102da0 0x7fde98198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ef5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde981036c0 0x7fde981989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ef5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 0x7fde9819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ef5c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde98111fa0 con 0x7fde9810c180 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ef5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fde98111e20 con 0x7fde98102da0 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ef5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fde98112120 con 0x7fde981036c0 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9d4f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 0x7fde9819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9d4f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 0x7fde9819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35464/0 (socket says 172.21.15.78:35464) 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9d4f9640 1 -- 172.21.15.78:0/1356325065 learned_addr learned my addr 172.21.15.78:0/1356325065 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde97fff640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde981036c0 0x7fde981989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9ccf8640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde98102da0 0x7fde98198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9d4f9640 1 -- 172.21.15.78:0/1356325065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde981036c0 msgr2=0x7fde981989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9d4f9640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde981036c0 0x7fde981989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9d4f9640 1 -- 172.21.15.78:0/1356325065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde98102da0 msgr2=0x7fde98198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9d4f9640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde98102da0 0x7fde98198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.270+0000 7fde9d4f9640 1 -- 172.21.15.78:0/1356325065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9819d430 con 0x7fde9810c180 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.271+0000 7fde9d4f9640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 0x7fde9819cd50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde80009230 tx=0x7fde800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.271+0000 7fde95ffb640 1 -- 172.21.15.78:0/1356325065 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8003f030 con 0x7fde9810c180 2024-09-16T00:12:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.271+0000 7fde95ffb640 1 -- 172.21.15.78:0/1356325065 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde80003ce0 con 0x7fde9810c180 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.271+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9819d6c0 con 0x7fde9810c180 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.271+0000 7fde95ffb640 1 -- 172.21.15.78:0/1356325065 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80035070 con 0x7fde9810c180 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.271+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981a5000 con 0x7fde9810c180 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.272+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde98103180 con 0x7fde9810c180 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.273+0000 7fde95ffb640 1 -- 172.21.15.78:0/1356325065 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fde80042030 con 0x7fde9810c180 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.274+0000 7fde95ffb640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fde840a3350 0x7fde840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.274+0000 7fde9ccf8640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fde840a3350 0x7fde840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.274+0000 7fde95ffb640 1 -- 172.21.15.78:0/1356325065 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fde800f0670 con 0x7fde9810c180 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.275+0000 7fde9ccf8640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fde840a3350 0x7fde840a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde900058c0 tx=0x7fde9000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:31.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.278+0000 7fde95ffb640 1 -- 172.21.15.78:0/1356325065 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde800bcef0 con 0x7fde9810c180 2024-09-16T00:12:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.448+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fde98107200 con 0x7fde9810c180 2024-09-16T00:12:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.448+0000 7fde95ffb640 1 -- 172.21.15.78:0/1356325065 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fde800bc7f0 con 0x7fde9810c180 2024-09-16T00:12:31.449 INFO:teuthology.orchestra.run.smithi078.stdout:133143986243 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fde840a3350 msgr2=0x7fde840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fde840a3350 0x7fde840a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde900058c0 tx=0x7fde9000a4c0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 msgr2=0x7fde9819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 0x7fde9819cd50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fde80009230 tx=0x7fde800042f0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 shutdown_connections 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fde840a3350 0x7fde840a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde9810c180 0x7fde9819cd50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde981036c0 0x7fde981989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 --2- 172.21.15.78:0/1356325065 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fde98102da0 0x7fde98198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 >> 172.21.15.78:0/1356325065 conn(0x7fde980fc7a0 msgr2=0x7fde980fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 shutdown_connections 2024-09-16T00:12:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.451+0000 7fde9ef5c640 1 -- 172.21.15.78:0/1356325065 wait complete. 2024-09-16T00:12:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 -- 172.21.15.78:0/1798365146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40106f10 msgr2=0x7f3a4010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/1798365146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40106f10 0x7f3a4010ea90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009ca0 tx=0x7f3a3802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 -- 172.21.15.78:0/1798365146 shutdown_connections 2024-09-16T00:12:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/1798365146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40106f10 0x7f3a4010ea90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/1798365146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a40102790 0x7f3a401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/1798365146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a40101de0 0x7f3a401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 -- 172.21.15.78:0/1798365146 >> 172.21.15.78:0/1798365146 conn(0x7f3a400fd8c0 msgr2=0x7f3a400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 -- 172.21.15.78:0/1798365146 shutdown_connections 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.501+0000 7f3a44f2e640 1 -- 172.21.15.78:0/1798365146 wait complete. 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a44f2e640 1 Processor -- start 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a44f2e640 1 -- start start 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a44f2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40101de0 0x7f3a400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a44f2e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a40102790 0x7f3a40074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a44f2e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a40106f10 0x7f3a40074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a44f2e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40111fc0 con 0x7f3a40101de0 2024-09-16T00:12:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a44f2e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40111e40 con 0x7f3a40106f10 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a44f2e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a40112140 con 0x7f3a40102790 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a3e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40101de0 0x7f3a400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a3e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40101de0 0x7f3a400744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35468/0 (socket says 172.21.15.78:35468) 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a3e59c640 1 -- 172.21.15.78:0/2220532537 learned_addr learned my addr 172.21.15.78:0/2220532537 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a3ed9d640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a40106f10 0x7f3a40074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a3e59c640 1 -- 172.21.15.78:0/2220532537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a40102790 msgr2=0x7f3a40074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a37fff640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a40102790 0x7f3a40074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.502+0000 7f3a3e59c640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a40102790 0x7f3a40074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a3e59c640 1 -- 172.21.15.78:0/2220532537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a40106f10 msgr2=0x7f3a40074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a3e59c640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a40106f10 0x7f3a40074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a3e59c640 1 -- 172.21.15.78:0/2220532537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a401a6890 con 0x7f3a40101de0 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a3e59c640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40101de0 0x7f3a400744f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000d950 tx=0x7f3a3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a377fe640 1 -- 172.21.15.78:0/2220532537 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30017030 con 0x7f3a40101de0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a377fe640 1 -- 172.21.15.78:0/2220532537 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a30004020 con 0x7f3a40101de0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a401a6b80 con 0x7f3a40101de0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a377fe640 1 -- 172.21.15.78:0/2220532537 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300047a0 con 0x7f3a40101de0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.503+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a401a70c0 con 0x7f3a40101de0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.504+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a401021c0 con 0x7f3a40101de0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.505+0000 7f3a377fe640 1 -- 172.21.15.78:0/2220532537 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3a300041d0 con 0x7f3a40101de0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.506+0000 7f3a377fe640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a2c0a32d0 0x7f3a2c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.506+0000 7f3a37fff640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a2c0a32d0 0x7f3a2c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.506+0000 7f3a377fe640 1 -- 172.21.15.78:0/2220532537 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f3a300cf8c0 con 0x7f3a40101de0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.506+0000 7f3a37fff640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a2c0a32d0 0x7f3a2c0a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a40075f80 tx=0x7f3a28005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.510+0000 7f3a377fe640 1 -- 172.21.15.78:0/2220532537 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a3009c0e0 con 0x7f3a40101de0 2024-09-16T00:12:31.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.681+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3a40075a20 con 0x7f3a40101de0 2024-09-16T00:12:31.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.681+0000 7f3a377fe640 1 -- 172.21.15.78:0/2220532537 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a3009b9e0 con 0x7f3a40101de0 2024-09-16T00:12:31.682 INFO:teuthology.orchestra.run.smithi078.stdout:34359738454 2024-09-16T00:12:31.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.683+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a2c0a32d0 msgr2=0x7f3a2c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.683+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a2c0a32d0 0x7f3a2c0a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3a40075f80 tx=0x7f3a28005ed0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.683+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40101de0 msgr2=0x7f3a400744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.683+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40101de0 0x7f3a400744f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000d950 tx=0x7f3a3000de10 comp rx=0 tx=0).stop 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.684+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 shutdown_connections 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.684+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a2c0a32d0 0x7f3a2c0a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.684+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a40106f10 0x7f3a40074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.684+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a40102790 0x7f3a40074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.684+0000 7f3a44f2e640 1 --2- 172.21.15.78:0/2220532537 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a40101de0 0x7f3a400744f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.684+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 >> 172.21.15.78:0/2220532537 conn(0x7f3a400fd8c0 msgr2=0x7f3a400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.684+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 shutdown_connections 2024-09-16T00:12:31.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:31.684+0000 7f3a44f2e640 1 -- 172.21.15.78:0/2220532537 wait complete. 2024-09-16T00:12:31.714 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:31 smithi078 ceph-mon[31157]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-16T00:12:31.714 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:31 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1356325065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:12:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:31 smithi193 ceph-mon[35484]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-16T00:12:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:31 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1356325065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:12:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:31 smithi148 ceph-mon[35599]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-16T00:12:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:31 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1356325065' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:12:32.430 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738454 for osd.0 2024-09-16T00:12:32.431 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:32.653 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:32 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2220532537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:12:32.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.844+0000 7f55a549b640 1 -- 172.21.15.78:0/2978777253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 msgr2=0x7f55a0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.844+0000 7f55a549b640 1 --2- 172.21.15.78:0/2978777253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 0x7f55a0100ef0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5588009930 tx=0x7f558802a420 comp rx=0 tx=0).stop 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 -- 172.21.15.78:0/2978777253 shutdown_connections 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 --2- 172.21.15.78:0/2978777253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0101430 0x7f55a010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 --2- 172.21.15.78:0/2978777253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 0x7f55a0100ef0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 --2- 172.21.15.78:0/2978777253 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a0106a90 0x7f55a0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 -- 172.21.15.78:0/2978777253 >> 172.21.15.78:0/2978777253 conn(0x7f55a00fc7d0 msgr2=0x7f55a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 -- 172.21.15.78:0/2978777253 shutdown_connections 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 -- 172.21.15.78:0/2978777253 wait complete. 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 Processor -- start 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 -- start start 2024-09-16T00:12:32.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 0x7f55a0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.845+0000 7f55a549b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a0101430 0x7f55a01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f55a549b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0106a90 0x7f55a019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f55a549b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0111fb0 con 0x7f55a0100a90 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f55a549b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0111e30 con 0x7f55a0101430 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f55a549b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55a0112130 con 0x7f55a0106a90 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 0x7f55a0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 0x7f55a0198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35490/0 (socket says 172.21.15.78:35490) 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 -- 172.21.15.78:0/3758991532 learned_addr learned my addr 172.21.15.78:0/3758991532 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559f7fe640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0106a90 0x7f55a019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559e7fc640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a0101430 0x7f55a01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 -- 172.21.15.78:0/3758991532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0106a90 msgr2=0x7f55a019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:32.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0106a90 0x7f55a019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 -- 172.21.15.78:0/3758991532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a0101430 msgr2=0x7f55a01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a0101430 0x7f55a01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 -- 172.21.15.78:0/3758991532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a019d440 con 0x7f55a0100a90 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559f7fe640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0106a90 0x7f55a019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f559effd640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 0x7f55a0198490 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f559000eab0 tx=0x7f559000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f5597fff640 1 -- 172.21.15.78:0/3758991532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559000ce40 con 0x7f55a0100a90 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f5597fff640 1 -- 172.21.15.78:0/3758991532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5590011950 con 0x7f55a0100a90 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a019d730 con 0x7f55a0100a90 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f5597fff640 1 -- 172.21.15.78:0/3758991532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590010ae0 con 0x7f55a0100a90 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.846+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a01a5010 con 0x7f55a0100a90 2024-09-16T00:12:32.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.847+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a01021d0 con 0x7f55a0100a90 2024-09-16T00:12:32.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.848+0000 7f5597fff640 1 -- 172.21.15.78:0/3758991532 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5590011b00 con 0x7f55a0100a90 2024-09-16T00:12:32.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.849+0000 7f5597fff640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f558c0a32d0 0x7f558c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:32.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.849+0000 7f559e7fc640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f558c0a32d0 0x7f558c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:32.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.849+0000 7f5597fff640 1 -- 172.21.15.78:0/3758991532 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f55900d04d0 con 0x7f55a0100a90 2024-09-16T00:12:32.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.850+0000 7f559e7fc640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f558c0a32d0 0x7f558c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5588000c00 tx=0x7f5588005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:32.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:32.853+0000 7f5597fff640 1 -- 172.21.15.78:0/3758991532 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5590016030 con 0x7f55a0100a90 2024-09-16T00:12:32.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:32 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2220532537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:12:32.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:32 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2220532537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:12:33.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.025+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f55a0109000 con 0x7f55a0100a90 2024-09-16T00:12:33.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.026+0000 7f5597fff640 1 -- 172.21.15.78:0/3758991532 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f559009cd50 con 0x7f55a0100a90 2024-09-16T00:12:33.027 INFO:teuthology.orchestra.run.smithi078.stdout:184683593784 2024-09-16T00:12:33.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.027+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f558c0a32d0 msgr2=0x7f558c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:33.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.027+0000 7f55a549b640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f558c0a32d0 0x7f558c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5588000c00 tx=0x7f5588005a20 comp rx=0 tx=0).stop 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.027+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 msgr2=0x7f55a0198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.027+0000 7f55a549b640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 0x7f55a0198490 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f559000eab0 tx=0x7f559000ef70 comp rx=0 tx=0).stop 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.028+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 shutdown_connections 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.028+0000 7f55a549b640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f558c0a32d0 0x7f558c0a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.028+0000 7f55a549b640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55a0106a90 0x7f55a019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.028+0000 7f55a549b640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55a0101430 0x7f55a01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.028+0000 7f55a549b640 1 --2- 172.21.15.78:0/3758991532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55a0100a90 0x7f55a0198490 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.028+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 >> 172.21.15.78:0/3758991532 conn(0x7f55a00fc7d0 msgr2=0x7f55a00fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.028+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 shutdown_connections 2024-09-16T00:12:33.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:33.028+0000 7f55a549b640 1 -- 172.21.15.78:0/3758991532 wait complete. 2024-09-16T00:12:33.731 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986239 got 133143986243 for osd.4 2024-09-16T00:12:33.731 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:33.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:33 smithi078 ceph-mon[31157]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T00:12:33.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:33 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3758991532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:12:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:33 smithi193 ceph-mon[35484]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T00:12:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:33 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3758991532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:12:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:33 smithi148 ceph-mon[35599]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-16T00:12:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:33 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3758991532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:12:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 -- 172.21.15.78:0/574685328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4102790 msgr2=0x7f35d41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 --2- 172.21.15.78:0/574685328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4102790 0x7f35d41067e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009ca0 tx=0x7f35c402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 -- 172.21.15.78:0/574685328 shutdown_connections 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 --2- 172.21.15.78:0/574685328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4106f10 0x7f35d410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 --2- 172.21.15.78:0/574685328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4102790 0x7f35d41067e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 --2- 172.21.15.78:0/574685328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35d4101de0 0x7f35d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 -- 172.21.15.78:0/574685328 >> 172.21.15.78:0/574685328 conn(0x7f35d40fd8c0 msgr2=0x7f35d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 -- 172.21.15.78:0/574685328 shutdown_connections 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.272+0000 7f35dacc7640 1 -- 172.21.15.78:0/574685328 wait complete. 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35dacc7640 1 Processor -- start 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35dacc7640 1 -- start start 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35dacc7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4101de0 0x7f35d41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35dacc7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35d4102790 0x7f35d4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35dacc7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4106f10 0x7f35d419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35dacc7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4112010 con 0x7f35d4101de0 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35dacc7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4111e90 con 0x7f35d4106f10 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35dacc7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4112190 con 0x7f35d4102790 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4101de0 0x7f35d41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4101de0 0x7f35d41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35500/0 (socket says 172.21.15.78:35500) 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 -- 172.21.15.78:0/4108109822 learned_addr learned my addr 172.21.15.78:0/4108109822 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d3fff640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35d4102790 0x7f35d4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 -- 172.21.15.78:0/4108109822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35d4102790 msgr2=0x7f35d4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35d4102790 0x7f35d4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d9264640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4106f10 0x7f35d419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 -- 172.21.15.78:0/4108109822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4106f10 msgr2=0x7f35d419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4106f10 0x7f35d419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 -- 172.21.15.78:0/4108109822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d419d560 con 0x7f35d4101de0 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d9264640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4106f10 0x7f35d419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d8a63640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4101de0 0x7f35d41985b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b4a0 tx=0x7f35cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.273+0000 7f35d1ffb640 1 -- 172.21.15.78:0/4108109822 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc014070 con 0x7f35d4101de0 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.274+0000 7f35d1ffb640 1 -- 172.21.15.78:0/4108109822 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35cc010af0 con 0x7f35d4101de0 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.274+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d419d850 con 0x7f35d4101de0 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.274+0000 7f35d1ffb640 1 -- 172.21.15.78:0/4108109822 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc017d10 con 0x7f35d4101de0 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.274+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41a5130 con 0x7f35d4101de0 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.275+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d4102240 con 0x7f35d4101de0 2024-09-16T00:12:34.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.276+0000 7f35d1ffb640 1 -- 172.21.15.78:0/4108109822 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f35cc017070 con 0x7f35d4101de0 2024-09-16T00:12:34.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.276+0000 7f35d1ffb640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35c00a3350 0x7f35c00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:34.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.277+0000 7f35d3fff640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35c00a3350 0x7f35c00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:34.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.277+0000 7f35d1ffb640 1 -- 172.21.15.78:0/4108109822 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f35cc0cf0e0 con 0x7f35d4101de0 2024-09-16T00:12:34.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.277+0000 7f35d3fff640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35c00a3350 0x7f35c00a5810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f35c4000f80 tx=0x7f35c402f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:34.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.281+0000 7f35d1ffb640 1 -- 172.21.15.78:0/4108109822 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35cc09ba70 con 0x7f35d4101de0 2024-09-16T00:12:34.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.450+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f35d4199910 con 0x7f35d4101de0 2024-09-16T00:12:34.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.450+0000 7f35d1ffb640 1 -- 172.21.15.78:0/4108109822 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f35cc09b370 con 0x7f35d4101de0 2024-09-16T00:12:34.451 INFO:teuthology.orchestra.run.smithi078.stdout:236223201327 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35c00a3350 msgr2=0x7f35c00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35c00a3350 0x7f35c00a5810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f35c4000f80 tx=0x7f35c402f310 comp rx=0 tx=0).stop 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4101de0 msgr2=0x7f35d41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4101de0 0x7f35d41985b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00b4a0 tx=0x7f35cc00b960 comp rx=0 tx=0).stop 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 shutdown_connections 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f35c00a3350 0x7f35c00a5810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f35d4106f10 0x7f35d419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35d4102790 0x7f35d4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 --2- 172.21.15.78:0/4108109822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35d4101de0 0x7f35d41985b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 >> 172.21.15.78:0/4108109822 conn(0x7f35d40fd8c0 msgr2=0x7f35d40fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 shutdown_connections 2024-09-16T00:12:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:34.452+0000 7f35dacc7640 1 -- 172.21.15.78:0/4108109822 wait complete. 2024-09-16T00:12:34.699 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593781 got 184683593784 for osd.6 2024-09-16T00:12:34.699 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:34.705 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:34 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4108109822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:12:34.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:34 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4108109822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:12:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:34 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4108109822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/2069649651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 msgr2=0x7f5d18103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/2069649651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 0x7f5d18103120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009900 tx=0x7f5d0c02a480 comp rx=0 tx=0).stop 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/2069649651 shutdown_connections 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/2069649651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1810c170 0x7f5d1810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/2069649651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d18103660 0x7f5d1810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/2069649651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 0x7f5d18103120 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/2069649651 >> 172.21.15.78:0/2069649651 conn(0x7f5d180fc7a0 msgr2=0x7f5d180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/2069649651 shutdown_connections 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.188+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/2069649651 wait complete. 2024-09-16T00:12:35.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ec8a640 1 Processor -- start 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ec8a640 1 -- start start 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ec8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 0x7f5d181985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ec8a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d18103660 0x7f5d18198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ec8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1810c170 0x7f5d1819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ec8a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18112030 con 0x7f5d18102d40 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ec8a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18111eb0 con 0x7f5d1810c170 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ec8a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d181121b0 con 0x7f5d18103660 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ca26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 0x7f5d181985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ca26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 0x7f5d181985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35514/0 (socket says 172.21.15.78:35514) 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ca26640 1 -- 172.21.15.78:0/3857350002 learned_addr learned my addr 172.21.15.78:0/3857350002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d17fff640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d18103660 0x7f5d18198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1d227640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1810c170 0x7f5d1819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:35.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ca26640 1 -- 172.21.15.78:0/3857350002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d18103660 msgr2=0x7f5d18198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ca26640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d18103660 0x7f5d18198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ca26640 1 -- 172.21.15.78:0/3857350002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1810c170 msgr2=0x7f5d1819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ca26640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1810c170 0x7f5d1819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1ca26640 1 -- 172.21.15.78:0/3857350002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1819d550 con 0x7f5d18102d40 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d1d227640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1810c170 0x7f5d1819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.189+0000 7f5d17fff640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d18103660 0x7f5d18198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.190+0000 7f5d1ca26640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 0x7f5d181985a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c005ab0 tx=0x7f5d0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.190+0000 7f5d15ffb640 1 -- 172.21.15.78:0/3857350002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c03f030 con 0x7f5d18102d40 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.190+0000 7f5d15ffb640 1 -- 172.21.15.78:0/3857350002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c003ce0 con 0x7f5d18102d40 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.190+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1819d7e0 con 0x7f5d18102d40 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.190+0000 7f5d15ffb640 1 -- 172.21.15.78:0/3857350002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c035070 con 0x7f5d18102d40 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.190+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d181a5120 con 0x7f5d18102d40 2024-09-16T00:12:35.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.191+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d181031a0 con 0x7f5d18102d40 2024-09-16T00:12:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.192+0000 7f5d15ffb640 1 -- 172.21.15.78:0/3857350002 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5d0c042030 con 0x7f5d18102d40 2024-09-16T00:12:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.193+0000 7f5d15ffb640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5d080a3350 0x7f5d080a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.193+0000 7f5d17fff640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5d080a3350 0x7f5d080a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.193+0000 7f5d15ffb640 1 -- 172.21.15.78:0/3857350002 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f5d0c0f0980 con 0x7f5d18102d40 2024-09-16T00:12:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.193+0000 7f5d17fff640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5d080a3350 0x7f5d080a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d180fee10 tx=0x7f5d04008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:35.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.196+0000 7f5d15ffb640 1 -- 172.21.15.78:0/3857350002 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d0c0bd200 con 0x7f5d18102d40 2024-09-16T00:12:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.368+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f5d181071e0 con 0x7f5d18102d40 2024-09-16T00:12:35.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.368+0000 7f5d15ffb640 1 -- 172.21.15.78:0/3857350002 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5d0c0bcb00 con 0x7f5d18102d40 2024-09-16T00:12:35.369 INFO:teuthology.orchestra.run.smithi078.stdout:317827579936 2024-09-16T00:12:35.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.370+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5d080a3350 msgr2=0x7f5d080a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.370+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5d080a3350 0x7f5d080a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d180fee10 tx=0x7f5d04008040 comp rx=0 tx=0).stop 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 msgr2=0x7f5d181985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 0x7f5d181985a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c005ab0 tx=0x7f5d0c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 shutdown_connections 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5d080a3350 0x7f5d080a5810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d1810c170 0x7f5d1819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d18103660 0x7f5d18198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 --2- 172.21.15.78:0/3857350002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d18102d40 0x7f5d181985a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 >> 172.21.15.78:0/3857350002 conn(0x7f5d180fc7a0 msgr2=0x7f5d180fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 shutdown_connections 2024-09-16T00:12:35.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:35.371+0000 7f5d1ec8a640 1 -- 172.21.15.78:0/3857350002 wait complete. 2024-09-16T00:12:35.756 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:35 smithi078 ceph-mon[31157]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:35.756 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:35 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3857350002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:12:35.784 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201324 got 236223201327 for osd.8 2024-09-16T00:12:35.785 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:35 smithi193 ceph-mon[35484]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:35 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3857350002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:12:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:35 smithi148 ceph-mon[35599]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:35 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3857350002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:12:36.477 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579933 got 317827579936 for osd.11 2024-09-16T00:12:36.477 DEBUG:teuthology.parallel:result is None 2024-09-16T00:12:36.477 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:12:36.477 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:12:36.730 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 -- 172.21.15.78:0/1316572685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 msgr2=0x7f66c0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 --2- 172.21.15.78:0/1316572685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 0x7f66c0106e60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009930 tx=0x7f66b002a420 comp rx=0 tx=0).stop 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 -- 172.21.15.78:0/1316572685 shutdown_connections 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 --2- 172.21.15.78:0/1316572685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66c0101420 0x7f66c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 --2- 172.21.15.78:0/1316572685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c0100a80 0x7f66c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 --2- 172.21.15.78:0/1316572685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 0x7f66c0106e60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 -- 172.21.15.78:0/1316572685 >> 172.21.15.78:0/1316572685 conn(0x7f66c00fc7a0 msgr2=0x7f66c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 -- 172.21.15.78:0/1316572685 shutdown_connections 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.210+0000 7f66c5c29640 1 -- 172.21.15.78:0/1316572685 wait complete. 2024-09-16T00:12:37.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66c5c29640 1 Processor -- start 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66c5c29640 1 -- start start 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66c5c29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c0100a80 0x7f66c00fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66c5c29640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66c0101420 0x7f66c00fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66c5c29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 0x7f66c00fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66c5c29640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0110ea0 con 0x7f66c0106a80 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66c5c29640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0110d20 con 0x7f66c0101420 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66c5c29640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66c0111020 con 0x7f66c0100a80 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 0x7f66c00fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66bffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 0x7f66c00fd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42440/0 (socket says 172.21.15.78:42440) 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.211+0000 7f66bffff640 1 -- 172.21.15.78:0/567215890 learned_addr learned my addr 172.21.15.78:0/567215890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bf7fe640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c0100a80 0x7f66c00fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bffff640 1 -- 172.21.15.78:0/567215890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c0100a80 msgr2=0x7f66c00fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:37.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bffff640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c0100a80 0x7f66c00fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bffff640 1 -- 172.21.15.78:0/567215890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66c0101420 msgr2=0x7f66c00fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bffff640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66c0101420 0x7f66c00fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bffff640 1 -- 172.21.15.78:0/567215890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c01a6940 con 0x7f66c0106a80 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bf7fe640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c0100a80 0x7f66c00fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bffff640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 0x7f66c00fd840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f66b800cce0 tx=0x7f66b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bcff9640 1 -- 172.21.15.78:0/567215890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b800f450 con 0x7f66c0106a80 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bcff9640 1 -- 172.21.15.78:0/567215890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66b800fad0 con 0x7f66c0106a80 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c01a6c30 con 0x7f66c0106a80 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66bcff9640 1 -- 172.21.15.78:0/567215890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8018c10 con 0x7f66c0106a80 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.212+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c01a7120 con 0x7f66c0106a80 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.213+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c0100ee0 con 0x7f66c0106a80 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.214+0000 7f66bcff9640 1 -- 172.21.15.78:0/567215890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f66b800f5f0 con 0x7f66c0106a80 2024-09-16T00:12:37.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.215+0000 7f66bcff9640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f66a80a32d0 0x7f66a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:37.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.215+0000 7f66bf7fe640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f66a80a32d0 0x7f66a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:37.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.215+0000 7f66bcff9640 1 -- 172.21.15.78:0/567215890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f66b80d0550 con 0x7f66c0106a80 2024-09-16T00:12:37.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.215+0000 7f66bf7fe640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f66a80a32d0 0x7f66a80a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:37.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.219+0000 7f66bcff9640 1 -- 172.21.15.78:0/567215890 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66b8014030 con 0x7f66c0106a80 2024-09-16T00:12:37.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.383+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66c0062630 con 0x7f66a80a32d0 2024-09-16T00:12:37.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.391+0000 7f66bcff9640 1 -- 172.21.15.78:0/567215890 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340395 (secure 0 0 0) 0x7f66c0062630 con 0x7f66a80a32d0 2024-09-16T00:12:37.396 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:37.398 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:12:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f66a80a32d0 msgr2=0x7f66a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f66a80a32d0 0x7f66a80a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b0005a20 comp rx=0 tx=0).stop 2024-09-16T00:12:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 msgr2=0x7f66c00fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 0x7f66c00fd840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f66b800cce0 tx=0x7f66b800a620 comp rx=0 tx=0).stop 2024-09-16T00:12:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 shutdown_connections 2024-09-16T00:12:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f66a80a32d0 0x7f66a80a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66c0106a80 0x7f66c00fd840 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f66c0101420 0x7f66c00fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 --2- 172.21.15.78:0/567215890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66c0100a80 0x7f66c00fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.394+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 >> 172.21.15.78:0/567215890 conn(0x7f66c00fc7a0 msgr2=0x7f66c006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.395+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 shutdown_connections 2024-09-16T00:12:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:37.395+0000 7f66c5c29640 1 -- 172.21.15.78:0/567215890 wait complete. 2024-09-16T00:12:37.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:37 smithi078 ceph-mon[31157]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:37 smithi193 ceph-mon[35484]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:37 smithi148 ceph-mon[35599]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:38.217 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-16T00:12:36.416797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660741,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":675,"num_read_kb":462,"num_write":437,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":356,"ondisk_log_size":356,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359116,"kb_used_data":13952,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582708,"statfs":{"total":1151940427776,"available":1151572692992,"internally_reserved":0,"allocated":14286848,"data_stored":7903937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003091"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:33.617217+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:12:33.617217+0000","last_peered":"2024-09-16T00:12:33.617217+0000","last_clean":"2024-09-16T00:12:33.617217+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:12:33.617217+0000","last_undegraded":"2024-09-16T00:12:33.617217+0000","last_fullsized":"2024-09-16T00:12:33.617217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131509+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:12:04.131509+0000","last_peered":"2024-09-16T00:12:04.131509+0000","last_clean":"2024-09-16T00:12:04.131509+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:12:04.131509+0000","last_undegraded":"2024-09-16T00:12:04.131509+0000","last_fullsized":"2024-09-16T00:12:04.131509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.132674+0000","last_change":"2024-09-16T00:12:04.132674+0000","last_active":"2024-09-16T00:12:04.132674+0000","last_peered":"2024-09-16T00:12:04.132674+0000","last_clean":"2024-09-16T00:12:04.132674+0000","last_became_active":"2024-09-16T00:12:04.132108+0000","last_became_peered":"2024-09-16T00:12:04.132108+0000","last_unstale":"2024-09-16T00:12:04.132674+0000","last_undegraded":"2024-09-16T00:12:04.132674+0000","last_fullsized":"2024-09-16T00:12:04.132674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:10.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131653+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:12:04.131653+0000","last_peered":"2024-09-16T00:12:04.131653+0000","last_clean":"2024-09-16T00:12:04.131653+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:12:04.131653+0000","last_undegraded":"2024-09-16T00:12:04.131653+0000","last_fullsized":"2024-09-16T00:12:04.131653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025410+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:12:04.025410+0000","last_peered":"2024-09-16T00:12:04.025410+0000","last_clean":"2024-09-16T00:12:04.025410+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:12:04.025410+0000","last_undegraded":"2024-09-16T00:12:04.025410+0000","last_fullsized":"2024-09-16T00:12:04.025410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024689+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:12:04.024689+0000","last_peered":"2024-09-16T00:12:04.024689+0000","last_clean":"2024-09-16T00:12:04.024689+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:12:04.024689+0000","last_undegraded":"2024-09-16T00:12:04.024689+0000","last_fullsized":"2024-09-16T00:12:04.024689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028354+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:12:04.028354+0000","last_peered":"2024-09-16T00:12:04.028354+0000","last_clean":"2024-09-16T00:12:04.028354+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:12:04.028354+0000","last_undegraded":"2024-09-16T00:12:04.028354+0000","last_fullsized":"2024-09-16T00:12:04.028354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027023+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:12:04.027023+0000","last_peered":"2024-09-16T00:12:04.027023+0000","last_clean":"2024-09-16T00:12:04.027023+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:12:04.027023+0000","last_undegraded":"2024-09-16T00:12:04.027023+0000","last_fullsized":"2024-09-16T00:12:04.027023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025689+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:12:04.025689+0000","last_peered":"2024-09-16T00:12:04.025689+0000","last_clean":"2024-09-16T00:12:04.025689+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:12:04.025689+0000","last_undegraded":"2024-09-16T00:12:04.025689+0000","last_fullsized":"2024-09-16T00:12:04.025689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028655+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:12:04.028655+0000","last_peered":"2024-09-16T00:12:04.028655+0000","last_clean":"2024-09-16T00:12:04.028655+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:12:04.028655+0000","last_undegraded":"2024-09-16T00:12:04.028655+0000","last_fullsized":"2024-09-16T00:12:04.028655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027065+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:12:04.027065+0000","last_peered":"2024-09-16T00:12:04.027065+0000","last_clean":"2024-09-16T00:12:04.027065+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:12:04.027065+0000","last_undegraded":"2024-09-16T00:12:04.027065+0000","last_fullsized":"2024-09-16T00:12:04.027065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025373+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:12:04.025373+0000","last_peered":"2024-09-16T00:12:04.025373+0000","last_clean":"2024-09-16T00:12:04.025373+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:12:04.025373+0000","last_undegraded":"2024-09-16T00:12:04.025373+0000","last_fullsized":"2024-09-16T00:12:04.025373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024110+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:12:04.024110+0000","last_peered":"2024-09-16T00:12:04.024110+0000","last_clean":"2024-09-16T00:12:04.024110+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:12:04.024110+0000","last_undegraded":"2024-09-16T00:12:04.024110+0000","last_fullsized":"2024-09-16T00:12:04.024110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.033872+0000","last_change":"2024-09-16T00:12:04.033464+0000","last_active":"2024-09-16T00:12:04.033872+0000","last_peered":"2024-09-16T00:12:04.033872+0000","last_clean":"2024-09-16T00:12:04.033872+0000","last_became_active":"2024-09-16T00:12:04.028916+0000","last_became_peered":"2024-09-16T00:12:04.028916+0000","last_unstale":"2024-09-16T00:12:04.033872+0000","last_undegraded":"2024-09-16T00:12:04.033872+0000","last_fullsized":"2024-09-16T00:12:04.033872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:47.359979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026145+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:12:04.026145+0000","last_peered":"2024-09-16T00:12:04.026145+0000","last_clean":"2024-09-16T00:12:04.026145+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:12:04.026145+0000","last_undegraded":"2024-09-16T00:12:04.026145+0000","last_fullsized":"2024-09-16T00:12:04.026145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131245+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:12:04.131245+0000","last_peered":"2024-09-16T00:12:04.131245+0000","last_clean":"2024-09-16T00:12:04.131245+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:12:04.131245+0000","last_undegraded":"2024-09-16T00:12:04.131245+0000","last_fullsized":"2024-09-16T00:12:04.131245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023148+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:12:04.023148+0000","last_peered":"2024-09-16T00:12:04.023148+0000","last_clean":"2024-09-16T00:12:04.023148+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:12:04.023148+0000","last_undegraded":"2024-09-16T00:12:04.023148+0000","last_fullsized":"2024-09-16T00:12:04.023148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026164+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:12:04.026164+0000","last_peered":"2024-09-16T00:12:04.026164+0000","last_clean":"2024-09-16T00:12:04.026164+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:12:04.026164+0000","last_undegraded":"2024-09-16T00:12:04.026164+0000","last_fullsized":"2024-09-16T00:12:04.026164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022802+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:12:04.022802+0000","last_peered":"2024-09-16T00:12:04.022802+0000","last_clean":"2024-09-16T00:12:04.022802+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:12:04.022802+0000","last_undegraded":"2024-09-16T00:12:04.022802+0000","last_fullsized":"2024-09-16T00:12:04.022802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131462+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:12:04.131462+0000","last_peered":"2024-09-16T00:12:04.131462+0000","last_clean":"2024-09-16T00:12:04.131462+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:12:04.131462+0000","last_undegraded":"2024-09-16T00:12:04.131462+0000","last_fullsized":"2024-09-16T00:12:04.131462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021468+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:12:04.021468+0000","last_peered":"2024-09-16T00:12:04.021468+0000","last_clean":"2024-09-16T00:12:04.021468+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:12:04.021468+0000","last_undegraded":"2024-09-16T00:12:04.021468+0000","last_fullsized":"2024-09-16T00:12:04.021468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026552+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:12:04.026552+0000","last_peered":"2024-09-16T00:12:04.026552+0000","last_clean":"2024-09-16T00:12:04.026552+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:12:04.026552+0000","last_undegraded":"2024-09-16T00:12:04.026552+0000","last_fullsized":"2024-09-16T00:12:04.026552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.034002+0000","last_change":"2024-09-16T00:12:04.034002+0000","last_active":"2024-09-16T00:12:04.034002+0000","last_peered":"2024-09-16T00:12:04.034002+0000","last_clean":"2024-09-16T00:12:04.034002+0000","last_became_active":"2024-09-16T00:12:04.030959+0000","last_became_peered":"2024-09-16T00:12:04.030959+0000","last_unstale":"2024-09-16T00:12:04.034002+0000","last_undegraded":"2024-09-16T00:12:04.034002+0000","last_fullsized":"2024-09-16T00:12:04.034002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:47.730692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.029906+0000","last_change":"2024-09-16T00:12:04.029906+0000","last_active":"2024-09-16T00:12:04.029906+0000","last_peered":"2024-09-16T00:12:04.029906+0000","last_clean":"2024-09-16T00:12:04.029906+0000","last_became_active":"2024-09-16T00:12:04.029259+0000","last_became_peered":"2024-09-16T00:12:04.029259+0000","last_unstale":"2024-09-16T00:12:04.029906+0000","last_undegraded":"2024-09-16T00:12:04.029906+0000","last_fullsized":"2024-09-16T00:12:04.029906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:23.399762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,1],"acting":[4,8,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025284+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:12:04.025284+0000","last_peered":"2024-09-16T00:12:04.025284+0000","last_clean":"2024-09-16T00:12:04.025284+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:12:04.025284+0000","last_undegraded":"2024-09-16T00:12:04.025284+0000","last_fullsized":"2024-09-16T00:12:04.025284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025227+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:12:04.025227+0000","last_peered":"2024-09-16T00:12:04.025227+0000","last_clean":"2024-09-16T00:12:04.025227+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:12:04.025227+0000","last_undegraded":"2024-09-16T00:12:04.025227+0000","last_fullsized":"2024-09-16T00:12:04.025227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023266+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:12:04.023266+0000","last_peered":"2024-09-16T00:12:04.023266+0000","last_clean":"2024-09-16T00:12:04.023266+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:12:04.023266+0000","last_undegraded":"2024-09-16T00:12:04.023266+0000","last_fullsized":"2024-09-16T00:12:04.023266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131687+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:12:04.131687+0000","last_peered":"2024-09-16T00:12:04.131687+0000","last_clean":"2024-09-16T00:12:04.131687+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:12:04.131687+0000","last_undegraded":"2024-09-16T00:12:04.131687+0000","last_fullsized":"2024-09-16T00:12:04.131687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'18","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021955+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:12:04.021955+0000","last_peered":"2024-09-16T00:12:04.021955+0000","last_clean":"2024-09-16T00:12:04.021955+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:12:04.021955+0000","last_undegraded":"2024-09-16T00:12:04.021955+0000","last_fullsized":"2024-09-16T00:12:04.021955+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131081+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:12:04.131081+0000","last_peered":"2024-09-16T00:12:04.131081+0000","last_clean":"2024-09-16T00:12:04.131081+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:12:04.131081+0000","last_undegraded":"2024-09-16T00:12:04.131081+0000","last_fullsized":"2024-09-16T00:12:04.131081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024873+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:12:04.024873+0000","last_peered":"2024-09-16T00:12:04.024873+0000","last_clean":"2024-09-16T00:12:04.024873+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:12:04.024873+0000","last_undegraded":"2024-09-16T00:12:04.024873+0000","last_fullsized":"2024-09-16T00:12:04.024873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028396+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:12:04.028396+0000","last_peered":"2024-09-16T00:12:04.028396+0000","last_clean":"2024-09-16T00:12:04.028396+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:12:04.028396+0000","last_undegraded":"2024-09-16T00:12:04.028396+0000","last_fullsized":"2024-09-16T00:12:04.028396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023644+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:12:04.023644+0000","last_peered":"2024-09-16T00:12:04.023644+0000","last_clean":"2024-09-16T00:12:04.023644+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:12:04.023644+0000","last_undegraded":"2024-09-16T00:12:04.023644+0000","last_fullsized":"2024-09-16T00:12:04.023644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022469+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:12:04.022469+0000","last_peered":"2024-09-16T00:12:04.022469+0000","last_clean":"2024-09-16T00:12:04.022469+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:12:04.022469+0000","last_undegraded":"2024-09-16T00:12:04.022469+0000","last_fullsized":"2024-09-16T00:12:04.022469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021573+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:12:04.021573+0000","last_peered":"2024-09-16T00:12:04.021573+0000","last_clean":"2024-09-16T00:12:04.021573+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:12:04.021573+0000","last_undegraded":"2024-09-16T00:12:04.021573+0000","last_fullsized":"2024-09-16T00:12:04.021573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025893+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:12:04.025893+0000","last_peered":"2024-09-16T00:12:04.025893+0000","last_clean":"2024-09-16T00:12:04.025893+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:12:04.025893+0000","last_undegraded":"2024-09-16T00:12:04.025893+0000","last_fullsized":"2024-09-16T00:12:04.025893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022530+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:12:04.022530+0000","last_peered":"2024-09-16T00:12:04.022530+0000","last_clean":"2024-09-16T00:12:04.022530+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:12:04.022530+0000","last_undegraded":"2024-09-16T00:12:04.022530+0000","last_fullsized":"2024-09-16T00:12:04.022530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:31.614855+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:12:31.614855+0000","last_peered":"2024-09-16T00:12:31.614855+0000","last_clean":"2024-09-16T00:12:31.614855+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:12:31.614855+0000","last_undegraded":"2024-09-16T00:12:31.614855+0000","last_fullsized":"2024-09-16T00:12:31.614855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024308+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:12:04.024308+0000","last_peered":"2024-09-16T00:12:04.024308+0000","last_clean":"2024-09-16T00:12:04.024308+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:12:04.024308+0000","last_undegraded":"2024-09-16T00:12:04.024308+0000","last_fullsized":"2024-09-16T00:12:04.024308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026567+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:12:04.026567+0000","last_peered":"2024-09-16T00:12:04.026567+0000","last_clean":"2024-09-16T00:12:04.026567+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:12:04.026567+0000","last_undegraded":"2024-09-16T00:12:04.026567+0000","last_fullsized":"2024-09-16T00:12:04.026567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:31.633667+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:12:31.633667+0000","last_peered":"2024-09-16T00:12:31.633667+0000","last_clean":"2024-09-16T00:12:31.633667+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:12:31.633667+0000","last_undegraded":"2024-09-16T00:12:31.633667+0000","last_fullsized":"2024-09-16T00:12:31.633667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025419+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:12:04.025419+0000","last_peered":"2024-09-16T00:12:04.025419+0000","last_clean":"2024-09-16T00:12:04.025419+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:12:04.025419+0000","last_undegraded":"2024-09-16T00:12:04.025419+0000","last_fullsized":"2024-09-16T00:12:04.025419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027859+0000","last_change":"2024-09-16T00:12:04.027859+0000","last_active":"2024-09-16T00:12:04.027859+0000","last_peered":"2024-09-16T00:12:04.027859+0000","last_clean":"2024-09-16T00:12:04.027859+0000","last_became_active":"2024-09-16T00:12:04.027506+0000","last_became_peered":"2024-09-16T00:12:04.027506+0000","last_unstale":"2024-09-16T00:12:04.027859+0000","last_undegraded":"2024-09-16T00:12:04.027859+0000","last_fullsized":"2024-09-16T00:12:04.027859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:34.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023822+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:12:04.023822+0000","last_peered":"2024-09-16T00:12:04.023822+0000","last_clean":"2024-09-16T00:12:04.023822+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:12:04.023822+0000","last_undegraded":"2024-09-16T00:12:04.023822+0000","last_fullsized":"2024-09-16T00:12:04.023822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.029471+0000","last_change":"2024-09-16T00:12:04.029471+0000","last_active":"2024-09-16T00:12:04.029471+0000","last_peered":"2024-09-16T00:12:04.029471+0000","last_clean":"2024-09-16T00:12:04.029471+0000","last_became_active":"2024-09-16T00:12:04.029054+0000","last_became_peered":"2024-09-16T00:12:04.029054+0000","last_unstale":"2024-09-16T00:12:04.029471+0000","last_undegraded":"2024-09-16T00:12:04.029471+0000","last_fullsized":"2024-09-16T00:12:04.029471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:52.054218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131251+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:12:04.131251+0000","last_peered":"2024-09-16T00:12:04.131251+0000","last_clean":"2024-09-16T00:12:04.131251+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:12:04.131251+0000","last_undegraded":"2024-09-16T00:12:04.131251+0000","last_fullsized":"2024-09-16T00:12:04.131251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026112+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:12:04.026112+0000","last_peered":"2024-09-16T00:12:04.026112+0000","last_clean":"2024-09-16T00:12:04.026112+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:12:04.026112+0000","last_undegraded":"2024-09-16T00:12:04.026112+0000","last_fullsized":"2024-09-16T00:12:04.026112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023973+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:12:04.023973+0000","last_peered":"2024-09-16T00:12:04.023973+0000","last_clean":"2024-09-16T00:12:04.023973+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:12:04.023973+0000","last_undegraded":"2024-09-16T00:12:04.023973+0000","last_fullsized":"2024-09-16T00:12:04.023973+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024398+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:12:04.024398+0000","last_peered":"2024-09-16T00:12:04.024398+0000","last_clean":"2024-09-16T00:12:04.024398+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:12:04.024398+0000","last_undegraded":"2024-09-16T00:12:04.024398+0000","last_fullsized":"2024-09-16T00:12:04.024398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025571+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:12:04.025571+0000","last_peered":"2024-09-16T00:12:04.025571+0000","last_clean":"2024-09-16T00:12:04.025571+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:12:04.025571+0000","last_undegraded":"2024-09-16T00:12:04.025571+0000","last_fullsized":"2024-09-16T00:12:04.025571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:33.619234+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:12:33.619234+0000","last_peered":"2024-09-16T00:12:33.619234+0000","last_clean":"2024-09-16T00:12:33.619234+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:12:33.619234+0000","last_undegraded":"2024-09-16T00:12:33.619234+0000","last_fullsized":"2024-09-16T00:12:33.619234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026989+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:12:04.026989+0000","last_peered":"2024-09-16T00:12:04.026989+0000","last_clean":"2024-09-16T00:12:04.026989+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:12:04.026989+0000","last_undegraded":"2024-09-16T00:12:04.026989+0000","last_fullsized":"2024-09-16T00:12:04.026989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024184+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:12:04.024184+0000","last_peered":"2024-09-16T00:12:04.024184+0000","last_clean":"2024-09-16T00:12:04.024184+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:12:04.024184+0000","last_undegraded":"2024-09-16T00:12:04.024184+0000","last_fullsized":"2024-09-16T00:12:04.024184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027184+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:12:04.027184+0000","last_peered":"2024-09-16T00:12:04.027184+0000","last_clean":"2024-09-16T00:12:04.027184+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:12:04.027184+0000","last_undegraded":"2024-09-16T00:12:04.027184+0000","last_fullsized":"2024-09-16T00:12:04.027184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024966+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:12:04.024966+0000","last_peered":"2024-09-16T00:12:04.024966+0000","last_clean":"2024-09-16T00:12:04.024966+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:12:04.024966+0000","last_undegraded":"2024-09-16T00:12:04.024966+0000","last_fullsized":"2024-09-16T00:12:04.024966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025278+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:12:04.025278+0000","last_peered":"2024-09-16T00:12:04.025278+0000","last_clean":"2024-09-16T00:12:04.025278+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:12:04.025278+0000","last_undegraded":"2024-09-16T00:12:04.025278+0000","last_fullsized":"2024-09-16T00:12:04.025278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025716+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:12:04.025716+0000","last_peered":"2024-09-16T00:12:04.025716+0000","last_clean":"2024-09-16T00:12:04.025716+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:12:04.025716+0000","last_undegraded":"2024-09-16T00:12:04.025716+0000","last_fullsized":"2024-09-16T00:12:04.025716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026174+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:12:04.026174+0000","last_peered":"2024-09-16T00:12:04.026174+0000","last_clean":"2024-09-16T00:12:04.026174+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:12:04.026174+0000","last_undegraded":"2024-09-16T00:12:04.026174+0000","last_fullsized":"2024-09-16T00:12:04.026174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024068+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:12:04.024068+0000","last_peered":"2024-09-16T00:12:04.024068+0000","last_clean":"2024-09-16T00:12:04.024068+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:12:04.024068+0000","last_undegraded":"2024-09-16T00:12:04.024068+0000","last_fullsized":"2024-09-16T00:12:04.024068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028185+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:12:04.028185+0000","last_peered":"2024-09-16T00:12:04.028185+0000","last_clean":"2024-09-16T00:12:04.028185+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:12:04.028185+0000","last_undegraded":"2024-09-16T00:12:04.028185+0000","last_fullsized":"2024-09-16T00:12:04.028185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025385+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:12:04.025385+0000","last_peered":"2024-09-16T00:12:04.025385+0000","last_clean":"2024-09-16T00:12:04.025385+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:12:04.025385+0000","last_undegraded":"2024-09-16T00:12:04.025385+0000","last_fullsized":"2024-09-16T00:12:04.025385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025738+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:12:04.025738+0000","last_peered":"2024-09-16T00:12:04.025738+0000","last_clean":"2024-09-16T00:12:04.025738+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:12:04.025738+0000","last_undegraded":"2024-09-16T00:12:04.025738+0000","last_fullsized":"2024-09-16T00:12:04.025738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025960+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:12:04.025960+0000","last_peered":"2024-09-16T00:12:04.025960+0000","last_clean":"2024-09-16T00:12:04.025960+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:12:04.025960+0000","last_undegraded":"2024-09-16T00:12:04.025960+0000","last_fullsized":"2024-09-16T00:12:04.025960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025608+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:12:04.025608+0000","last_peered":"2024-09-16T00:12:04.025608+0000","last_clean":"2024-09-16T00:12:04.025608+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:12:04.025608+0000","last_undegraded":"2024-09-16T00:12:04.025608+0000","last_fullsized":"2024-09-16T00:12:04.025608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023685+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:12:04.023685+0000","last_peered":"2024-09-16T00:12:04.023685+0000","last_clean":"2024-09-16T00:12:04.023685+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:12:04.023685+0000","last_undegraded":"2024-09-16T00:12:04.023685+0000","last_fullsized":"2024-09-16T00:12:04.023685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:31.634469+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:12:31.634469+0000","last_peered":"2024-09-16T00:12:31.634469+0000","last_clean":"2024-09-16T00:12:31.634469+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:12:31.634469+0000","last_undegraded":"2024-09-16T00:12:31.634469+0000","last_fullsized":"2024-09-16T00:12:31.634469+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021569+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:12:04.021569+0000","last_peered":"2024-09-16T00:12:04.021569+0000","last_clean":"2024-09-16T00:12:04.021569+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:12:04.021569+0000","last_undegraded":"2024-09-16T00:12:04.021569+0000","last_fullsized":"2024-09-16T00:12:04.021569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024056+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:12:04.024056+0000","last_peered":"2024-09-16T00:12:04.024056+0000","last_clean":"2024-09-16T00:12:04.024056+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:12:04.024056+0000","last_undegraded":"2024-09-16T00:12:04.024056+0000","last_fullsized":"2024-09-16T00:12:04.024056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026671+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:12:04.026671+0000","last_peered":"2024-09-16T00:12:04.026671+0000","last_clean":"2024-09-16T00:12:04.026671+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:12:04.026671+0000","last_undegraded":"2024-09-16T00:12:04.026671+0000","last_fullsized":"2024-09-16T00:12:04.026671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023704+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:12:04.023704+0000","last_peered":"2024-09-16T00:12:04.023704+0000","last_clean":"2024-09-16T00:12:04.023704+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:12:04.023704+0000","last_undegraded":"2024-09-16T00:12:04.023704+0000","last_fullsized":"2024-09-16T00:12:04.023704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024155+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:12:04.024155+0000","last_peered":"2024-09-16T00:12:04.024155+0000","last_clean":"2024-09-16T00:12:04.024155+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:12:04.024155+0000","last_undegraded":"2024-09-16T00:12:04.024155+0000","last_fullsized":"2024-09-16T00:12:04.024155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023404+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:12:04.023404+0000","last_peered":"2024-09-16T00:12:04.023404+0000","last_clean":"2024-09-16T00:12:04.023404+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:12:04.023404+0000","last_undegraded":"2024-09-16T00:12:04.023404+0000","last_fullsized":"2024-09-16T00:12:04.023404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026986+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:12:04.026986+0000","last_peered":"2024-09-16T00:12:04.026986+0000","last_clean":"2024-09-16T00:12:04.026986+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:12:04.026986+0000","last_undegraded":"2024-09-16T00:12:04.026986+0000","last_fullsized":"2024-09-16T00:12:04.026986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028251+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:12:04.028251+0000","last_peered":"2024-09-16T00:12:04.028251+0000","last_clean":"2024-09-16T00:12:04.028251+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:12:04.028251+0000","last_undegraded":"2024-09-16T00:12:04.028251+0000","last_fullsized":"2024-09-16T00:12:04.028251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":182,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:34.619871+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:12:34.619871+0000","last_peered":"2024-09-16T00:12:34.619871+0000","last_clean":"2024-09-16T00:12:34.619871+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:12:34.619871+0000","last_undegraded":"2024-09-16T00:12:34.619871+0000","last_fullsized":"2024-09-16T00:12:34.619871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62931,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":117,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021641+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:12:04.021641+0000","last_peered":"2024-09-16T00:12:04.021641+0000","last_clean":"2024-09-16T00:12:04.021641+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:12:04.021641+0000","last_undegraded":"2024-09-16T00:12:04.021641+0000","last_fullsized":"2024-09-16T00:12:04.021641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027169+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:12:04.027169+0000","last_peered":"2024-09-16T00:12:04.027169+0000","last_clean":"2024-09-16T00:12:04.027169+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:12:04.027169+0000","last_undegraded":"2024-09-16T00:12:04.027169+0000","last_fullsized":"2024-09-16T00:12:04.027169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027157+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:12:04.027157+0000","last_peered":"2024-09-16T00:12:04.027157+0000","last_clean":"2024-09-16T00:12:04.027157+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:12:04.027157+0000","last_undegraded":"2024-09-16T00:12:04.027157+0000","last_fullsized":"2024-09-16T00:12:04.027157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025924+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:12:04.025924+0000","last_peered":"2024-09-16T00:12:04.025924+0000","last_clean":"2024-09-16T00:12:04.025924+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:12:04.025924+0000","last_undegraded":"2024-09-16T00:12:04.025924+0000","last_fullsized":"2024-09-16T00:12:04.025924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025502+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:12:04.025502+0000","last_peered":"2024-09-16T00:12:04.025502+0000","last_clean":"2024-09-16T00:12:04.025502+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:12:04.025502+0000","last_undegraded":"2024-09-16T00:12:04.025502+0000","last_fullsized":"2024-09-16T00:12:04.025502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025140+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:12:04.025140+0000","last_peered":"2024-09-16T00:12:04.025140+0000","last_clean":"2024-09-16T00:12:04.025140+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:12:04.025140+0000","last_undegraded":"2024-09-16T00:12:04.025140+0000","last_fullsized":"2024-09-16T00:12:04.025140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021165+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:12:04.021165+0000","last_peered":"2024-09-16T00:12:04.021165+0000","last_clean":"2024-09-16T00:12:04.021165+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:12:04.021165+0000","last_undegraded":"2024-09-16T00:12:04.021165+0000","last_fullsized":"2024-09-16T00:12:04.021165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027316+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:12:04.027316+0000","last_peered":"2024-09-16T00:12:04.027316+0000","last_clean":"2024-09-16T00:12:04.027316+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:12:04.027316+0000","last_undegraded":"2024-09-16T00:12:04.027316+0000","last_fullsized":"2024-09-16T00:12:04.027316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:34.619808+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:12:34.619808+0000","last_peered":"2024-09-16T00:12:34.619808+0000","last_clean":"2024-09-16T00:12:34.619808+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:12:34.619808+0000","last_undegraded":"2024-09-16T00:12:34.619808+0000","last_fullsized":"2024-09-16T00:12:34.619808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131339+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:12:04.131339+0000","last_peered":"2024-09-16T00:12:04.131339+0000","last_clean":"2024-09-16T00:12:04.131339+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:12:04.131339+0000","last_undegraded":"2024-09-16T00:12:04.131339+0000","last_fullsized":"2024-09-16T00:12:04.131339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025106+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:12:04.025106+0000","last_peered":"2024-09-16T00:12:04.025106+0000","last_clean":"2024-09-16T00:12:04.025106+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:12:04.025106+0000","last_undegraded":"2024-09-16T00:12:04.025106+0000","last_fullsized":"2024-09-16T00:12:04.025106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021346+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:12:04.021346+0000","last_peered":"2024-09-16T00:12:04.021346+0000","last_clean":"2024-09-16T00:12:04.021346+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:12:04.021346+0000","last_undegraded":"2024-09-16T00:12:04.021346+0000","last_fullsized":"2024-09-16T00:12:04.021346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026679+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:12:04.026679+0000","last_peered":"2024-09-16T00:12:04.026679+0000","last_clean":"2024-09-16T00:12:04.026679+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:12:04.026679+0000","last_undegraded":"2024-09-16T00:12:04.026679+0000","last_fullsized":"2024-09-16T00:12:04.026679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024106+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:12:04.024106+0000","last_peered":"2024-09-16T00:12:04.024106+0000","last_clean":"2024-09-16T00:12:04.024106+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:12:04.024106+0000","last_undegraded":"2024-09-16T00:12:04.024106+0000","last_fullsized":"2024-09-16T00:12:04.024106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131191+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:12:04.131191+0000","last_peered":"2024-09-16T00:12:04.131191+0000","last_clean":"2024-09-16T00:12:04.131191+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:12:04.131191+0000","last_undegraded":"2024-09-16T00:12:04.131191+0000","last_fullsized":"2024-09-16T00:12:04.131191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023936+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:12:04.023936+0000","last_peered":"2024-09-16T00:12:04.023936+0000","last_clean":"2024-09-16T00:12:04.023936+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:12:04.023936+0000","last_undegraded":"2024-09-16T00:12:04.023936+0000","last_fullsized":"2024-09-16T00:12:04.023936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022320+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:12:04.022320+0000","last_peered":"2024-09-16T00:12:04.022320+0000","last_clean":"2024-09-16T00:12:04.022320+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:12:04.022320+0000","last_undegraded":"2024-09-16T00:12:04.022320+0000","last_fullsized":"2024-09-16T00:12:04.022320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025656+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:12:04.025656+0000","last_peered":"2024-09-16T00:12:04.025656+0000","last_clean":"2024-09-16T00:12:04.025656+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:12:04.025656+0000","last_undegraded":"2024-09-16T00:12:04.025656+0000","last_fullsized":"2024-09-16T00:12:04.025656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023608+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:12:04.023608+0000","last_peered":"2024-09-16T00:12:04.023608+0000","last_clean":"2024-09-16T00:12:04.023608+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:12:04.023608+0000","last_undegraded":"2024-09-16T00:12:04.023608+0000","last_fullsized":"2024-09-16T00:12:04.023608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:29.613715+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:12:29.613715+0000","last_peered":"2024-09-16T00:12:29.613715+0000","last_clean":"2024-09-16T00:12:29.613715+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:12:29.613715+0000","last_undegraded":"2024-09-16T00:12:29.613715+0000","last_fullsized":"2024-09-16T00:12:29.613715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027324+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:12:04.027324+0000","last_peered":"2024-09-16T00:12:04.027324+0000","last_clean":"2024-09-16T00:12:04.027324+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:12:04.027324+0000","last_undegraded":"2024-09-16T00:12:04.027324+0000","last_fullsized":"2024-09-16T00:12:04.027324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023080+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:12:04.023080+0000","last_peered":"2024-09-16T00:12:04.023080+0000","last_clean":"2024-09-16T00:12:04.023080+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:12:04.023080+0000","last_undegraded":"2024-09-16T00:12:04.023080+0000","last_fullsized":"2024-09-16T00:12:04.023080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022647+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:12:04.022647+0000","last_peered":"2024-09-16T00:12:04.022647+0000","last_clean":"2024-09-16T00:12:04.022647+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:12:04.022647+0000","last_undegraded":"2024-09-16T00:12:04.022647+0000","last_fullsized":"2024-09-16T00:12:04.022647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024254+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:12:04.024254+0000","last_peered":"2024-09-16T00:12:04.024254+0000","last_clean":"2024-09-16T00:12:04.024254+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:12:04.024254+0000","last_undegraded":"2024-09-16T00:12:04.024254+0000","last_fullsized":"2024-09-16T00:12:04.024254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022993+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:12:04.022993+0000","last_peered":"2024-09-16T00:12:04.022993+0000","last_clean":"2024-09-16T00:12:04.022993+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:12:04.022993+0000","last_undegraded":"2024-09-16T00:12:04.022993+0000","last_fullsized":"2024-09-16T00:12:04.022993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.029280+0000","last_change":"2024-09-16T00:12:04.029280+0000","last_active":"2024-09-16T00:12:04.029280+0000","last_peered":"2024-09-16T00:12:04.029280+0000","last_clean":"2024-09-16T00:12:04.029280+0000","last_became_active":"2024-09-16T00:12:04.028973+0000","last_became_peered":"2024-09-16T00:12:04.028973+0000","last_unstale":"2024-09-16T00:12:04.029280+0000","last_undegraded":"2024-09-16T00:12:04.029280+0000","last_fullsized":"2024-09-16T00:12:04.029280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028319+0000","last_change":"2024-09-16T00:12:04.028319+0000","last_active":"2024-09-16T00:12:04.028319+0000","last_peered":"2024-09-16T00:12:04.028319+0000","last_clean":"2024-09-16T00:12:04.028319+0000","last_became_active":"2024-09-16T00:12:04.027985+0000","last_became_peered":"2024-09-16T00:12:04.027985+0000","last_unstale":"2024-09-16T00:12:04.028319+0000","last_undegraded":"2024-09-16T00:12:04.028319+0000","last_fullsized":"2024-09-16T00:12:04.028319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.119428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023573+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:12:04.023573+0000","last_peered":"2024-09-16T00:12:04.023573+0000","last_clean":"2024-09-16T00:12:04.023573+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:12:04.023573+0000","last_undegraded":"2024-09-16T00:12:04.023573+0000","last_fullsized":"2024-09-16T00:12:04.023573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024142+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:12:04.024142+0000","last_peered":"2024-09-16T00:12:04.024142+0000","last_clean":"2024-09-16T00:12:04.024142+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:12:04.024142+0000","last_undegraded":"2024-09-16T00:12:04.024142+0000","last_fullsized":"2024-09-16T00:12:04.024142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131625+0000","last_change":"2024-09-16T00:11:04.649155+0000","last_active":"2024-09-16T00:12:04.131625+0000","last_peered":"2024-09-16T00:12:04.131625+0000","last_clean":"2024-09-16T00:12:04.131625+0000","last_became_active":"2024-09-16T00:11:04.648844+0000","last_became_peered":"2024-09-16T00:11:04.648844+0000","last_unstale":"2024-09-16T00:12:04.131625+0000","last_undegraded":"2024-09-16T00:12:04.131625+0000","last_fullsized":"2024-09-16T00:12:04.131625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:20.205113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026514+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:12:04.026514+0000","last_peered":"2024-09-16T00:12:04.026514+0000","last_clean":"2024-09-16T00:12:04.026514+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:12:04.026514+0000","last_undegraded":"2024-09-16T00:12:04.026514+0000","last_fullsized":"2024-09-16T00:12:04.026514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131398+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:12:04.131398+0000","last_peered":"2024-09-16T00:12:04.131398+0000","last_clean":"2024-09-16T00:12:04.131398+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:12:04.131398+0000","last_undegraded":"2024-09-16T00:12:04.131398+0000","last_fullsized":"2024-09-16T00:12:04.131398+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027313+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:12:04.027313+0000","last_peered":"2024-09-16T00:12:04.027313+0000","last_clean":"2024-09-16T00:12:04.027313+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:12:04.027313+0000","last_undegraded":"2024-09-16T00:12:04.027313+0000","last_fullsized":"2024-09-16T00:12:04.027313+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024287+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:12:04.024287+0000","last_peered":"2024-09-16T00:12:04.024287+0000","last_clean":"2024-09-16T00:12:04.024287+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:12:04.024287+0000","last_undegraded":"2024-09-16T00:12:04.024287+0000","last_fullsized":"2024-09-16T00:12:04.024287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023821+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:12:04.023821+0000","last_peered":"2024-09-16T00:12:04.023821+0000","last_clean":"2024-09-16T00:12:04.023821+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:12:04.023821+0000","last_undegraded":"2024-09-16T00:12:04.023821+0000","last_fullsized":"2024-09-16T00:12:04.023821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023729+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:12:04.023729+0000","last_peered":"2024-09-16T00:12:04.023729+0000","last_clean":"2024-09-16T00:12:04.023729+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:12:04.023729+0000","last_undegraded":"2024-09-16T00:12:04.023729+0000","last_fullsized":"2024-09-16T00:12:04.023729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021859+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:12:04.021859+0000","last_peered":"2024-09-16T00:12:04.021859+0000","last_clean":"2024-09-16T00:12:04.021859+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:12:04.021859+0000","last_undegraded":"2024-09-16T00:12:04.021859+0000","last_fullsized":"2024-09-16T00:12:04.021859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024995+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:12:04.024995+0000","last_peered":"2024-09-16T00:12:04.024995+0000","last_clean":"2024-09-16T00:12:04.024995+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:12:04.024995+0000","last_undegraded":"2024-09-16T00:12:04.024995+0000","last_fullsized":"2024-09-16T00:12:04.024995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027097+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:12:04.027097+0000","last_peered":"2024-09-16T00:12:04.027097+0000","last_clean":"2024-09-16T00:12:04.027097+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:12:04.027097+0000","last_undegraded":"2024-09-16T00:12:04.027097+0000","last_fullsized":"2024-09-16T00:12:04.027097+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024431+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:12:04.024431+0000","last_peered":"2024-09-16T00:12:04.024431+0000","last_clean":"2024-09-16T00:12:04.024431+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:12:04.024431+0000","last_undegraded":"2024-09-16T00:12:04.024431+0000","last_fullsized":"2024-09-16T00:12:04.024431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024478+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:12:04.024478+0000","last_peered":"2024-09-16T00:12:04.024478+0000","last_clean":"2024-09-16T00:12:04.024478+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:12:04.024478+0000","last_undegraded":"2024-09-16T00:12:04.024478+0000","last_fullsized":"2024-09-16T00:12:04.024478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024195+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:12:04.024195+0000","last_peered":"2024-09-16T00:12:04.024195+0000","last_clean":"2024-09-16T00:12:04.024195+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:12:04.024195+0000","last_undegraded":"2024-09-16T00:12:04.024195+0000","last_fullsized":"2024-09-16T00:12:04.024195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026638+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:12:04.026638+0000","last_peered":"2024-09-16T00:12:04.026638+0000","last_clean":"2024-09-16T00:12:04.026638+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:12:04.026638+0000","last_undegraded":"2024-09-16T00:12:04.026638+0000","last_fullsized":"2024-09-16T00:12:04.026638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025490+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:12:04.025490+0000","last_peered":"2024-09-16T00:12:04.025490+0000","last_clean":"2024-09-16T00:12:04.025490+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:12:04.025490+0000","last_undegraded":"2024-09-16T00:12:04.025490+0000","last_fullsized":"2024-09-16T00:12:04.025490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023272+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:12:04.023272+0000","last_peered":"2024-09-16T00:12:04.023272+0000","last_clean":"2024-09-16T00:12:04.023272+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:12:04.023272+0000","last_undegraded":"2024-09-16T00:12:04.023272+0000","last_fullsized":"2024-09-16T00:12:04.023272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027537+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:12:04.027537+0000","last_peered":"2024-09-16T00:12:04.027537+0000","last_clean":"2024-09-16T00:12:04.027537+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:12:04.027537+0000","last_undegraded":"2024-09-16T00:12:04.027537+0000","last_fullsized":"2024-09-16T00:12:04.027537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131159+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:12:04.131159+0000","last_peered":"2024-09-16T00:12:04.131159+0000","last_clean":"2024-09-16T00:12:04.131159+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:12:04.131159+0000","last_undegraded":"2024-09-16T00:12:04.131159+0000","last_fullsized":"2024-09-16T00:12:04.131159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028342+0000","last_change":"2024-09-16T00:12:04.028342+0000","last_active":"2024-09-16T00:12:04.028342+0000","last_peered":"2024-09-16T00:12:04.028342+0000","last_clean":"2024-09-16T00:12:04.028342+0000","last_became_active":"2024-09-16T00:12:04.027889+0000","last_became_peered":"2024-09-16T00:12:04.027889+0000","last_unstale":"2024-09-16T00:12:04.028342+0000","last_undegraded":"2024-09-16T00:12:04.028342+0000","last_fullsized":"2024-09-16T00:12:04.028342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:47.840428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022264+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:12:04.022264+0000","last_peered":"2024-09-16T00:12:04.022264+0000","last_clean":"2024-09-16T00:12:04.022264+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:12:04.022264+0000","last_undegraded":"2024-09-16T00:12:04.022264+0000","last_fullsized":"2024-09-16T00:12:04.022264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027341+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:12:04.027341+0000","last_peered":"2024-09-16T00:12:04.027341+0000","last_clean":"2024-09-16T00:12:04.027341+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:12:04.027341+0000","last_undegraded":"2024-09-16T00:12:04.027341+0000","last_fullsized":"2024-09-16T00:12:04.027341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026539+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:12:04.026539+0000","last_peered":"2024-09-16T00:12:04.026539+0000","last_clean":"2024-09-16T00:12:04.026539+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:12:04.026539+0000","last_undegraded":"2024-09-16T00:12:04.026539+0000","last_fullsized":"2024-09-16T00:12:04.026539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025058+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:12:04.025058+0000","last_peered":"2024-09-16T00:12:04.025058+0000","last_clean":"2024-09-16T00:12:04.025058+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:12:04.025058+0000","last_undegraded":"2024-09-16T00:12:04.025058+0000","last_fullsized":"2024-09-16T00:12:04.025058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025200+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:12:04.025200+0000","last_peered":"2024-09-16T00:12:04.025200+0000","last_clean":"2024-09-16T00:12:04.025200+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:12:04.025200+0000","last_undegraded":"2024-09-16T00:12:04.025200+0000","last_fullsized":"2024-09-16T00:12:04.025200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024950+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:12:04.024950+0000","last_peered":"2024-09-16T00:12:04.024950+0000","last_clean":"2024-09-16T00:12:04.024950+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:12:04.024950+0000","last_undegraded":"2024-09-16T00:12:04.024950+0000","last_fullsized":"2024-09-16T00:12:04.024950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70139,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":549,"num_read_kb":353,"num_write":254,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579936,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1007616,"data_stored":496485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1703936,"data_stored":1146232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1036288,"data_stored":500118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1028096,"data_stored":496400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1626112,"data_stored":1092899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593785,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":491526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1634304,"data_stored":1088088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986244,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":495993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":491234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1097728,"data_stored":556100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1077248,"data_stored":555487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":493375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:12:38.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:12:38.475 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:38.780 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:38 smithi078 ceph-mon[31157]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:38.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:38 smithi193 ceph-mon[35484]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:38.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:38 smithi148 ceph-mon[35599]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:38.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.977+0000 7fe40b76d640 1 -- 172.21.15.78:0/2759510669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4041036e0 msgr2=0x7fe404107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.977+0000 7fe40b76d640 1 --2- 172.21.15.78:0/2759510669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4041036e0 0x7fe404107b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009930 tx=0x7fe3f802a420 comp rx=0 tx=0).stop 2024-09-16T00:12:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 -- 172.21.15.78:0/2759510669 shutdown_connections 2024-09-16T00:12:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 --2- 172.21.15.78:0/2759510669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4041082a0 0x7fe40410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:38.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 --2- 172.21.15.78:0/2759510669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4041036e0 0x7fe404107b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 --2- 172.21.15.78:0/2759510669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe404102d30 0x7fe404103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 -- 172.21.15.78:0/2759510669 >> 172.21.15.78:0/2759510669 conn(0x7fe4040fc7d0 msgr2=0x7fe4040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 -- 172.21.15.78:0/2759510669 shutdown_connections 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 -- 172.21.15.78:0/2759510669 wait complete. 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 Processor -- start 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 -- start start 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe404102d30 0x7fe404198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4041036e0 0x7fe404198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4041082a0 0x7fe40419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4041120a0 con 0x7fe404102d30 2024-09-16T00:12:38.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe404111f20 con 0x7fe4041082a0 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.978+0000 7fe40b76d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe404112220 con 0x7fe4041036e0 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe404102d30 0x7fe404198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe404102d30 0x7fe404198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42458/0 (socket says 172.21.15.78:42458) 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 -- 172.21.15.78:0/3948617159 learned_addr learned my addr 172.21.15.78:0/3948617159 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409d0a640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4041082a0 0x7fe40419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe408d08640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4041036e0 0x7fe404198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 -- 172.21.15.78:0/3948617159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4041036e0 msgr2=0x7fe404198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4041036e0 0x7fe404198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 -- 172.21.15.78:0/3948617159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4041082a0 msgr2=0x7fe40419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4041082a0 0x7fe40419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:38.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 -- 172.21.15.78:0/3948617159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40419d530 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe409509640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe404102d30 0x7fe404198580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400cce0 tx=0x7fe3f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe4027fc640 1 -- 172.21.15.78:0/3948617159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4004b20 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40419d820 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe4027fc640 1 -- 172.21.15.78:0/3948617159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f4004cc0 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe4027fc640 1 -- 172.21.15.78:0/3948617159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400f8c0 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.979+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4041a5100 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.980+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe404103110 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.981+0000 7fe4027fc640 1 -- 172.21.15.78:0/3948617159 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe3f400fa60 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.982+0000 7fe4027fc640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3ec0a32d0 0x7fe3ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.982+0000 7fe408d08640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3ec0a32d0 0x7fe3ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.982+0000 7fe4027fc640 1 -- 172.21.15.78:0/3948617159 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fe3f40d0360 con 0x7fe404102d30 2024-09-16T00:12:38.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.983+0000 7fe408d08640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3ec0a32d0 0x7fe3ec0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009900 tx=0x7fe3f8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:38.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:38.985+0000 7fe4027fc640 1 -- 172.21.15.78:0/3948617159 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3f4014030 con 0x7fe404102d30 2024-09-16T00:12:39.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.149+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe404062630 con 0x7fe3ec0a32d0 2024-09-16T00:12:39.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.157+0000 7fe4027fc640 1 -- 172.21.15.78:0/3948617159 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340395 (secure 0 0 0) 0x7fe404062630 con 0x7fe3ec0a32d0 2024-09-16T00:12:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:39.164 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:12:39.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3ec0a32d0 msgr2=0x7fe3ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:39.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3ec0a32d0 0x7fe3ec0a5790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009900 tx=0x7fe3f8000bc0 comp rx=0 tx=0).stop 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe404102d30 msgr2=0x7fe404198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe404102d30 0x7fe404198580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400cce0 tx=0x7fe3f400a620 comp rx=0 tx=0).stop 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 shutdown_connections 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3ec0a32d0 0x7fe3ec0a5790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4041082a0 0x7fe40419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4041036e0 0x7fe404198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 --2- 172.21.15.78:0/3948617159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe404102d30 0x7fe404198580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.160+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 >> 172.21.15.78:0/3948617159 conn(0x7fe4040fc7d0 msgr2=0x7fe404105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.161+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 shutdown_connections 2024-09-16T00:12:39.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:39.161+0000 7fe40b76d640 1 -- 172.21.15.78:0/3948617159 wait complete. 2024-09-16T00:12:39.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:39 smithi078 ceph-mon[31157]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:39.919 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-16T00:12:38.417306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660741,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":690,"num_read_kb":467,"num_write":437,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":356,"ondisk_log_size":356,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359116,"kb_used_data":13952,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582708,"statfs":{"total":1151940427776,"available":1151572692992,"internally_reserved":0,"allocated":14286848,"data_stored":7903937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003139"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:33.617217+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:12:33.617217+0000","last_peered":"2024-09-16T00:12:33.617217+0000","last_clean":"2024-09-16T00:12:33.617217+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:12:33.617217+0000","last_undegraded":"2024-09-16T00:12:33.617217+0000","last_fullsized":"2024-09-16T00:12:33.617217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131509+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:12:04.131509+0000","last_peered":"2024-09-16T00:12:04.131509+0000","last_clean":"2024-09-16T00:12:04.131509+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:12:04.131509+0000","last_undegraded":"2024-09-16T00:12:04.131509+0000","last_fullsized":"2024-09-16T00:12:04.131509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.132674+0000","last_change":"2024-09-16T00:12:04.132674+0000","last_active":"2024-09-16T00:12:04.132674+0000","last_peered":"2024-09-16T00:12:04.132674+0000","last_clean":"2024-09-16T00:12:04.132674+0000","last_became_active":"2024-09-16T00:12:04.132108+0000","last_became_peered":"2024-09-16T00:12:04.132108+0000","last_unstale":"2024-09-16T00:12:04.132674+0000","last_undegraded":"2024-09-16T00:12:04.132674+0000","last_fullsized":"2024-09-16T00:12:04.132674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:10.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131653+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:12:04.131653+0000","last_peered":"2024-09-16T00:12:04.131653+0000","last_clean":"2024-09-16T00:12:04.131653+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:12:04.131653+0000","last_undegraded":"2024-09-16T00:12:04.131653+0000","last_fullsized":"2024-09-16T00:12:04.131653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025410+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:12:04.025410+0000","last_peered":"2024-09-16T00:12:04.025410+0000","last_clean":"2024-09-16T00:12:04.025410+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:12:04.025410+0000","last_undegraded":"2024-09-16T00:12:04.025410+0000","last_fullsized":"2024-09-16T00:12:04.025410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024689+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:12:04.024689+0000","last_peered":"2024-09-16T00:12:04.024689+0000","last_clean":"2024-09-16T00:12:04.024689+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:12:04.024689+0000","last_undegraded":"2024-09-16T00:12:04.024689+0000","last_fullsized":"2024-09-16T00:12:04.024689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028354+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:12:04.028354+0000","last_peered":"2024-09-16T00:12:04.028354+0000","last_clean":"2024-09-16T00:12:04.028354+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:12:04.028354+0000","last_undegraded":"2024-09-16T00:12:04.028354+0000","last_fullsized":"2024-09-16T00:12:04.028354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027023+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:12:04.027023+0000","last_peered":"2024-09-16T00:12:04.027023+0000","last_clean":"2024-09-16T00:12:04.027023+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:12:04.027023+0000","last_undegraded":"2024-09-16T00:12:04.027023+0000","last_fullsized":"2024-09-16T00:12:04.027023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025689+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:12:04.025689+0000","last_peered":"2024-09-16T00:12:04.025689+0000","last_clean":"2024-09-16T00:12:04.025689+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:12:04.025689+0000","last_undegraded":"2024-09-16T00:12:04.025689+0000","last_fullsized":"2024-09-16T00:12:04.025689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028655+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:12:04.028655+0000","last_peered":"2024-09-16T00:12:04.028655+0000","last_clean":"2024-09-16T00:12:04.028655+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:12:04.028655+0000","last_undegraded":"2024-09-16T00:12:04.028655+0000","last_fullsized":"2024-09-16T00:12:04.028655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027065+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:12:04.027065+0000","last_peered":"2024-09-16T00:12:04.027065+0000","last_clean":"2024-09-16T00:12:04.027065+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:12:04.027065+0000","last_undegraded":"2024-09-16T00:12:04.027065+0000","last_fullsized":"2024-09-16T00:12:04.027065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025373+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:12:04.025373+0000","last_peered":"2024-09-16T00:12:04.025373+0000","last_clean":"2024-09-16T00:12:04.025373+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:12:04.025373+0000","last_undegraded":"2024-09-16T00:12:04.025373+0000","last_fullsized":"2024-09-16T00:12:04.025373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024110+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:12:04.024110+0000","last_peered":"2024-09-16T00:12:04.024110+0000","last_clean":"2024-09-16T00:12:04.024110+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:12:04.024110+0000","last_undegraded":"2024-09-16T00:12:04.024110+0000","last_fullsized":"2024-09-16T00:12:04.024110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.033872+0000","last_change":"2024-09-16T00:12:04.033464+0000","last_active":"2024-09-16T00:12:04.033872+0000","last_peered":"2024-09-16T00:12:04.033872+0000","last_clean":"2024-09-16T00:12:04.033872+0000","last_became_active":"2024-09-16T00:12:04.028916+0000","last_became_peered":"2024-09-16T00:12:04.028916+0000","last_unstale":"2024-09-16T00:12:04.033872+0000","last_undegraded":"2024-09-16T00:12:04.033872+0000","last_fullsized":"2024-09-16T00:12:04.033872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:47.359979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026145+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:12:04.026145+0000","last_peered":"2024-09-16T00:12:04.026145+0000","last_clean":"2024-09-16T00:12:04.026145+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:12:04.026145+0000","last_undegraded":"2024-09-16T00:12:04.026145+0000","last_fullsized":"2024-09-16T00:12:04.026145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131245+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:12:04.131245+0000","last_peered":"2024-09-16T00:12:04.131245+0000","last_clean":"2024-09-16T00:12:04.131245+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:12:04.131245+0000","last_undegraded":"2024-09-16T00:12:04.131245+0000","last_fullsized":"2024-09-16T00:12:04.131245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023148+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:12:04.023148+0000","last_peered":"2024-09-16T00:12:04.023148+0000","last_clean":"2024-09-16T00:12:04.023148+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:12:04.023148+0000","last_undegraded":"2024-09-16T00:12:04.023148+0000","last_fullsized":"2024-09-16T00:12:04.023148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026164+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:12:04.026164+0000","last_peered":"2024-09-16T00:12:04.026164+0000","last_clean":"2024-09-16T00:12:04.026164+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:12:04.026164+0000","last_undegraded":"2024-09-16T00:12:04.026164+0000","last_fullsized":"2024-09-16T00:12:04.026164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022802+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:12:04.022802+0000","last_peered":"2024-09-16T00:12:04.022802+0000","last_clean":"2024-09-16T00:12:04.022802+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:12:04.022802+0000","last_undegraded":"2024-09-16T00:12:04.022802+0000","last_fullsized":"2024-09-16T00:12:04.022802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131462+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:12:04.131462+0000","last_peered":"2024-09-16T00:12:04.131462+0000","last_clean":"2024-09-16T00:12:04.131462+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:12:04.131462+0000","last_undegraded":"2024-09-16T00:12:04.131462+0000","last_fullsized":"2024-09-16T00:12:04.131462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021468+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:12:04.021468+0000","last_peered":"2024-09-16T00:12:04.021468+0000","last_clean":"2024-09-16T00:12:04.021468+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:12:04.021468+0000","last_undegraded":"2024-09-16T00:12:04.021468+0000","last_fullsized":"2024-09-16T00:12:04.021468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026552+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:12:04.026552+0000","last_peered":"2024-09-16T00:12:04.026552+0000","last_clean":"2024-09-16T00:12:04.026552+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:12:04.026552+0000","last_undegraded":"2024-09-16T00:12:04.026552+0000","last_fullsized":"2024-09-16T00:12:04.026552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.034002+0000","last_change":"2024-09-16T00:12:04.034002+0000","last_active":"2024-09-16T00:12:04.034002+0000","last_peered":"2024-09-16T00:12:04.034002+0000","last_clean":"2024-09-16T00:12:04.034002+0000","last_became_active":"2024-09-16T00:12:04.030959+0000","last_became_peered":"2024-09-16T00:12:04.030959+0000","last_unstale":"2024-09-16T00:12:04.034002+0000","last_undegraded":"2024-09-16T00:12:04.034002+0000","last_fullsized":"2024-09-16T00:12:04.034002+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:47.730692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.029906+0000","last_change":"2024-09-16T00:12:04.029906+0000","last_active":"2024-09-16T00:12:04.029906+0000","last_peered":"2024-09-16T00:12:04.029906+0000","last_clean":"2024-09-16T00:12:04.029906+0000","last_became_active":"2024-09-16T00:12:04.029259+0000","last_became_peered":"2024-09-16T00:12:04.029259+0000","last_unstale":"2024-09-16T00:12:04.029906+0000","last_undegraded":"2024-09-16T00:12:04.029906+0000","last_fullsized":"2024-09-16T00:12:04.029906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:23.399762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,1],"acting":[4,8,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025284+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:12:04.025284+0000","last_peered":"2024-09-16T00:12:04.025284+0000","last_clean":"2024-09-16T00:12:04.025284+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:12:04.025284+0000","last_undegraded":"2024-09-16T00:12:04.025284+0000","last_fullsized":"2024-09-16T00:12:04.025284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025227+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:12:04.025227+0000","last_peered":"2024-09-16T00:12:04.025227+0000","last_clean":"2024-09-16T00:12:04.025227+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:12:04.025227+0000","last_undegraded":"2024-09-16T00:12:04.025227+0000","last_fullsized":"2024-09-16T00:12:04.025227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023266+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:12:04.023266+0000","last_peered":"2024-09-16T00:12:04.023266+0000","last_clean":"2024-09-16T00:12:04.023266+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:12:04.023266+0000","last_undegraded":"2024-09-16T00:12:04.023266+0000","last_fullsized":"2024-09-16T00:12:04.023266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131687+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:12:04.131687+0000","last_peered":"2024-09-16T00:12:04.131687+0000","last_clean":"2024-09-16T00:12:04.131687+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:12:04.131687+0000","last_undegraded":"2024-09-16T00:12:04.131687+0000","last_fullsized":"2024-09-16T00:12:04.131687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'18","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021955+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:12:04.021955+0000","last_peered":"2024-09-16T00:12:04.021955+0000","last_clean":"2024-09-16T00:12:04.021955+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:12:04.021955+0000","last_undegraded":"2024-09-16T00:12:04.021955+0000","last_fullsized":"2024-09-16T00:12:04.021955+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131081+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:12:04.131081+0000","last_peered":"2024-09-16T00:12:04.131081+0000","last_clean":"2024-09-16T00:12:04.131081+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:12:04.131081+0000","last_undegraded":"2024-09-16T00:12:04.131081+0000","last_fullsized":"2024-09-16T00:12:04.131081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024873+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:12:04.024873+0000","last_peered":"2024-09-16T00:12:04.024873+0000","last_clean":"2024-09-16T00:12:04.024873+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:12:04.024873+0000","last_undegraded":"2024-09-16T00:12:04.024873+0000","last_fullsized":"2024-09-16T00:12:04.024873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028396+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:12:04.028396+0000","last_peered":"2024-09-16T00:12:04.028396+0000","last_clean":"2024-09-16T00:12:04.028396+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:12:04.028396+0000","last_undegraded":"2024-09-16T00:12:04.028396+0000","last_fullsized":"2024-09-16T00:12:04.028396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023644+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:12:04.023644+0000","last_peered":"2024-09-16T00:12:04.023644+0000","last_clean":"2024-09-16T00:12:04.023644+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:12:04.023644+0000","last_undegraded":"2024-09-16T00:12:04.023644+0000","last_fullsized":"2024-09-16T00:12:04.023644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022469+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:12:04.022469+0000","last_peered":"2024-09-16T00:12:04.022469+0000","last_clean":"2024-09-16T00:12:04.022469+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:12:04.022469+0000","last_undegraded":"2024-09-16T00:12:04.022469+0000","last_fullsized":"2024-09-16T00:12:04.022469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021573+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:12:04.021573+0000","last_peered":"2024-09-16T00:12:04.021573+0000","last_clean":"2024-09-16T00:12:04.021573+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:12:04.021573+0000","last_undegraded":"2024-09-16T00:12:04.021573+0000","last_fullsized":"2024-09-16T00:12:04.021573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025893+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:12:04.025893+0000","last_peered":"2024-09-16T00:12:04.025893+0000","last_clean":"2024-09-16T00:12:04.025893+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:12:04.025893+0000","last_undegraded":"2024-09-16T00:12:04.025893+0000","last_fullsized":"2024-09-16T00:12:04.025893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022530+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:12:04.022530+0000","last_peered":"2024-09-16T00:12:04.022530+0000","last_clean":"2024-09-16T00:12:04.022530+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:12:04.022530+0000","last_undegraded":"2024-09-16T00:12:04.022530+0000","last_fullsized":"2024-09-16T00:12:04.022530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:31.614855+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:12:31.614855+0000","last_peered":"2024-09-16T00:12:31.614855+0000","last_clean":"2024-09-16T00:12:31.614855+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:12:31.614855+0000","last_undegraded":"2024-09-16T00:12:31.614855+0000","last_fullsized":"2024-09-16T00:12:31.614855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024308+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:12:04.024308+0000","last_peered":"2024-09-16T00:12:04.024308+0000","last_clean":"2024-09-16T00:12:04.024308+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:12:04.024308+0000","last_undegraded":"2024-09-16T00:12:04.024308+0000","last_fullsized":"2024-09-16T00:12:04.024308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026567+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:12:04.026567+0000","last_peered":"2024-09-16T00:12:04.026567+0000","last_clean":"2024-09-16T00:12:04.026567+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:12:04.026567+0000","last_undegraded":"2024-09-16T00:12:04.026567+0000","last_fullsized":"2024-09-16T00:12:04.026567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:36.642031+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:12:36.642031+0000","last_peered":"2024-09-16T00:12:36.642031+0000","last_clean":"2024-09-16T00:12:36.642031+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:12:36.642031+0000","last_undegraded":"2024-09-16T00:12:36.642031+0000","last_fullsized":"2024-09-16T00:12:36.642031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025419+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:12:04.025419+0000","last_peered":"2024-09-16T00:12:04.025419+0000","last_clean":"2024-09-16T00:12:04.025419+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:12:04.025419+0000","last_undegraded":"2024-09-16T00:12:04.025419+0000","last_fullsized":"2024-09-16T00:12:04.025419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027859+0000","last_change":"2024-09-16T00:12:04.027859+0000","last_active":"2024-09-16T00:12:04.027859+0000","last_peered":"2024-09-16T00:12:04.027859+0000","last_clean":"2024-09-16T00:12:04.027859+0000","last_became_active":"2024-09-16T00:12:04.027506+0000","last_became_peered":"2024-09-16T00:12:04.027506+0000","last_unstale":"2024-09-16T00:12:04.027859+0000","last_undegraded":"2024-09-16T00:12:04.027859+0000","last_fullsized":"2024-09-16T00:12:04.027859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:34.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023822+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:12:04.023822+0000","last_peered":"2024-09-16T00:12:04.023822+0000","last_clean":"2024-09-16T00:12:04.023822+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:12:04.023822+0000","last_undegraded":"2024-09-16T00:12:04.023822+0000","last_fullsized":"2024-09-16T00:12:04.023822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.029471+0000","last_change":"2024-09-16T00:12:04.029471+0000","last_active":"2024-09-16T00:12:04.029471+0000","last_peered":"2024-09-16T00:12:04.029471+0000","last_clean":"2024-09-16T00:12:04.029471+0000","last_became_active":"2024-09-16T00:12:04.029054+0000","last_became_peered":"2024-09-16T00:12:04.029054+0000","last_unstale":"2024-09-16T00:12:04.029471+0000","last_undegraded":"2024-09-16T00:12:04.029471+0000","last_fullsized":"2024-09-16T00:12:04.029471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:52.054218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131251+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:12:04.131251+0000","last_peered":"2024-09-16T00:12:04.131251+0000","last_clean":"2024-09-16T00:12:04.131251+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:12:04.131251+0000","last_undegraded":"2024-09-16T00:12:04.131251+0000","last_fullsized":"2024-09-16T00:12:04.131251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026112+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:12:04.026112+0000","last_peered":"2024-09-16T00:12:04.026112+0000","last_clean":"2024-09-16T00:12:04.026112+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:12:04.026112+0000","last_undegraded":"2024-09-16T00:12:04.026112+0000","last_fullsized":"2024-09-16T00:12:04.026112+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023973+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:12:04.023973+0000","last_peered":"2024-09-16T00:12:04.023973+0000","last_clean":"2024-09-16T00:12:04.023973+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:12:04.023973+0000","last_undegraded":"2024-09-16T00:12:04.023973+0000","last_fullsized":"2024-09-16T00:12:04.023973+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024398+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:12:04.024398+0000","last_peered":"2024-09-16T00:12:04.024398+0000","last_clean":"2024-09-16T00:12:04.024398+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:12:04.024398+0000","last_undegraded":"2024-09-16T00:12:04.024398+0000","last_fullsized":"2024-09-16T00:12:04.024398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025571+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:12:04.025571+0000","last_peered":"2024-09-16T00:12:04.025571+0000","last_clean":"2024-09-16T00:12:04.025571+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:12:04.025571+0000","last_undegraded":"2024-09-16T00:12:04.025571+0000","last_fullsized":"2024-09-16T00:12:04.025571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:33.619234+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:12:33.619234+0000","last_peered":"2024-09-16T00:12:33.619234+0000","last_clean":"2024-09-16T00:12:33.619234+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:12:33.619234+0000","last_undegraded":"2024-09-16T00:12:33.619234+0000","last_fullsized":"2024-09-16T00:12:33.619234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026989+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:12:04.026989+0000","last_peered":"2024-09-16T00:12:04.026989+0000","last_clean":"2024-09-16T00:12:04.026989+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:12:04.026989+0000","last_undegraded":"2024-09-16T00:12:04.026989+0000","last_fullsized":"2024-09-16T00:12:04.026989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024184+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:12:04.024184+0000","last_peered":"2024-09-16T00:12:04.024184+0000","last_clean":"2024-09-16T00:12:04.024184+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:12:04.024184+0000","last_undegraded":"2024-09-16T00:12:04.024184+0000","last_fullsized":"2024-09-16T00:12:04.024184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027184+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:12:04.027184+0000","last_peered":"2024-09-16T00:12:04.027184+0000","last_clean":"2024-09-16T00:12:04.027184+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:12:04.027184+0000","last_undegraded":"2024-09-16T00:12:04.027184+0000","last_fullsized":"2024-09-16T00:12:04.027184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024966+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:12:04.024966+0000","last_peered":"2024-09-16T00:12:04.024966+0000","last_clean":"2024-09-16T00:12:04.024966+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:12:04.024966+0000","last_undegraded":"2024-09-16T00:12:04.024966+0000","last_fullsized":"2024-09-16T00:12:04.024966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025278+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:12:04.025278+0000","last_peered":"2024-09-16T00:12:04.025278+0000","last_clean":"2024-09-16T00:12:04.025278+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:12:04.025278+0000","last_undegraded":"2024-09-16T00:12:04.025278+0000","last_fullsized":"2024-09-16T00:12:04.025278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025716+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:12:04.025716+0000","last_peered":"2024-09-16T00:12:04.025716+0000","last_clean":"2024-09-16T00:12:04.025716+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:12:04.025716+0000","last_undegraded":"2024-09-16T00:12:04.025716+0000","last_fullsized":"2024-09-16T00:12:04.025716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026174+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:12:04.026174+0000","last_peered":"2024-09-16T00:12:04.026174+0000","last_clean":"2024-09-16T00:12:04.026174+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:12:04.026174+0000","last_undegraded":"2024-09-16T00:12:04.026174+0000","last_fullsized":"2024-09-16T00:12:04.026174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024068+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:12:04.024068+0000","last_peered":"2024-09-16T00:12:04.024068+0000","last_clean":"2024-09-16T00:12:04.024068+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:12:04.024068+0000","last_undegraded":"2024-09-16T00:12:04.024068+0000","last_fullsized":"2024-09-16T00:12:04.024068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028185+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:12:04.028185+0000","last_peered":"2024-09-16T00:12:04.028185+0000","last_clean":"2024-09-16T00:12:04.028185+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:12:04.028185+0000","last_undegraded":"2024-09-16T00:12:04.028185+0000","last_fullsized":"2024-09-16T00:12:04.028185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025385+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:12:04.025385+0000","last_peered":"2024-09-16T00:12:04.025385+0000","last_clean":"2024-09-16T00:12:04.025385+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:12:04.025385+0000","last_undegraded":"2024-09-16T00:12:04.025385+0000","last_fullsized":"2024-09-16T00:12:04.025385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025738+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:12:04.025738+0000","last_peered":"2024-09-16T00:12:04.025738+0000","last_clean":"2024-09-16T00:12:04.025738+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:12:04.025738+0000","last_undegraded":"2024-09-16T00:12:04.025738+0000","last_fullsized":"2024-09-16T00:12:04.025738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025960+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:12:04.025960+0000","last_peered":"2024-09-16T00:12:04.025960+0000","last_clean":"2024-09-16T00:12:04.025960+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:12:04.025960+0000","last_undegraded":"2024-09-16T00:12:04.025960+0000","last_fullsized":"2024-09-16T00:12:04.025960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025608+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:12:04.025608+0000","last_peered":"2024-09-16T00:12:04.025608+0000","last_clean":"2024-09-16T00:12:04.025608+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:12:04.025608+0000","last_undegraded":"2024-09-16T00:12:04.025608+0000","last_fullsized":"2024-09-16T00:12:04.025608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023685+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:12:04.023685+0000","last_peered":"2024-09-16T00:12:04.023685+0000","last_clean":"2024-09-16T00:12:04.023685+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:12:04.023685+0000","last_undegraded":"2024-09-16T00:12:04.023685+0000","last_fullsized":"2024-09-16T00:12:04.023685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:36.642727+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:12:36.642727+0000","last_peered":"2024-09-16T00:12:36.642727+0000","last_clean":"2024-09-16T00:12:36.642727+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:12:36.642727+0000","last_undegraded":"2024-09-16T00:12:36.642727+0000","last_fullsized":"2024-09-16T00:12:36.642727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021569+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:12:04.021569+0000","last_peered":"2024-09-16T00:12:04.021569+0000","last_clean":"2024-09-16T00:12:04.021569+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:12:04.021569+0000","last_undegraded":"2024-09-16T00:12:04.021569+0000","last_fullsized":"2024-09-16T00:12:04.021569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024056+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:12:04.024056+0000","last_peered":"2024-09-16T00:12:04.024056+0000","last_clean":"2024-09-16T00:12:04.024056+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:12:04.024056+0000","last_undegraded":"2024-09-16T00:12:04.024056+0000","last_fullsized":"2024-09-16T00:12:04.024056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026671+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:12:04.026671+0000","last_peered":"2024-09-16T00:12:04.026671+0000","last_clean":"2024-09-16T00:12:04.026671+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:12:04.026671+0000","last_undegraded":"2024-09-16T00:12:04.026671+0000","last_fullsized":"2024-09-16T00:12:04.026671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023704+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:12:04.023704+0000","last_peered":"2024-09-16T00:12:04.023704+0000","last_clean":"2024-09-16T00:12:04.023704+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:12:04.023704+0000","last_undegraded":"2024-09-16T00:12:04.023704+0000","last_fullsized":"2024-09-16T00:12:04.023704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024155+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:12:04.024155+0000","last_peered":"2024-09-16T00:12:04.024155+0000","last_clean":"2024-09-16T00:12:04.024155+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:12:04.024155+0000","last_undegraded":"2024-09-16T00:12:04.024155+0000","last_fullsized":"2024-09-16T00:12:04.024155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023404+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:12:04.023404+0000","last_peered":"2024-09-16T00:12:04.023404+0000","last_clean":"2024-09-16T00:12:04.023404+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:12:04.023404+0000","last_undegraded":"2024-09-16T00:12:04.023404+0000","last_fullsized":"2024-09-16T00:12:04.023404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026986+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:12:04.026986+0000","last_peered":"2024-09-16T00:12:04.026986+0000","last_clean":"2024-09-16T00:12:04.026986+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:12:04.026986+0000","last_undegraded":"2024-09-16T00:12:04.026986+0000","last_fullsized":"2024-09-16T00:12:04.026986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028251+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:12:04.028251+0000","last_peered":"2024-09-16T00:12:04.028251+0000","last_clean":"2024-09-16T00:12:04.028251+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:12:04.028251+0000","last_undegraded":"2024-09-16T00:12:04.028251+0000","last_fullsized":"2024-09-16T00:12:04.028251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":182,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:34.619871+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:12:34.619871+0000","last_peered":"2024-09-16T00:12:34.619871+0000","last_clean":"2024-09-16T00:12:34.619871+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:12:34.619871+0000","last_undegraded":"2024-09-16T00:12:34.619871+0000","last_fullsized":"2024-09-16T00:12:34.619871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62931,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":117,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021641+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:12:04.021641+0000","last_peered":"2024-09-16T00:12:04.021641+0000","last_clean":"2024-09-16T00:12:04.021641+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:12:04.021641+0000","last_undegraded":"2024-09-16T00:12:04.021641+0000","last_fullsized":"2024-09-16T00:12:04.021641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027169+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:12:04.027169+0000","last_peered":"2024-09-16T00:12:04.027169+0000","last_clean":"2024-09-16T00:12:04.027169+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:12:04.027169+0000","last_undegraded":"2024-09-16T00:12:04.027169+0000","last_fullsized":"2024-09-16T00:12:04.027169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027157+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:12:04.027157+0000","last_peered":"2024-09-16T00:12:04.027157+0000","last_clean":"2024-09-16T00:12:04.027157+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:12:04.027157+0000","last_undegraded":"2024-09-16T00:12:04.027157+0000","last_fullsized":"2024-09-16T00:12:04.027157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025924+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:12:04.025924+0000","last_peered":"2024-09-16T00:12:04.025924+0000","last_clean":"2024-09-16T00:12:04.025924+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:12:04.025924+0000","last_undegraded":"2024-09-16T00:12:04.025924+0000","last_fullsized":"2024-09-16T00:12:04.025924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025502+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:12:04.025502+0000","last_peered":"2024-09-16T00:12:04.025502+0000","last_clean":"2024-09-16T00:12:04.025502+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:12:04.025502+0000","last_undegraded":"2024-09-16T00:12:04.025502+0000","last_fullsized":"2024-09-16T00:12:04.025502+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025140+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:12:04.025140+0000","last_peered":"2024-09-16T00:12:04.025140+0000","last_clean":"2024-09-16T00:12:04.025140+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:12:04.025140+0000","last_undegraded":"2024-09-16T00:12:04.025140+0000","last_fullsized":"2024-09-16T00:12:04.025140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021165+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:12:04.021165+0000","last_peered":"2024-09-16T00:12:04.021165+0000","last_clean":"2024-09-16T00:12:04.021165+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:12:04.021165+0000","last_undegraded":"2024-09-16T00:12:04.021165+0000","last_fullsized":"2024-09-16T00:12:04.021165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027316+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:12:04.027316+0000","last_peered":"2024-09-16T00:12:04.027316+0000","last_clean":"2024-09-16T00:12:04.027316+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:12:04.027316+0000","last_undegraded":"2024-09-16T00:12:04.027316+0000","last_fullsized":"2024-09-16T00:12:04.027316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:34.619808+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:12:34.619808+0000","last_peered":"2024-09-16T00:12:34.619808+0000","last_clean":"2024-09-16T00:12:34.619808+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:12:34.619808+0000","last_undegraded":"2024-09-16T00:12:34.619808+0000","last_fullsized":"2024-09-16T00:12:34.619808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131339+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:12:04.131339+0000","last_peered":"2024-09-16T00:12:04.131339+0000","last_clean":"2024-09-16T00:12:04.131339+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:12:04.131339+0000","last_undegraded":"2024-09-16T00:12:04.131339+0000","last_fullsized":"2024-09-16T00:12:04.131339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025106+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:12:04.025106+0000","last_peered":"2024-09-16T00:12:04.025106+0000","last_clean":"2024-09-16T00:12:04.025106+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:12:04.025106+0000","last_undegraded":"2024-09-16T00:12:04.025106+0000","last_fullsized":"2024-09-16T00:12:04.025106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021346+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:12:04.021346+0000","last_peered":"2024-09-16T00:12:04.021346+0000","last_clean":"2024-09-16T00:12:04.021346+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:12:04.021346+0000","last_undegraded":"2024-09-16T00:12:04.021346+0000","last_fullsized":"2024-09-16T00:12:04.021346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026679+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:12:04.026679+0000","last_peered":"2024-09-16T00:12:04.026679+0000","last_clean":"2024-09-16T00:12:04.026679+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:12:04.026679+0000","last_undegraded":"2024-09-16T00:12:04.026679+0000","last_fullsized":"2024-09-16T00:12:04.026679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024106+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:12:04.024106+0000","last_peered":"2024-09-16T00:12:04.024106+0000","last_clean":"2024-09-16T00:12:04.024106+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:12:04.024106+0000","last_undegraded":"2024-09-16T00:12:04.024106+0000","last_fullsized":"2024-09-16T00:12:04.024106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131191+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:12:04.131191+0000","last_peered":"2024-09-16T00:12:04.131191+0000","last_clean":"2024-09-16T00:12:04.131191+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:12:04.131191+0000","last_undegraded":"2024-09-16T00:12:04.131191+0000","last_fullsized":"2024-09-16T00:12:04.131191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023936+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:12:04.023936+0000","last_peered":"2024-09-16T00:12:04.023936+0000","last_clean":"2024-09-16T00:12:04.023936+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:12:04.023936+0000","last_undegraded":"2024-09-16T00:12:04.023936+0000","last_fullsized":"2024-09-16T00:12:04.023936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022320+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:12:04.022320+0000","last_peered":"2024-09-16T00:12:04.022320+0000","last_clean":"2024-09-16T00:12:04.022320+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:12:04.022320+0000","last_undegraded":"2024-09-16T00:12:04.022320+0000","last_fullsized":"2024-09-16T00:12:04.022320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025656+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:12:04.025656+0000","last_peered":"2024-09-16T00:12:04.025656+0000","last_clean":"2024-09-16T00:12:04.025656+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:12:04.025656+0000","last_undegraded":"2024-09-16T00:12:04.025656+0000","last_fullsized":"2024-09-16T00:12:04.025656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023608+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:12:04.023608+0000","last_peered":"2024-09-16T00:12:04.023608+0000","last_clean":"2024-09-16T00:12:04.023608+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:12:04.023608+0000","last_undegraded":"2024-09-16T00:12:04.023608+0000","last_fullsized":"2024-09-16T00:12:04.023608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:34.620532+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:12:34.620532+0000","last_peered":"2024-09-16T00:12:34.620532+0000","last_clean":"2024-09-16T00:12:34.620532+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:12:34.620532+0000","last_undegraded":"2024-09-16T00:12:34.620532+0000","last_fullsized":"2024-09-16T00:12:34.620532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027324+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:12:04.027324+0000","last_peered":"2024-09-16T00:12:04.027324+0000","last_clean":"2024-09-16T00:12:04.027324+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:12:04.027324+0000","last_undegraded":"2024-09-16T00:12:04.027324+0000","last_fullsized":"2024-09-16T00:12:04.027324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023080+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:12:04.023080+0000","last_peered":"2024-09-16T00:12:04.023080+0000","last_clean":"2024-09-16T00:12:04.023080+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:12:04.023080+0000","last_undegraded":"2024-09-16T00:12:04.023080+0000","last_fullsized":"2024-09-16T00:12:04.023080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022647+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:12:04.022647+0000","last_peered":"2024-09-16T00:12:04.022647+0000","last_clean":"2024-09-16T00:12:04.022647+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:12:04.022647+0000","last_undegraded":"2024-09-16T00:12:04.022647+0000","last_fullsized":"2024-09-16T00:12:04.022647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024254+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:12:04.024254+0000","last_peered":"2024-09-16T00:12:04.024254+0000","last_clean":"2024-09-16T00:12:04.024254+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:12:04.024254+0000","last_undegraded":"2024-09-16T00:12:04.024254+0000","last_fullsized":"2024-09-16T00:12:04.024254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022993+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:12:04.022993+0000","last_peered":"2024-09-16T00:12:04.022993+0000","last_clean":"2024-09-16T00:12:04.022993+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:12:04.022993+0000","last_undegraded":"2024-09-16T00:12:04.022993+0000","last_fullsized":"2024-09-16T00:12:04.022993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.029280+0000","last_change":"2024-09-16T00:12:04.029280+0000","last_active":"2024-09-16T00:12:04.029280+0000","last_peered":"2024-09-16T00:12:04.029280+0000","last_clean":"2024-09-16T00:12:04.029280+0000","last_became_active":"2024-09-16T00:12:04.028973+0000","last_became_peered":"2024-09-16T00:12:04.028973+0000","last_unstale":"2024-09-16T00:12:04.029280+0000","last_undegraded":"2024-09-16T00:12:04.029280+0000","last_fullsized":"2024-09-16T00:12:04.029280+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028319+0000","last_change":"2024-09-16T00:12:04.028319+0000","last_active":"2024-09-16T00:12:04.028319+0000","last_peered":"2024-09-16T00:12:04.028319+0000","last_clean":"2024-09-16T00:12:04.028319+0000","last_became_active":"2024-09-16T00:12:04.027985+0000","last_became_peered":"2024-09-16T00:12:04.027985+0000","last_unstale":"2024-09-16T00:12:04.028319+0000","last_undegraded":"2024-09-16T00:12:04.028319+0000","last_fullsized":"2024-09-16T00:12:04.028319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.119428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023573+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:12:04.023573+0000","last_peered":"2024-09-16T00:12:04.023573+0000","last_clean":"2024-09-16T00:12:04.023573+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:12:04.023573+0000","last_undegraded":"2024-09-16T00:12:04.023573+0000","last_fullsized":"2024-09-16T00:12:04.023573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024142+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:12:04.024142+0000","last_peered":"2024-09-16T00:12:04.024142+0000","last_clean":"2024-09-16T00:12:04.024142+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:12:04.024142+0000","last_undegraded":"2024-09-16T00:12:04.024142+0000","last_fullsized":"2024-09-16T00:12:04.024142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131625+0000","last_change":"2024-09-16T00:11:04.649155+0000","last_active":"2024-09-16T00:12:04.131625+0000","last_peered":"2024-09-16T00:12:04.131625+0000","last_clean":"2024-09-16T00:12:04.131625+0000","last_became_active":"2024-09-16T00:11:04.648844+0000","last_became_peered":"2024-09-16T00:11:04.648844+0000","last_unstale":"2024-09-16T00:12:04.131625+0000","last_undegraded":"2024-09-16T00:12:04.131625+0000","last_fullsized":"2024-09-16T00:12:04.131625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:20.205113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026514+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:12:04.026514+0000","last_peered":"2024-09-16T00:12:04.026514+0000","last_clean":"2024-09-16T00:12:04.026514+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:12:04.026514+0000","last_undegraded":"2024-09-16T00:12:04.026514+0000","last_fullsized":"2024-09-16T00:12:04.026514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131398+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:12:04.131398+0000","last_peered":"2024-09-16T00:12:04.131398+0000","last_clean":"2024-09-16T00:12:04.131398+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:12:04.131398+0000","last_undegraded":"2024-09-16T00:12:04.131398+0000","last_fullsized":"2024-09-16T00:12:04.131398+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027313+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:12:04.027313+0000","last_peered":"2024-09-16T00:12:04.027313+0000","last_clean":"2024-09-16T00:12:04.027313+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:12:04.027313+0000","last_undegraded":"2024-09-16T00:12:04.027313+0000","last_fullsized":"2024-09-16T00:12:04.027313+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024287+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:12:04.024287+0000","last_peered":"2024-09-16T00:12:04.024287+0000","last_clean":"2024-09-16T00:12:04.024287+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:12:04.024287+0000","last_undegraded":"2024-09-16T00:12:04.024287+0000","last_fullsized":"2024-09-16T00:12:04.024287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023821+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:12:04.023821+0000","last_peered":"2024-09-16T00:12:04.023821+0000","last_clean":"2024-09-16T00:12:04.023821+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:12:04.023821+0000","last_undegraded":"2024-09-16T00:12:04.023821+0000","last_fullsized":"2024-09-16T00:12:04.023821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023729+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:12:04.023729+0000","last_peered":"2024-09-16T00:12:04.023729+0000","last_clean":"2024-09-16T00:12:04.023729+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:12:04.023729+0000","last_undegraded":"2024-09-16T00:12:04.023729+0000","last_fullsized":"2024-09-16T00:12:04.023729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.021859+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:12:04.021859+0000","last_peered":"2024-09-16T00:12:04.021859+0000","last_clean":"2024-09-16T00:12:04.021859+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:12:04.021859+0000","last_undegraded":"2024-09-16T00:12:04.021859+0000","last_fullsized":"2024-09-16T00:12:04.021859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024995+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:12:04.024995+0000","last_peered":"2024-09-16T00:12:04.024995+0000","last_clean":"2024-09-16T00:12:04.024995+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:12:04.024995+0000","last_undegraded":"2024-09-16T00:12:04.024995+0000","last_fullsized":"2024-09-16T00:12:04.024995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027097+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:12:04.027097+0000","last_peered":"2024-09-16T00:12:04.027097+0000","last_clean":"2024-09-16T00:12:04.027097+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:12:04.027097+0000","last_undegraded":"2024-09-16T00:12:04.027097+0000","last_fullsized":"2024-09-16T00:12:04.027097+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024431+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:12:04.024431+0000","last_peered":"2024-09-16T00:12:04.024431+0000","last_clean":"2024-09-16T00:12:04.024431+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:12:04.024431+0000","last_undegraded":"2024-09-16T00:12:04.024431+0000","last_fullsized":"2024-09-16T00:12:04.024431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024478+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:12:04.024478+0000","last_peered":"2024-09-16T00:12:04.024478+0000","last_clean":"2024-09-16T00:12:04.024478+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:12:04.024478+0000","last_undegraded":"2024-09-16T00:12:04.024478+0000","last_fullsized":"2024-09-16T00:12:04.024478+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024195+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:12:04.024195+0000","last_peered":"2024-09-16T00:12:04.024195+0000","last_clean":"2024-09-16T00:12:04.024195+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:12:04.024195+0000","last_undegraded":"2024-09-16T00:12:04.024195+0000","last_fullsized":"2024-09-16T00:12:04.024195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026638+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:12:04.026638+0000","last_peered":"2024-09-16T00:12:04.026638+0000","last_clean":"2024-09-16T00:12:04.026638+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:12:04.026638+0000","last_undegraded":"2024-09-16T00:12:04.026638+0000","last_fullsized":"2024-09-16T00:12:04.026638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025490+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:12:04.025490+0000","last_peered":"2024-09-16T00:12:04.025490+0000","last_clean":"2024-09-16T00:12:04.025490+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:12:04.025490+0000","last_undegraded":"2024-09-16T00:12:04.025490+0000","last_fullsized":"2024-09-16T00:12:04.025490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.023272+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:12:04.023272+0000","last_peered":"2024-09-16T00:12:04.023272+0000","last_clean":"2024-09-16T00:12:04.023272+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:12:04.023272+0000","last_undegraded":"2024-09-16T00:12:04.023272+0000","last_fullsized":"2024-09-16T00:12:04.023272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027537+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:12:04.027537+0000","last_peered":"2024-09-16T00:12:04.027537+0000","last_clean":"2024-09-16T00:12:04.027537+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:12:04.027537+0000","last_undegraded":"2024-09-16T00:12:04.027537+0000","last_fullsized":"2024-09-16T00:12:04.027537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.131159+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:12:04.131159+0000","last_peered":"2024-09-16T00:12:04.131159+0000","last_clean":"2024-09-16T00:12:04.131159+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:12:04.131159+0000","last_undegraded":"2024-09-16T00:12:04.131159+0000","last_fullsized":"2024-09-16T00:12:04.131159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.028342+0000","last_change":"2024-09-16T00:12:04.028342+0000","last_active":"2024-09-16T00:12:04.028342+0000","last_peered":"2024-09-16T00:12:04.028342+0000","last_clean":"2024-09-16T00:12:04.028342+0000","last_became_active":"2024-09-16T00:12:04.027889+0000","last_became_peered":"2024-09-16T00:12:04.027889+0000","last_unstale":"2024-09-16T00:12:04.028342+0000","last_undegraded":"2024-09-16T00:12:04.028342+0000","last_fullsized":"2024-09-16T00:12:04.028342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:47.840428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.022264+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:12:04.022264+0000","last_peered":"2024-09-16T00:12:04.022264+0000","last_clean":"2024-09-16T00:12:04.022264+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:12:04.022264+0000","last_undegraded":"2024-09-16T00:12:04.022264+0000","last_fullsized":"2024-09-16T00:12:04.022264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.027341+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:12:04.027341+0000","last_peered":"2024-09-16T00:12:04.027341+0000","last_clean":"2024-09-16T00:12:04.027341+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:12:04.027341+0000","last_undegraded":"2024-09-16T00:12:04.027341+0000","last_fullsized":"2024-09-16T00:12:04.027341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.026539+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:12:04.026539+0000","last_peered":"2024-09-16T00:12:04.026539+0000","last_clean":"2024-09-16T00:12:04.026539+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:12:04.026539+0000","last_undegraded":"2024-09-16T00:12:04.026539+0000","last_fullsized":"2024-09-16T00:12:04.026539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025058+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:12:04.025058+0000","last_peered":"2024-09-16T00:12:04.025058+0000","last_clean":"2024-09-16T00:12:04.025058+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:12:04.025058+0000","last_undegraded":"2024-09-16T00:12:04.025058+0000","last_fullsized":"2024-09-16T00:12:04.025058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.025200+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:12:04.025200+0000","last_peered":"2024-09-16T00:12:04.025200+0000","last_clean":"2024-09-16T00:12:04.025200+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:12:04.025200+0000","last_undegraded":"2024-09-16T00:12:04.025200+0000","last_fullsized":"2024-09-16T00:12:04.025200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:12:04.024950+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:12:04.024950+0000","last_peered":"2024-09-16T00:12:04.024950+0000","last_clean":"2024-09-16T00:12:04.024950+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:12:04.024950+0000","last_undegraded":"2024-09-16T00:12:04.024950+0000","last_fullsized":"2024-09-16T00:12:04.024950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70139,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":564,"num_read_kb":358,"num_write":254,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579937,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1007616,"data_stored":496485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1703936,"data_stored":1146232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972395,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1036288,"data_stored":500118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1028096,"data_stored":496400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1626112,"data_stored":1092899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":491526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1634304,"data_stored":1088088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":495993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215178,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":491234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1097728,"data_stored":556100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1077248,"data_stored":555487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1032192,"data_stored":493375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:12:39.925 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:12:39.925 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T00:12:39.925 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T00:12:39.926 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T00:12:39.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:39 smithi193 ceph-mon[35484]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:39 smithi148 ceph-mon[35599]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:40.178 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:40.611 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:40 smithi078 ceph-mon[31157]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.692+0000 7f8e335a5640 1 -- 172.21.15.78:0/64924320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c10c170 msgr2=0x7f8e2c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.692+0000 7f8e335a5640 1 --2- 172.21.15.78:0/64924320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c10c170 0x7f8e2c10e610 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009ca0 tx=0x7f8e2402a950 comp rx=0 tx=0).stop 2024-09-16T00:12:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.692+0000 7f8e335a5640 1 -- 172.21.15.78:0/64924320 shutdown_connections 2024-09-16T00:12:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 --2- 172.21.15.78:0/64924320 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c10c170 0x7f8e2c10e610 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 --2- 172.21.15.78:0/64924320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c103660 0x7f8e2c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 --2- 172.21.15.78:0/64924320 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e2c102d40 0x7f8e2c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 -- 172.21.15.78:0/64924320 >> 172.21.15.78:0/64924320 conn(0x7f8e2c0fc7a0 msgr2=0x7f8e2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 -- 172.21.15.78:0/64924320 shutdown_connections 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 -- 172.21.15.78:0/64924320 wait complete. 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 Processor -- start 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 -- start start 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e2c102d40 0x7f8e2c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c103660 0x7f8e2c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c10c170 0x7f8e2c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c111f70 con 0x7f8e2c103660 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c111df0 con 0x7f8e2c102d40 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.693+0000 7f8e335a5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1120f0 con 0x7f8e2c10c170 2024-09-16T00:12:40.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c103660 0x7f8e2c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c103660 0x7f8e2c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42486/0 (socket says 172.21.15.78:42486) 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 -- 172.21.15.78:0/2573246997 learned_addr learned my addr 172.21.15.78:0/2573246997 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e31b42640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c10c170 0x7f8e2c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e31341640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e2c102d40 0x7f8e2c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 -- 172.21.15.78:0/2573246997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c10c170 msgr2=0x7f8e2c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c10c170 0x7f8e2c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 -- 172.21.15.78:0/2573246997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e2c102d40 msgr2=0x7f8e2c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e2c102d40 0x7f8e2c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 -- 172.21.15.78:0/2573246997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c19d3a0 con 0x7f8e2c103660 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e31b42640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c10c170 0x7f8e2c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:40.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e31341640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e2c102d40 0x7f8e2c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e30b40640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c103660 0x7f8e2c198930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00d750 tx=0x7f8e1c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e227fc640 1 -- 172.21.15.78:0/2573246997 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c017070 con 0x7f8e2c103660 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e227fc640 1 -- 172.21.15.78:0/2573246997 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e1c005020 con 0x7f8e2c103660 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e227fc640 1 -- 172.21.15.78:0/2573246997 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c004d00 con 0x7f8e2c103660 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c19d690 con 0x7f8e2c103660 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.694+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2c1a4f70 con 0x7f8e2c103660 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.695+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e2c103120 con 0x7f8e2c103660 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.697+0000 7f8e227fc640 1 -- 172.21.15.78:0/2573246997 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8e1c009a30 con 0x7f8e2c103660 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.697+0000 7f8e227fc640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8e180a3350 0x7f8e180a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.697+0000 7f8e31341640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8e180a3350 0x7f8e180a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.697+0000 7f8e227fc640 1 -- 172.21.15.78:0/2573246997 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f8e1c0cfbd0 con 0x7f8e2c103660 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.698+0000 7f8e31341640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8e180a3350 0x7f8e180a5810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009b40 tx=0x7f8e28005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:40.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.701+0000 7f8e227fc640 1 -- 172.21.15.78:0/2573246997 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e1c09c4d0 con 0x7f8e2c103660 2024-09-16T00:12:40.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.903+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8e2c107240 con 0x7f8e2c103660 2024-09-16T00:12:40.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.903+0000 7f8e227fc640 1 -- 172.21.15.78:0/2573246997 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8e1c09bdd0 con 0x7f8e2c103660 2024-09-16T00:12:40.904 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:40.905 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T00:12:40.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8e180a3350 msgr2=0x7f8e180a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:40.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8e180a3350 0x7f8e180a5810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009b40 tx=0x7f8e28005e50 comp rx=0 tx=0).stop 2024-09-16T00:12:40.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c103660 msgr2=0x7f8e2c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:40.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c103660 0x7f8e2c198930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00d750 tx=0x7f8e1c00dc10 comp rx=0 tx=0).stop 2024-09-16T00:12:40.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 shutdown_connections 2024-09-16T00:12:40.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8e180a3350 0x7f8e180a5810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e2c10c170 0x7f8e2c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e2c103660 0x7f8e2c198930 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 --2- 172.21.15.78:0/2573246997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e2c102d40 0x7f8e2c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:40.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.905+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 >> 172.21.15.78:0/2573246997 conn(0x7f8e2c0fc7a0 msgr2=0x7f8e2c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:40.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.906+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 shutdown_connections 2024-09-16T00:12:40.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:40.906+0000 7f8e335a5640 1 -- 172.21.15.78:0/2573246997 wait complete. 2024-09-16T00:12:40.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:40 smithi193 ceph-mon[35484]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:40.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:40 smithi148 ceph-mon[35599]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:12:41.645 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T00:12:41.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:12:41.831 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:41 smithi193 ceph-mon[35484]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:12:41.831 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:41 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2573246997' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:12:41.855 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:41 smithi148 ceph-mon[35599]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:12:41.855 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:41 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2573246997' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:12:41.892 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:42.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:41 smithi078 ceph-mon[31157]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:12:42.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:41 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2573246997' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:12:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 -- 172.21.15.78:0/1988417609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 msgr2=0x7fe76010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1988417609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 0x7fe76010ea90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe758009ca0 tx=0x7fe75802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 -- 172.21.15.78:0/1988417609 shutdown_connections 2024-09-16T00:12:42.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1988417609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 0x7fe76010ea90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1988417609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe760102790 0x7fe7601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1988417609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760101de0 0x7fe7601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 -- 172.21.15.78:0/1988417609 >> 172.21.15.78:0/1988417609 conn(0x7fe7600fd8c0 msgr2=0x7fe7600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 -- 172.21.15.78:0/1988417609 shutdown_connections 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.391+0000 7fe765d0f640 1 -- 172.21.15.78:0/1988417609 wait complete. 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe765d0f640 1 Processor -- start 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe765d0f640 1 -- start start 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe765d0f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe760101de0 0x7fe7600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe765d0f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760102790 0x7fe760074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe765d0f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 0x7fe760074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe765d0f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe760111fb0 con 0x7fe760106f10 2024-09-16T00:12:42.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe765d0f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe760111e30 con 0x7fe760102790 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe765d0f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe760112130 con 0x7fe760101de0 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe75ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 0x7fe760074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe75ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 0x7fe760074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42506/0 (socket says 172.21.15.78:42506) 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.392+0000 7fe75ffff640 1 -- 172.21.15.78:0/1278852200 learned_addr learned my addr 172.21.15.78:0/1278852200 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75effd640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760102790 0x7fe760074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75ffff640 1 -- 172.21.15.78:0/1278852200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe760101de0 msgr2=0x7fe7600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75ffff640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe760101de0 0x7fe7600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75ffff640 1 -- 172.21.15.78:0/1278852200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760102790 msgr2=0x7fe760074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75ffff640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760102790 0x7fe760074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75ffff640 1 -- 172.21.15.78:0/1278852200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7601a6950 con 0x7fe760106f10 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75effd640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760102790 0x7fe760074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:42.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75ffff640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 0x7fe760074f70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe758031a60 tx=0x7fe758031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75cff9640 1 -- 172.21.15.78:0/1278852200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758003c30 con 0x7fe760106f10 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7601a6be0 con 0x7fe760106f10 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75cff9640 1 -- 172.21.15.78:0/1278852200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe758003dd0 con 0x7fe760106f10 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe75cff9640 1 -- 172.21.15.78:0/1278852200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580047e0 con 0x7fe760106f10 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.393+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7601a7120 con 0x7fe760106f10 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.394+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7601021c0 con 0x7fe760106f10 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.395+0000 7fe75cff9640 1 -- 172.21.15.78:0/1278852200 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe758004b40 con 0x7fe760106f10 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.396+0000 7fe75cff9640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7480a32d0 0x7fe7480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.396+0000 7fe75f7fe640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7480a32d0 0x7fe7480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.396+0000 7fe75cff9640 1 -- 172.21.15.78:0/1278852200 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fe7580f12f0 con 0x7fe760106f10 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.396+0000 7fe75f7fe640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7480a32d0 0x7fe7480a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe750009710 tx=0x7fe750006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:42.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.399+0000 7fe75cff9640 1 -- 172.21.15.78:0/1278852200 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7580bdc80 con 0x7fe760106f10 2024-09-16T00:12:42.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.602+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe760075a20 con 0x7fe760106f10 2024-09-16T00:12:42.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.603+0000 7fe75cff9640 1 -- 172.21.15.78:0/1278852200 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11986 (secure 0 0 0) 0x7fe7580bd580 con 0x7fe760106f10 2024-09-16T00:12:42.604 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:42.604 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-16T00:11:44:544077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:11:44.262714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:12:42.605 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-16T00:12:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7480a32d0 msgr2=0x7fe7480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7480a32d0 0x7fe7480a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe750009710 tx=0x7fe750006c90 comp rx=0 tx=0).stop 2024-09-16T00:12:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 msgr2=0x7fe760074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 0x7fe760074f70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe758031a60 tx=0x7fe758031a90 comp rx=0 tx=0).stop 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 shutdown_connections 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7480a32d0 0x7fe7480a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe760106f10 0x7fe760074f70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe760102790 0x7fe760074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 --2- 172.21.15.78:0/1278852200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe760101de0 0x7fe7600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 >> 172.21.15.78:0/1278852200 conn(0x7fe7600fd8c0 msgr2=0x7fe7600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 shutdown_connections 2024-09-16T00:12:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:42.605+0000 7fe765d0f640 1 -- 172.21.15.78:0/1278852200 wait complete. 2024-09-16T00:12:43.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:12:43.678 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:43.830 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:43 smithi078 ceph-mon[31157]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:12:43.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:43 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1278852200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:43 smithi193 ceph-mon[35484]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:12:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:43 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1278852200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:43.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:43 smithi148 ceph-mon[35599]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:12:43.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:43 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1278852200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 -- 172.21.15.78:0/3682257599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 msgr2=0x7fa5381029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3682257599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 0x7fa5381029b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa534009900 tx=0x7fa53402a480 comp rx=0 tx=0).stop 2024-09-16T00:12:44.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 -- 172.21.15.78:0/3682257599 shutdown_connections 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3682257599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa538102ef0 0x7fa53810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3682257599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 0x7fa5381029b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3682257599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa538100e20 0x7fa538101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 -- 172.21.15.78:0/3682257599 >> 172.21.15.78:0/3682257599 conn(0x7fa5380fc7d0 msgr2=0x7fa5380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 -- 172.21.15.78:0/3682257599 shutdown_connections 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 -- 172.21.15.78:0/3682257599 wait complete. 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.220+0000 7fa53fad5640 1 Processor -- start 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53fad5640 1 -- start start 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53fad5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa538100e20 0x7fa5381993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53fad5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 0x7fa5381998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53fad5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa538102ef0 0x7fa5381a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:44.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53fad5640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa538110f80 con 0x7fa538102570 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53fad5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa538110e00 con 0x7fa538102ef0 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53fad5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa538111100 con 0x7fa538100e20 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 0x7fa5381998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 0x7fa5381998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42532/0 (socket says 172.21.15.78:42532) 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 -- 172.21.15.78:0/3119567654 learned_addr learned my addr 172.21.15.78:0/3119567654 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d871640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa538100e20 0x7fa5381993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53e072640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa538102ef0 0x7fa5381a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 -- 172.21.15.78:0/3119567654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa538100e20 msgr2=0x7fa5381993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa538100e20 0x7fa5381993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 -- 172.21.15.78:0/3119567654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa538102ef0 msgr2=0x7fa5381a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa538102ef0 0x7fa5381a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 -- 172.21.15.78:0/3119567654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5381a4970 con 0x7fa538102570 2024-09-16T00:12:44.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa53d070640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 0x7fa5381998f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa534006010 tx=0x7fa534003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.221+0000 7fa52affd640 1 -- 172.21.15.78:0/3119567654 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53403f070 con 0x7fa538102570 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.222+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5381a4b40 con 0x7fa538102570 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.222+0000 7fa52affd640 1 -- 172.21.15.78:0/3119567654 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa534031c30 con 0x7fa538102570 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.222+0000 7fa52affd640 1 -- 172.21.15.78:0/3119567654 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534035030 con 0x7fa538102570 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.222+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381a4fa0 con 0x7fa538102570 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.222+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa538101200 con 0x7fa538102570 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.224+0000 7fa52affd640 1 -- 172.21.15.78:0/3119567654 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa534042070 con 0x7fa538102570 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.224+0000 7fa52affd640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa5240a32d0 0x7fa5240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.224+0000 7fa53d871640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa5240a32d0 0x7fa5240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.224+0000 7fa52affd640 1 -- 172.21.15.78:0/3119567654 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fa5340f0b10 con 0x7fa538102570 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.225+0000 7fa53d871640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa5240a32d0 0x7fa5240a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa52c005fd0 tx=0x7fa52c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:44.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.227+0000 7fa52affd640 1 -- 172.21.15.78:0/3119567654 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa534031de0 con 0x7fa538102570 2024-09-16T00:12:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.428+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa538109090 con 0x7fa538102570 2024-09-16T00:12:44.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.429+0000 7fa52affd640 1 -- 172.21.15.78:0/3119567654 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11986 (secure 0 0 0) 0x7fa5340d4ef0 con 0x7fa538102570 2024-09-16T00:12:44.430 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:44.430 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-16T00:11:44:544077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:11:44.262714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:12:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-16T00:12:44.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.431+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa5240a32d0 msgr2=0x7fa5240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:44.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.431+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa5240a32d0 0x7fa5240a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa52c005fd0 tx=0x7fa52c005850 comp rx=0 tx=0).stop 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.431+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 msgr2=0x7fa5381998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.431+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 0x7fa5381998f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa534006010 tx=0x7fa534003c10 comp rx=0 tx=0).stop 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.432+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 shutdown_connections 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.432+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa5240a32d0 0x7fa5240a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.432+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa538102ef0 0x7fa5381a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.432+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa538102570 0x7fa5381998f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.432+0000 7fa53fad5640 1 --2- 172.21.15.78:0/3119567654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa538100e20 0x7fa5381993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.432+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 >> 172.21.15.78:0/3119567654 conn(0x7fa5380fc7d0 msgr2=0x7fa5380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.432+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 shutdown_connections 2024-09-16T00:12:44.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:44.432+0000 7fa53fad5640 1 -- 172.21.15.78:0/3119567654 wait complete. 2024-09-16T00:12:44.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:44 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3119567654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:44 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3119567654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:44 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3119567654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:45.114 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:12:45.361 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:45.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:45 smithi078 ceph-mon[31157]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 -- 172.21.15.78:0/3097187220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80106a80 msgr2=0x7fdb80106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 --2- 172.21.15.78:0/3097187220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80106a80 0x7fdb80106e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009930 tx=0x7fdb7002a420 comp rx=0 tx=0).stop 2024-09-16T00:12:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 -- 172.21.15.78:0/3097187220 shutdown_connections 2024-09-16T00:12:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 --2- 172.21.15.78:0/3097187220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb80101420 0x7fdb8010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 --2- 172.21.15.78:0/3097187220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb80100a80 0x7fdb80100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 --2- 172.21.15.78:0/3097187220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80106a80 0x7fdb80106e60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 -- 172.21.15.78:0/3097187220 >> 172.21.15.78:0/3097187220 conn(0x7fdb800fc7a0 msgr2=0x7fdb800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 -- 172.21.15.78:0/3097187220 shutdown_connections 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 -- 172.21.15.78:0/3097187220 wait complete. 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.902+0000 7fdb85321640 1 Processor -- start 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb85321640 1 -- start start 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb85321640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb80100a80 0x7fdb801984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb85321640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80101420 0x7fdb801989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb85321640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb80106a80 0x7fdb8019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb85321640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80110ec0 con 0x7fdb80101420 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb85321640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80110d40 con 0x7fdb80106a80 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb85321640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80111040 con 0x7fdb80100a80 2024-09-16T00:12:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80101420 0x7fdb801989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80101420 0x7fdb801989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42560/0 (socket says 172.21.15.78:42560) 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7e7fc640 1 -- 172.21.15.78:0/3624876697 learned_addr learned my addr 172.21.15.78:0/3624876697 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7f7fe640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb80106a80 0x7fdb8019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7e7fc640 1 -- 172.21.15.78:0/3624876697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb80100a80 msgr2=0x7fdb801984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7effd640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb80100a80 0x7fdb801984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7e7fc640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb80100a80 0x7fdb801984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7e7fc640 1 -- 172.21.15.78:0/3624876697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb80106a80 msgr2=0x7fdb8019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7e7fc640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb80106a80 0x7fdb8019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.903+0000 7fdb7e7fc640 1 -- 172.21.15.78:0/3624876697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8019d450 con 0x7fdb80101420 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.904+0000 7fdb7e7fc640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80101420 0x7fdb801989e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800d770 tx=0x7fdb6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.904+0000 7fdb77fff640 1 -- 172.21.15.78:0/3624876697 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68017070 con 0x7fdb80101420 2024-09-16T00:12:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.904+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8019d740 con 0x7fdb80101420 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.904+0000 7fdb77fff640 1 -- 172.21.15.78:0/3624876697 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb68005500 con 0x7fdb80101420 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.904+0000 7fdb77fff640 1 -- 172.21.15.78:0/3624876697 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68004e00 con 0x7fdb80101420 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.904+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801a5020 con 0x7fdb80101420 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.905+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb80100ee0 con 0x7fdb80101420 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.906+0000 7fdb77fff640 1 -- 172.21.15.78:0/3624876697 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdb68004fa0 con 0x7fdb80101420 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.906+0000 7fdb77fff640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb6c0a3350 0x7fdb6c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.907+0000 7fdb7effd640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb6c0a3350 0x7fdb6c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.907+0000 7fdb77fff640 1 -- 172.21.15.78:0/3624876697 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fdb680cfcc0 con 0x7fdb80101420 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.907+0000 7fdb7effd640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb6c0a3350 0x7fdb6c0a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb700096c0 tx=0x7fdb70000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:45.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:45.910+0000 7fdb77fff640 1 -- 172.21.15.78:0/3624876697 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb6809c5c0 con 0x7fdb80101420 2024-09-16T00:12:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:45 smithi193 ceph-mon[35484]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:45 smithi148 ceph-mon[35599]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.077+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb8010a030 con 0x7fdb80101420 2024-09-16T00:12:46.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.079+0000 7fdb77fff640 1 -- 172.21.15.78:0/3624876697 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24710 (secure 0 0 0) 0x7fdb6809bec0 con 0x7fdb80101420 2024-09-16T00:12:46.080 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:46.080 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":88,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:12:04.018109+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb6c0a3350 msgr2=0x7fdb6c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb6c0a3350 0x7fdb6c0a5810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb700096c0 tx=0x7fdb70000bc0 comp rx=0 tx=0).stop 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80101420 msgr2=0x7fdb801989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80101420 0x7fdb801989e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800d770 tx=0x7fdb6800dc30 comp rx=0 tx=0).stop 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 shutdown_connections 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb6c0a3350 0x7fdb6c0a5810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb80106a80 0x7fdb8019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb80101420 0x7fdb801989e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:46.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 --2- 172.21.15.78:0/3624876697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb80100a80 0x7fdb801984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:46.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 >> 172.21.15.78:0/3624876697 conn(0x7fdb800fc7a0 msgr2=0x7fdb8006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:46.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 shutdown_connections 2024-09-16T00:12:46.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:46.081+0000 7fdb85321640 1 -- 172.21.15.78:0/3624876697 wait complete. 2024-09-16T00:12:46.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:46 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3624876697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:46.893 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:12:46.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:46 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3624876697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:46.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:46 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3624876697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:12:47.139 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:47.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 -- 172.21.15.78:0/73772142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614106b80 msgr2=0x7ff614106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:47.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 --2- 172.21.15.78:0/73772142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614106b80 0x7ff614106f60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff604009900 tx=0x7ff60402a410 comp rx=0 tx=0).stop 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 -- 172.21.15.78:0/73772142 shutdown_connections 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 --2- 172.21.15.78:0/73772142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff614100a50 0x7ff61410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 --2- 172.21.15.78:0/73772142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6141000b0 0x7ff614100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 --2- 172.21.15.78:0/73772142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614106b80 0x7ff614106f60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 -- 172.21.15.78:0/73772142 >> 172.21.15.78:0/73772142 conn(0x7ff6140fbee0 msgr2=0x7ff6140fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 -- 172.21.15.78:0/73772142 shutdown_connections 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.663+0000 7ff6192b3640 1 -- 172.21.15.78:0/73772142 wait complete. 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6192b3640 1 Processor -- start 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6192b3640 1 -- start start 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6192b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6141000b0 0x7ff614198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6192b3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614100a50 0x7ff614198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:47.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6192b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff614106b80 0x7ff61419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6192b3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff614112070 con 0x7ff614100a50 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6192b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff614111ef0 con 0x7ff6141000b0 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6192b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff6141121f0 con 0x7ff614106b80 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6127fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614100a50 0x7ff614198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6127fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614100a50 0x7ff614198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57694/0 (socket says 172.21.15.78:57694) 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6127fc640 1 -- 172.21.15.78:0/793113866 learned_addr learned my addr 172.21.15.78:0/793113866 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff612ffd640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6141000b0 0x7ff614198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6127fc640 1 -- 172.21.15.78:0/793113866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff614106b80 msgr2=0x7ff61419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6127fc640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff614106b80 0x7ff61419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6127fc640 1 -- 172.21.15.78:0/793113866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6141000b0 msgr2=0x7ff614198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6127fc640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6141000b0 0x7ff614198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.664+0000 7ff6127fc640 1 -- 172.21.15.78:0/793113866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61419d400 con 0x7ff614100a50 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.665+0000 7ff6127fc640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614100a50 0x7ff614198990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c890 tx=0x7ff60800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:47.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.665+0000 7ff5fbfff640 1 -- 172.21.15.78:0/793113866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608004220 con 0x7ff614100a50 2024-09-16T00:12:47.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.665+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61419d6f0 con 0x7ff614100a50 2024-09-16T00:12:47.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.665+0000 7ff5fbfff640 1 -- 172.21.15.78:0/793113866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6080043c0 con 0x7ff614100a50 2024-09-16T00:12:47.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.665+0000 7ff5fbfff640 1 -- 172.21.15.78:0/793113866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608004a00 con 0x7ff614100a50 2024-09-16T00:12:47.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.665+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6141a4fd0 con 0x7ff614100a50 2024-09-16T00:12:47.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.666+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff614100510 con 0x7ff614100a50 2024-09-16T00:12:47.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.667+0000 7ff5fbfff640 1 -- 172.21.15.78:0/793113866 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff6080079b0 con 0x7ff614100a50 2024-09-16T00:12:47.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.668+0000 7ff5fbfff640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5fc0a32d0 0x7ff5fc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:47.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.668+0000 7ff612ffd640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5fc0a32d0 0x7ff5fc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:47.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.668+0000 7ff5fbfff640 1 -- 172.21.15.78:0/793113866 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7ff6080cfae0 con 0x7ff614100a50 2024-09-16T00:12:47.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.669+0000 7ff612ffd640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5fc0a32d0 0x7ff5fc0a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff604000c00 tx=0x7ff6040058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:47.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.672+0000 7ff5fbfff640 1 -- 172.21.15.78:0/793113866 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff608014030 con 0x7ff614100a50 2024-09-16T00:12:47.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.874+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff61410a070 con 0x7ff614100a50 2024-09-16T00:12:47.875 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:47 smithi078 ceph-mon[31157]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:47.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.875+0000 7ff5fbfff640 1 -- 172.21.15.78:0/793113866 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11986 (secure 0 0 0) 0x7ff60809c360 con 0x7ff614100a50 2024-09-16T00:12:47.876 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:12:47.876 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-16T00:11:44:544077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:11:44.262714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:12:47.876 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-16T00:12:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5fc0a32d0 msgr2=0x7ff5fc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5fc0a32d0 0x7ff5fc0a5790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff604000c00 tx=0x7ff6040058d0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614100a50 msgr2=0x7ff614198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614100a50 0x7ff614198990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c890 tx=0x7ff60800cd50 comp rx=0 tx=0).stop 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 shutdown_connections 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5fc0a32d0 0x7ff5fc0a5790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff614106b80 0x7ff61419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff614100a50 0x7ff614198990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 --2- 172.21.15.78:0/793113866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff6141000b0 0x7ff614198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.877+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 >> 172.21.15.78:0/793113866 conn(0x7ff6140fbee0 msgr2=0x7ff6140fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.878+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 shutdown_connections 2024-09-16T00:12:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:47.878+0000 7ff6192b3640 1 -- 172.21.15.78:0/793113866 wait complete. 2024-09-16T00:12:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:47 smithi193 ceph-mon[35484]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:47 smithi148 ceph-mon[35599]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:48.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:11:05.234467+0000', 'modified': '2024-09-16T00:11:44.262714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24445, 'mds_2': 24440, 'mds_3': 24427, 'mds_4': 24421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/1764484583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 1764484583}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 1764484583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14583': {'gid': 14583, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6841/1822173762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 1822173762}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 1822173762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/1629662299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 1629662299}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 1629662299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6833/3203333506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3203333506}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 3203333506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/903408403', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 903408403}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 903408403}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.193:6835/2292206006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 2292206006}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 2292206006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.193:6837/1139945107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1139945107}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1139945107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.148:6837/3780846668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3780846668}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3780846668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.193:6839/3313451522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3313451522}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3313451522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.148:6839/831570873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 831570873}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 831570873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24421, 24427, 24440, 24445, 24470]} 2024-09-16T00:12:48.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-16T00:12:48.677 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-16T00:12:48.920 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:48.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:48 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/793113866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:48.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:48 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/793113866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:49.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:48 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/793113866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 -- 172.21.15.78:0/1639747270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 msgr2=0x7f1f641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1639747270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 0x7f1f641067e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c009900 tx=0x7f1f4c02a410 comp rx=0 tx=0).stop 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 -- 172.21.15.78:0/1639747270 shutdown_connections 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1639747270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f64106f10 0x7f1f6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1639747270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 0x7f1f641067e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1639747270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64101de0 0x7f1f641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 -- 172.21.15.78:0/1639747270 >> 172.21.15.78:0/1639747270 conn(0x7f1f640fd8c0 msgr2=0x7f1f640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 -- 172.21.15.78:0/1639747270 shutdown_connections 2024-09-16T00:12:49.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 -- 172.21.15.78:0/1639747270 wait complete. 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 Processor -- start 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.416+0000 7f1f694c9640 1 -- start start 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f694c9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f64101de0 0x7f1f64198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f694c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 0x7f1f641989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f694c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64106f10 0x7f1f6419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f694c9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64111f90 con 0x7f1f64102790 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f694c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64111e10 con 0x7f1f64106f10 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f694c9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64112110 con 0x7f1f64101de0 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 0x7f1f641989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 0x7f1f641989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57704/0 (socket says 172.21.15.78:57704) 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 -- 172.21.15.78:0/1574415214 learned_addr learned my addr 172.21.15.78:0/1574415214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f64101de0 0x7f1f64198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f637fe640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64106f10 0x7f1f6419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 -- 172.21.15.78:0/1574415214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f64101de0 msgr2=0x7f1f64198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f64101de0 0x7f1f64198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 -- 172.21.15.78:0/1574415214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64106f10 msgr2=0x7f1f6419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64106f10 0x7f1f6419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 -- 172.21.15.78:0/1574415214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6419d440 con 0x7f1f64102790 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f64101de0 0x7f1f64198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 0x7f1f641989d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005ab0 tx=0x7f1f4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f60ff9640 1 -- 172.21.15.78:0/1574415214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c03f030 con 0x7f1f64102790 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f60ff9640 1 -- 172.21.15.78:0/1574415214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4c003ce0 con 0x7f1f64102790 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f694c9640 1 -- 172.21.15.78:0/1574415214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6419d6d0 con 0x7f1f64102790 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f60ff9640 1 -- 172.21.15.78:0/1574415214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c035070 con 0x7f1f64102790 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.417+0000 7f1f694c9640 1 -- 172.21.15.78:0/1574415214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f641a5010 con 0x7f1f64102790 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f60ff9640 1 -- 172.21.15.78:0/1574415214 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1f4c042030 con 0x7f1f64102790 2024-09-16T00:12:49.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f6b8a2740 1 Processor -- start 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f6b8a2740 1 -- start start 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 0x561c4e98cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98d1b0 0x561c4e99ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e99bfc0 0x561c4e99e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c4e8f3090 con 0x561c4e99bfc0 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561c4e89aa70 con 0x561c4e98d1b0 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.420+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561c4e8ef050 con 0x561c4e98c890 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e99bfc0 0x561c4e99e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e99bfc0 0x561c4e99e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57720/0 (socket says 172.21.15.78:57720) 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 -- 172.21.15.78:0/644390967 learned_addr learned my addr 172.21.15.78:0/644390967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98d1b0 0x561c4e99ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f60ff9640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f500a3350 0x7f1f500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f637fe640 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 0x561c4e98cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f637fe640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f500a3350 0x7f1f500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 -- 172.21.15.78:0/644390967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 msgr2=0x561c4e98cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 0x561c4e98cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 -- 172.21.15.78:0/644390967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98d1b0 msgr2=0x561c4e99ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98d1b0 0x561c4e99ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 -- 172.21.15.78:0/644390967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4e8a7130 con 0x561c4e99bfc0 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98d1b0 0x561c4e99ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f60ff9640 1 -- 172.21.15.78:0/1574415214 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f1f4c0f0720 con 0x7f1f64102790 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e99bfc0 0x561c4e99e3b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0b8d00 tx=0x7f1f4c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c199b7339f3685d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f637fe640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f500a3350 0x7f1f500a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0070b0 tx=0x7f1f5c00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f597fa640 1 -- 172.21.15.78:0/644390967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c049070 con 0x561c4e99bfc0 2024-09-16T00:12:49.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f597fa640 1 -- 172.21.15.78:0/644390967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4c0bcfa0 con 0x561c4e99bfc0 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.421+0000 7f1f597fa640 1 -- 172.21.15.78:0/644390967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0c0c50 con 0x561c4e99bfc0 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/644390967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e99bfc0 msgr2=0x561c4e99e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e99bfc0 0x561c4e99e3b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0b8d00 tx=0x7f1f4c04c000 comp rx=0 tx=0).stop 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/644390967 shutdown_connections 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e99bfc0 0x561c4e99e3b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98d1b0 0x561c4e99ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/644390967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 0x561c4e98cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/644390967 >> 172.21.15.78:0/644390967 conn(0x561c4e988f60 msgr2=0x561c4e989340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/644390967 shutdown_connections 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.422+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/644390967 wait complete. 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 1 -- start start 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 1 Processor -- start 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 0x561c4e99b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e998bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 0x561c4e999110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c4e8f3090 con 0x561c4e98d1b0 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561c4e8adf20 con 0x561c4e99bfc0 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561c4e99e950 con 0x561c4e98c890 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f6b8a2740 10 client.0 _ll_register_callbacks cb 0x561c4e877030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e998bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e998bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57732/0 (socket says 172.21.15.78:57732) 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f5bfff640 1 -- 172.21.15.78:0/4236530107 learned_addr learned my addr 172.21.15.78:0/4236530107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f5bfff640 1 -- 172.21.15.78:0/4236530107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 msgr2=0x561c4e99b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 0x561c4e99b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f5bfff640 1 -- 172.21.15.78:0/4236530107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 msgr2=0x561c4e999110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 0x561c4e999110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.423+0000 7f1f5bfff640 1 -- 172.21.15.78:0/4236530107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4e6642b0 con 0x561c4e98d1b0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e998bd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0bc550 tx=0x7f1f4c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b6cd84ca05ef76 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/4236530107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c049040 con 0x561c4e98d1b0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/4236530107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f1f4c03b460 con 0x561c4e98d1b0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/4236530107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f4c0bcfa0 con 0x561c4e98d1b0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/4236530107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f4c0c5510 con 0x561c4e98d1b0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 10 client.15039 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/4236530107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c4e9aa140 con 0x561c4e98d1b0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/4236530107 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4c0bcfa0 con 0x561c4e98d1b0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/4236530107 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1f4c0bfca0 con 0x561c4e98d1b0 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 10 client.15039 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 10 client.15039 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 20 client.15039 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:49.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561c4e9ac5f0 0x561c4e9ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 4 client.15039 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.424+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/4236530107 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c4e900600 con 0x561c4e9ac5f0 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.425+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561c4e9ac5f0 0x561c4e9ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.425+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561c4e9ac5f0 0x561c4e9ada60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21b6ccaa49955d9 server_cookie=6044555fcb723718 in_seq=0 out_seq=0 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.425+0000 7f1f4a7fc640 10 client.15039 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/4236530107 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c4e900600 con 0x561c4e9ac5f0 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f4a7fc640 10 client.15039 handle_command_reply: tid=0 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 client.15039 shutdown 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/4236530107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 msgr2=0x561c4e998bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e998bd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0bc550 tx=0x7f1f4c02a410 comp rx=0 tx=0).stop 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/4236530107 shutdown_connections 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 0x561c4e999110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e998bd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561c4e9ac5f0 0x561c4e9ada60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/4236530107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98c890 0x561c4e99b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f4a7fc640 0 client.15039 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:12:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/4236530107 >> 172.21.15.78:0/4236530107 conn(0x561c4e988f60 msgr2=0x561c4e98ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/4236530107 shutdown_connections 2024-09-16T00:12:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/4236530107 wait complete. 2024-09-16T00:12:49.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.426+0000 7f1f6b8a2740 20 client.15039 trim_cache size 0 max 16384 2024-09-16T00:12:49.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f6b8a2740 1 Processor -- start 2024-09-16T00:12:49.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f6b8a2740 1 -- start start 2024-09-16T00:12:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98c890 0x561c4e995e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e996370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e99bfc0 0x561c4e993a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c4e9abda0 con 0x561c4e98d1b0 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561c4e9ac470 con 0x561c4e98c890 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561c4e9acb40 con 0x561c4e99bfc0 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e996370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e996370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57748/0 (socket says 172.21.15.78:57748) 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.444+0000 7f1f5bfff640 1 -- 172.21.15.78:0/332030531 learned_addr learned my addr 172.21.15.78:0/332030531 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f637fe640 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e99bfc0 0x561c4e993a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98c890 0x561c4e995e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f5bfff640 1 -- 172.21.15.78:0/332030531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e99bfc0 msgr2=0x561c4e993a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e99bfc0 0x561c4e993a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f5bfff640 1 -- 172.21.15.78:0/332030531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98c890 msgr2=0x561c4e995e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98c890 0x561c4e995e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f5bfff640 1 -- 172.21.15.78:0/332030531 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4e997120 con 0x561c4e98d1b0 2024-09-16T00:12:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f637fe640 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e99bfc0 0x561c4e993a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e996370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0bb710 tx=0x7f1f4c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2467dab1ab1987c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f58ff9640 1 -- 172.21.15.78:0/332030531 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c049070 con 0x561c4e98d1b0 2024-09-16T00:12:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f58ff9640 1 -- 172.21.15.78:0/332030531 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4c0bcfa0 con 0x561c4e98d1b0 2024-09-16T00:12:49.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.445+0000 7f1f58ff9640 1 -- 172.21.15.78:0/332030531 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c03b460 con 0x561c4e98d1b0 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/332030531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 msgr2=0x561c4e996370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e996370 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0bb710 tx=0x7f1f4c0045e0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/332030531 shutdown_connections 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e99bfc0 0x561c4e993a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98d1b0 0x561c4e996370 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/332030531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e98c890 0x561c4e995e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/332030531 >> 172.21.15.78:0/332030531 conn(0x561c4e988f60 msgr2=0x561c4e9974a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/332030531 shutdown_connections 2024-09-16T00:12:49.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/332030531 wait complete. 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 -- start start 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.446+0000 7f1f6b8a2740 1 Processor -- start 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98c890 0x561c4e9aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98d1b0 0x561c4e9af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f6b8a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 0x561c4ea14cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561c4e9ad8e0 con 0x561c4e98c890 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561c4e9ad210 con 0x561c4e99bfc0 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f6b8a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561c4e9adfb0 con 0x561c4e98d1b0 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f6b8a2740 10 client.0 _ll_register_callbacks cb 0x7f1f5400c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98c890 0x561c4e9aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98c890 0x561c4e9aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57752/0 (socket says 172.21.15.78:57752) 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 -- 172.21.15.78:0/3862871567 learned_addr learned my addr 172.21.15.78:0/3862871567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:49.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98d1b0 0x561c4e9af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f637fe640 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 0x561c4ea14cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 -- 172.21.15.78:0/3862871567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98d1b0 msgr2=0x561c4e9af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98d1b0 0x561c4e9af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 -- 172.21.15.78:0/3862871567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 msgr2=0x561c4ea14cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 0x561c4ea14cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 -- 172.21.15.78:0/3862871567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4e9afbd0 con 0x561c4e98c890 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f637fe640 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 0x561c4ea14cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f62ffd640 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98c890 0x561c4e9aed90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x561c4e98d690 tx=0x7f1f54005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c149f94139f1a68b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/3862871567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54010630 con 0x561c4e98c890 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/3862871567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f1f540107d0 con 0x561c4e98c890 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.447+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/3862871567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f4c0bcfa0 con 0x561c4e98c890 2024-09-16T00:12:49.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/3862871567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f5402a050 con 0x561c4e98c890 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f6b8a2740 10 client.15045 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/3862871567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561c4ea163c0 con 0x561c4e98c890 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/3862871567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4c0bcfa0 con 0x561c4e98c890 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/3862871567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1f540219e0 con 0x561c4e98c890 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f6b8a2740 10 client.15045 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f6b8a2740 10 client.15045 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f6b8a2740 20 client.15045 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561c4ea1ae80 0x561c4ea19b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f6b8a2740 4 client.15045 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/3862871567 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "status"}) -- 0x561c4e8fab10 con 0x561c4ea1ae80 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.448+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561c4ea1ae80 0x561c4ea19b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f5bfff640 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561c4ea1ae80 0x561c4ea19b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b3c2e388b0d999a server_cookie=d5159bc6a6f905f0 in_seq=0 out_seq=0 2024-09-16T00:12:49.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f4a7fc640 10 client.15045 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f4a7fc640 1 -- 172.21.15.78:0/3862871567 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x561c4e8fab10 con 0x561c4ea1ae80 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f4a7fc640 10 client.15045 handle_command_reply: tid=0 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 client.15045 shutdown 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/3862871567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98c890 msgr2=0x561c4e9aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98c890 0x561c4e9aed90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x561c4e98d690 tx=0x7f1f54005410 comp rx=0 tx=0).stop 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/3862871567 shutdown_connections 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561c4e99bfc0 0x561c4ea14cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561c4ea1ae80 0x561c4ea19b90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c4e98d1b0 0x561c4e9af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 --2- 172.21.15.78:0/3862871567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561c4e98c890 0x561c4e9aed90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f4a7fc640 0 client.15045 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/3862871567 >> 172.21.15.78:0/3862871567 conn(0x561c4e988f60 msgr2=0x561c4e994240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/3862871567 shutdown_connections 2024-09-16T00:12:49.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 1 -- 172.21.15.78:0/3862871567 wait complete. 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.449+0000 7f1f6b8a2740 20 client.15045 trim_cache size 0 max 16384 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24421, 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 66.914504334, 2024-09-16T00:12:49.457 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:12:49.458 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 88, 2024-09-16T00:12:49.458 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:12:49.458 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 133.59864903600001 2024-09-16T00:12:49.458 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:12:49.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 -- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f500a3350 msgr2=0x7f1f500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f500a3350 0x7f1f500a5810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0070b0 tx=0x7f1f5c00c040 comp rx=0 tx=0).stop 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 -- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 msgr2=0x7f1f641989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 0x7f1f641989d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c005ab0 tx=0x7f1f4c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 -- 172.21.15.78:0/1574415214 shutdown_connections 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f500a3350 0x7f1f500a5810 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f64106f10 0x7f1f6419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f64102790 0x7f1f641989d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 --2- 172.21.15.78:0/1574415214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f64101de0 0x7f1f64198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 -- 172.21.15.78:0/1574415214 >> 172.21.15.78:0/1574415214 conn(0x7f1f640fd8c0 msgr2=0x7f1f640fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 -- 172.21.15.78:0/1574415214 shutdown_connections 2024-09-16T00:12:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:49.450+0000 7f1f694c9640 1 -- 172.21.15.78:0/1574415214 wait complete. 2024-09-16T00:12:49.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:49 smithi078 ceph-mon[31157]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:49 smithi193 ceph-mon[35484]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:49 smithi148 ceph-mon[35599]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:50.347 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-16T00:12:50.594 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 -- 172.21.15.78:0/2861125275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 msgr2=0x7fc75810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 --2- 172.21.15.78:0/2861125275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 0x7fc75810e790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc750009900 tx=0x7fc75002a410 comp rx=0 tx=0).stop 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 -- 172.21.15.78:0/2861125275 shutdown_connections 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 --2- 172.21.15.78:0/2861125275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 0x7fc75810e790 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 --2- 172.21.15.78:0/2861125275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7581037a0 0x7fc75810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 --2- 172.21.15.78:0/2861125275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc758102df0 0x7fc7581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 -- 172.21.15.78:0/2861125275 >> 172.21.15.78:0/2861125275 conn(0x7fc7580fc7d0 msgr2=0x7fc7580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 -- 172.21.15.78:0/2861125275 shutdown_connections 2024-09-16T00:12:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.117+0000 7fc75d77a640 1 -- 172.21.15.78:0/2861125275 wait complete. 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.118+0000 7fc75d77a640 1 Processor -- start 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.118+0000 7fc75d77a640 1 -- start start 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.118+0000 7fc75d77a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc758102df0 0x7fc758100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.118+0000 7fc75d77a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7581037a0 0x7fc758100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.118+0000 7fc75d77a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 0x7fc7581014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.118+0000 7fc75d77a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581120f0 con 0x7fc75810c2f0 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.118+0000 7fc75d77a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc758111f70 con 0x7fc758102df0 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.118+0000 7fc75d77a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc758112270 con 0x7fc7581037a0 2024-09-16T00:12:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 0x7fc7581014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 0x7fc7581014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57774/0 (socket says 172.21.15.78:57774) 2024-09-16T00:12:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 -- 172.21.15.78:0/4225154314 learned_addr learned my addr 172.21.15.78:0/4225154314 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc756ffd640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc758102df0 0x7fc758100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7567fc640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7581037a0 0x7fc758100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 -- 172.21.15.78:0/4225154314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7581037a0 msgr2=0x7fc758100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7581037a0 0x7fc758100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 -- 172.21.15.78:0/4225154314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc758102df0 msgr2=0x7fc758100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc758102df0 0x7fc758100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 -- 172.21.15.78:0/4225154314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7581a6af0 con 0x7fc75810c2f0 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7567fc640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7581037a0 0x7fc758100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc7577fe640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 0x7fc7581014d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc750005ab0 tx=0x7fc7500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc74ffff640 1 -- 172.21.15.78:0/4225154314 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75003f030 con 0x7fc75810c2f0 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc75d77a640 1 -- 172.21.15.78:0/4225154314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7581a6d80 con 0x7fc75810c2f0 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc74ffff640 1 -- 172.21.15.78:0/4225154314 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc750003ce0 con 0x7fc75810c2f0 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc74ffff640 1 -- 172.21.15.78:0/4225154314 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750035070 con 0x7fc75810c2f0 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.119+0000 7fc75d77a640 1 -- 172.21.15.78:0/4225154314 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7581a7260 con 0x7fc75810c2f0 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.121+0000 7fc74ffff640 1 -- 172.21.15.78:0/4225154314 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc750042030 con 0x7fc75810c2f0 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc75fb53740 1 Processor -- start 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc75fb53740 1 -- start start 2024-09-16T00:12:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc74ffff640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7440a32d0 0x7fc7440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 0x55e8c8d64db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc75fb53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8ccb090 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc75fb53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8c72a70 con 0x55e8c8d652f0 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc75fb53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8cc7050 con 0x55e8c8d649d0 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.122+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7567fc640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7440a32d0 0x7fc7440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d71fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57784/0 (socket says 172.21.15.78:57784) 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7577fe640 1 -- 172.21.15.78:0/3413828683 learned_addr learned my addr 172.21.15.78:0/3413828683 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc74ffff640 1 -- 172.21.15.78:0/4225154314 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fc7500f05c0 con 0x7fc75810c2f0 2024-09-16T00:12:51.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc756ffd640 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 0x55e8c8d64db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7567fc640 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7577fe640 1 -- 172.21.15.78:0/3413828683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 msgr2=0x55e8c8d64db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7577fe640 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 0x55e8c8d64db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7577fe640 1 -- 172.21.15.78:0/3413828683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d652f0 msgr2=0x55e8c8d6f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7577fe640 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7577fe640 1 -- 172.21.15.78:0/3413828683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c8c7f130 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7567fc640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7440a32d0 0x7fc7440a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7581024e0 tx=0x7fc7400127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc7577fe640 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d71fa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc750009a20 tx=0x7fc75003b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f971b0e34870238 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc74d7fa640 1 -- 172.21.15.78:0/3413828683 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750049070 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc74d7fa640 1 -- 172.21.15.78:0/3413828683 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7500bce40 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.123+0000 7fc74d7fa640 1 -- 172.21.15.78:0/3413828683 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7500c0af0 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 -- 172.21.15.78:0/3413828683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 msgr2=0x55e8c8d71fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d71fa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc750009a20 tx=0x7fc75003b6b0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 -- 172.21.15.78:0/3413828683 shutdown_connections 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d71fa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 --2- 172.21.15.78:0/3413828683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 0x55e8c8d64db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 -- 172.21.15.78:0/3413828683 >> 172.21.15.78:0/3413828683 conn(0x55e8c8d60f60 msgr2=0x55e8c8d61340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 -- 172.21.15.78:0/3413828683 shutdown_connections 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 -- 172.21.15.78:0/3413828683 wait complete. 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 -- start start 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.124+0000 7fc75fb53740 1 Processor -- start 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d6f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d6cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc75fb53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8d72540 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc75fb53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8c85f20 con 0x55e8c8d649d0 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc75fb53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8d726c0 con 0x55e8c8d652f0 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc75fb53740 10 client.0 _ll_register_callbacks cb 0x55e8c8c4f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d6cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d6cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57788/0 (socket says 172.21.15.78:57788) 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 -- 172.21.15.78:0/1915594872 learned_addr learned my addr 172.21.15.78:0/1915594872 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7567fc640 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc756ffd640 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d6f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 -- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 msgr2=0x55e8c8d6c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 -- 172.21.15.78:0/1915594872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 msgr2=0x55e8c8d6f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d6f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 -- 172.21.15.78:0/1915594872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c8a3c2b0 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7567fc640 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc7577fe640 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d6cba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc7500b9110 tx=0x7fc750043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce934577134b6c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc73e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc73e7fc640 1 -- 172.21.15.78:0/1915594872 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750049070 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc73e7fc640 1 -- 172.21.15.78:0/1915594872 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fc75003b460 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.125+0000 7fc75fb53740 1 -- 172.21.15.78:0/1915594872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7500bce40 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc73e7fc640 1 -- 172.21.15.78:0/1915594872 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc7500868f0 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc75fb53740 10 client.15057 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc75fb53740 1 -- 172.21.15.78:0/1915594872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e8c8d72540 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc73e7fc640 1 -- 172.21.15.78:0/1915594872 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7500bce40 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc73e7fc640 1 -- 172.21.15.78:0/1915594872 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc7500b8db0 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc75fb53740 10 client.15057 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc75fb53740 10 client.15057 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc75fb53740 20 client.15057 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc75fb53740 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e8c8d80090 0x55e8c8d81500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc75fb53740 4 client.15057 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc75fb53740 1 -- 172.21.15.78:0/1915594872 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8c8cd8600 con 0x55e8c8d80090 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.126+0000 7fc756ffd640 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e8c8d80090 0x55e8c8d81500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.127+0000 7fc73e7fc640 10 client.15057 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.127+0000 7fc756ffd640 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e8c8d80090 0x55e8c8d81500 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ae06d359cc22e5b7 server_cookie=a0c898f8a0461372 in_seq=0 out_seq=0 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc73e7fc640 1 -- 172.21.15.78:0/1915594872 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8c8cd8600 con 0x55e8c8d80090 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc73e7fc640 10 client.15057 handle_command_reply: tid=0 2024-09-16T00:12:51.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 client.15057 shutdown 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 -- 172.21.15.78:0/1915594872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 msgr2=0x55e8c8d6cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d6cba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc7500b9110 tx=0x7fc750043000 comp rx=0 tx=0).stop 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 -- 172.21.15.78:0/1915594872 shutdown_connections 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d6cba0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d6c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e8c8d80090 0x55e8c8d81500 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 --2- 172.21.15.78:0/1915594872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d6f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 -- 172.21.15.78:0/1915594872 >> 172.21.15.78:0/1915594872 conn(0x55e8c8d60f60 msgr2=0x55e8c8d66c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc73e7fc640 0 client.15057 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 -- 172.21.15.78:0/1915594872 shutdown_connections 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 1 -- 172.21.15.78:0/1915594872 wait complete. 2024-09-16T00:12:51.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.128+0000 7fc75fb53740 20 client.15057 trim_cache size 0 max 16384 2024-09-16T00:12:51.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.145+0000 7fc75fb53740 1 Processor -- start 2024-09-16T00:12:51.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.145+0000 7fc75fb53740 1 -- start start 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d67b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc75fb53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8d7f840 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc75fb53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8d7ff10 con 0x55e8c8d649d0 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc75fb53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8d805e0 con 0x55e8c8d652f0 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d67b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d67b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57794/0 (socket says 172.21.15.78:57794) 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 -- 172.21.15.78:0/4169345502 learned_addr learned my addr 172.21.15.78:0/4169345502 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc756ffd640 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7567fc640 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 -- 172.21.15.78:0/4169345502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 msgr2=0x55e8c8d675c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d675c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 -- 172.21.15.78:0/4169345502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 msgr2=0x55e8c8d80090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d80090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 -- 172.21.15.78:0/4169345502 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c8d607f0 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc756ffd640 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d80090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc7577fe640 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d67b00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc750038020 tx=0x7fc7500f0550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9248f3be9dd35229 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc74cff9640 1 -- 172.21.15.78:0/4169345502 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750049070 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc74cff9640 1 -- 172.21.15.78:0/4169345502 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7500db0c0 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.146+0000 7fc74cff9640 1 -- 172.21.15.78:0/4169345502 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7500b8db0 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 -- 172.21.15.78:0/4169345502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 msgr2=0x55e8c8d67b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d67b00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc750038020 tx=0x7fc7500f0550 comp rx=0 tx=0).stop 2024-09-16T00:12:51.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 -- 172.21.15.78:0/4169345502 shutdown_connections 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d67b00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d652f0 0x55e8c8d675c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 --2- 172.21.15.78:0/4169345502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d649d0 0x55e8c8d80090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 -- 172.21.15.78:0/4169345502 >> 172.21.15.78:0/4169345502 conn(0x55e8c8d60f60 msgr2=0x55e8c8d6af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 -- 172.21.15.78:0/4169345502 shutdown_connections 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.147+0000 7fc75fb53740 1 -- 172.21.15.78:0/4169345502 wait complete. 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 1 -- start start 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 1 Processor -- start 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 0x55e8c8d82740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d652f0 0x55e8c8d82c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8d81380 con 0x55e8c8d652f0 2024-09-16T00:12:51.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8d80cb0 con 0x55e8c8d6fbb0 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e8c8d81a50 con 0x55e8c8d649d0 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc75fb53740 10 client.0 _ll_register_callbacks cb 0x7fc7500bc0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d652f0 0x55e8c8d82c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7567fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d652f0 0x55e8c8d82c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57798/0 (socket says 172.21.15.78:57798) 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7567fc640 1 -- 172.21.15.78:0/122308327 learned_addr learned my addr 172.21.15.78:0/122308327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc756ffd640 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 0x55e8c8d82740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7577fe640 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7567fc640 1 -- 172.21.15.78:0/122308327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 msgr2=0x55e8c8d82740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7567fc640 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 0x55e8c8d82740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7567fc640 1 -- 172.21.15.78:0/122308327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d6fbb0 msgr2=0x55e8c8d831c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7567fc640 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d831c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7567fc640 1 -- 172.21.15.78:0/122308327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c8de8990 con 0x55e8c8d652f0 2024-09-16T00:12:51.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.148+0000 7fc7577fe640 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc7567fc640 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d652f0 0x55e8c8d82c80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc7400058c0 tx=0x7fc740004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a6e158558dd03a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc73e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc73e7fc640 1 -- 172.21.15.78:0/122308327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74001fcf0 con 0x55e8c8d652f0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc73e7fc640 1 -- 172.21.15.78:0/122308327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fc740005020 con 0x55e8c8d652f0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 1 -- 172.21.15.78:0/122308327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7500db0c0 con 0x55e8c8d652f0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc73e7fc640 1 -- 172.21.15.78:0/122308327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc740033050 con 0x55e8c8d652f0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 10 client.15063 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 1 -- 172.21.15.78:0/122308327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e8c8de9c70 con 0x55e8c8d652f0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc73e7fc640 1 -- 172.21.15.78:0/122308327 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7500db0c0 con 0x55e8c8d652f0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc73e7fc640 1 -- 172.21.15.78:0/122308327 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc740020400 con 0x55e8c8d652f0 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 10 client.15063 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T00:12:51.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 20 client.15063 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e8c8dee880 0x55e8c8ded590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 4 client.15063 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.149+0000 7fc75fb53740 1 -- 172.21.15.78:0/122308327 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "status"}) -- 0x55e8c8cd2b10 con 0x55e8c8dee880 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.150+0000 7fc756ffd640 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e8c8dee880 0x55e8c8ded590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.150+0000 7fc73e7fc640 10 client.15063 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.150+0000 7fc756ffd640 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e8c8dee880 0x55e8c8ded590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=27a1fd715984c19d server_cookie=56cffead077384a7 in_seq=0 out_seq=0 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.150+0000 7fc73e7fc640 1 -- 172.21.15.78:0/122308327 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e8c8cd2b10 con 0x55e8c8dee880 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.150+0000 7fc73e7fc640 10 client.15063 handle_command_reply: tid=0 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.150+0000 7fc75fb53740 1 client.15063 shutdown 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.150+0000 7fc75fb53740 1 -- 172.21.15.78:0/122308327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d652f0 msgr2=0x55e8c8d82c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.150+0000 7fc75fb53740 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d652f0 0x55e8c8d82c80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc7400058c0 tx=0x7fc740004730 comp rx=0 tx=0).stop 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 1 -- 172.21.15.78:0/122308327 shutdown_connections 2024-09-16T00:12:51.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e8c8dee880 0x55e8c8ded590 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e8c8d6fbb0 0x55e8c8d831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc73e7fc640 0 client.15063 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e8c8d652f0 0x55e8c8d82c80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 1 --2- 172.21.15.78:0/122308327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8c8d649d0 0x55e8c8d82740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 1 -- 172.21.15.78:0/122308327 >> 172.21.15.78:0/122308327 conn(0x55e8c8d60f60 msgr2=0x55e8c8d6aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 1 -- 172.21.15.78:0/122308327 shutdown_connections 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 1 -- 172.21.15.78:0/122308327 wait complete. 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75fb53740 20 client.15063 trim_cache size 0 max 16384 2024-09-16T00:12:51.162 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24427, 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 69.630348538000007, 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 88, 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 132.68539779400001 2024-09-16T00:12:51.163 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:12:51.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 -- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7440a32d0 msgr2=0x7fc7440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7440a32d0 0x7fc7440a5790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7581024e0 tx=0x7fc7400127c0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 -- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 msgr2=0x7fc7581014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:51.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 0x7fc7581014d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc750005ab0 tx=0x7fc7500042f0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 -- 172.21.15.78:0/4225154314 shutdown_connections 2024-09-16T00:12:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7440a32d0 0x7fc7440a5790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc75810c2f0 0x7fc7581014d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7581037a0 0x7fc758100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 --2- 172.21.15.78:0/4225154314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc758102df0 0x7fc758100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 -- 172.21.15.78:0/4225154314 >> 172.21.15.78:0/4225154314 conn(0x7fc7580fc7d0 msgr2=0x7fc7580fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 -- 172.21.15.78:0/4225154314 shutdown_connections 2024-09-16T00:12:51.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:51.151+0000 7fc75d77a640 1 -- 172.21.15.78:0/4225154314 wait complete. 2024-09-16T00:12:51.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:51 smithi078 ceph-mon[31157]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:12:51.898 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:12:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:51 smithi193 ceph-mon[35484]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:12:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:51 smithi148 ceph-mon[35599]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:12:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 -- 172.21.15.78:0/3283169710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918102ef0 msgr2=0x7f491810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 --2- 172.21.15.78:0/3283169710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918102ef0 0x7f491810da40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4910009ca0 tx=0x7f491002a950 comp rx=0 tx=0).stop 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 -- 172.21.15.78:0/3283169710 shutdown_connections 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 --2- 172.21.15.78:0/3283169710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918102ef0 0x7f491810da40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 --2- 172.21.15.78:0/3283169710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4918102570 0x7f49181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 --2- 172.21.15.78:0/3283169710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918100e20 0x7f4918101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 -- 172.21.15.78:0/3283169710 >> 172.21.15.78:0/3283169710 conn(0x7f49180fc7d0 msgr2=0x7f49180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 -- 172.21.15.78:0/3283169710 shutdown_connections 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.655+0000 7f491ea8e640 1 -- 172.21.15.78:0/3283169710 wait complete. 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491ea8e640 1 Processor -- start 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491ea8e640 1 -- start start 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491ea8e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918100e20 0x7f49181974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491ea8e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918102570 0x7f4918197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491ea8e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4918102ef0 0x7f491819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491ea8e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4918111020 con 0x7f4918100e20 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491ea8e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4918110ea0 con 0x7f4918102570 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491ea8e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49181111a0 con 0x7f4918102ef0 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491c82a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918100e20 0x7f49181974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491c82a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918100e20 0x7f49181974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57818/0 (socket says 172.21.15.78:57818) 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.656+0000 7f491c82a640 1 -- 172.21.15.78:0/2221170599 learned_addr learned my addr 172.21.15.78:0/2221170599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f491c82a640 1 -- 172.21.15.78:0/2221170599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4918102ef0 msgr2=0x7f491819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f491c82a640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4918102ef0 0x7f491819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f491c82a640 1 -- 172.21.15.78:0/2221170599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918102570 msgr2=0x7f4918197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f491c82a640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918102570 0x7f4918197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f491c82a640 1 -- 172.21.15.78:0/2221170599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491819c490 con 0x7f4918100e20 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f491c82a640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918100e20 0x7f49181974e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f490c00b4a0 tx=0x7f490c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:52.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f4915ffb640 1 -- 172.21.15.78:0/2221170599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c014070 con 0x7f4918100e20 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f491ea8e640 1 -- 172.21.15.78:0/2221170599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491819c780 con 0x7f4918100e20 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f4915ffb640 1 -- 172.21.15.78:0/2221170599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c010af0 con 0x7f4918100e20 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f4915ffb640 1 -- 172.21.15.78:0/2221170599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c017d10 con 0x7f4918100e20 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.657+0000 7f491ea8e640 1 -- 172.21.15.78:0/2221170599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49181a4060 con 0x7f4918100e20 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4915ffb640 1 -- 172.21.15.78:0/2221170599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f490c017070 con 0x7f4918100e20 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4920e67740 1 Processor -- start 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4920e67740 1 -- start start 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4920e67740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf49db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4920e67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf54670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4920e67740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 0x5649fbf56fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4920e67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649fbeb0090 con 0x5649fbf4a2f0 2024-09-16T00:12:52.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4920e67740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5649fbe57a70 con 0x5649fbf499d0 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.660+0000 7f4920e67740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649fbeac050 con 0x5649fbf54bb0 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf54670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf54670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57834/0 (socket says 172.21.15.78:57834) 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 -- 172.21.15.78:0/2207549259 learned_addr learned my addr 172.21.15.78:0/2207549259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491d02b640 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf49db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f4917fff640 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 0x5649fbf56fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 -- 172.21.15.78:0/2207549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 msgr2=0x5649fbf56fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 0x5649fbf56fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 -- 172.21.15.78:0/2207549259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 msgr2=0x5649fbf49db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf49db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 -- 172.21.15.78:0/2207549259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649fbe64130 con 0x5649fbf4a2f0 2024-09-16T00:12:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491d02b640 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf49db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f4915ffb640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f49080a32d0 0x7f49080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491c82a640 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf54670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f490c0d2fd0 tx=0x7f490c0d7910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc516467560f141 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f491d02b640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f49080a32d0 0x7f49080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f4915ffb640 1 -- 172.21.15.78:0/2221170599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f490c0cf1b0 con 0x7f4918100e20 2024-09-16T00:12:52.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f4902ffd640 1 -- 172.21.15.78:0/2207549259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c097680 con 0x5649fbf4a2f0 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f4902ffd640 1 -- 172.21.15.78:0/2207549259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c09bae0 con 0x5649fbf4a2f0 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.661+0000 7f4902ffd640 1 -- 172.21.15.78:0/2207549259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c09ad40 con 0x5649fbf4a2f0 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 -- 172.21.15.78:0/2207549259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 msgr2=0x5649fbf54670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f491d02b640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f49080a32d0 0x7f49080a5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4910005cc0 tx=0x7f49100093c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf54670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f490c0d2fd0 tx=0x7f490c0d7910 comp rx=0 tx=0).stop 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 -- 172.21.15.78:0/2207549259 shutdown_connections 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 0x5649fbf56fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf54670 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 --2- 172.21.15.78:0/2207549259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf49db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 -- 172.21.15.78:0/2207549259 >> 172.21.15.78:0/2207549259 conn(0x5649fbf45f60 msgr2=0x5649fbf46340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 -- 172.21.15.78:0/2207549259 shutdown_connections 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 -- 172.21.15.78:0/2207549259 wait complete. 2024-09-16T00:12:52.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.662+0000 7f4920e67740 1 -- start start 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4920e67740 1 Processor -- start 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4920e67740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf499d0 0x5649fbf53ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4920e67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4920e67740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf54bb0 0x5649fbf5b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4920e67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649fbf57780 con 0x5649fbf4a2f0 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4920e67740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5649fbe6af20 con 0x5649fbf54bb0 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4920e67740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649fbf57900 con 0x5649fbf499d0 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4920e67740 10 client.0 _ll_register_callbacks cb 0x5649fbe34030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4917fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4917fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57846/0 (socket says 172.21.15.78:57846) 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4917fff640 1 -- 172.21.15.78:0/1758498338 learned_addr learned my addr 172.21.15.78:0/1758498338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f491d02b640 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf54bb0 0x5649fbf5b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4917fff640 1 -- 172.21.15.78:0/1758498338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf499d0 msgr2=0x5649fbf53ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4917fff640 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf499d0 0x5649fbf53ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4917fff640 1 -- 172.21.15.78:0/1758498338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf54bb0 msgr2=0x5649fbf5b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4917fff640 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf54bb0 0x5649fbf5b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.663+0000 7f4917fff640 1 -- 172.21.15.78:0/1758498338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649fbc212b0 con 0x5649fbf4a2f0 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4917fff640 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf540e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f490400d190 tx=0x7f490400d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09371ec5ded641e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f48f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f48f7fff640 1 -- 172.21.15.78:0/1758498338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49040125f0 con 0x5649fbf4a2f0 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f48f7fff640 1 -- 172.21.15.78:0/1758498338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f490400f040 con 0x5649fbf4a2f0 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 1 -- 172.21.15.78:0/1758498338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490c09bae0 con 0x5649fbf4a2f0 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f48f7fff640 1 -- 172.21.15.78:0/1758498338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4904021050 con 0x5649fbf4a2f0 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 10 client.15075 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:52.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 1 -- 172.21.15.78:0/1758498338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649fbf652f0 con 0x5649fbf4a2f0 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f48f7fff640 1 -- 172.21.15.78:0/1758498338 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c09bae0 con 0x5649fbf4a2f0 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f48f7fff640 1 -- 172.21.15.78:0/1758498338 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4904013070 con 0x5649fbf4a2f0 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 10 client.15075 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 10 client.15075 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 20 client.15075 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5649fbf67a00 0x5649fbf68e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 4 client.15075 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.664+0000 7f4920e67740 1 -- 172.21.15.78:0/1758498338 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649fbebd600 con 0x5649fbf67a00 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.665+0000 7f491c82a640 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5649fbf67a00 0x5649fbf68e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.665+0000 7f491c82a640 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5649fbf67a00 0x5649fbf68e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d25d2442787e2cb server_cookie=b470fb357937c43e in_seq=0 out_seq=0 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.665+0000 7f48f7fff640 10 client.15075 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f48f7fff640 1 -- 172.21.15.78:0/1758498338 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649fbebd600 con 0x5649fbf67a00 2024-09-16T00:12:52.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f48f7fff640 10 client.15075 handle_command_reply: tid=0 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 client.15075 shutdown 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 -- 172.21.15.78:0/1758498338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 msgr2=0x5649fbf540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf540e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f490400d190 tx=0x7f490400d650 comp rx=0 tx=0).stop 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 -- 172.21.15.78:0/1758498338 shutdown_connections 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5649fbf67a00 0x5649fbf68e70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf54bb0 0x5649fbf5b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf540e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 --2- 172.21.15.78:0/1758498338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf499d0 0x5649fbf53ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f48f7fff640 0 client.15075 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 -- 172.21.15.78:0/1758498338 >> 172.21.15.78:0/1758498338 conn(0x5649fbf45f60 msgr2=0x5649fbf4bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 -- 172.21.15.78:0/1758498338 shutdown_connections 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.666+0000 7f4920e67740 1 -- 172.21.15.78:0/1758498338 wait complete. 2024-09-16T00:12:52.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.667+0000 7f4920e67740 20 client.15075 trim_cache size 0 max 16384 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4920e67740 1 Processor -- start 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4920e67740 1 -- start start 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4920e67740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4920e67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf5b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4920e67740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 0x5649fbf71150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4920e67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649fbf671b0 con 0x5649fbf4a2f0 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4920e67740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5649fbf67880 con 0x5649fbf499d0 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4920e67740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649fbf67f50 con 0x5649fbf54bb0 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.684+0000 7f4917fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf5b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f4917fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf5b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57860/0 (socket says 172.21.15.78:57860) 2024-09-16T00:12:52.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f4917fff640 1 -- 172.21.15.78:0/2001131107 learned_addr learned my addr 172.21.15.78:0/2001131107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f491c82a640 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f491d02b640 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 0x5649fbf71150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f4917fff640 1 -- 172.21.15.78:0/2001131107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 msgr2=0x5649fbf71150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f4917fff640 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 0x5649fbf71150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f4917fff640 1 -- 172.21.15.78:0/2001131107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 msgr2=0x5649fbf687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f4917fff640 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f4917fff640 1 -- 172.21.15.78:0/2001131107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649fbf68400 con 0x5649fbf4a2f0 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f4917fff640 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf5b110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4904016020 tx=0x7f490400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51003dc989b1526e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f49027fc640 1 -- 172.21.15.78:0/2001131107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904029070 con 0x5649fbf4a2f0 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f49027fc640 1 -- 172.21.15.78:0/2001131107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490400db50 con 0x5649fbf4a2f0 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.685+0000 7f49027fc640 1 -- 172.21.15.78:0/2001131107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490401eb40 con 0x5649fbf4a2f0 2024-09-16T00:12:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 -- 172.21.15.78:0/2001131107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 msgr2=0x5649fbf5b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf5b110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4904016020 tx=0x7f490400bb90 comp rx=0 tx=0).stop 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 -- 172.21.15.78:0/2001131107 shutdown_connections 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf54bb0 0x5649fbf71150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf4a2f0 0x5649fbf5b110 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 --2- 172.21.15.78:0/2001131107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 -- 172.21.15.78:0/2001131107 >> 172.21.15.78:0/2001131107 conn(0x5649fbf45f60 msgr2=0x5649fbf53ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 -- 172.21.15.78:0/2001131107 shutdown_connections 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 -- 172.21.15.78:0/2001131107 wait complete. 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 -- start start 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.686+0000 7f4920e67740 1 Processor -- start 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4920e67740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4920e67740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf4a2f0 0x5649fbf4fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4920e67740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf54bb0 0x5649fbf50150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4920e67740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5649fbf68cf0 con 0x5649fbf54bb0 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4920e67740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5649fbf68620 con 0x5649fbf499d0 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4920e67740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649fbf693c0 con 0x5649fbf4a2f0 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4920e67740 10 client.0 _ll_register_callbacks cb 0x7f490400b120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf54bb0 0x5649fbf50150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf54bb0 0x5649fbf50150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57868/0 (socket says 172.21.15.78:57868) 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 -- 172.21.15.78:0/651085466 learned_addr learned my addr 172.21.15.78:0/651085466 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491c82a640 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4917fff640 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf4a2f0 0x5649fbf4fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 -- 172.21.15.78:0/651085466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf4a2f0 msgr2=0x5649fbf4fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf4a2f0 0x5649fbf4fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 -- 172.21.15.78:0/651085466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 msgr2=0x5649fbf513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:52.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 -- 172.21.15.78:0/651085466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649fbf63370 con 0x5649fbf54bb0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f4917fff640 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf4a2f0 0x5649fbf4fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f491d02b640 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf54bb0 0x5649fbf50150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4910009ca0 tx=0x7f4910005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8981bc62a2e93490 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.687+0000 7f48f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f48f7fff640 1 -- 172.21.15.78:0/651085466 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910034040 con 0x5649fbf54bb0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 1 -- 172.21.15.78:0/651085466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490400db50 con 0x5649fbf54bb0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f48f7fff640 1 -- 172.21.15.78:0/651085466 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f4910004250 con 0x5649fbf54bb0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f48f7fff640 1 -- 172.21.15.78:0/651085466 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4910004450 con 0x5649fbf54bb0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 10 client.15081 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 1 -- 172.21.15.78:0/651085466 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649fbf62970 con 0x5649fbf54bb0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f48f7fff640 1 -- 172.21.15.78:0/651085466 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490400db50 con 0x5649fbf54bb0 2024-09-16T00:12:52.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f48f7fff640 1 -- 172.21.15.78:0/651085466 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4910042040 con 0x5649fbf54bb0 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 10 client.15081 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 20 client.15081 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5649fbfd67d0 0x5649fbfd54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 4 client.15081 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.688+0000 7f4920e67740 1 -- 172.21.15.78:0/651085466 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "status"}) -- 0x5649fbeb7b10 con 0x5649fbfd67d0 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f491c82a640 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5649fbfd67d0 0x5649fbfd54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f491c82a640 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5649fbfd67d0 0x5649fbfd54e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5747f5b30d8a88d server_cookie=cb01bf54136cab01 in_seq=0 out_seq=0 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f48f7fff640 10 client.15081 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f48f7fff640 1 -- 172.21.15.78:0/651085466 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5649fbeb7b10 con 0x5649fbfd67d0 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f48f7fff640 10 client.15081 handle_command_reply: tid=0 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 client.15081 shutdown 2024-09-16T00:12:52.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 -- 172.21.15.78:0/651085466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf54bb0 msgr2=0x5649fbf50150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf54bb0 0x5649fbf50150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4910009ca0 tx=0x7f4910005c50 comp rx=0 tx=0).stop 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 -- 172.21.15.78:0/651085466 shutdown_connections 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5649fbf54bb0 0x5649fbf50150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649fbf4a2f0 0x5649fbf4fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5649fbfd67d0 0x5649fbfd54e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 --2- 172.21.15.78:0/651085466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5649fbf499d0 0x5649fbf513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f48f7fff640 0 client.15081 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 -- 172.21.15.78:0/651085466 >> 172.21.15.78:0/651085466 conn(0x5649fbf45f60 msgr2=0x5649fbf52270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 -- 172.21.15.78:0/651085466 shutdown_connections 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.689+0000 7f4920e67740 1 -- 172.21.15.78:0/651085466 wait complete. 2024-09-16T00:12:52.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f4920e67740 20 client.15081 trim_cache size 0 max 16384 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24440, 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 72.181383424000003, 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 88, 2024-09-16T00:12:52.698 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:12:52.699 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 131.61854400999999 2024-09-16T00:12:52.699 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:12:52.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 -- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f49080a32d0 msgr2=0x7f49080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f49080a32d0 0x7f49080a5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4910005cc0 tx=0x7f49100093c0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 -- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918100e20 msgr2=0x7f49181974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:52.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918100e20 0x7f49181974e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f490c00b4a0 tx=0x7f490c00b960 comp rx=0 tx=0).stop 2024-09-16T00:12:52.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 -- 172.21.15.78:0/2221170599 shutdown_connections 2024-09-16T00:12:52.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f49080a32d0 0x7f49080a5790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4918102ef0 0x7f491819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4918102570 0x7f4918197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 --2- 172.21.15.78:0/2221170599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4918100e20 0x7f49181974e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:52.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 -- 172.21.15.78:0/2221170599 >> 172.21.15.78:0/2221170599 conn(0x7f49180fc7d0 msgr2=0x7f49180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:52.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 -- 172.21.15.78:0/2221170599 shutdown_connections 2024-09-16T00:12:52.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:52.690+0000 7f491ea8e640 1 -- 172.21.15.78:0/2221170599 wait complete. 2024-09-16T00:12:53.519 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T00:12:53.767 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:53.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:53 smithi078 ceph-mon[31157]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:12:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:53 smithi193 ceph-mon[35484]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:12:53.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:53 smithi148 ceph-mon[35599]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:12:54.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.265+0000 7f3226941640 1 -- 172.21.15.78:0/3815975952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 msgr2=0x7f322010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.265+0000 7f3226941640 1 --2- 172.21.15.78:0/3815975952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 0x7f322010ea90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3218009ca0 tx=0x7f321802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 -- 172.21.15.78:0/3815975952 shutdown_connections 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 --2- 172.21.15.78:0/3815975952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 0x7f322010ea90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 --2- 172.21.15.78:0/3815975952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220102790 0x7f32201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 --2- 172.21.15.78:0/3815975952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3220101de0 0x7f32201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 -- 172.21.15.78:0/3815975952 >> 172.21.15.78:0/3815975952 conn(0x7f32200fd8c0 msgr2=0x7f32200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 -- 172.21.15.78:0/3815975952 shutdown_connections 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 -- 172.21.15.78:0/3815975952 wait complete. 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 Processor -- start 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 -- start start 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3220101de0 0x7f3220198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220102790 0x7f3220198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 0x7f322019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3220111f70 con 0x7f3220106f10 2024-09-16T00:12:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3220111df0 con 0x7f3220101de0 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.266+0000 7f3226941640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32201120f0 con 0x7f3220102790 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 0x7f322019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 0x7f322019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57876/0 (socket says 172.21.15.78:57876) 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 -- 172.21.15.78:0/3028427396 learned_addr learned my addr 172.21.15.78:0/3028427396 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f321ffff640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3220101de0 0x7f3220198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3217fff640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220102790 0x7f3220198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 -- 172.21.15.78:0/3028427396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220102790 msgr2=0x7f3220198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220102790 0x7f3220198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 -- 172.21.15.78:0/3028427396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3220101de0 msgr2=0x7f3220198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3220101de0 0x7f3220198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 -- 172.21.15.78:0/3028427396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322019d2e0 con 0x7f3220106f10 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3217fff640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220102790 0x7f3220198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:54.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3224ede640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 0x7f322019cc00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3218031a60 tx=0x7f3218031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f321dffb640 1 -- 172.21.15.78:0/3028427396 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218003c30 con 0x7f3220106f10 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3226941640 1 -- 172.21.15.78:0/3028427396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322019d570 con 0x7f3220106f10 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f321dffb640 1 -- 172.21.15.78:0/3028427396 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3218003dd0 con 0x7f3220106f10 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f321dffb640 1 -- 172.21.15.78:0/3028427396 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32180047e0 con 0x7f3220106f10 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.267+0000 7f3226941640 1 -- 172.21.15.78:0/3028427396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32201a4eb0 con 0x7f3220106f10 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.270+0000 7f321dffb640 1 -- 172.21.15.78:0/3028427396 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f321802f790 con 0x7f3220106f10 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.270+0000 7f3228d1a740 1 Processor -- start 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.270+0000 7f3228d1a740 1 -- start start 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.270+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d52ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.270+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d538670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d53afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f3228d1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b9d494090 con 0x559b9d52e2f0 2024-09-16T00:12:54.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f3228d1a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559b9d43ba70 con 0x559b9d52d9d0 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f3228d1a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559b9d490050 con 0x559b9d538bb0 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d538670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d538670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57884/0 (socket says 172.21.15.78:57884) 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 -- 172.21.15.78:0/3499369018 learned_addr learned my addr 172.21.15.78:0/3499369018 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f3217fff640 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d53afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f3217fff640 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d52ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 -- 172.21.15.78:0/3499369018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 msgr2=0x559b9d53afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d53afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 -- 172.21.15.78:0/3499369018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 msgr2=0x559b9d52ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d52ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 -- 172.21.15.78:0/3499369018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b9d448130 con 0x559b9d52e2f0 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321dffb640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f320c0a32d0 0x7f320c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321ffff640 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d538670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f321000cbf0 tx=0x7f321000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151091bd915f773 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:54.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f3217fff640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f320c0a32d0 0x7f320c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f32167fc640 1 -- 172.21.15.78:0/3499369018 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210010630 con 0x559b9d52e2f0 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f321dffb640 1 -- 172.21.15.78:0/3028427396 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f32180f11b0 con 0x7f3220106f10 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f32167fc640 1 -- 172.21.15.78:0/3499369018 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3210010cb0 con 0x559b9d52e2f0 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.271+0000 7f32167fc640 1 -- 172.21.15.78:0/3499369018 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210019c90 con 0x559b9d52e2f0 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 -- 172.21.15.78:0/3499369018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 msgr2=0x559b9d538670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3217fff640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f320c0a32d0 0x7f320c0a5790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3208005940 tx=0x7f3208005890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d538670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f321000cbf0 tx=0x7f321000e720 comp rx=0 tx=0).stop 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 -- 172.21.15.78:0/3499369018 shutdown_connections 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d53afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d538670 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3499369018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d52ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 -- 172.21.15.78:0/3499369018 >> 172.21.15.78:0/3499369018 conn(0x559b9d529f60 msgr2=0x559b9d52a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:54.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 -- 172.21.15.78:0/3499369018 shutdown_connections 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 -- 172.21.15.78:0/3499369018 wait complete. 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.272+0000 7f3228d1a740 1 -- start start 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3228d1a740 1 Processor -- start 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d538410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d52e2f0 0x559b9d535760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d538bb0 0x559b9d535ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3228d1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b9d53b780 con 0x559b9d52d9d0 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3228d1a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559b9d44ef20 con 0x559b9d538bb0 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3228d1a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559b9d53b900 con 0x559b9d52e2f0 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3228d1a740 10 client.0 _ll_register_callbacks cb 0x559b9d418030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f321ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d538410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f321ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d538410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57900/0 (socket says 172.21.15.78:57900) 2024-09-16T00:12:54.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f321ffff640 1 -- 172.21.15.78:0/3799228885 learned_addr learned my addr 172.21.15.78:0/3799228885 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3224ede640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d538bb0 0x559b9d535ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f321ffff640 1 -- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d52e2f0 msgr2=0x559b9d535760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3217fff640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d52e2f0 0x559b9d535760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f321ffff640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d52e2f0 0x559b9d535760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f321ffff640 1 -- 172.21.15.78:0/3799228885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d538bb0 msgr2=0x559b9d535ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f321ffff640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d538bb0 0x559b9d535ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f321ffff640 1 -- 172.21.15.78:0/3799228885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b9d2052b0 con 0x559b9d52d9d0 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3224ede640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d538bb0 0x559b9d535ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.273+0000 7f3217fff640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d52e2f0 0x559b9d535760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f321ffff640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d538410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3210019810 tx=0x7f321000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a9d2f5b40da5ab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f31ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3799228885 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210010e60 con 0x559b9d52d9d0 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3799228885 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f3210005250 con 0x559b9d52d9d0 2024-09-16T00:12:54.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f3228d1a740 1 -- 172.21.15.78:0/3799228885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3210010cb0 con 0x559b9d52d9d0 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3799228885 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f321002e050 con 0x559b9d52d9d0 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f3228d1a740 10 client.15093 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f3228d1a740 1 -- 172.21.15.78:0/3799228885 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b9d547070 con 0x559b9d52d9d0 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3799228885 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3210010cb0 con 0x559b9d52d9d0 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.274+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3799228885 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f32100046c0 con 0x559b9d52d9d0 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f3228d1a740 10 client.15093 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f3228d1a740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f3228d1a740 20 client.15093 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559b9d549320 0x559b9d54a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f3228d1a740 4 client.15093 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f3228d1a740 1 -- 172.21.15.78:0/3799228885 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b9d4a1600 con 0x559b9d549320 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f3217fff640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559b9d549320 0x559b9d54a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f31ff7fe640 10 client.15093 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:12:54.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.275+0000 7f3217fff640 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559b9d549320 0x559b9d54a790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0daba2ccc823ebb server_cookie=ce2c178428ca38e in_seq=0 out_seq=0 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3799228885 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b9d4a1600 con 0x559b9d549320 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f31ff7fe640 10 client.15093 handle_command_reply: tid=0 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 client.15093 shutdown 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 -- 172.21.15.78:0/3799228885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 msgr2=0x559b9d538410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d538410 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3210019810 tx=0x7f321000c960 comp rx=0 tx=0).stop 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 -- 172.21.15.78:0/3799228885 shutdown_connections 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559b9d549320 0x559b9d54a790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d538bb0 0x559b9d535ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d52e2f0 0x559b9d535760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3799228885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d538410 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f3228d1a740 1 -- 172.21.15.78:0/3799228885 >> 172.21.15.78:0/3799228885 conn(0x559b9d529f60 msgr2=0x559b9d52fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.276+0000 7f31ff7fe640 0 client.15093 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:12:54.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.277+0000 7f3228d1a740 1 -- 172.21.15.78:0/3799228885 shutdown_connections 2024-09-16T00:12:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.277+0000 7f3228d1a740 1 -- 172.21.15.78:0/3799228885 wait complete. 2024-09-16T00:12:54.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.277+0000 7f3228d1a740 20 client.15093 trim_cache size 0 max 16384 2024-09-16T00:12:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.294+0000 7f3228d1a740 1 Processor -- start 2024-09-16T00:12:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.294+0000 7f3228d1a740 1 -- start start 2024-09-16T00:12:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d5499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d530cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d5329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3228d1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b9d548ad0 con 0x559b9d52e2f0 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3228d1a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559b9d5491a0 con 0x559b9d52d9d0 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3228d1a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559b9d549870 con 0x559b9d538bb0 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d530cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d530cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57906/0 (socket says 172.21.15.78:57906) 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 -- 172.21.15.78:0/1403201660 learned_addr learned my addr 172.21.15.78:0/1403201660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f321ffff640 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d5499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3224ede640 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d5329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 -- 172.21.15.78:0/1403201660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 msgr2=0x559b9d5329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d5329c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 -- 172.21.15.78:0/1403201660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 msgr2=0x559b9d5499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d5499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 -- 172.21.15.78:0/1403201660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b9d529d90 con 0x559b9d52e2f0 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f321ffff640 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d5499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3224ede640 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d5329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3217fff640 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d530cd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x559b9d52b2e0 tx=0x7f3208014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3057130587e804f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3215ffb640 1 -- 172.21.15.78:0/1403201660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320802a030 con 0x559b9d52e2f0 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3215ffb640 1 -- 172.21.15.78:0/1403201660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320801fd20 con 0x559b9d52e2f0 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.295+0000 7f3215ffb640 1 -- 172.21.15.78:0/1403201660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208018070 con 0x559b9d52e2f0 2024-09-16T00:12:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 -- 172.21.15.78:0/1403201660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 msgr2=0x559b9d530cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d530cd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x559b9d52b2e0 tx=0x7f3208014000 comp rx=0 tx=0).stop 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 -- 172.21.15.78:0/1403201660 shutdown_connections 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d5329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52e2f0 0x559b9d530cd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 --2- 172.21.15.78:0/1403201660 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52d9d0 0x559b9d5499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 -- 172.21.15.78:0/1403201660 >> 172.21.15.78:0/1403201660 conn(0x559b9d529f60 msgr2=0x559b9d5305c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 -- 172.21.15.78:0/1403201660 shutdown_connections 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 -- 172.21.15.78:0/1403201660 wait complete. 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.296+0000 7f3228d1a740 1 -- start start 2024-09-16T00:12:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3228d1a740 1 Processor -- start 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d54bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52e2f0 0x559b9d54c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3228d1a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d5b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3228d1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559b9d54a610 con 0x559b9d52d9d0 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3228d1a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559b9d549f40 con 0x559b9d52e2f0 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3228d1a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559b9d54ace0 con 0x559b9d538bb0 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3228d1a740 10 client.0 _ll_register_callbacks cb 0x7f3210014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d54bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d54bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57910/0 (socket says 172.21.15.78:57910) 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 -- 172.21.15.78:0/3930820551 learned_addr learned my addr 172.21.15.78:0/3930820551 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3217fff640 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52e2f0 0x559b9d54c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 -- 172.21.15.78:0/3930820551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 msgr2=0x559b9d5b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d5b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 -- 172.21.15.78:0/3930820551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52e2f0 msgr2=0x559b9d54c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52e2f0 0x559b9d54c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 -- 172.21.15.78:0/3930820551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b9d5b1f60 con 0x559b9d52d9d0 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f3217fff640 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52e2f0 0x559b9d54c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f321ffff640 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d54bf80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f32100058c0 tx=0x7f3210004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4430a5a11ad29015 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.297+0000 7f31ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3930820551 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321001e020 con 0x559b9d52d9d0 2024-09-16T00:12:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3930820551 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f3210004660 con 0x559b9d52d9d0 2024-09-16T00:12:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 1 -- 172.21.15.78:0/3930820551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f320801fd20 con 0x559b9d52d9d0 2024-09-16T00:12:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3930820551 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f321000c470 con 0x559b9d52d9d0 2024-09-16T00:12:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 10 client.15099 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 1 -- 172.21.15.78:0/3930820551 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b9d5b3090 con 0x559b9d52d9d0 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3930820551 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320801fd20 con 0x559b9d52d9d0 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3930820551 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f32100193c0 con 0x559b9d52d9d0 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 10 client.15099 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 20 client.15099 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559b9d5b7d10 0x559b9d5b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 4 client.15099 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.298+0000 7f3228d1a740 1 -- 172.21.15.78:0/3930820551 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "status"}) -- 0x559b9d49bb10 con 0x559b9d5b7d10 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3217fff640 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559b9d5b7d10 0x559b9d5b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3217fff640 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559b9d5b7d10 0x559b9d5b6a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68c3aa8868d4d8aa server_cookie=d1303d58275ab47 in_seq=0 out_seq=0 2024-09-16T00:12:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f31ff7fe640 10 client.15099 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:12:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f31ff7fe640 1 -- 172.21.15.78:0/3930820551 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559b9d49bb10 con 0x559b9d5b7d10 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f31ff7fe640 10 client.15099 handle_command_reply: tid=0 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 client.15099 shutdown 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 -- 172.21.15.78:0/3930820551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 msgr2=0x559b9d54bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d54bf80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f32100058c0 tx=0x7f3210004e30 comp rx=0 tx=0).stop 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 -- 172.21.15.78:0/3930820551 shutdown_connections 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559b9d5b7d10 0x559b9d5b6a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559b9d538bb0 0x559b9d5b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559b9d52e2f0 0x559b9d54c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 --2- 172.21.15.78:0/3930820551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559b9d52d9d0 0x559b9d54bf80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 -- 172.21.15.78:0/3930820551 >> 172.21.15.78:0/3930820551 conn(0x559b9d529f60 msgr2=0x559b9d535810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f31ff7fe640 0 client.15099 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.299+0000 7f3228d1a740 1 -- 172.21.15.78:0/3930820551 shutdown_connections 2024-09-16T00:12:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3228d1a740 1 -- 172.21.15.78:0/3930820551 wait complete. 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3228d1a740 20 client.15099 trim_cache size 0 max 16384 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24445, 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:12:54.312 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 74.800921408999997, 2024-09-16T00:12:54.313 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:12:54.313 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 88, 2024-09-16T00:12:54.313 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:12:54.313 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 130.627240865 2024-09-16T00:12:54.313 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:12:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 -- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f320c0a32d0 msgr2=0x7f320c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f320c0a32d0 0x7f320c0a5790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3208005940 tx=0x7f3208005890 comp rx=0 tx=0).stop 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 -- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 msgr2=0x7f322019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 0x7f322019cc00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3218031a60 tx=0x7f3218031a90 comp rx=0 tx=0).stop 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 -- 172.21.15.78:0/3028427396 shutdown_connections 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f320c0a32d0 0x7f320c0a5790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3220106f10 0x7f322019cc00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3220102790 0x7f3220198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 --2- 172.21.15.78:0/3028427396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3220101de0 0x7f3220198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 -- 172.21.15.78:0/3028427396 >> 172.21.15.78:0/3028427396 conn(0x7f32200fd8c0 msgr2=0x7f32200ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 -- 172.21.15.78:0/3028427396 shutdown_connections 2024-09-16T00:12:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:54.300+0000 7f3226941640 1 -- 172.21.15.78:0/3028427396 wait complete. 2024-09-16T00:12:55.103 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:12:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:55.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:55 smithi078 ceph-mon[31157]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 -- 172.21.15.78:0/940012053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0101df0 msgr2=0x7fb7a01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 --2- 172.21.15.78:0/940012053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0101df0 0x7fb7a01021d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb790009900 tx=0x7fb79002a410 comp rx=0 tx=0).stop 2024-09-16T00:12:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 -- 172.21.15.78:0/940012053 shutdown_connections 2024-09-16T00:12:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 --2- 172.21.15.78:0/940012053 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a0106f20 0x7fb7a010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 --2- 172.21.15.78:0/940012053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a01027a0 0x7fb7a01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 --2- 172.21.15.78:0/940012053 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0101df0 0x7fb7a01021d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 -- 172.21.15.78:0/940012053 >> 172.21.15.78:0/940012053 conn(0x7fb7a00fd8b0 msgr2=0x7fb7a00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 -- 172.21.15.78:0/940012053 shutdown_connections 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.850+0000 7fb7a6311640 1 -- 172.21.15.78:0/940012053 wait complete. 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.851+0000 7fb7a6311640 1 Processor -- start 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.851+0000 7fb7a6311640 1 -- start start 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.851+0000 7fb7a6311640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0101df0 0x7fb7a0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.851+0000 7fb7a6311640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a01027a0 0x7fb7a01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.851+0000 7fb7a6311640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0106f20 0x7fb7a019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.851+0000 7fb7a6311640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0111fd0 con 0x7fb7a0106f20 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.851+0000 7fb7a6311640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0111e50 con 0x7fb7a01027a0 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.851+0000 7fb7a6311640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0112150 con 0x7fb7a0101df0 2024-09-16T00:12:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0106f20 0x7fb7a019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0106f20 0x7fb7a019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57934/0 (socket says 172.21.15.78:57934) 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 -- 172.21.15.78:0/3475114794 learned_addr learned my addr 172.21.15.78:0/3475114794 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a01027a0 0x7fb7a01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb79ffff640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0101df0 0x7fb7a0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 -- 172.21.15.78:0/3475114794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0101df0 msgr2=0x7fb7a0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0101df0 0x7fb7a0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 -- 172.21.15.78:0/3475114794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a01027a0 msgr2=0x7fb7a01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a01027a0 0x7fb7a01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 -- 172.21.15.78:0/3475114794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a019d420 con 0x7fb7a0106f20 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb79ffff640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0101df0 0x7fb7a0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a48ae640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0106f20 0x7fb7a019cd40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb79800eb10 tx=0x7fb79800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb79d7fa640 1 -- 172.21.15.78:0/3475114794 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798004b20 con 0x7fb7a0106f20 2024-09-16T00:12:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a6311640 1 -- 172.21.15.78:0/3475114794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a019d710 con 0x7fb7a0106f20 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb79d7fa640 1 -- 172.21.15.78:0/3475114794 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb798004cc0 con 0x7fb7a0106f20 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb79d7fa640 1 -- 172.21.15.78:0/3475114794 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb798010760 con 0x7fb7a0106f20 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.852+0000 7fb7a6311640 1 -- 172.21.15.78:0/3475114794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a01a4ff0 con 0x7fb7a0106f20 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.854+0000 7fb79d7fa640 1 -- 172.21.15.78:0/3475114794 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb798005020 con 0x7fb7a0106f20 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a86ea740 1 Processor -- start 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a86ea740 1 -- start start 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad08b0 0x55eb9fad0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad11d0 0x55eb9fadfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fae23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb9fa37090 con 0x55eb9fadffc0 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb9f9dea70 con 0x55eb9fad08b0 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb79d7fa640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7880a32d0 0x7fb7880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb9fa33050 con 0x55eb9fad11d0 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb79f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fae23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb79f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fae23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57936/0 (socket says 172.21.15.78:57936) 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb79f7fe640 1 -- 172.21.15.78:0/3445747652 learned_addr learned my addr 172.21.15.78:0/3445747652 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad08b0 0x55eb9fad0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb79d7fa640 1 -- 172.21.15.78:0/3475114794 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fb7980d0380 con 0x7fb7a0106f20 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.855+0000 7fb7a48ae640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7880a32d0 0x7fb7880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb79ffff640 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad11d0 0x55eb9fadfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb79f7fe640 1 -- 172.21.15.78:0/3445747652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad11d0 msgr2=0x55eb9fadfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad11d0 0x55eb9fadfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb79f7fe640 1 -- 172.21.15.78:0/3445747652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad08b0 msgr2=0x55eb9fad0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad08b0 0x55eb9fad0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb79f7fe640 1 -- 172.21.15.78:0/3445747652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9f9eb130 con 0x55eb9fadffc0 2024-09-16T00:12:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fae23b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb7940127c0 tx=0x7fb79400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b1d95209a07bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb7a48ae640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7880a32d0 0x7fb7880a5790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb7980049d0 tx=0x7fb798005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb7867fc640 1 -- 172.21.15.78:0/3445747652 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794012e00 con 0x55eb9fadffc0 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb7867fc640 1 -- 172.21.15.78:0/3445747652 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb794010c50 con 0x55eb9fadffc0 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.856+0000 7fb7867fc640 1 -- 172.21.15.78:0/3445747652 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79401eb30 con 0x55eb9fadffc0 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3445747652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 msgr2=0x55eb9fae23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fae23b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb7940127c0 tx=0x7fb79400b7c0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3445747652 shutdown_connections 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fae23b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad11d0 0x55eb9fadfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3445747652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad08b0 0x55eb9fad0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3445747652 >> 172.21.15.78:0/3445747652 conn(0x55eb9faccf60 msgr2=0x55eb9facd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3445747652 shutdown_connections 2024-09-16T00:12:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3445747652 wait complete. 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 -- start start 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.857+0000 7fb7a86ea740 1 Processor -- start 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fadf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9fadcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fadd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb9fae2b90 con 0x55eb9fadffc0 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb9f9f1f20 con 0x55eb9fad11d0 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb9fae2d10 con 0x55eb9fad08b0 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb7a86ea740 10 client.0 _ll_register_callbacks cb 0x55eb9f9bb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fadd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fadd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57950/0 (socket says 172.21.15.78:57950) 2024-09-16T00:12:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 -- 172.21.15.78:0/3131373463 learned_addr learned my addr 172.21.15.78:0/3131373463 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fadf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79ffff640 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9fadcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 -- 172.21.15.78:0/3131373463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 msgr2=0x55eb9fadf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fadf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 -- 172.21.15.78:0/3131373463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 msgr2=0x55eb9fadcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9fadcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 -- 172.21.15.78:0/3131373463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9f7a82b0 con 0x55eb9fadffc0 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fadd0e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b450 tx=0x7fb794014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9943369db6039e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb77f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.858+0000 7fb77f7fe640 1 -- 172.21.15.78:0/3131373463 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794024040 con 0x55eb9fadffc0 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb77f7fe640 1 -- 172.21.15.78:0/3131373463 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fb79401e8b0 con 0x55eb9fadffc0 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3131373463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb794010c50 con 0x55eb9fadffc0 2024-09-16T00:12:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb77f7fe640 1 -- 172.21.15.78:0/3131373463 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb79401eab0 con 0x55eb9fadffc0 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 10 client.15111 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3131373463 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb9faee4e0 con 0x55eb9fadffc0 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb77f7fe640 1 -- 172.21.15.78:0/3131373463 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb794010c50 con 0x55eb9fadffc0 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb77f7fe640 1 -- 172.21.15.78:0/3131373463 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb79402a070 con 0x55eb9fadffc0 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 10 client.15111 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 10 client.15111 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 20 client.15111 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55eb9faf0840 0x55eb9faf1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 4 client.15111 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3131373463 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb9fa44600 con 0x55eb9faf0840 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.859+0000 7fb79ffff640 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55eb9faf0840 0x55eb9faf1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.860+0000 7fb77f7fe640 10 client.15111 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:12:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.860+0000 7fb79ffff640 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55eb9faf0840 0x55eb9faf1cb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6efac5bee7afa04b server_cookie=b253ec437e2b9a1f in_seq=0 out_seq=0 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.860+0000 7fb77f7fe640 1 -- 172.21.15.78:0/3131373463 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb9fa44600 con 0x55eb9faf0840 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.860+0000 7fb77f7fe640 10 client.15111 handle_command_reply: tid=0 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 client.15111 shutdown 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3131373463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 msgr2=0x55eb9fadd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fadd0e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b450 tx=0x7fb794014040 comp rx=0 tx=0).stop 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3131373463 shutdown_connections 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55eb9faf0840 0x55eb9faf1cb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fadd0e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9fadcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/3131373463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fadf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb77f7fe640 0 client.15111 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3131373463 >> 172.21.15.78:0/3131373463 conn(0x55eb9faccf60 msgr2=0x55eb9fad2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3131373463 shutdown_connections 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 1 -- 172.21.15.78:0/3131373463 wait complete. 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.861+0000 7fb7a86ea740 20 client.15111 trim_cache size 0 max 16384 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.878+0000 7fb7a86ea740 1 Processor -- start 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.878+0000 7fb7a86ea740 1 -- start start 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.878+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9faf0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.878+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9faf0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.878+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fac7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.878+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb9faefff0 con 0x55eb9fadffc0 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.878+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb9faf06c0 con 0x55eb9fad11d0 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.878+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb9faf0d90 con 0x55eb9fad08b0 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fac7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fac7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57956/0 (socket says 172.21.15.78:57956) 2024-09-16T00:12:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 -- 172.21.15.78:0/4057584508 learned_addr learned my addr 172.21.15.78:0/4057584508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79ffff640 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9faf0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 -- 172.21.15.78:0/4057584508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 msgr2=0x55eb9faf0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9faf0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 -- 172.21.15.78:0/4057584508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 msgr2=0x55eb9faf0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9faf0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 -- 172.21.15.78:0/4057584508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9fadb0f0 con 0x55eb9fadffc0 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79ffff640 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9faf0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fac7d50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb7940127c0 tx=0x7fb79400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e42da647515f5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb785ffb640 1 -- 172.21.15.78:0/4057584508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794010c50 con 0x55eb9fadffc0 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb785ffb640 1 -- 172.21.15.78:0/4057584508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb794016030 con 0x55eb9fadffc0 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.879+0000 7fb785ffb640 1 -- 172.21.15.78:0/4057584508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79401e780 con 0x55eb9fadffc0 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 -- 172.21.15.78:0/4057584508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 msgr2=0x55eb9fac7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fac7d50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb7940127c0 tx=0x7fb79400bf40 comp rx=0 tx=0).stop 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 -- 172.21.15.78:0/4057584508 shutdown_connections 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fadffc0 0x55eb9fac7d50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fad11d0 0x55eb9faf0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/4057584508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9faf0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 -- 172.21.15.78:0/4057584508 >> 172.21.15.78:0/4057584508 conn(0x55eb9faccf60 msgr2=0x55eb9facd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 -- 172.21.15.78:0/4057584508 shutdown_connections 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 -- 172.21.15.78:0/4057584508 wait complete. 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.880+0000 7fb7a86ea740 1 -- start start 2024-09-16T00:12:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb7a86ea740 1 Processor -- start 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fafa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fad11d0 0x55eb9fafa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb7a86ea740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fadffc0 0x55eb9faf3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb9faf1b30 con 0x55eb9fad11d0 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb9faf1460 con 0x55eb9fadffc0 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb7a86ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb9faf2200 con 0x55eb9fad08b0 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb7a86ea740 10 client.0 _ll_register_callbacks cb 0x7fb79400be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fad11d0 0x55eb9fafa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fad11d0 0x55eb9fafa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57966/0 (socket says 172.21.15.78:57966) 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 -- 172.21.15.78:0/502724669 learned_addr learned my addr 172.21.15.78:0/502724669 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:55.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fafa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fadffc0 0x55eb9faf3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 -- 172.21.15.78:0/502724669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 msgr2=0x55eb9fafa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:55.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fafa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 -- 172.21.15.78:0/502724669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fadffc0 msgr2=0x55eb9faf3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fadffc0 0x55eb9faf3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 -- 172.21.15.78:0/502724669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9faf3f20 con 0x55eb9fad11d0 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fafa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb79ffff640 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fad11d0 0x55eb9fafa7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb79002a9d0 tx=0x7fb790004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52712db3bf43ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.881+0000 7fb77f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb77f7fe640 1 -- 172.21.15.78:0/502724669 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790035040 con 0x55eb9fad11d0 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb77f7fe640 1 -- 172.21.15.78:0/502724669 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fb790005450 con 0x55eb9fad11d0 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 1 -- 172.21.15.78:0/502724669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb794016030 con 0x55eb9fad11d0 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb77f7fe640 1 -- 172.21.15.78:0/502724669 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb790052050 con 0x55eb9fad11d0 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 10 client.15117 fetch_fsmap learned FSMap version 28 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 1 -- 172.21.15.78:0/502724669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb9faf88c0 con 0x55eb9fad11d0 2024-09-16T00:12:55.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb77f7fe640 1 -- 172.21.15.78:0/502724669 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb794016030 con 0x55eb9fad11d0 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb77f7fe640 1 -- 172.21.15.78:0/502724669 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb790031880 con 0x55eb9fad11d0 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 10 client.15117 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 20 client.15117 populate_metadata read hostname 'smithi078' 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55eb9fb5f6a0 0x55eb9fb5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 4 client.15117 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.882+0000 7fb7a86ea740 1 -- 172.21.15.78:0/502724669 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "status"}) -- 0x55eb9fa3eb10 con 0x55eb9fb5f6a0 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55eb9fb5f6a0 0x55eb9fb5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb77f7fe640 10 client.15117 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb79f7fe640 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55eb9fb5f6a0 0x55eb9fb5e3b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7e6e586b035ec63 server_cookie=2336721e0a22fe17 in_seq=0 out_seq=0 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb77f7fe640 1 -- 172.21.15.78:0/502724669 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55eb9fa3eb10 con 0x55eb9fb5f6a0 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb77f7fe640 10 client.15117 handle_command_reply: tid=0 2024-09-16T00:12:55.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 client.15117 shutdown 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 -- 172.21.15.78:0/502724669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fad11d0 msgr2=0x55eb9fafa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fad11d0 0x55eb9fafa7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb79002a9d0 tx=0x7fb790004540 comp rx=0 tx=0).stop 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 -- 172.21.15.78:0/502724669 shutdown_connections 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55eb9fb5f6a0 0x55eb9fb5e3b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb9fadffc0 0x55eb9faf3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb9fad11d0 0x55eb9fafa7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 --2- 172.21.15.78:0/502724669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb9fad08b0 0x55eb9fafa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.883+0000 7fb7a86ea740 1 -- 172.21.15.78:0/502724669 >> 172.21.15.78:0/502724669 conn(0x55eb9faccf60 msgr2=0x55eb9fac7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb77f7fe640 0 client.15117 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a86ea740 1 -- 172.21.15.78:0/502724669 shutdown_connections 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a86ea740 1 -- 172.21.15.78:0/502724669 wait complete. 2024-09-16T00:12:55.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a86ea740 20 client.15117 trim_cache size 0 max 16384 2024-09-16T00:12:55.902 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:12:55.902 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:12:55.902 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-16T00:12:55.902 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24470, 2024-09-16T00:12:55.902 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:12:55.903 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:12:55.903 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:12:55.903 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 110.25698126, 2024-09-16T00:12:55.903 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:12:55.903 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 88, 2024-09-16T00:12:55.903 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:12:55.903 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 122.050214576 2024-09-16T00:12:55.903 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:12:55.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 -- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7880a32d0 msgr2=0x7fb7880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7880a32d0 0x7fb7880a5790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb7980049d0 tx=0x7fb798005b40 comp rx=0 tx=0).stop 2024-09-16T00:12:55.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 -- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0106f20 msgr2=0x7fb7a019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:55.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0106f20 0x7fb7a019cd40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb79800eb10 tx=0x7fb79800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 -- 172.21.15.78:0/3475114794 shutdown_connections 2024-09-16T00:12:55.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7880a32d0 0x7fb7880a5790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7a0106f20 0x7fb7a019cd40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7a01027a0 0x7fb7a01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 --2- 172.21.15.78:0/3475114794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a0101df0 0x7fb7a0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:55.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 -- 172.21.15.78:0/3475114794 >> 172.21.15.78:0/3475114794 conn(0x7fb7a00fd8b0 msgr2=0x7fb7a00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:55.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 -- 172.21.15.78:0/3475114794 shutdown_connections 2024-09-16T00:12:55.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:55.884+0000 7fb7a6311640 1 -- 172.21.15.78:0/3475114794 wait complete. 2024-09-16T00:12:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:55 smithi193 ceph-mon[35484]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:55 smithi148 ceph-mon[35599]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:56.698 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T00:12:56.698 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:12:56.716 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-09-16T00:12:56.716 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T00:12:56.961 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 -- 172.21.15.78:0/1912525104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 msgr2=0x7f53a0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/1912525104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 0x7f53a0108f60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5398009be0 tx=0x7f539802a6c0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 -- 172.21.15.78:0/1912525104 shutdown_connections 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/1912525104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 0x7f53a0108f60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/1912525104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a0101d10 0x7f53a0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/1912525104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0100b10 0x7f53a0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 -- 172.21.15.78:0/1912525104 >> 172.21.15.78:0/1912525104 conn(0x7f53a00fc2a0 msgr2=0x7f53a00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 -- 172.21.15.78:0/1912525104 shutdown_connections 2024-09-16T00:12:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.504+0000 7f53a6e3e640 1 -- 172.21.15.78:0/1912525104 wait complete. 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a6e3e640 1 Processor -- start 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a6e3e640 1 -- start start 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a6e3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a0100b10 0x7f53a0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a6e3e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0101d10 0x7f53a01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a6e3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 0x7f53a019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a6e3e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53a010c720 con 0x7f53a01026d0 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a6e3e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53a010c5a0 con 0x7f53a0100b10 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a6e3e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53a010c8a0 con 0x7f53a0101d10 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 0x7f53a019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 0x7f53a019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60034/0 (socket says 172.21.15.78:60034) 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 -- 172.21.15.78:0/3989930580 learned_addr learned my addr 172.21.15.78:0/3989930580 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:57.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a4bda640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a0100b10 0x7f53a0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f539ffff640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0101d10 0x7f53a01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 -- 172.21.15.78:0/3989930580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0101d10 msgr2=0x7f53a01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0101d10 0x7f53a01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 -- 172.21.15.78:0/3989930580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a0100b10 msgr2=0x7f53a0197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a0100b10 0x7f53a0197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 -- 172.21.15.78:0/3989930580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a019fac0 con 0x7f53a01026d0 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f53a53db640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 0x7f53a019f450 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f53980305b0 tx=0x7f53980305e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.505+0000 7f539dffb640 1 -- 172.21.15.78:0/3989930580 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398041020 con 0x7f53a01026d0 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.506+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a019fd50 con 0x7f53a01026d0 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.506+0000 7f539dffb640 1 -- 172.21.15.78:0/3989930580 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5398037040 con 0x7f53a01026d0 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.506+0000 7f539dffb640 1 -- 172.21.15.78:0/3989930580 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398036070 con 0x7f53a01026d0 2024-09-16T00:12:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.506+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a01a01d0 con 0x7f53a01026d0 2024-09-16T00:12:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.507+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a01a04c0 con 0x7f53a01026d0 2024-09-16T00:12:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.508+0000 7f539dffb640 1 -- 172.21.15.78:0/3989930580 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5398044020 con 0x7f53a01026d0 2024-09-16T00:12:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.509+0000 7f539dffb640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53900a3300 0x7f53900a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.509+0000 7f53a4bda640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53900a3300 0x7f53900a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.509+0000 7f539dffb640 1 -- 172.21.15.78:0/3989930580 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f53980f0ac0 con 0x7f53a01026d0 2024-09-16T00:12:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.509+0000 7f53a4bda640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53900a3300 0x7f53900a57c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f53a0101b70 tx=0x7f5394009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.513+0000 7f539dffb640 1 -- 172.21.15.78:0/3989930580 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f539804b020 con 0x7f53a01026d0 2024-09-16T00:12:57.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.696+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f53a0062630 con 0x7f53900a3300 2024-09-16T00:12:57.697 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:57 smithi078 ceph-mon[31157]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:57.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.697+0000 7f539dffb640 1 -- 172.21.15.78:0/3989930580 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f53a0062630 con 0x7f53900a3300 2024-09-16T00:12:57.698 INFO:teuthology.orchestra.run.smithi078.stdout:Backend: cephadm 2024-09-16T00:12:57.698 INFO:teuthology.orchestra.run.smithi078.stdout:Available: Yes 2024-09-16T00:12:57.698 INFO:teuthology.orchestra.run.smithi078.stdout:Paused: No 2024-09-16T00:12:57.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.699+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53900a3300 msgr2=0x7f53900a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:57.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.699+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53900a3300 0x7f53900a57c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f53a0101b70 tx=0x7f5394009260 comp rx=0 tx=0).stop 2024-09-16T00:12:57.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.699+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 msgr2=0x7f53a019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.699+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 0x7f53a019f450 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f53980305b0 tx=0x7f53980305e0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.700+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 shutdown_connections 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.700+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53900a3300 0x7f53900a57c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.700+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53a01026d0 0x7f53a019f450 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.700+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53a0101d10 0x7f53a01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.700+0000 7f53a6e3e640 1 --2- 172.21.15.78:0/3989930580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a0100b10 0x7f53a0197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.700+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 >> 172.21.15.78:0/3989930580 conn(0x7f53a00fc2a0 msgr2=0x7f53a00fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.700+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 shutdown_connections 2024-09-16T00:12:57.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:57.700+0000 7f53a6e3e640 1 -- 172.21.15.78:0/3989930580 wait complete. 2024-09-16T00:12:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:57 smithi193 ceph-mon[35484]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:57 smithi148 ceph-mon[35599]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-16T00:12:58.475 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T00:12:58.723 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:12:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 -- 172.21.15.78:0/1941506232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c101cd0 msgr2=0x7f510c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 --2- 172.21.15.78:0/1941506232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c101cd0 0x7f510c102150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009900 tx=0x7f50fc02a410 comp rx=0 tx=0).stop 2024-09-16T00:12:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 -- 172.21.15.78:0/1941506232 shutdown_connections 2024-09-16T00:12:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 --2- 172.21.15.78:0/1941506232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f510c102690 0x7f510c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 --2- 172.21.15.78:0/1941506232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c101cd0 0x7f510c102150 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 --2- 172.21.15.78:0/1941506232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510c100ad0 0x7f510c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 -- 172.21.15.78:0/1941506232 >> 172.21.15.78:0/1941506232 conn(0x7f510c0fc280 msgr2=0x7f510c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:59.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 -- 172.21.15.78:0/1941506232 shutdown_connections 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 -- 172.21.15.78:0/1941506232 wait complete. 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.247+0000 7f51134b1640 1 Processor -- start 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 -- start start 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510c100ad0 0x7f510c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f510c101cd0 0x7f510c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c102690 0x7f510c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f510c10d960 con 0x7f510c102690 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f510c10d7e0 con 0x7f510c100ad0 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f510c10dae0 con 0x7f510c101cd0 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c102690 0x7f510c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c102690 0x7f510c19f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60044/0 (socket says 172.21.15.78:60044) 2024-09-16T00:12:59.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 -- 172.21.15.78:0/3093220420 learned_addr learned my addr 172.21.15.78:0/3093220420 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f511124d640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510c100ad0 0x7f510c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5110a4c640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f510c101cd0 0x7f510c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 -- 172.21.15.78:0/3093220420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f510c101cd0 msgr2=0x7f510c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f510c101cd0 0x7f510c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 -- 172.21.15.78:0/3093220420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510c100ad0 msgr2=0x7f510c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510c100ad0 0x7f510c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 -- 172.21.15.78:0/3093220420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c19fcd0 con 0x7f510c102690 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f511124d640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510c100ad0 0x7f510c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f5111a4e640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c102690 0x7f510c19f660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f510400b970 tx=0x7f510400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51027fc640 1 -- 172.21.15.78:0/3093220420 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104019070 con 0x7f510c102690 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51027fc640 1 -- 172.21.15.78:0/3093220420 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f510400cb40 con 0x7f510c102690 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c19ffc0 con 0x7f510c102690 2024-09-16T00:12:59.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51027fc640 1 -- 172.21.15.78:0/3093220420 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104007650 con 0x7f510c102690 2024-09-16T00:12:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.248+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1a04f0 con 0x7f510c102690 2024-09-16T00:12:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.250+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c107f60 con 0x7f510c102690 2024-09-16T00:12:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.251+0000 7f51027fc640 1 -- 172.21.15.78:0/3093220420 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f510401c070 con 0x7f510c102690 2024-09-16T00:12:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.251+0000 7f51027fc640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f50f80a3300 0x7f50f80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:12:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.251+0000 7f51027fc640 1 -- 172.21.15.78:0/3093220420 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f51040cf780 con 0x7f510c102690 2024-09-16T00:12:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.252+0000 7f511124d640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f50f80a3300 0x7f50f80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:12:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.252+0000 7f511124d640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f50f80a3300 0x7f50f80a57c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f510c101b30 tx=0x7f5108008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:12:59.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.255+0000 7f51027fc640 1 -- 172.21.15.78:0/3093220420 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f510409c110 con 0x7f510c102690 2024-09-16T00:12:59.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.429+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f510c062630 con 0x7f50f80a3300 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.441+0000 7f51027fc640 1 -- 172.21.15.78:0/3093220420 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f510c062630 con 0x7f50f80a3300 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:mds.a smithi078 running (2m) 58s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c 38e369dd5f43 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:mds.b smithi148 running (2m) 59s ago 2m 28.9M - 19.1.1-326-g8921c8ef f1455bd0219c 3899f630a044 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:mds.c smithi193 running (2m) 59s ago 2m 16.8M - 19.1.1-326-g8921c8ef f1455bd0219c f4a64c186697 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:mds.d smithi078 running (2m) 58s ago 2m 13.4M - 19.1.1-326-g8921c8ef f1455bd0219c 1d0226a243e8 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:mds.e smithi148 running (2m) 59s ago 2m 16.6M - 19.1.1-326-g8921c8ef f1455bd0219c 4052c5131330 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:mds.f smithi193 running (2m) 59s ago 2m 19.2M - 19.1.1-326-g8921c8ef f1455bd0219c 3e9c74b0cac8 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:mds.g smithi078 running (2m) 58s ago 2m 15.7M - 19.1.1-326-g8921c8ef f1455bd0219c 0560a45c8536 2024-09-16T00:12:59.442 INFO:teuthology.orchestra.run.smithi078.stdout:mds.h smithi148 running (2m) 59s ago 2m 17.2M - 19.1.1-326-g8921c8ef f1455bd0219c 2744b0beb612 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mds.i smithi193 running (2m) 59s ago 2m 16.2M - 19.1.1-326-g8921c8ef f1455bd0219c 39672d44980d 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mds.j smithi078 running (2m) 58s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c 980434cf6e6d 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mds.k smithi148 running (2m) 59s ago 2m 17.8M - 19.1.1-326-g8921c8ef f1455bd0219c 3b76efa29f97 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mds.l smithi193 running (2m) 59s ago 2m 14.8M - 19.1.1-326-g8921c8ef f1455bd0219c 6b6940ac814d 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.x smithi078 *:9283,8765 running (9m) 58s ago 9m 508M - 19.1.1-326-g8921c8ef f1455bd0219c f1c3878bc64b 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.y smithi148 *:8443,8765 running (7m) 59s ago 7m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 151c57f5ff20 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mgr.z smithi193 *:8443,8765 running (7m) 59s ago 7m 441M - 19.1.1-326-g8921c8ef f1455bd0219c e3d4387421df 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mon.a smithi078 running (9m) 58s ago 9m 50.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 16361528f3de 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mon.b smithi148 running (8m) 59s ago 8m 40.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 99365af9c893 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:mon.c smithi193 running (8m) 59s ago 8m 41.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 3076c2d30b7b 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 smithi078 running (7m) 58s ago 7m 45.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 6c698f6b413a 2024-09-16T00:12:59.443 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 smithi148 running (7m) 59s ago 7m 46.2M 4272M 19.1.1-326-g8921c8ef f1455bd0219c de6bab0137e0 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 smithi193 running (6m) 59s ago 6m 55.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b68b92a398e6 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 smithi078 running (6m) 58s ago 6m 46.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c e97064930bcb 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 smithi148 running (5m) 59s ago 5m 42.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 3e355e46559c 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 smithi193 running (5m) 59s ago 5m 53.2M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0727ae72ae9f 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 smithi078 running (5m) 58s ago 5m 42.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c a92f2939fed2 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 smithi148 running (4m) 59s ago 4m 44.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c d06d5b99e1c5 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 smithi193 running (4m) 59s ago 4m 44.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 4db43b5b1c28 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 smithi078 running (3m) 58s ago 3m 42.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5e239e9ceb47 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 smithi148 running (3m) 59s ago 3m 43.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 65a6c56fcc95 2024-09-16T00:12:59.444 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 smithi193 running (2m) 59s ago 2m 48.9M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b73a94c50e13 2024-09-16T00:12:59.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f50f80a3300 msgr2=0x7f50f80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:59.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f50f80a3300 0x7f50f80a57c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f510c101b30 tx=0x7f5108008040 comp rx=0 tx=0).stop 2024-09-16T00:12:59.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c102690 msgr2=0x7f510c19f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:12:59.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c102690 0x7f510c19f660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f510400b970 tx=0x7f510400be30 comp rx=0 tx=0).stop 2024-09-16T00:12:59.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 shutdown_connections 2024-09-16T00:12:59.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f50f80a3300 0x7f50f80a57c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f510c102690 0x7f510c19f660 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f510c101cd0 0x7f510c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 --2- 172.21.15.78:0/3093220420 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f510c100ad0 0x7f510c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:12:59.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 >> 172.21.15.78:0/3093220420 conn(0x7f510c0fc280 msgr2=0x7f510c105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:12:59.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 shutdown_connections 2024-09-16T00:12:59.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:12:59.443+0000 7f51134b1640 1 -- 172.21.15.78:0/3093220420 wait complete. 2024-09-16T00:12:59.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:59 smithi078 ceph-mon[31157]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:12:59.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:12:59 smithi078 ceph-mon[31157]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:59 smithi193 ceph-mon[35484]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:12:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:12:59 smithi193 ceph-mon[35484]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:12:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:59 smithi148 ceph-mon[35599]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:12:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:12:59 smithi148 ceph-mon[35599]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:00.236 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T00:13:00.480 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:00.916 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:00 smithi078 ceph-mon[31157]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:00.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:00 smithi193 ceph-mon[35484]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:00.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:00 smithi148 ceph-mon[35599]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:01.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.995+0000 7f3adb5d4640 1 -- 172.21.15.78:0/1570460603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4102690 msgr2=0x7f3ad4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:01.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.995+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/1570460603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4102690 0x7f3ad4108f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009ca0 tx=0x7f3acc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 -- 172.21.15.78:0/1570460603 shutdown_connections 2024-09-16T00:13:01.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/1570460603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4102690 0x7f3ad4108f20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/1570460603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad4101cd0 0x7f3ad4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/1570460603 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad4100ad0 0x7f3ad4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 -- 172.21.15.78:0/1570460603 >> 172.21.15.78:0/1570460603 conn(0x7f3ad40fc280 msgr2=0x7f3ad40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 -- 172.21.15.78:0/1570460603 shutdown_connections 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 -- 172.21.15.78:0/1570460603 wait complete. 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 Processor -- start 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 -- start start 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad4100ad0 0x7f3ad4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4101cd0 0x7f3ad41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad4102690 0x7f3ad419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad410c750 con 0x7f3ad4101cd0 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad410c5d0 con 0x7f3ad4102690 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3adb5d4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad410c8d0 con 0x7f3ad4100ad0 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.996+0000 7f3ad8b6f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4101cd0 0x7f3ad41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:01.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad8b6f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4101cd0 0x7f3ad41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60050/0 (socket says 172.21.15.78:60050) 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad8b6f640 1 -- 172.21.15.78:0/791064141 learned_addr learned my addr 172.21.15.78:0/791064141 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad9370640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad4100ad0 0x7f3ad4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad9b71640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad4102690 0x7f3ad419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad8b6f640 1 -- 172.21.15.78:0/791064141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad4100ad0 msgr2=0x7f3ad4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad8b6f640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad4100ad0 0x7f3ad4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad8b6f640 1 -- 172.21.15.78:0/791064141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad4102690 msgr2=0x7f3ad419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad8b6f640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad4102690 0x7f3ad419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad8b6f640 1 -- 172.21.15.78:0/791064141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad4069ff0 con 0x7f3ad4101cd0 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad9370640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad4100ad0 0x7f3ad4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad9b71640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad4102690 0x7f3ad419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3ad8b6f640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4101cd0 0x7f3ad41972d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400d770 tx=0x7f3ac400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3aca7fc640 1 -- 172.21.15.78:0/791064141 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4017070 con 0x7f3ad4101cd0 2024-09-16T00:13:01.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad406a2e0 con 0x7f3ad4101cd0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3aca7fc640 1 -- 172.21.15.78:0/791064141 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac4005500 con 0x7f3ad4101cd0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3aca7fc640 1 -- 172.21.15.78:0/791064141 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4004e00 con 0x7f3ad4101cd0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.997+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad406a7d0 con 0x7f3ad4101cd0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.998+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad4107e50 con 0x7f3ad4101cd0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:00.999+0000 7f3aca7fc640 1 -- 172.21.15.78:0/791064141 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3ac4004fa0 con 0x7f3ad4101cd0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.000+0000 7f3aca7fc640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac00a32d0 0x7f3ac00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.000+0000 7f3aca7fc640 1 -- 172.21.15.78:0/791064141 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7f3ac40cf910 con 0x7f3ad4101cd0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.000+0000 7f3ad9370640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac00a32d0 0x7f3ac00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.000+0000 7f3ad9370640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac00a32d0 0x7f3ac00a5790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005fd0 tx=0x7f3ad0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:01.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.004+0000 7f3aca7fc640 1 -- 172.21.15.78:0/791064141 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ac409c190 con 0x7f3ad4101cd0 2024-09-16T00:13:01.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.180+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f3ad406aaa0 con 0x7f3ac00a32d0 2024-09-16T00:13:01.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.185+0000 7f3aca7fc640 1 -- 172.21.15.78:0/791064141 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f3ad406aaa0 con 0x7f3ac00a32d0 2024-09-16T00:13:01.186 INFO:teuthology.orchestra.run.smithi078.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T00:13:01.187 INFO:teuthology.orchestra.run.smithi078.stdout:mds.all 12/12 61s ago 2m smithi078=a;smithi078=d;smithi078=g;smithi078=j;smithi148=b;smithi148=e;smithi148=h;smithi148=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-16T00:13:01.187 INFO:teuthology.orchestra.run.smithi078.stdout:mgr 3/3 61s ago 8m smithi078=x;smithi148=y;smithi193=z;count:3 2024-09-16T00:13:01.187 INFO:teuthology.orchestra.run.smithi078.stdout:mon 3/3 61s ago 8m smithi078:172.21.15.78=a;smithi148:172.21.15.148=b;smithi193:172.21.15.193=c;count:3 2024-09-16T00:13:01.187 INFO:teuthology.orchestra.run.smithi078.stdout:osd 12 61s ago - 2024-09-16T00:13:01.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac00a32d0 msgr2=0x7f3ac00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:01.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac00a32d0 0x7f3ac00a5790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0005fd0 tx=0x7f3ad0005850 comp rx=0 tx=0).stop 2024-09-16T00:13:01.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4101cd0 msgr2=0x7f3ad41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:01.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4101cd0 0x7f3ad41972d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400d770 tx=0x7f3ac400dc30 comp rx=0 tx=0).stop 2024-09-16T00:13:01.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 shutdown_connections 2024-09-16T00:13:01.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac00a32d0 0x7f3ac00a5790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ad4102690 0x7f3ad419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ad4101cd0 0x7f3ad41972d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 --2- 172.21.15.78:0/791064141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ad4100ad0 0x7f3ad4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:01.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 >> 172.21.15.78:0/791064141 conn(0x7f3ad40fc280 msgr2=0x7f3ad4105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:01.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 shutdown_connections 2024-09-16T00:13:01.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:01.188+0000 7f3adb5d4640 1 -- 172.21.15.78:0/791064141 wait complete. 2024-09-16T00:13:01.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:01 smithi078 ceph-mon[31157]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:01 smithi193 ceph-mon[35484]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:01 smithi148 ceph-mon[35599]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:01.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T00:13:02.247 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:02.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:02 smithi078 ceph-mon[31157]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:02.747 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:02 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:13:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:02 smithi193 ceph-mon[35484]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:02.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:02 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:13:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:02 smithi148 ceph-mon[35599]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:02 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:13:03.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 -- 172.21.15.78:0/2830563888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08102690 msgr2=0x7fcb08108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:03.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 --2- 172.21.15.78:0/2830563888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08102690 0x7fcb08108f20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009ca0 tx=0x7fcb0002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 -- 172.21.15.78:0/2830563888 shutdown_connections 2024-09-16T00:13:03.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 --2- 172.21.15.78:0/2830563888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08102690 0x7fcb08108f20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 --2- 172.21.15.78:0/2830563888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb08101cd0 0x7fcb08102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 --2- 172.21.15.78:0/2830563888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08100ad0 0x7fcb08100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 -- 172.21.15.78:0/2830563888 >> 172.21.15.78:0/2830563888 conn(0x7fcb080fc280 msgr2=0x7fcb080fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 -- 172.21.15.78:0/2830563888 shutdown_connections 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.093+0000 7fcb0e526640 1 -- 172.21.15.78:0/2830563888 wait complete. 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0e526640 1 Processor -- start 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0e526640 1 -- start start 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0e526640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08100ad0 0x7fcb0819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0e526640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08101cd0 0x7fcb0819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0e526640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb08102690 0x7fcb081a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0e526640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0810d880 con 0x7fcb08100ad0 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0e526640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0810d700 con 0x7fcb08101cd0 2024-09-16T00:13:03.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0e526640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0810da00 con 0x7fcb08102690 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08100ad0 0x7fcb0819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08100ad0 0x7fcb0819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60064/0 (socket says 172.21.15.78:60064) 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 -- 172.21.15.78:0/112171694 learned_addr learned my addr 172.21.15.78:0/112171694 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcafffff640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08101cd0 0x7fcb0819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0cac3640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb08102690 0x7fcb081a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 -- 172.21.15.78:0/112171694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb08102690 msgr2=0x7fcb081a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb08102690 0x7fcb081a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 -- 172.21.15.78:0/112171694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08101cd0 msgr2=0x7fcb0819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08101cd0 0x7fcb0819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 -- 172.21.15.78:0/112171694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb081a1cc0 con 0x7fcb08100ad0 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb0cac3640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb08102690 0x7fcb081a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.094+0000 7fcb07fff640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08100ad0 0x7fcb0819a090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800d770 tx=0x7fcaf800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:03.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.095+0000 7fcb05ffb640 1 -- 172.21.15.78:0/112171694 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8017070 con 0x7fcb08100ad0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.095+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb081a1fb0 con 0x7fcb08100ad0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.095+0000 7fcb05ffb640 1 -- 172.21.15.78:0/112171694 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf8009c30 con 0x7fcb08100ad0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.095+0000 7fcb05ffb640 1 -- 172.21.15.78:0/112171694 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8003dd0 con 0x7fcb08100ad0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.095+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb081a2400 con 0x7fcb08100ad0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.096+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb081a2660 con 0x7fcb08100ad0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.097+0000 7fcb05ffb640 1 -- 172.21.15.78:0/112171694 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fcaf8003f70 con 0x7fcb08100ad0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.098+0000 7fcb05ffb640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf40a3270 0x7fcaf40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.098+0000 7fcafffff640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf40a3270 0x7fcaf40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.098+0000 7fcb05ffb640 1 -- 172.21.15.78:0/112171694 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9071+0+0 (secure 0 0 0) 0x7fcaf80cf9e0 con 0x7fcb08100ad0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.098+0000 7fcafffff640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf40a3270 0x7fcaf40a5730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb0819b5b0 tx=0x7fcaf000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:03.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.101+0000 7fcb05ffb640 1 -- 172.21.15.78:0/112171694 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcaf8009de0 con 0x7fcb08100ad0 2024-09-16T00:13:03.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.277+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fcb0810b000 con 0x7fcaf40a3270 2024-09-16T00:13:03.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.278+0000 7fcb05ffb640 1 -- 172.21.15.78:0/112171694 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fcb0810b000 con 0x7fcaf40a3270 2024-09-16T00:13:03.279 INFO:teuthology.orchestra.run.smithi078.stdout:HOST ADDR LABELS STATUS 2024-09-16T00:13:03.279 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 172.21.15.78 2024-09-16T00:13:03.279 INFO:teuthology.orchestra.run.smithi078.stdout:smithi148 172.21.15.148 2024-09-16T00:13:03.279 INFO:teuthology.orchestra.run.smithi078.stdout:smithi193 172.21.15.193 2024-09-16T00:13:03.279 INFO:teuthology.orchestra.run.smithi078.stdout:3 hosts in cluster 2024-09-16T00:13:03.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf40a3270 msgr2=0x7fcaf40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:03.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf40a3270 0x7fcaf40a5730 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcb0819b5b0 tx=0x7fcaf000b410 comp rx=0 tx=0).stop 2024-09-16T00:13:03.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08100ad0 msgr2=0x7fcb0819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:03.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08100ad0 0x7fcb0819a090 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800d770 tx=0x7fcaf800dc30 comp rx=0 tx=0).stop 2024-09-16T00:13:03.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 shutdown_connections 2024-09-16T00:13:03.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf40a3270 0x7fcaf40a5730 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb08102690 0x7fcb081a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb08101cd0 0x7fcb0819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 --2- 172.21.15.78:0/112171694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb08100ad0 0x7fcb0819a090 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:03.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 >> 172.21.15.78:0/112171694 conn(0x7fcb080fc280 msgr2=0x7fcb081098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:03.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 shutdown_connections 2024-09-16T00:13:03.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:03.280+0000 7fcb0e526640 1 -- 172.21.15.78:0/112171694 wait complete. 2024-09-16T00:13:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:03 smithi078 ceph-mon[31157]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:13:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:03 smithi078 ceph-mon[31157]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:13:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-16T00:13:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.606 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.607 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:03 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:03 smithi193 ceph-mon[35484]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:13:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:03 smithi193 ceph-mon[35484]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:13:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-16T00:13:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:03 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:03 smithi148 ceph-mon[35599]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:13:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:03 smithi148 ceph-mon[35599]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:13:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-16T00:13:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:03.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:03 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:04.186 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T00:13:04.454 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:04.921 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:04 smithi078 ceph-mon[31157]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:04.921 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:04 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-16T00:13:04.921 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:04 smithi078 ceph-mon[31157]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:13:04.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:04 smithi193 ceph-mon[35484]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:04.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:04 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-16T00:13:04.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:04 smithi193 ceph-mon[35484]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:13:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:04 smithi148 ceph-mon[35599]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:04 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-16T00:13:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:04 smithi148 ceph-mon[35599]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:13:05.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 -- 172.21.15.78:0/1506601542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac102ef0 msgr2=0x7f55ac103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 --2- 172.21.15.78:0/1506601542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac102ef0 0x7f55ac103370 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5594009930 tx=0x7f559402a420 comp rx=0 tx=0).stop 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 -- 172.21.15.78:0/1506601542 shutdown_connections 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 --2- 172.21.15.78:0/1506601542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ac1038b0 0x7f55ac10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 --2- 172.21.15.78:0/1506601542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac102ef0 0x7f55ac103370 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 --2- 172.21.15.78:0/1506601542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ac0fff10 0x7f55ac100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 -- 172.21.15.78:0/1506601542 >> 172.21.15.78:0/1506601542 conn(0x7f55ac0fba80 msgr2=0x7f55ac0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 -- 172.21.15.78:0/1506601542 shutdown_connections 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.006+0000 7f55b17f6640 1 -- 172.21.15.78:0/1506601542 wait complete. 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55b17f6640 1 Processor -- start 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55b17f6640 1 -- start start 2024-09-16T00:13:05.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55b17f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ac0fff10 0x7f55ac1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55b17f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ac102ef0 0x7f55ac1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55b17f6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac1038b0 0x7f55ac19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55b17f6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac10d9d0 con 0x7f55ac1038b0 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55b17f6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac10d850 con 0x7f55ac0fff10 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55b17f6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac10db50 con 0x7f55ac102ef0 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55ab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac1038b0 0x7f55ac19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55ab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac1038b0 0x7f55ac19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60090/0 (socket says 172.21.15.78:60090) 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.007+0000 7f55ab7fe640 1 -- 172.21.15.78:0/2782339774 learned_addr learned my addr 172.21.15.78:0/2782339774 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55a3fff640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ac102ef0 0x7f55ac1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55aaffd640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ac0fff10 0x7f55ac1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55ab7fe640 1 -- 172.21.15.78:0/2782339774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ac102ef0 msgr2=0x7f55ac1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55ab7fe640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ac102ef0 0x7f55ac1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55ab7fe640 1 -- 172.21.15.78:0/2782339774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ac0fff10 msgr2=0x7f55ac1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55ab7fe640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ac0fff10 0x7f55ac1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55ab7fe640 1 -- 172.21.15.78:0/2782339774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac19fce0 con 0x7f55ac1038b0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55ab7fe640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac1038b0 0x7f55ac19f670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f55a400b970 tx=0x7f55a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55a8ff9640 1 -- 172.21.15.78:0/2782339774 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4019070 con 0x7f55ac1038b0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55a8ff9640 1 -- 172.21.15.78:0/2782339774 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55a400cb40 con 0x7f55ac1038b0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ac19ffd0 con 0x7f55ac1038b0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.008+0000 7f55a8ff9640 1 -- 172.21.15.78:0/2782339774 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4007650 con 0x7f55ac1038b0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.009+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ac1a04b0 con 0x7f55ac1038b0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.010+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ac109180 con 0x7f55ac1038b0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.010+0000 7f55a8ff9640 1 -- 172.21.15.78:0/2782339774 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f55a401c070 con 0x7f55ac1038b0 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.011+0000 7f55a8ff9640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55980a3270 0x7f55980a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:05.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.011+0000 7f55a8ff9640 1 -- 172.21.15.78:0/2782339774 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f55a40cf270 con 0x7f55ac1038b0 2024-09-16T00:13:05.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.011+0000 7f55aaffd640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55980a3270 0x7f55980a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:05.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.012+0000 7f55aaffd640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55980a3270 0x7f55980a5730 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55ac069d70 tx=0x7f559c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:05.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.016+0000 7f55a8ff9640 1 -- 172.21.15.78:0/2782339774 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55a409bc30 con 0x7f55ac1038b0 2024-09-16T00:13:05.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.198+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f55ac062630 con 0x7f55980a3270 2024-09-16T00:13:05.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.200+0000 7f55a8ff9640 1 -- 172.21.15.78:0/2782339774 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f55ac062630 con 0x7f55980a3270 2024-09-16T00:13:05.201 INFO:teuthology.orchestra.run.smithi078.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T00:13:05.201 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T00:13:05.201 INFO:teuthology.orchestra.run.smithi078.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T00:13:05.201 INFO:teuthology.orchestra.run.smithi078.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T00:13:05.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.202+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55980a3270 msgr2=0x7f55980a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:05.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.202+0000 7f55b17f6640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55980a3270 0x7f55980a5730 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55ac069d70 tx=0x7f559c008040 comp rx=0 tx=0).stop 2024-09-16T00:13:05.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.202+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac1038b0 msgr2=0x7f55ac19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.202+0000 7f55b17f6640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac1038b0 0x7f55ac19f670 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f55a400b970 tx=0x7f55a400be30 comp rx=0 tx=0).stop 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.203+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 shutdown_connections 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.203+0000 7f55b17f6640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55980a3270 0x7f55980a5730 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.203+0000 7f55b17f6640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55ac1038b0 0x7f55ac19f670 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.203+0000 7f55b17f6640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ac102ef0 0x7f55ac1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.203+0000 7f55b17f6640 1 --2- 172.21.15.78:0/2782339774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55ac0fff10 0x7f55ac1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.203+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 >> 172.21.15.78:0/2782339774 conn(0x7f55ac0fba80 msgr2=0x7f55ac106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.203+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 shutdown_connections 2024-09-16T00:13:05.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:05.203+0000 7f55b17f6640 1 -- 172.21.15.78:0/2782339774 wait complete. 2024-09-16T00:13:05.881 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:05 smithi078 ceph-mon[31157]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:13:05.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:05 smithi078 ceph-mon[31157]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:13:05.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:05.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:05.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:13:05.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:13:05.882 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:05.909 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:13:05.918 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-09-16T00:13:05.918 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T00:13:05.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:05 smithi193 ceph-mon[35484]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:13:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:05 smithi193 ceph-mon[35484]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:13:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:13:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:13:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:05 smithi148 ceph-mon[35599]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-16T00:13:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:05 smithi148 ceph-mon[35599]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:13:05.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:05.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:05.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:13:05.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:13:05.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:13:06.163 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:06.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 -- 172.21.15.78:0/3000765858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 msgr2=0x7f04d8104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:06.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 --2- 172.21.15.78:0/3000765858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 0x7f04d8104a80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009ca0 tx=0x7f04c802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 -- 172.21.15.78:0/3000765858 shutdown_connections 2024-09-16T00:13:06.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 --2- 172.21.15.78:0/3000765858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d81078f0 0x7f04d8109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 --2- 172.21.15.78:0/3000765858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04d8104fc0 0x7f04d81073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 --2- 172.21.15.78:0/3000765858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 0x7f04d8104a80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 -- 172.21.15.78:0/3000765858 >> 172.21.15.78:0/3000765858 conn(0x7f04d80fc290 msgr2=0x7f04d80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:06.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 -- 172.21.15.78:0/3000765858 shutdown_connections 2024-09-16T00:13:06.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.658+0000 7f04df287640 1 -- 172.21.15.78:0/3000765858 wait complete. 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04df287640 1 Processor -- start 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04df287640 1 -- start start 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04df287640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 0x7f04d8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04df287640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04d8104fc0 0x7f04d81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04df287640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d81078f0 0x7f04d819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04df287640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810d980 con 0x7f04d8102690 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04df287640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810d800 con 0x7f04d8104fc0 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04df287640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04d810db00 con 0x7f04d81078f0 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04dd023640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 0x7f04d8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04dd023640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 0x7f04d8198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47948/0 (socket says 172.21.15.78:47948) 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.659+0000 7f04dd023640 1 -- 172.21.15.78:0/976535294 learned_addr learned my addr 172.21.15.78:0/976535294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dd824640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d81078f0 0x7f04d819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:06.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dd023640 1 -- 172.21.15.78:0/976535294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d81078f0 msgr2=0x7f04d819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dc822640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04d8104fc0 0x7f04d81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dd023640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d81078f0 0x7f04d819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dd023640 1 -- 172.21.15.78:0/976535294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04d8104fc0 msgr2=0x7f04d81985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dd023640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04d8104fc0 0x7f04d81985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dd023640 1 -- 172.21.15.78:0/976535294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d819fcc0 con 0x7f04d8102690 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dd824640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d81078f0 0x7f04d819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dc822640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04d8104fc0 0x7f04d81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04dd023640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 0x7f04d8198090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04c8031a40 tx=0x7f04c802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04d67fc640 1 -- 172.21.15.78:0/976535294 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c802fdc0 con 0x7f04d8102690 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d819ff50 con 0x7f04d8102690 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81a0480 con 0x7f04d8102690 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04d67fc640 1 -- 172.21.15.78:0/976535294 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c8030aa0 con 0x7f04d8102690 2024-09-16T00:13:06.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.660+0000 7f04d67fc640 1 -- 172.21.15.78:0/976535294 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8046a10 con 0x7f04d8102690 2024-09-16T00:13:06.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.661+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c40052c0 con 0x7f04d8102690 2024-09-16T00:13:06.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.662+0000 7f04d67fc640 1 -- 172.21.15.78:0/976535294 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f04c8030c50 con 0x7f04d8102690 2024-09-16T00:13:06.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.663+0000 7f04d67fc640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04c00a3270 0x7f04c00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:06.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.663+0000 7f04dc822640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04c00a3270 0x7f04c00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:06.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.663+0000 7f04d67fc640 1 -- 172.21.15.78:0/976535294 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f04c80f1580 con 0x7f04d8102690 2024-09-16T00:13:06.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.663+0000 7f04dc822640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04c00a3270 0x7f04c00a5730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006fd0 tx=0x7f04cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:06.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.666+0000 7f04d67fc640 1 -- 172.21.15.78:0/976535294 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04c80bdf40 con 0x7f04d8102690 2024-09-16T00:13:06.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.866+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f04c4005880 con 0x7f04d8102690 2024-09-16T00:13:06.867 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:06 smithi078 ceph-mon[31157]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:06.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.866+0000 7f04d67fc640 1 -- 172.21.15.78:0/976535294 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3138 (secure 0 0 0) 0x7f04c80bd840 con 0x7f04d8102690 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:e28 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:btime 2024-09-16T00:11:44:544077+0000 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:legacy client fscid: 1 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:fs_name cephfs 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 28 2024-09-16T00:13:06.868 INFO:teuthology.orchestra.run.smithi078.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-09-16T00:11:05.234467+0000 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-09-16T00:11:44.262714+0000 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:tableserver 0 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:root 0 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:session_timeout 300 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:session_autoclose 300 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:max_file_size 1099511627776 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:max_xattr_size 65536 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:required_client_features {} 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure 0 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:last_failure_osd_epoch 0 2024-09-16T00:13:06.869 INFO:teuthology.orchestra.run.smithi078.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:max_mds 5 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:in 0,1,2,3,4 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:up {0=24470,1=24445,2=24440,3=24427,4=24421} 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:failed 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:damaged 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:stopped 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:data_pools [3,4] 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:metadata_pool 2 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:inline_data disabled 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:balancer 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:bal_rank_mask -1 2024-09-16T00:13:06.870 INFO:teuthology.orchestra.run.smithi078.stdout:standby_count_wanted 0 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:qdb_cluster leader: 24470 members: 24421,24427,24440,24445,24470 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b{0:24470} state up:active seq 4 addr [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.e{0:24416} state up:standby-replay seq 1 addr [v2:172.21.15.148:6834/903408403,v1:172.21.15.148:6835/903408403] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.i{1:24445} state up:active seq 15 addr [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.k{1:24398} state up:standby-replay seq 1 addr [v2:172.21.15.148:6832/3203333506,v1:172.21.15.148:6833/3203333506] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.h{2:24440} state up:active seq 16 addr [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.l{2:24391} state up:standby-replay seq 1 addr [v2:172.21.15.193:6832/1629662299,v1:172.21.15.193:6833/1629662299] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c{3:24427} state up:active seq 17 addr [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a{3:14583} state up:standby-replay seq 1 addr [v2:172.21.15.78:6840/1822173762,v1:172.21.15.78:6841/1822173762] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.f{4:24421} state up:active seq 18 addr [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.871 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.g{4:14571} state up:standby-replay seq 1 addr [v2:172.21.15.78:6838/1764484583,v1:172.21.15.78:6839/1764484583] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.872 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:06.872 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:06.872 INFO:teuthology.orchestra.run.smithi078.stdout:Standby daemons: 2024-09-16T00:13:06.872 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:06.872 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.j{-1:14505} state up:standby seq 1 addr [v2:172.21.15.78:6834/4001113137,v1:172.21.15.78:6835/4001113137] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.872 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d{-1:14517} state up:standby seq 1 addr [v2:172.21.15.78:6836/3192121522,v1:172.21.15.78:6837/3192121522] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:13:06.873 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-16T00:13:06.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04c00a3270 msgr2=0x7f04c00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:06.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04c00a3270 0x7f04c00a5730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f04cc006fd0 tx=0x7f04cc008040 comp rx=0 tx=0).stop 2024-09-16T00:13:06.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 msgr2=0x7f04d8198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:06.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 0x7f04d8198090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04c8031a40 tx=0x7f04c802f840 comp rx=0 tx=0).stop 2024-09-16T00:13:06.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 shutdown_connections 2024-09-16T00:13:06.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04c00a3270 0x7f04c00a5730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d81078f0 0x7f04d819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04d8104fc0 0x7f04d81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 --2- 172.21.15.78:0/976535294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04d8102690 0x7f04d8198090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:06.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 >> 172.21.15.78:0/976535294 conn(0x7f04d80fc290 msgr2=0x7f04d8103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:06.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 shutdown_connections 2024-09-16T00:13:06.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:06.869+0000 7f04df287640 1 -- 172.21.15.78:0/976535294 wait complete. 2024-09-16T00:13:06.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:06 smithi193 ceph-mon[35484]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:06.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:06 smithi148 ceph-mon[35599]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:07.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-16T00:13:07.836 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:07 smithi193 ceph-mon[35484]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 12 op/s 2024-09-16T00:13:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:07 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/976535294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:13:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:07 smithi148 ceph-mon[35599]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 12 op/s 2024-09-16T00:13:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:07 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/976535294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:13:07.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:07 smithi078 ceph-mon[31157]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 12 op/s 2024-09-16T00:13:07.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:07 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/976535294' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:13:08.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 -- 172.21.15.78:0/476590645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24441037c0 msgr2=0x7f244410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:08.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 --2- 172.21.15.78:0/476590645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24441037c0 0x7f244410a050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f243c009ca0 tx=0x7f243c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 -- 172.21.15.78:0/476590645 shutdown_connections 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 --2- 172.21.15.78:0/476590645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24441037c0 0x7f244410a050 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 --2- 172.21.15.78:0/476590645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2444102e00 0x7f2444103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 --2- 172.21.15.78:0/476590645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2444101c00 0x7f2444102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 -- 172.21.15.78:0/476590645 >> 172.21.15.78:0/476590645 conn(0x7f24440fd3b0 msgr2=0x7f24440ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 -- 172.21.15.78:0/476590645 shutdown_connections 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 -- 172.21.15.78:0/476590645 wait complete. 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.391+0000 7f244b655640 1 Processor -- start 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f244b655640 1 -- start start 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f244b655640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2444101c00 0x7f2444197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f244b655640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2444102e00 0x7f2444198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f244b655640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24441037c0 0x7f244419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:08.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f244b655640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f244410d900 con 0x7f2444101c00 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f244b655640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f244410d780 con 0x7f2444102e00 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f244b655640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f244410da80 con 0x7f24441037c0 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2444101c00 0x7f2444197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2444101c00 0x7f2444197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47966/0 (socket says 172.21.15.78:47966) 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 -- 172.21.15.78:0/198292397 learned_addr learned my addr 172.21.15.78:0/198292397 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f2448bf0640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2444102e00 0x7f2444198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f2449bf2640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24441037c0 0x7f244419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 -- 172.21.15.78:0/198292397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24441037c0 msgr2=0x7f244419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24441037c0 0x7f244419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 -- 172.21.15.78:0/198292397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2444102e00 msgr2=0x7f2444198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2444102e00 0x7f2444198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 -- 172.21.15.78:0/198292397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244419fb20 con 0x7f2444101c00 2024-09-16T00:13:08.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f2449bf2640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24441037c0 0x7f244419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.392+0000 7f24493f1640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2444101c00 0x7f2444197ef0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f243400d950 tx=0x7f243400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.393+0000 7f243a7fc640 1 -- 172.21.15.78:0/198292397 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434017070 con 0x7f2444101c00 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.393+0000 7f243a7fc640 1 -- 172.21.15.78:0/198292397 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2434009c30 con 0x7f2444101c00 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.393+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244419fe10 con 0x7f2444101c00 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.393+0000 7f243a7fc640 1 -- 172.21.15.78:0/198292397 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243400f600 con 0x7f2444101c00 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.393+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24441a02e0 con 0x7f2444101c00 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.394+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2444108f30 con 0x7f2444101c00 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.395+0000 7f243a7fc640 1 -- 172.21.15.78:0/198292397 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f243401a070 con 0x7f2444101c00 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.395+0000 7f243a7fc640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24300a32f0 0x7f24300a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.395+0000 7f2448bf0640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24300a32f0 0x7f24300a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.396+0000 7f243a7fc640 1 -- 172.21.15.78:0/198292397 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f24340cf6a0 con 0x7f2444101c00 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.396+0000 7f2448bf0640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24300a32f0 0x7f24300a57b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2444199410 tx=0x7f2440008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:08.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.399+0000 7f243a7fc640 1 -- 172.21.15.78:0/198292397 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f243409c0e0 con 0x7f2444101c00 2024-09-16T00:13:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.573+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f244402da10 con 0x7f2444101c00 2024-09-16T00:13:08.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.574+0000 7f243a7fc640 1 -- 172.21.15.78:0/198292397 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5602 (secure 0 0 0) 0x7f243409b9e0 con 0x7f2444101c00 2024-09-16T00:13:08.575 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 90 2024-09-16T00:13:08.575 INFO:teuthology.orchestra.run.smithi078.stdout:fsid eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:13:08.575 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-09-16T00:03:16.287057+0000 2024-09-16T00:13:08.575 INFO:teuthology.orchestra.run.smithi078.stdout:modified 2024-09-16T00:13:04.600749+0000 2024-09-16T00:13:08.575 INFO:teuthology.orchestra.run.smithi078.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T00:13:08.575 INFO:teuthology.orchestra.run.smithi078.stdout:crush_version 27 2024-09-16T00:13:08.575 INFO:teuthology.orchestra.run.smithi078.stdout:full_ratio 0.95 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:backfillfull_ratio 0.9 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:nearfull_ratio 0.85 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:require_min_compat_client luminous 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:min_compat_client luminous 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:require_osd_release squid 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:stretch_mode_enabled false 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:max_osd 12 2024-09-16T00:13:08.576 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6802/1264645608,v1:172.21.15.78:6803/1264645608] [v2:172.21.15.78:6804/1264645608,v1:172.21.15.78:6805/1264645608] exists,up 03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/2952692564,v1:172.21.15.148:6801/2952692564] [v2:172.21.15.148:6802/2952692564,v1:172.21.15.148:6803/2952692564] exists,up c2638a47-f838-4300-848b-e731ac3cce78 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/2579590991,v1:172.21.15.193:6801/2579590991] [v2:172.21.15.193:6802/2579590991,v1:172.21.15.193:6803/2579590991] exists,up 863c25a6-76c4-4097-9c6f-20e838ddb3e9 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 up in weight 1 up_from 24 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6810/2314919115,v1:172.21.15.78:6811/2314919115] [v2:172.21.15.78:6812/2314919115,v1:172.21.15.78:6813/2314919115] exists,up f34ec822-59f0-44e6-a33b-ae877c648c67 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 up in weight 1 up_from 31 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/336734377,v1:172.21.15.148:6809/336734377] [v2:172.21.15.148:6810/336734377,v1:172.21.15.148:6811/336734377] exists,up 928fd68c-669c-4de2-aa53-11d764c071ed 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 up in weight 1 up_from 36 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/2332816249,v1:172.21.15.193:6809/2332816249] [v2:172.21.15.193:6810/2332816249,v1:172.21.15.193:6811/2332816249] exists,up 538b1b72-96f3-41ed-bfc6-9796b5ae0578 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 up in weight 1 up_from 43 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6818/445240021,v1:172.21.15.78:6819/445240021] [v2:172.21.15.78:6820/445240021,v1:172.21.15.78:6821/445240021] exists,up d19221b9-f4f0-42ca-ae31-7b1935b52a50 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 up in weight 1 up_from 48 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/155815362,v1:172.21.15.148:6817/155815362] [v2:172.21.15.148:6818/155815362,v1:172.21.15.148:6819/155815362] exists,up c7efa41b-836b-419a-b2c0-8e2861cddeaa 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.8 up in weight 1 up_from 55 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/3026218583,v1:172.21.15.193:6817/3026218583] [v2:172.21.15.193:6818/3026218583,v1:172.21.15.193:6819/3026218583] exists,up f8e53741-a657-48d0-a667-fb8a28f6f922 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.9 up in weight 1 up_from 62 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6826/1947247517,v1:172.21.15.78:6827/1947247517] [v2:172.21.15.78:6828/1947247517,v1:172.21.15.78:6829/1947247517] exists,up 7e15eca2-aea8-4693-807d-bfa2b2439c48 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/1827220524,v1:172.21.15.148:6825/1827220524] [v2:172.21.15.148:6826/1827220524,v1:172.21.15.148:6827/1827220524] exists,up 92a86462-ac20-4b68-b8eb-d18ba5c604b3 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:osd.11 up in weight 1 up_from 74 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/872081348,v1:172.21.15.193:6825/872081348] [v2:172.21.15.193:6826/872081348,v1:172.21.15.193:6827/872081348] exists,up 41c5e26a-ddcf-4933-95f1-5c868521acfd 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.8 [2,11] 2024-09-16T00:13:08.577 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 2.b [8,11] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.1 [4,9] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.23 [4,8] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.24 [4,9] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.27 [3,9] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 3.39 [5,6] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.6 [9,11] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.e [9,1,5,8] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:pg_upmap_items 4.19 [9,8] 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/1052494441 expires 2024-09-17T00:04:02.295421+0000 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/3861325161 expires 2024-09-17T00:04:02.295421+0000 2024-09-16T00:13:08.578 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1335742179 expires 2024-09-17T00:03:37.955119+0000 2024-09-16T00:13:08.579 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/3957981561 expires 2024-09-17T00:03:37.955119+0000 2024-09-16T00:13:08.579 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/2949703267 expires 2024-09-17T00:03:37.955119+0000 2024-09-16T00:13:08.579 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/1052494441 expires 2024-09-17T00:04:02.295421+0000 2024-09-16T00:13:08.579 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6801/1515675480 expires 2024-09-17T00:03:37.955119+0000 2024-09-16T00:13:08.579 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/486633606 expires 2024-09-17T00:04:02.295421+0000 2024-09-16T00:13:08.579 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:0/1597437728 expires 2024-09-17T00:04:02.295421+0000 2024-09-16T00:13:08.579 INFO:teuthology.orchestra.run.smithi078.stdout:blocklist 172.21.15.78:6800/1515675480 expires 2024-09-17T00:03:37.955119+0000 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24300a32f0 msgr2=0x7f24300a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24300a32f0 0x7f24300a57b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2444199410 tx=0x7f2440008040 comp rx=0 tx=0).stop 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2444101c00 msgr2=0x7f2444197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2444101c00 0x7f2444197ef0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f243400d950 tx=0x7f243400de10 comp rx=0 tx=0).stop 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 shutdown_connections 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24300a32f0 0x7f24300a57b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24441037c0 0x7f244419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2444102e00 0x7f2444198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 --2- 172.21.15.78:0/198292397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2444101c00 0x7f2444197ef0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:08.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 >> 172.21.15.78:0/198292397 conn(0x7f24440fd3b0 msgr2=0x7f2444106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:08.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 shutdown_connections 2024-09-16T00:13:08.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:08.576+0000 7f244b655640 1 -- 172.21.15.78:0/198292397 wait complete. 2024-09-16T00:13:08.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:08 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/198292397' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:13:08.946 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:08 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/198292397' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:13:08.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:08 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/198292397' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:13:09.300 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-16T00:13:09.311 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:13:09.568 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:09.844 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:09 smithi078 ceph-mon[31157]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-09-16T00:13:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:09 smithi193 ceph-mon[35484]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-09-16T00:13:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:09 smithi148 ceph-mon[35599]: pgmap v353: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-09-16T00:13:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 -- 172.21.15.78:0/972170872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102570 msgr2=0x7ff2341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 --2- 172.21.15.78:0/972170872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102570 0x7ff2341029b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009900 tx=0x7ff21c02a480 comp rx=0 tx=0).stop 2024-09-16T00:13:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 -- 172.21.15.78:0/972170872 shutdown_connections 2024-09-16T00:13:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 --2- 172.21.15.78:0/972170872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff234102ef0 0x7ff23410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 --2- 172.21.15.78:0/972170872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102570 0x7ff2341029b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 --2- 172.21.15.78:0/972170872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff234100e20 0x7ff234101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 -- 172.21.15.78:0/972170872 >> 172.21.15.78:0/972170872 conn(0x7ff2340fc7d0 msgr2=0x7ff2340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 -- 172.21.15.78:0/972170872 shutdown_connections 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 -- 172.21.15.78:0/972170872 wait complete. 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.072+0000 7ff239da4640 1 Processor -- start 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff239da4640 1 -- start start 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff239da4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff234100e20 0x7ff234197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff239da4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff234102570 0x7ff2341978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff239da4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102ef0 0x7ff23419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff239da4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff234110f80 con 0x7ff234102ef0 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff239da4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff234110e00 con 0x7ff234102570 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff239da4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff234111100 con 0x7ff234100e20 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102ef0 0x7ff23419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102ef0 0x7ff23419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47980/0 (socket says 172.21.15.78:47980) 2024-09-16T00:13:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 -- 172.21.15.78:0/928903668 learned_addr learned my addr 172.21.15.78:0/928903668 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff22affd640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff234102570 0x7ff2341978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 -- 172.21.15.78:0/928903668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff234100e20 msgr2=0x7ff234197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff234100e20 0x7ff234197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 -- 172.21.15.78:0/928903668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff234102570 msgr2=0x7ff2341978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff234102570 0x7ff2341978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 -- 172.21.15.78:0/928903668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23419c330 con 0x7ff234102ef0 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff22affd640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff234102570 0x7ff2341978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.073+0000 7ff233fff640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102ef0 0x7ff23419bc50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00cad0 tx=0x7ff22c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.074+0000 7ff2317fa640 1 -- 172.21.15.78:0/928903668 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00ae20 con 0x7ff234102ef0 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.074+0000 7ff2317fa640 1 -- 172.21.15.78:0/928903668 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff22c005020 con 0x7ff234102ef0 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.074+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23419c620 con 0x7ff234102ef0 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.074+0000 7ff2317fa640 1 -- 172.21.15.78:0/928903668 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00f6b0 con 0x7ff234102ef0 2024-09-16T00:13:10.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.074+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff234069f70 con 0x7ff234102ef0 2024-09-16T00:13:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.075+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff234101200 con 0x7ff234102ef0 2024-09-16T00:13:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.076+0000 7ff2317fa640 1 -- 172.21.15.78:0/928903668 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff22c0051d0 con 0x7ff234102ef0 2024-09-16T00:13:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.077+0000 7ff2317fa640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2200a32c0 0x7ff2200a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.077+0000 7ff2337fe640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2200a32c0 0x7ff2200a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.077+0000 7ff2317fa640 1 -- 172.21.15.78:0/928903668 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff22c0d0020 con 0x7ff234102ef0 2024-09-16T00:13:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.077+0000 7ff2337fe640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2200a32c0 0x7ff2200a5780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff22400a990 tx=0x7ff224005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:10.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.080+0000 7ff2317fa640 1 -- 172.21.15.78:0/928903668 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff22c014030 con 0x7ff234102ef0 2024-09-16T00:13:10.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.282+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff23406a230 con 0x7ff234102ef0 2024-09-16T00:13:10.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.283+0000 7ff2317fa640 1 -- 172.21.15.78:0/928903668 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11986 (secure 0 0 0) 0x7ff22c09c9e0 con 0x7ff234102ef0 2024-09-16T00:13:10.284 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:10.284 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-16T00:11:44:544077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:11:44.262714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:13:10.285 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2200a32c0 msgr2=0x7ff2200a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2200a32c0 0x7ff2200a5780 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff22400a990 tx=0x7ff224005c70 comp rx=0 tx=0).stop 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102ef0 msgr2=0x7ff23419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102ef0 0x7ff23419bc50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00cad0 tx=0x7ff22c00cf90 comp rx=0 tx=0).stop 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 shutdown_connections 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2200a32c0 0x7ff2200a5780 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff234102ef0 0x7ff23419bc50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff234102570 0x7ff2341978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 --2- 172.21.15.78:0/928903668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff234100e20 0x7ff234197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 >> 172.21.15.78:0/928903668 conn(0x7ff2340fc7d0 msgr2=0x7ff234106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 shutdown_connections 2024-09-16T00:13:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:10.285+0000 7ff239da4640 1 -- 172.21.15.78:0/928903668 wait complete. 2024-09-16T00:13:10.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:10 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/928903668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:10.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:10 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/928903668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:10.974 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:10 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/928903668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:11.005 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:13:11.253 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 -- 172.21.15.78:0/1176784858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 msgr2=0x7f08c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/1176784858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 0x7f08c01067e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009900 tx=0x7f08b402a410 comp rx=0 tx=0).stop 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 -- 172.21.15.78:0/1176784858 shutdown_connections 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/1176784858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c0106f10 0x7f08c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/1176784858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 0x7f08c01067e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/1176784858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0101de0 0x7f08c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 -- 172.21.15.78:0/1176784858 >> 172.21.15.78:0/1176784858 conn(0x7f08c00fd8c0 msgr2=0x7f08c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 -- 172.21.15.78:0/1176784858 shutdown_connections 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 -- 172.21.15.78:0/1176784858 wait complete. 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.744+0000 7f08c5b9e640 1 Processor -- start 2024-09-16T00:13:11.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08c5b9e640 1 -- start start 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08c5b9e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c0101de0 0x7f08c0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08c5b9e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 0x7f08c0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08c5b9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0106f10 0x7f08c019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08c5b9e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0111f70 con 0x7f08c0102790 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08c5b9e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08c0111df0 con 0x7f08c0101de0 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08c5b9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08c01120f0 con 0x7f08c0106f10 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08beffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 0x7f08c0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08beffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 0x7f08c0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48002/0 (socket says 172.21.15.78:48002) 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08beffd640 1 -- 172.21.15.78:0/3208532061 learned_addr learned my addr 172.21.15.78:0/3208532061 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08bf7fe640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c0101de0 0x7f08c0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08bffff640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0106f10 0x7f08c019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08beffd640 1 -- 172.21.15.78:0/3208532061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0106f10 msgr2=0x7f08c019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:11.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08beffd640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0106f10 0x7f08c019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08beffd640 1 -- 172.21.15.78:0/3208532061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c0101de0 msgr2=0x7f08c0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08beffd640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c0101de0 0x7f08c0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08beffd640 1 -- 172.21.15.78:0/3208532061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c019d4c0 con 0x7f08c0102790 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.745+0000 7f08bf7fe640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c0101de0 0x7f08c0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.746+0000 7f08beffd640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 0x7f08c0198a50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f08b4006010 tx=0x7f08b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.746+0000 7f08bcff9640 1 -- 172.21.15.78:0/3208532061 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b403f030 con 0x7f08c0102790 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.746+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c019d750 con 0x7f08c0102790 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.746+0000 7f08bcff9640 1 -- 172.21.15.78:0/3208532061 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b4003ce0 con 0x7f08c0102790 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.746+0000 7f08bcff9640 1 -- 172.21.15.78:0/3208532061 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4035070 con 0x7f08c0102790 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.746+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c01a5090 con 0x7f08c0102790 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.747+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c01032c0 con 0x7f08c0102790 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.748+0000 7f08bcff9640 1 -- 172.21.15.78:0/3208532061 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f08b4042030 con 0x7f08c0102790 2024-09-16T00:13:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.748+0000 7f08bcff9640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08a80a3240 0x7f08a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:11.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.748+0000 7f08bcff9640 1 -- 172.21.15.78:0/3208532061 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f08b40f0680 con 0x7f08c0102790 2024-09-16T00:13:11.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.749+0000 7f08bf7fe640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08a80a3240 0x7f08a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:11.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.749+0000 7f08bf7fe640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08a80a3240 0x7f08a80a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08b00058c0 tx=0x7f08b0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:11.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.752+0000 7f08bcff9640 1 -- 172.21.15.78:0/3208532061 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08b40bd100 con 0x7f08c0102790 2024-09-16T00:13:11.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:11 smithi193 ceph-mon[35484]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 9 op/s 2024-09-16T00:13:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:11 smithi148 ceph-mon[35599]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 9 op/s 2024-09-16T00:13:11.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.956+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08c01021c0 con 0x7f08c0102790 2024-09-16T00:13:11.957 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:11 smithi078 ceph-mon[31157]: pgmap v354: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 9 op/s 2024-09-16T00:13:11.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.957+0000 7f08bcff9640 1 -- 172.21.15.78:0/3208532061 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11986 (secure 0 0 0) 0x7f08c01021c0 con 0x7f08c0102790 2024-09-16T00:13:11.958 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:11.958 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":28,"btime":"2024-09-16T00:11:44:544077+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:11:44.262714+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:13:11.958 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 28 2024-09-16T00:13:11.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08a80a3240 msgr2=0x7f08a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:11.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08a80a3240 0x7f08a80a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08b00058c0 tx=0x7f08b0005850 comp rx=0 tx=0).stop 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 msgr2=0x7f08c0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 0x7f08c0198a50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f08b4006010 tx=0x7f08b40042f0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 shutdown_connections 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08a80a3240 0x7f08a80a5700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08c0106f10 0x7f08c019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08c0102790 0x7f08c0198a50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 --2- 172.21.15.78:0/3208532061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f08c0101de0 0x7f08c0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 >> 172.21.15.78:0/3208532061 conn(0x7f08c00fd8c0 msgr2=0x7f08c00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 shutdown_connections 2024-09-16T00:13:11.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:11.959+0000 7f08c5b9e640 1 -- 172.21.15.78:0/3208532061 wait complete. 2024-09-16T00:13:12.648 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:13:12.893 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:12.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:12 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3208532061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:12.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:12 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3208532061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:13.014 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:12 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3208532061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 -- 172.21.15.78:0/3698295729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c106f10 msgr2=0x7f630c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 --2- 172.21.15.78:0/3698295729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c106f10 0x7f630c10ea90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6304009ca0 tx=0x7f630402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 -- 172.21.15.78:0/3698295729 shutdown_connections 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 --2- 172.21.15.78:0/3698295729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c106f10 0x7f630c10ea90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 --2- 172.21.15.78:0/3698295729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f630c102790 0x7f630c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 --2- 172.21.15.78:0/3698295729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630c101de0 0x7f630c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 -- 172.21.15.78:0/3698295729 >> 172.21.15.78:0/3698295729 conn(0x7f630c0fd8c0 msgr2=0x7f630c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 -- 172.21.15.78:0/3698295729 shutdown_connections 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.415+0000 7f6314721640 1 -- 172.21.15.78:0/3698295729 wait complete. 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6314721640 1 Processor -- start 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6314721640 1 -- start start 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6314721640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c101de0 0x7f630c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6314721640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f630c102790 0x7f630c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:13.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6314721640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630c106f10 0x7f630c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6314721640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f630c111f70 con 0x7f630c101de0 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6314721640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f630c111df0 con 0x7f630c106f10 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6314721640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f630c1120f0 con 0x7f630c102790 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f63124bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c101de0 0x7f630c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f63124bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c101de0 0x7f630c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48010/0 (socket says 172.21.15.78:48010) 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f63124bd640 1 -- 172.21.15.78:0/3471932010 learned_addr learned my addr 172.21.15.78:0/3471932010 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6312cbe640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630c106f10 0x7f630c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f6311cbc640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f630c102790 0x7f630c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f63124bd640 1 -- 172.21.15.78:0/3471932010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f630c102790 msgr2=0x7f630c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f63124bd640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f630c102790 0x7f630c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f63124bd640 1 -- 172.21.15.78:0/3471932010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630c106f10 msgr2=0x7f630c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f63124bd640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630c106f10 0x7f630c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.416+0000 7f63124bd640 1 -- 172.21.15.78:0/3471932010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c19d3c0 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.417+0000 7f63124bd640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c101de0 0x7f630c198410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f630000d950 tx=0x7f630000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.417+0000 7f62ff7fe640 1 -- 172.21.15.78:0/3471932010 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300017030 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.417+0000 7f62ff7fe640 1 -- 172.21.15.78:0/3471932010 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6300004020 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.417+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630c19d6b0 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.417+0000 7f62ff7fe640 1 -- 172.21.15.78:0/3471932010 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63000047a0 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.417+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630c1a4f90 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.418+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630c1021c0 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.419+0000 7f62ff7fe640 1 -- 172.21.15.78:0/3471932010 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f6300009a30 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.419+0000 7f62ff7fe640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f62f80a32c0 0x7f62f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.420+0000 7f6311cbc640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f62f80a32c0 0x7f62f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.420+0000 7f62ff7fe640 1 -- 172.21.15.78:0/3471932010 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f63000cf6b0 con 0x7f630c101de0 2024-09-16T00:13:13.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.420+0000 7f6311cbc640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f62f80a32c0 0x7f62f80a5780 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f630c0fdce0 tx=0x7f6308005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:13.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.424+0000 7f62ff7fe640 1 -- 172.21.15.78:0/3471932010 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6300019310 con 0x7f630c101de0 2024-09-16T00:13:13.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.591+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f630c10a1b0 con 0x7f630c101de0 2024-09-16T00:13:13.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.592+0000 7f62ff7fe640 1 -- 172.21.15.78:0/3471932010 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24757 (secure 0 0 0) 0x7f630009c070 con 0x7f630c101de0 2024-09-16T00:13:13.594 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:13.594 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:13:04.600749+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:13:13.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f62f80a32c0 msgr2=0x7f62f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:13.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f62f80a32c0 0x7f62f80a5780 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f630c0fdce0 tx=0x7f6308005be0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c101de0 msgr2=0x7f630c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c101de0 0x7f630c198410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f630000d950 tx=0x7f630000de10 comp rx=0 tx=0).stop 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 shutdown_connections 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f62f80a32c0 0x7f62f80a5780 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f630c106f10 0x7f630c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f630c102790 0x7f630c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 --2- 172.21.15.78:0/3471932010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f630c101de0 0x7f630c198410 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 >> 172.21.15.78:0/3471932010 conn(0x7f630c0fd8c0 msgr2=0x7f630c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 shutdown_connections 2024-09-16T00:13:13.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:13.594+0000 7f6314721640 1 -- 172.21.15.78:0/3471932010 wait complete. 2024-09-16T00:13:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:13 smithi193 ceph-mon[35484]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 580 B/s wr, 10 op/s 2024-09-16T00:13:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:13 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3471932010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:13 smithi148 ceph-mon[35599]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 580 B/s wr, 10 op/s 2024-09-16T00:13:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:13 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3471932010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:13.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:13 smithi078 ceph-mon[31157]: pgmap v355: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 580 B/s wr, 10 op/s 2024-09-16T00:13:13.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:13 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3471932010' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:14.307 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:11:05.234467+0000', 'modified': '2024-09-16T00:11:44.262714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24445, 'mds_2': 24440, 'mds_3': 24427, 'mds_4': 24421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/1764484583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 1764484583}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 1764484583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14583': {'gid': 14583, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6841/1822173762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 1822173762}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 1822173762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/1629662299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 1629662299}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 1629662299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6833/3203333506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3203333506}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 3203333506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/903408403', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 903408403}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 903408403}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.193:6835/2292206006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 2292206006}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 2292206006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.193:6837/1139945107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1139945107}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1139945107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.148:6837/3780846668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3780846668}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3780846668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.193:6839/3313451522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3313451522}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3313451522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.148:6839/831570873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 831570873}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 831570873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24421, 24427, 24440, 24445, 24470]} 2024-09-16T00:13:14.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-16T00:13:14.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-16T00:13:14.557 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:15.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 -- 172.21.15.78:0/3325280599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 msgr2=0x7f09101021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 --2- 172.21.15.78:0/3325280599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 0x7f09101021b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0900009ca0 tx=0x7f090002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 -- 172.21.15.78:0/3325280599 shutdown_connections 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 --2- 172.21.15.78:0/3325280599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0910106f00 0x7f091010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 --2- 172.21.15.78:0/3325280599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0910102780 0x7f09101067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 --2- 172.21.15.78:0/3325280599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 0x7f09101021b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 -- 172.21.15.78:0/3325280599 >> 172.21.15.78:0/3325280599 conn(0x7f09100fd890 msgr2=0x7f09100ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 -- 172.21.15.78:0/3325280599 shutdown_connections 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.069+0000 7f09185bf640 1 -- 172.21.15.78:0/3325280599 wait complete. 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f09185bf640 1 Processor -- start 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f09185bf640 1 -- start start 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f09185bf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 0x7f09101a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f09185bf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0910102780 0x7f09101a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f09185bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0910106f00 0x7f091019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f09185bf640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0910112000 con 0x7f0910101dd0 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f09185bf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0910111e80 con 0x7f0910106f00 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f09185bf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0910112180 con 0x7f0910102780 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 0x7f09101a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 0x7f09101a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48032/0 (socket says 172.21.15.78:48032) 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 -- 172.21.15.78:0/1756241285 learned_addr learned my addr 172.21.15.78:0/1756241285 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f0915b5a640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0910102780 0x7f09101a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f0916b5c640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0910106f00 0x7f091019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 -- 172.21.15.78:0/1756241285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0910102780 msgr2=0x7f09101a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0910102780 0x7f09101a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 -- 172.21.15.78:0/1756241285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0910106f00 msgr2=0x7f091019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0910106f00 0x7f091019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 -- 172.21.15.78:0/1756241285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091019aec0 con 0x7f0910101dd0 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f0915b5a640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0910102780 0x7f09101a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f0916b5c640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0910106f00 0x7f091019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.070+0000 7f091635b640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 0x7f09101a0580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f09000097b0 tx=0x7f0900031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.071+0000 7f090f7fe640 1 -- 172.21.15.78:0/1756241285 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090002fba0 con 0x7f0910101dd0 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.071+0000 7f090f7fe640 1 -- 172.21.15.78:0/1756241285 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f090002fd40 con 0x7f0910101dd0 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.071+0000 7f09185bf640 1 -- 172.21.15.78:0/1756241285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091019b150 con 0x7f0910101dd0 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.071+0000 7f090f7fe640 1 -- 172.21.15.78:0/1756241285 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090003b930 con 0x7f0910101dd0 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.071+0000 7f09185bf640 1 -- 172.21.15.78:0/1756241285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09101a7330 con 0x7f0910101dd0 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.073+0000 7f090f7fe640 1 -- 172.21.15.78:0/1756241285 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0900030cc0 con 0x7f0910101dd0 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.073+0000 7f091a998740 1 Processor -- start 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.073+0000 7f091a998740 1 -- start start 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f090f7fe640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08f80a3240 0x7f08f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f091a998740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5eedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f091a998740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f091a998740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f091a998740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd555090 con 0x55e1dd5f9bb0 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f091a998740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd4fca70 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f091a998740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd551050 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08f80a3240 0x7f08f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f090f7fe640 1 -- 172.21.15.78:0/1756241285 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f09000f1440 con 0x7f0910101dd0 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5fbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48048/0 (socket says 172.21.15.78:48048) 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 -- 172.21.15.78:0/3349319482 learned_addr learned my addr 172.21.15.78:0/3349319482 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f091635b640 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5eedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0915b5a640 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 -- 172.21.15.78:0/3349319482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 msgr2=0x55e1dd5f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:15.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 -- 172.21.15.78:0/3349319482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 msgr2=0x55e1dd5eedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5eedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 -- 172.21.15.78:0/3349319482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1dd509130 con 0x55e1dd5f9bb0 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f091635b640 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5eedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08f80a3240 0x7f08f80a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f09080154f0 tx=0x7f0908009860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.074+0000 7f0916b5c640 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5fbfa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f09080176e0 tx=0x7f0908018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63be8a39d5d10fad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f090cff9640 1 -- 172.21.15.78:0/3349319482 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090800f040 con 0x55e1dd5f9bb0 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f090cff9640 1 -- 172.21.15.78:0/3349319482 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0908018b60 con 0x55e1dd5f9bb0 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f090cff9640 1 -- 172.21.15.78:0/3349319482 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908022910 con 0x55e1dd5f9bb0 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f091a998740 1 -- 172.21.15.78:0/3349319482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5f9bb0 msgr2=0x55e1dd5fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f091a998740 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5fbfa0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f09080176e0 tx=0x7f0908018000 comp rx=0 tx=0).stop 2024-09-16T00:13:15.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f091a998740 1 -- 172.21.15.78:0/3349319482 shutdown_connections 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f091a998740 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5fbfa0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f091a998740 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f091a998740 1 --2- 172.21.15.78:0/3349319482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5eedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f091a998740 1 -- 172.21.15.78:0/3349319482 >> 172.21.15.78:0/3349319482 conn(0x55e1dd5eaf60 msgr2=0x55e1dd5eb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.075+0000 7f091a998740 1 -- 172.21.15.78:0/3349319482 shutdown_connections 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 -- 172.21.15.78:0/3349319482 wait complete. 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 -- start start 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 Processor -- start 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd5fc720 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd50ff20 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd5fc8a0 con 0x55e1dd5f9bb0 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091a998740 10 client.0 _ll_register_callbacks cb 0x55e1dd4d9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f0915b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f0915b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48054/0 (socket says 172.21.15.78:48054) 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f0915b5a640 1 -- 172.21.15.78:0/2387189725 learned_addr learned my addr 172.21.15.78:0/2387189725 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f091635b640 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f0916b5c640 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f0915b5a640 1 -- 172.21.15.78:0/2387189725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 msgr2=0x55e1dd5f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.076+0000 7f0915b5a640 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f0915b5a640 1 -- 172.21.15.78:0/2387189725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 msgr2=0x55e1dd5f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f0915b5a640 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f0915b5a640 1 -- 172.21.15.78:0/2387189725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1dd2c62b0 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f091635b640 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:15.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f0915b5a640 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f6800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f09040104d0 tx=0x7f0904010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc2535b7ed649f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f08f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f08f5ffb640 1 -- 172.21.15.78:0/2387189725 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904018070 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f08f5ffb640 1 -- 172.21.15.78:0/2387189725 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0904014640 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f091a998740 1 -- 172.21.15.78:0/2387189725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0908018b60 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f08f5ffb640 1 -- 172.21.15.78:0/2387189725 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0904024050 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f091a998740 10 client.15189 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f091a998740 1 -- 172.21.15.78:0/2387189725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e1dd608120 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f08f5ffb640 1 -- 172.21.15.78:0/2387189725 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0908018b60 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f08f5ffb640 1 -- 172.21.15.78:0/2387189725 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0904012a00 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.077+0000 7f091a998740 10 client.15189 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.078+0000 7f091a998740 10 client.15189 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.078+0000 7f091a998740 20 client.15189 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.078+0000 7f091a998740 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e1dd60a7b0 0x55e1dd60bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.078+0000 7f091a998740 4 client.15189 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.078+0000 7f091a998740 1 -- 172.21.15.78:0/2387189725 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1dd562600 con 0x55e1dd60a7b0 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.078+0000 7f091635b640 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e1dd60a7b0 0x55e1dd60bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.078+0000 7f091635b640 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e1dd60a7b0 0x55e1dd60bc20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b751aa7247d6a5d4 server_cookie=737842f8eb0a85aa in_seq=0 out_seq=0 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.078+0000 7f08f5ffb640 10 client.15189 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f08f5ffb640 1 -- 172.21.15.78:0/2387189725 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1dd562600 con 0x55e1dd60a7b0 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f08f5ffb640 10 client.15189 handle_command_reply: tid=0 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 client.15189 shutdown 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 -- 172.21.15.78:0/2387189725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ef2f0 msgr2=0x55e1dd5f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f6800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f09040104d0 tx=0x7f0904010990 comp rx=0 tx=0).stop 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 -- 172.21.15.78:0/2387189725 shutdown_connections 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e1dd60a7b0 0x55e1dd60bc20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f6800 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 --2- 172.21.15.78:0/2387189725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f08f5ffb640 0 client.15189 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:13:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 -- 172.21.15.78:0/2387189725 >> 172.21.15.78:0/2387189725 conn(0x55e1dd5eaf60 msgr2=0x55e1dd5f0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 -- 172.21.15.78:0/2387189725 shutdown_connections 2024-09-16T00:13:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 1 -- 172.21.15.78:0/2387189725 wait complete. 2024-09-16T00:13:15.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.079+0000 7f091a998740 20 client.15189 trim_cache size 0 max 16384 2024-09-16T00:13:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091a998740 1 Processor -- start 2024-09-16T00:13:15.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091a998740 1 -- start start 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091a998740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091a998740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091a998740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091a998740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd609f60 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091a998740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd60a630 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091a998740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd60ad00 con 0x55e1dd5f9bb0 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091635b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091635b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48068/0 (socket says 172.21.15.78:48068) 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091635b640 1 -- 172.21.15.78:0/2120939092 learned_addr learned my addr 172.21.15.78:0/2120939092 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f0915b5a640 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091635b640 1 -- 172.21.15.78:0/2120939092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 msgr2=0x55e1dd5f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091635b640 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091635b640 1 -- 172.21.15.78:0/2120939092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ef2f0 msgr2=0x55e1dd5f09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091635b640 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f091635b640 1 -- 172.21.15.78:0/2120939092 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1dd5ee800 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.097+0000 7f0915b5a640 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091635b640 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f0470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f09000bd690 tx=0x7f09000a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8978b1017f79477d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f08f7fff640 1 -- 172.21.15.78:0/2120939092 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09000bde00 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f08f7fff640 1 -- 172.21.15.78:0/2120939092 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0900036040 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f08f7fff640 1 -- 172.21.15.78:0/2120939092 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09000bba00 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 -- 172.21.15.78:0/2120939092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 msgr2=0x55e1dd5f0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f0470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f09000bd690 tx=0x7f09000a8040 comp rx=0 tx=0).stop 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 -- 172.21.15.78:0/2120939092 shutdown_connections 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd5f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 --2- 172.21.15.78:0/2120939092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd5f0470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 -- 172.21.15.78:0/2120939092 >> 172.21.15.78:0/2120939092 conn(0x55e1dd5eaf60 msgr2=0x55e1dd5f1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 -- 172.21.15.78:0/2120939092 shutdown_connections 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.098+0000 7f091a998740 1 -- 172.21.15.78:0/2120939092 wait complete. 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 1 -- start start 2024-09-16T00:13:15.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 1 Processor -- start 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd614160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd6146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd60baa0 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd60b3d0 con 0x55e1dd5f9bb0 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1dd60c170 con 0x55e1dd5ef2f0 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091a998740 10 client.0 _ll_register_callbacks cb 0x7f0904007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091635b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd614160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091635b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd614160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48078/0 (socket says 172.21.15.78:48078) 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091635b640 1 -- 172.21.15.78:0/656213655 learned_addr learned my addr 172.21.15.78:0/656213655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f0916b5c640 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091635b640 1 -- 172.21.15.78:0/656213655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 msgr2=0x55e1dd6146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:15.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091635b640 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd6146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091635b640 1 -- 172.21.15.78:0/656213655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5f9bb0 msgr2=0x55e1dd5f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091635b640 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f091635b640 1 -- 172.21.15.78:0/656213655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1dd5f39e0 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.099+0000 7f0916b5c640 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091635b640 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd614160 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f090002ade0 tx=0x7f090003b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d40513e1e2c404a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f08f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f08f5ffb640 1 -- 172.21.15.78:0/656213655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09000c1ab0 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f08f5ffb640 1 -- 172.21.15.78:0/656213655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f09000bba00 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 1 -- 172.21.15.78:0/656213655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0900036040 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f08f5ffb640 1 -- 172.21.15.78:0/656213655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09000bde00 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 10 client.15195 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 1 -- 172.21.15.78:0/656213655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e1dd6139d0 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f08f5ffb640 1 -- 172.21.15.78:0/656213655 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0900036040 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f08f5ffb640 1 -- 172.21.15.78:0/656213655 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f0900004b40 con 0x55e1dd5ee9d0 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 10 client.15195 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 10 client.15195 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 20 client.15195 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e1dd6794f0 0x55e1dd678200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 4 client.15195 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.100+0000 7f091a998740 1 -- 172.21.15.78:0/656213655 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "status"}) -- 0x55e1dd55cb10 con 0x55e1dd6794f0 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f0915b5a640 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e1dd6794f0 0x55e1dd678200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f0915b5a640 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e1dd6794f0 0x55e1dd678200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5145880092b2eb9b server_cookie=b89c095c35e7d4cb in_seq=0 out_seq=0 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f08f5ffb640 10 client.15195 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f08f5ffb640 1 -- 172.21.15.78:0/656213655 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e1dd55cb10 con 0x55e1dd6794f0 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f08f5ffb640 10 client.15195 handle_command_reply: tid=0 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 client.15195 shutdown 2024-09-16T00:13:15.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 -- 172.21.15.78:0/656213655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 msgr2=0x55e1dd614160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd614160 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f090002ade0 tx=0x7f090003b400 comp rx=0 tx=0).stop 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 -- 172.21.15.78:0/656213655 shutdown_connections 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1dd5f9bb0 0x55e1dd5f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e1dd6794f0 0x55e1dd678200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1dd5ef2f0 0x55e1dd6146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 --2- 172.21.15.78:0/656213655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1dd5ee9d0 0x55e1dd614160 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f08f5ffb640 0 client.15195 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.101+0000 7f091a998740 1 -- 172.21.15.78:0/656213655 >> 172.21.15.78:0/656213655 conn(0x55e1dd5eaf60 msgr2=0x55e1dd5f15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f091a998740 1 -- 172.21.15.78:0/656213655 shutdown_connections 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f091a998740 1 -- 172.21.15.78:0/656213655 wait complete. 2024-09-16T00:13:15.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f091a998740 20 client.15195 trim_cache size 0 max 16384 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24421, 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 92.567674858000004, 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:13:15.110 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 159.25182001100001 2024-09-16T00:13:15.111 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:15.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 -- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08f80a3240 msgr2=0x7f08f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08f80a3240 0x7f08f80a5700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f09080154f0 tx=0x7f0908009860 comp rx=0 tx=0).stop 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 -- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 msgr2=0x7f09101a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 0x7f09101a0580 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f09000097b0 tx=0x7f0900031b10 comp rx=0 tx=0).stop 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 -- 172.21.15.78:0/1756241285 shutdown_connections 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08f80a3240 0x7f08f80a5700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0910106f00 0x7f091019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0910102780 0x7f09101a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 --2- 172.21.15.78:0/1756241285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0910101dd0 0x7f09101a0580 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 -- 172.21.15.78:0/1756241285 >> 172.21.15.78:0/1756241285 conn(0x7f09100fd890 msgr2=0x7f09100fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 -- 172.21.15.78:0/1756241285 shutdown_connections 2024-09-16T00:13:15.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:15.102+0000 7f09185bf640 1 -- 172.21.15.78:0/1756241285 wait complete. 2024-09-16T00:13:15.817 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-16T00:13:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:15 smithi193 ceph-mon[35484]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T00:13:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:15 smithi148 ceph-mon[35599]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T00:13:16.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:15 smithi078 ceph-mon[31157]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T00:13:16.062 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:16.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 -- 172.21.15.78:0/2996678449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734103780 msgr2=0x7f2734107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 --2- 172.21.15.78:0/2996678449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734103780 0x7f2734107bc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2728009930 tx=0x7f272802a420 comp rx=0 tx=0).stop 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 -- 172.21.15.78:0/2996678449 shutdown_connections 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 --2- 172.21.15.78:0/2996678449 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2734108300 0x7f273410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 --2- 172.21.15.78:0/2996678449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734103780 0x7f2734107bc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 --2- 172.21.15.78:0/2996678449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2734102dd0 0x7f27341031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 -- 172.21.15.78:0/2996678449 >> 172.21.15.78:0/2996678449 conn(0x7f27340fc7d0 msgr2=0x7f27340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 -- 172.21.15.78:0/2996678449 shutdown_connections 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.620+0000 7f2739366640 1 -- 172.21.15.78:0/2996678449 wait complete. 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.621+0000 7f2739366640 1 Processor -- start 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.621+0000 7f2739366640 1 -- start start 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.621+0000 7f2739366640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734102dd0 0x7f27341985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.621+0000 7f2739366640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2734103780 0x7f2734198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.621+0000 7f2739366640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2734108300 0x7f273419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.621+0000 7f2739366640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27341120d0 con 0x7f2734102dd0 2024-09-16T00:13:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.621+0000 7f2739366640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2734111f50 con 0x7f2734108300 2024-09-16T00:13:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.621+0000 7f2739366640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2734112250 con 0x7f2734103780 2024-09-16T00:13:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734102dd0 0x7f27341985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734102dd0 0x7f27341985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51214/0 (socket says 172.21.15.78:51214) 2024-09-16T00:13:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 -- 172.21.15.78:0/1816249967 learned_addr learned my addr 172.21.15.78:0/1816249967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:16.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f27327fc640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2734103780 0x7f2734198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f27337fe640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2734108300 0x7f273419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 -- 172.21.15.78:0/1816249967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2734103780 msgr2=0x7f2734198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2734103780 0x7f2734198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 -- 172.21.15.78:0/1816249967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2734108300 msgr2=0x7f273419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2734108300 0x7f273419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 -- 172.21.15.78:0/1816249967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273419d560 con 0x7f2734102dd0 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f27327fc640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2734103780 0x7f2734198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f27337fe640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2734108300 0x7f273419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2732ffd640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734102dd0 0x7f27341985b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f272400cce0 tx=0x7f272400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f271bfff640 1 -- 172.21.15.78:0/1816249967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272400f650 con 0x7f2734102dd0 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2739366640 1 -- 172.21.15.78:0/1816249967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273419d850 con 0x7f2734102dd0 2024-09-16T00:13:16.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f271bfff640 1 -- 172.21.15.78:0/1816249967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f272400fcd0 con 0x7f2734102dd0 2024-09-16T00:13:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f271bfff640 1 -- 172.21.15.78:0/1816249967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724018c70 con 0x7f2734102dd0 2024-09-16T00:13:16.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.622+0000 7f2739366640 1 -- 172.21.15.78:0/1816249967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27341a5130 con 0x7f2734102dd0 2024-09-16T00:13:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.625+0000 7f271bfff640 1 -- 172.21.15.78:0/1816249967 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f272400f7f0 con 0x7f2734102dd0 2024-09-16T00:13:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f273b73f740 1 Processor -- start 2024-09-16T00:13:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f271bfff640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f271c0a3240 0x7f271c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f273b73f740 1 -- start start 2024-09-16T00:13:16.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f271bfff640 1 -- 172.21.15.78:0/1816249967 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f27240d01e0 con 0x7f2734102dd0 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f27337fe640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f271c0a3240 0x7f271c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f273b73f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf4cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f273b73f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 0x55a7daf5ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f273b73f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f273b73f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7daeb3090 con 0x55a7daf4c890 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f273b73f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a7dae5aa70 con 0x55a7daf4d1b0 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f273b73f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a7daeaf050 con 0x55a7daf5bfc0 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf4cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf4cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51230/0 (socket says 172.21.15.78:51230) 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f2732ffd640 1 -- 172.21.15.78:0/1724381211 learned_addr learned my addr 172.21.15.78:0/1724381211 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.626+0000 7f27327fc640 1 --2- 172.21.15.78:0/1724381211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 0x55a7daf5ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f2732ffd640 1 -- 172.21.15.78:0/1724381211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 msgr2=0x55a7daf5e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f2732ffd640 1 --2- 172.21.15.78:0/1724381211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f2732ffd640 1 -- 172.21.15.78:0/1724381211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 msgr2=0x55a7daf5ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f2732ffd640 1 --2- 172.21.15.78:0/1724381211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 0x55a7daf5ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f27337fe640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f271c0a3240 0x7f271c0a5700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f272c00a900 tx=0x7f272c00ad00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f2732ffd640 1 -- 172.21.15.78:0/1724381211 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7dae67130 con 0x55a7daf4c890 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f2732ffd640 1 --2- 172.21.15.78:0/1724381211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf4cc70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2724098900 tx=0x7f272409c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1345749b1cbf5b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f27197fa640 1 -- 172.21.15.78:0/1724381211 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27240b5070 con 0x55a7daf4c890 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f27197fa640 1 -- 172.21.15.78:0/1724381211 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f272409c4a0 con 0x55a7daf4c890 2024-09-16T00:13:16.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.627+0000 7f27197fa640 1 -- 172.21.15.78:0/1724381211 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724098b10 con 0x55a7daf4c890 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- 172.21.15.78:0/1724381211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 msgr2=0x55a7daf4cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 --2- 172.21.15.78:0/1724381211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf4cc70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2724098900 tx=0x7f272409c330 comp rx=0 tx=0).stop 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- 172.21.15.78:0/1724381211 shutdown_connections 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 --2- 172.21.15.78:0/1724381211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf5e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 --2- 172.21.15.78:0/1724381211 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 0x55a7daf5ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 --2- 172.21.15.78:0/1724381211 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf4cc70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- 172.21.15.78:0/1724381211 >> 172.21.15.78:0/1724381211 conn(0x55a7daf48f60 msgr2=0x55a7daf49340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- 172.21.15.78:0/1724381211 shutdown_connections 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- 172.21.15.78:0/1724381211 wait complete. 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- start start 2024-09-16T00:13:16.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 Processor -- start 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf5b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf4d1b0 0x55a7daf58c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf5bfc0 0x55a7daf591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7daf5eb90 con 0x55a7daf5bfc0 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a7dae6df20 con 0x55a7daf4c890 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.628+0000 7f273b73f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a7daf5ed10 con 0x55a7daf4d1b0 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f273b73f740 10 client.0 _ll_register_callbacks cb 0x55a7dae37030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf5bfc0 0x55a7daf591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf5bfc0 0x55a7daf591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51242/0 (socket says 172.21.15.78:51242) 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 -- 172.21.15.78:0/2443475557 learned_addr learned my addr 172.21.15.78:0/2443475557 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f2732ffd640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf5b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27327fc640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf4d1b0 0x55a7daf58c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 -- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf4d1b0 msgr2=0x55a7daf58c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf4d1b0 0x55a7daf58c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 -- 172.21.15.78:0/2443475557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 msgr2=0x55a7daf5b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf5b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 -- 172.21.15.78:0/2443475557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dac242b0 con 0x55a7daf5bfc0 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27327fc640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf4d1b0 0x55a7daf58c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f2732ffd640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf5b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f27337fe640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf5bfc0 0x55a7daf591d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f272c016680 tx=0x7f272c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e9f2bb44d82bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f2716ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f2716ff5640 1 -- 172.21.15.78:0/2443475557 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c025060 con 0x55a7daf5bfc0 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f2716ff5640 1 -- 172.21.15.78:0/2443475557 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f272c016e00 con 0x55a7daf5bfc0 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f273b73f740 1 -- 172.21.15.78:0/2443475557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272409c4a0 con 0x55a7daf5bfc0 2024-09-16T00:13:16.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f2716ff5640 1 -- 172.21.15.78:0/2443475557 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f272c02c050 con 0x55a7daf5bfc0 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f273b73f740 10 client.15207 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.629+0000 7f273b73f740 1 -- 172.21.15.78:0/2443475557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7daf6a600 con 0x55a7daf5bfc0 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f2716ff5640 1 -- 172.21.15.78:0/2443475557 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f272409c4a0 con 0x55a7daf5bfc0 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f2716ff5640 1 -- 172.21.15.78:0/2443475557 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f272c012030 con 0x55a7daf5bfc0 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f273b73f740 10 client.15207 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f273b73f740 10 client.15207 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f273b73f740 20 client.15207 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f273b73f740 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a7daf6cc20 0x55a7daf6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f273b73f740 4 client.15207 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f273b73f740 1 -- 172.21.15.78:0/2443475557 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7daec0600 con 0x55a7daf6cc20 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.630+0000 7f2732ffd640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a7daf6cc20 0x55a7daf6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f2732ffd640 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a7daf6cc20 0x55a7daf6e090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79f0909359870a76 server_cookie=3134778a1256e1a9 in_seq=0 out_seq=0 2024-09-16T00:13:16.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f2716ff5640 10 client.15207 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f2716ff5640 1 -- 172.21.15.78:0/2443475557 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7daec0600 con 0x55a7daf6cc20 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f2716ff5640 10 client.15207 handle_command_reply: tid=0 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f273b73f740 1 client.15207 shutdown 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f273b73f740 1 -- 172.21.15.78:0/2443475557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf5bfc0 msgr2=0x55a7daf591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f273b73f740 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf5bfc0 0x55a7daf591d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f272c016680 tx=0x7f272c0047d0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f273b73f740 1 -- 172.21.15.78:0/2443475557 shutdown_connections 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f273b73f740 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf5bfc0 0x55a7daf591d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f273b73f740 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf4d1b0 0x55a7daf58c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.631+0000 7f273b73f740 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a7daf6cc20 0x55a7daf6e090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.632+0000 7f273b73f740 1 --2- 172.21.15.78:0/2443475557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf5b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.632+0000 7f2716ff5640 0 client.15207 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:13:16.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.632+0000 7f273b73f740 1 -- 172.21.15.78:0/2443475557 >> 172.21.15.78:0/2443475557 conn(0x55a7daf48f60 msgr2=0x55a7daf4ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:16.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.632+0000 7f273b73f740 1 -- 172.21.15.78:0/2443475557 shutdown_connections 2024-09-16T00:13:16.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.632+0000 7f273b73f740 1 -- 172.21.15.78:0/2443475557 wait complete. 2024-09-16T00:13:16.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.632+0000 7f273b73f740 20 client.15207 trim_cache size 0 max 16384 2024-09-16T00:13:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.649+0000 7f273b73f740 1 Processor -- start 2024-09-16T00:13:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.649+0000 7f273b73f740 1 -- start start 2024-09-16T00:13:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.649+0000 7f273b73f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf6c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.649+0000 7f273b73f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.649+0000 7f273b73f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf58c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.649+0000 7f273b73f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7daf6c3d0 con 0x55a7daf4c890 2024-09-16T00:13:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f273b73f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a7daf6caa0 con 0x55a7daf4d1b0 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f273b73f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a7daf6d170 con 0x55a7daf5bfc0 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf6c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf6c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51250/0 (socket says 172.21.15.78:51250) 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 -- 172.21.15.78:0/24615818 learned_addr learned my addr 172.21.15.78:0/24615818 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f27327fc640 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f27337fe640 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf58c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 -- 172.21.15.78:0/24615818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 msgr2=0x55a7daf58c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf58c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 -- 172.21.15.78:0/24615818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 msgr2=0x55a7daf6e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 -- 172.21.15.78:0/24615818 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7daf5bc90 con 0x55a7daf4c890 2024-09-16T00:13:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2732ffd640 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf6c550 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f27240cd9d0 tx=0x7f272409c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c547d87dca9027a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2718ff9640 1 -- 172.21.15.78:0/24615818 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27240b5040 con 0x55a7daf4c890 2024-09-16T00:13:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2718ff9640 1 -- 172.21.15.78:0/24615818 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27240a5110 con 0x55a7daf4c890 2024-09-16T00:13:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.650+0000 7f2718ff9640 1 -- 172.21.15.78:0/24615818 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272409c700 con 0x55a7daf4c890 2024-09-16T00:13:16.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 -- 172.21.15.78:0/24615818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 msgr2=0x55a7daf6c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf6c550 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f27240cd9d0 tx=0x7f272409c3a0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 -- 172.21.15.78:0/24615818 shutdown_connections 2024-09-16T00:13:16.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf58c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 --2- 172.21.15.78:0/24615818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4c890 0x55a7daf6c550 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 -- 172.21.15.78:0/24615818 >> 172.21.15.78:0/24615818 conn(0x55a7daf48f60 msgr2=0x55a7daf43ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 -- 172.21.15.78:0/24615818 shutdown_connections 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 -- 172.21.15.78:0/24615818 wait complete. 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.651+0000 7f273b73f740 1 -- start start 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f273b73f740 1 Processor -- start 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f273b73f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf6eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f273b73f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f273b73f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf6fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f273b73f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7daf6df10 con 0x55a7daf4d1b0 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f273b73f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a7daf6d840 con 0x55a7daf4c890 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f273b73f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a7daf6e5e0 con 0x55a7daf5bfc0 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f273b73f740 10 client.0 _ll_register_callbacks cb 0x7f272c00cd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51266/0 (socket says 172.21.15.78:51266) 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 -- 172.21.15.78:0/2867704521 learned_addr learned my addr 172.21.15.78:0/2867704521 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f2732ffd640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf6eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27337fe640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf6fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 -- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 msgr2=0x55a7daf6fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf6fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 -- 172.21.15.78:0/2867704521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 msgr2=0x55a7daf6eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf6eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 -- 172.21.15.78:0/2867704521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dafd5320 con 0x55a7daf4d1b0 2024-09-16T00:13:16.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27337fe640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf6fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f2732ffd640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf6eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f27327fc640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6f530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2728005ab0 tx=0x7f2728004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53133d315be1609f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.652+0000 7f2716ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f2716ff5640 1 -- 172.21.15.78:0/2867704521 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728035040 con 0x55a7daf4d1b0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 1 -- 172.21.15.78:0/2867704521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27240a5110 con 0x55a7daf4d1b0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f2716ff5640 1 -- 172.21.15.78:0/2867704521 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2728005220 con 0x55a7daf4d1b0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f2716ff5640 1 -- 172.21.15.78:0/2867704521 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2728005420 con 0x55a7daf4d1b0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 10 client.15213 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 1 -- 172.21.15.78:0/2867704521 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7dafd65c0 con 0x55a7daf4d1b0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f2716ff5640 1 -- 172.21.15.78:0/2867704521 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27240a5110 con 0x55a7daf4d1b0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f2716ff5640 1 -- 172.21.15.78:0/2867704521 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f2728044070 con 0x55a7daf4d1b0 2024-09-16T00:13:16.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 10 client.15213 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 10 client.15213 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 20 client.15213 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a7dafdb180 0x55a7dafd9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 4 client.15213 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.653+0000 7f273b73f740 1 -- 172.21.15.78:0/2867704521 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "status"}) -- 0x55a7daebab10 con 0x55a7dafdb180 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f2732ffd640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a7dafdb180 0x55a7dafd9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f2732ffd640 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a7dafdb180 0x55a7dafd9e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=618b803614f05de9 server_cookie=af55adeb06c647f8 in_seq=0 out_seq=0 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f2716ff5640 10 client.15213 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f2716ff5640 1 -- 172.21.15.78:0/2867704521 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a7daebab10 con 0x55a7dafdb180 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f2716ff5640 10 client.15213 handle_command_reply: tid=0 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 client.15213 shutdown 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 -- 172.21.15.78:0/2867704521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4d1b0 msgr2=0x55a7daf6f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6f530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2728005ab0 tx=0x7f2728004270 comp rx=0 tx=0).stop 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 -- 172.21.15.78:0/2867704521 shutdown_connections 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7daf5bfc0 0x55a7daf6fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7daf4d1b0 0x55a7daf6f530 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a7dafdb180 0x55a7dafd9e90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 --2- 172.21.15.78:0/2867704521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7daf4c890 0x55a7daf6eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.654+0000 7f273b73f740 1 -- 172.21.15.78:0/2867704521 >> 172.21.15.78:0/2867704521 conn(0x55a7daf48f60 msgr2=0x55a7daf6c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2716ff5640 0 client.15213 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f273b73f740 1 -- 172.21.15.78:0/2867704521 shutdown_connections 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f273b73f740 1 -- 172.21.15.78:0/2867704521 wait complete. 2024-09-16T00:13:16.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f273b73f740 20 client.15213 trim_cache size 0 max 16384 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24427, 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 95.135215586000001, 2024-09-16T00:13:16.666 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:13:16.667 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:16.667 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:13:16.667 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 158.19026526900001 2024-09-16T00:13:16.667 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 -- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f271c0a3240 msgr2=0x7f271c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f271c0a3240 0x7f271c0a5700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f272c00a900 tx=0x7f272c00ad00 comp rx=0 tx=0).stop 2024-09-16T00:13:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 -- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734102dd0 msgr2=0x7f27341985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734102dd0 0x7f27341985b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f272400cce0 tx=0x7f272400a620 comp rx=0 tx=0).stop 2024-09-16T00:13:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 -- 172.21.15.78:0/1816249967 shutdown_connections 2024-09-16T00:13:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f271c0a3240 0x7f271c0a5700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2734108300 0x7f273419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2734103780 0x7f2734198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 --2- 172.21.15.78:0/1816249967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2734102dd0 0x7f27341985b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 -- 172.21.15.78:0/1816249967 >> 172.21.15.78:0/1816249967 conn(0x7f27340fc7d0 msgr2=0x7f2734105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 -- 172.21.15.78:0/1816249967 shutdown_connections 2024-09-16T00:13:16.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:16.655+0000 7f2739366640 1 -- 172.21.15.78:0/1816249967 wait complete. 2024-09-16T00:13:17.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:13:17.716 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:17.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:17 smithi078 ceph-mon[31157]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 433 B/s wr, 11 op/s 2024-09-16T00:13:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:17 smithi193 ceph-mon[35484]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 433 B/s wr, 11 op/s 2024-09-16T00:13:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:17 smithi148 ceph-mon[35599]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 433 B/s wr, 11 op/s 2024-09-16T00:13:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 -- 172.21.15.78:0/1397005061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102570 msgr2=0x7f29a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/1397005061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102570 0x7f29a81029b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2990009900 tx=0x7f299002a480 comp rx=0 tx=0).stop 2024-09-16T00:13:18.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 -- 172.21.15.78:0/1397005061 shutdown_connections 2024-09-16T00:13:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/1397005061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a8102ef0 0x7f29a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/1397005061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102570 0x7f29a81029b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/1397005061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8100e20 0x7f29a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 -- 172.21.15.78:0/1397005061 >> 172.21.15.78:0/1397005061 conn(0x7f29a80fc7d0 msgr2=0x7f29a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:18.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 -- 172.21.15.78:0/1397005061 shutdown_connections 2024-09-16T00:13:18.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.205+0000 7f29ad8a8640 1 -- 172.21.15.78:0/1397005061 wait complete. 2024-09-16T00:13:18.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29ad8a8640 1 Processor -- start 2024-09-16T00:13:18.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29ad8a8640 1 -- start start 2024-09-16T00:13:18.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29ad8a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a8100e20 0x7f29a8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29ad8a8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8102570 0x7f29a8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29ad8a8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102ef0 0x7f29a819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29ad8a8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8110fa0 con 0x7f29a8102ef0 2024-09-16T00:13:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29ad8a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8110e20 con 0x7f29a8100e20 2024-09-16T00:13:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29ad8a8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8111120 con 0x7f29a8102570 2024-09-16T00:13:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102ef0 0x7f29a819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102ef0 0x7f29a819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51272/0 (socket says 172.21.15.78:51272) 2024-09-16T00:13:18.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f29a77fe640 1 -- 172.21.15.78:0/3680328289 learned_addr learned my addr 172.21.15.78:0/3680328289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:18.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.206+0000 7f299e7fc640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8102570 0x7f29a8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a8100e20 0x7f29a8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29a77fe640 1 -- 172.21.15.78:0/3680328289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8102570 msgr2=0x7f29a8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29a77fe640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8102570 0x7f29a8198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29a77fe640 1 -- 172.21.15.78:0/3680328289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a8100e20 msgr2=0x7f29a8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29a77fe640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a8100e20 0x7f29a8198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29a77fe640 1 -- 172.21.15.78:0/3680328289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a819d4f0 con 0x7f29a8102ef0 2024-09-16T00:13:18.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f299e7fc640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8102570 0x7f29a8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:18.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a8100e20 0x7f29a8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:18.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29a77fe640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102ef0 0x7f29a819ce10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29a000b970 tx=0x7f29a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:18.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29ac8a6640 1 -- 172.21.15.78:0/3680328289 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0019070 con 0x7f29a8102ef0 2024-09-16T00:13:18.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29ad8a8640 1 -- 172.21.15.78:0/3680328289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a819d7e0 con 0x7f29a8102ef0 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29ac8a6640 1 -- 172.21.15.78:0/3680328289 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a000cb40 con 0x7f29a8102ef0 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29ac8a6640 1 -- 172.21.15.78:0/3680328289 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0007650 con 0x7f29a8102ef0 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.207+0000 7f29ad8a8640 1 -- 172.21.15.78:0/3680328289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a81a50c0 con 0x7f29a8102ef0 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29ac8a6640 1 -- 172.21.15.78:0/3680328289 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f29a001c070 con 0x7f29a8102ef0 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29afc81740 1 Processor -- start 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29afc81740 1 -- start start 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29afc81740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29afc81740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29afc81740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0d5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29afc81740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb02f090 con 0x55cdeb0c92f0 2024-09-16T00:13:18.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29afc81740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cdeafd6a70 con 0x55cdeb0c89d0 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29afc81740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb02b050 con 0x55cdeb0d3bb0 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51282/0 (socket says 172.21.15.78:51282) 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.210+0000 7f29a6ffd640 1 -- 172.21.15.78:0/1548401188 learned_addr learned my addr 172.21.15.78:0/1548401188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f29a6ffd640 1 -- 172.21.15.78:0/1548401188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 msgr2=0x55cdeb0d5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/1548401188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0d5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f29a6ffd640 1 -- 172.21.15.78:0/1548401188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 msgr2=0x55cdeb0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f299e7fc640 1 --2- 172.21.15.78:0/1548401188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f29ac8a6640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29940a3240 0x7f29940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/1548401188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f29a6ffd640 1 -- 172.21.15.78:0/1548401188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdeafe3130 con 0x55cdeb0c92f0 2024-09-16T00:13:18.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f299e7fc640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29940a3240 0x7f29940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f29ac8a6640 1 -- 172.21.15.78:0/3680328289 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f29a00cf330 con 0x7f29a8102ef0 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/1548401188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d3670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2998009c50 tx=0x7f299800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d71f068d10ae21 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f299dffb640 1 -- 172.21.15.78:0/1548401188 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998016070 con 0x55cdeb0c92f0 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f299dffb640 1 -- 172.21.15.78:0/1548401188 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2998005500 con 0x55cdeb0c92f0 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.211+0000 7f299dffb640 1 -- 172.21.15.78:0/1548401188 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29980116d0 con 0x55cdeb0c92f0 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 -- 172.21.15.78:0/1548401188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 msgr2=0x55cdeb0d3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f299e7fc640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29940a3240 0x7f29940a5700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f29a80fee70 tx=0x7f2990005730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 --2- 172.21.15.78:0/1548401188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d3670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2998009c50 tx=0x7f299800e6c0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 -- 172.21.15.78:0/1548401188 shutdown_connections 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 --2- 172.21.15.78:0/1548401188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0d5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 --2- 172.21.15.78:0/1548401188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d3670 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 --2- 172.21.15.78:0/1548401188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 -- 172.21.15.78:0/1548401188 >> 172.21.15.78:0/1548401188 conn(0x55cdeb0c4f60 msgr2=0x55cdeb0c5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 -- 172.21.15.78:0/1548401188 shutdown_connections 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 -- 172.21.15.78:0/1548401188 wait complete. 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 -- start start 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.212+0000 7f29afc81740 1 Processor -- start 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29afc81740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29afc81740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29afc81740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29afc81740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb0d6780 con 0x55cdeb0c92f0 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29afc81740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cdeafe9f20 con 0x55cdeb0c89d0 2024-09-16T00:13:18.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29afc81740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb0d6900 con 0x55cdeb0d3bb0 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29afc81740 10 client.0 _ll_register_callbacks cb 0x55cdeafb3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51292/0 (socket says 172.21.15.78:51292) 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 -- 172.21.15.78:0/2888306047 learned_addr learned my addr 172.21.15.78:0/2888306047 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29a77fe640 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 -- 172.21.15.78:0/2888306047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 msgr2=0x55cdeb0d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 -- 172.21.15.78:0/2888306047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 msgr2=0x55cdeb0d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0d33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 -- 172.21.15.78:0/2888306047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdeada02b0 con 0x55cdeb0c92f0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f299e7fc640 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2990033000 tx=0x7f2990005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804eab98d88ef2bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.213+0000 7f2986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f2986ffd640 1 -- 172.21.15.78:0/2888306047 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990040070 con 0x55cdeb0c92f0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f2986ffd640 1 -- 172.21.15.78:0/2888306047 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2990031950 con 0x55cdeb0c92f0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 1 -- 172.21.15.78:0/2888306047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2998005500 con 0x55cdeb0c92f0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f2986ffd640 1 -- 172.21.15.78:0/2888306047 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2990035040 con 0x55cdeb0c92f0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 10 client.15225 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 1 -- 172.21.15.78:0/2888306047 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cdeb0e2040 con 0x55cdeb0c92f0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f2986ffd640 1 -- 172.21.15.78:0/2888306047 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2998005500 con 0x55cdeb0c92f0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f2986ffd640 1 -- 172.21.15.78:0/2888306047 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f299003cb30 con 0x55cdeb0c92f0 2024-09-16T00:13:18.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 10 client.15225 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 10 client.15225 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 20 client.15225 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55cdeb0e46a0 0x55cdeb0e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 4 client.15225 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29afc81740 1 -- 172.21.15.78:0/2888306047 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdeb03c600 con 0x55cdeb0e46a0 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.214+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55cdeb0e46a0 0x55cdeb0e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.215+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55cdeb0e46a0 0x55cdeb0e5b10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c5c5a1174c95c4d server_cookie=72d4de359f3f59ff in_seq=0 out_seq=0 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.215+0000 7f2986ffd640 10 client.15225 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f2986ffd640 1 -- 172.21.15.78:0/2888306047 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdeb03c600 con 0x55cdeb0e46a0 2024-09-16T00:13:18.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f2986ffd640 10 client.15225 handle_command_reply: tid=0 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 client.15225 shutdown 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 -- 172.21.15.78:0/2888306047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 msgr2=0x55cdeb0d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2990033000 tx=0x7f2990005a20 comp rx=0 tx=0).stop 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 -- 172.21.15.78:0/2888306047 shutdown_connections 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0740 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55cdeb0e46a0 0x55cdeb0e5b10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 --2- 172.21.15.78:0/2888306047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 -- 172.21.15.78:0/2888306047 >> 172.21.15.78:0/2888306047 conn(0x55cdeb0c4f60 msgr2=0x55cdeb0cac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f2986ffd640 0 client.15225 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 -- 172.21.15.78:0/2888306047 shutdown_connections 2024-09-16T00:13:18.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 1 -- 172.21.15.78:0/2888306047 wait complete. 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.216+0000 7f29afc81740 20 client.15225 trim_cache size 0 max 16384 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.233+0000 7f29afc81740 1 Processor -- start 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.233+0000 7f29afc81740 1 -- start start 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f29afc81740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f29afc81740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f29afc81740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f29afc81740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb0e3e50 con 0x55cdeb0c92f0 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f29afc81740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb0e4520 con 0x55cdeb0c89d0 2024-09-16T00:13:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f29afc81740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb0e4bf0 con 0x55cdeb0d3bb0 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51294/0 (socket says 172.21.15.78:51294) 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 -- 172.21.15.78:0/1948235727 learned_addr learned my addr 172.21.15.78:0/1948235727 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 -- 172.21.15.78:0/1948235727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 msgr2=0x55cdeb0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f29a77fe640 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 -- 172.21.15.78:0/1948235727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 msgr2=0x55cdeb0e5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 -- 172.21.15.78:0/1948235727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdeb0c4d90 con 0x55cdeb0c92f0 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f29a77fe640 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299e7fc640 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0000 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2990006010 tx=0x7f2990004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed58fe61a5e93247 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.234+0000 7f299d7fa640 1 -- 172.21.15.78:0/1948235727 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990040070 con 0x55cdeb0c92f0 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.235+0000 7f299d7fa640 1 -- 172.21.15.78:0/1948235727 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2990003dc0 con 0x55cdeb0c92f0 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.235+0000 7f299d7fa640 1 -- 172.21.15.78:0/1948235727 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299003c6a0 con 0x55cdeb0c92f0 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.235+0000 7f29afc81740 1 -- 172.21.15.78:0/1948235727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 msgr2=0x55cdeb0d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.235+0000 7f29afc81740 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0000 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2990006010 tx=0x7f2990004290 comp rx=0 tx=0).stop 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 -- 172.21.15.78:0/1948235727 shutdown_connections 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0d0000 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 --2- 172.21.15.78:0/1948235727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 -- 172.21.15.78:0/1948235727 >> 172.21.15.78:0/1948235727 conn(0x55cdeb0c4f60 msgr2=0x55cdeb0d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 -- 172.21.15.78:0/1948235727 shutdown_connections 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 -- 172.21.15.78:0/1948235727 wait complete. 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 -- start start 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 Processor -- start 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb0e5990 con 0x55cdeb0c89d0 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb0e52c0 con 0x55cdeb0d3bb0 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.236+0000 7f29afc81740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cdeb0e6060 con 0x55cdeb0c92f0 2024-09-16T00:13:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29afc81740 10 client.0 _ll_register_callbacks cb 0x7f2990009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51302/0 (socket says 172.21.15.78:51302) 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 -- 172.21.15.78:0/1359862421 learned_addr learned my addr 172.21.15.78:0/1359862421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f299e7fc640 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a77fe640 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb14cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 -- 172.21.15.78:0/1359862421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0c92f0 msgr2=0x55cdeb0e77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0e77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 -- 172.21.15.78:0/1359862421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0d3bb0 msgr2=0x55cdeb14cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb14cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 -- 172.21.15.78:0/1359862421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdeb14d360 con 0x55cdeb0c89d0 2024-09-16T00:13:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29a6ffd640 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e7270 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55cdeb0c5dc0 tx=0x7f2998009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5426cbcdad826ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f2986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f2986ffd640 1 -- 172.21.15.78:0/1359862421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998016040 con 0x55cdeb0c89d0 2024-09-16T00:13:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f2986ffd640 1 -- 172.21.15.78:0/1359862421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2998004880 con 0x55cdeb0c89d0 2024-09-16T00:13:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29afc81740 1 -- 172.21.15.78:0/1359862421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2990003dc0 con 0x55cdeb0c89d0 2024-09-16T00:13:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f2986ffd640 1 -- 172.21.15.78:0/1359862421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2998004a80 con 0x55cdeb0c89d0 2024-09-16T00:13:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29afc81740 10 client.15231 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.237+0000 7f29afc81740 1 -- 172.21.15.78:0/1359862421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cdeb14e630 con 0x55cdeb0c89d0 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f2986ffd640 1 -- 172.21.15.78:0/1359862421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2990003dc0 con 0x55cdeb0c89d0 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f2986ffd640 1 -- 172.21.15.78:0/1359862421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f299800f040 con 0x55cdeb0c89d0 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f29afc81740 10 client.15231 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f29afc81740 10 client.15231 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f29afc81740 20 client.15231 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f29afc81740 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55cdeb1530a0 0x55cdeb151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f29afc81740 4 client.15231 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f29afc81740 1 -- 172.21.15.78:0/1359862421 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "status"}) -- 0x55cdeb036b10 con 0x55cdeb1530a0 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.238+0000 7f299e7fc640 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55cdeb1530a0 0x55cdeb151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f299e7fc640 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55cdeb1530a0 0x55cdeb151db0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e48ca3f2088ff686 server_cookie=cc8a6b82750a3d85 in_seq=0 out_seq=0 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f2986ffd640 10 client.15231 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:13:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f2986ffd640 1 -- 172.21.15.78:0/1359862421 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55cdeb036b10 con 0x55cdeb1530a0 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f2986ffd640 10 client.15231 handle_command_reply: tid=0 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 client.15231 shutdown 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 -- 172.21.15.78:0/1359862421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c89d0 msgr2=0x55cdeb0e7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e7270 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55cdeb0c5dc0 tx=0x7f2998009540 comp rx=0 tx=0).stop 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 -- 172.21.15.78:0/1359862421 shutdown_connections 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55cdeb1530a0 0x55cdeb151db0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cdeb0d3bb0 0x55cdeb14cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeb0c92f0 0x55cdeb0e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 --2- 172.21.15.78:0/1359862421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cdeb0c89d0 0x55cdeb0e7270 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f2986ffd640 0 client.15231 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:13:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 -- 172.21.15.78:0/1359862421 >> 172.21.15.78:0/1359862421 conn(0x55cdeb0c4f60 msgr2=0x55cdeb0d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 -- 172.21.15.78:0/1359862421 shutdown_connections 2024-09-16T00:13:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 1 -- 172.21.15.78:0/1359862421 wait complete. 2024-09-16T00:13:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.239+0000 7f29afc81740 20 client.15231 trim_cache size 0 max 16384 2024-09-16T00:13:18.265 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:18.265 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:18.265 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-09-16T00:13:18.265 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24440, 2024-09-16T00:13:18.266 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:18.266 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:18.266 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:18.266 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 97.732023085999998, 2024-09-16T00:13:18.266 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:13:18.266 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:18.267 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:13:18.267 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 157.16918368 2024-09-16T00:13:18.267 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 -- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29940a3240 msgr2=0x7f29940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29940a3240 0x7f29940a5700 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f29a80fee70 tx=0x7f2990005730 comp rx=0 tx=0).stop 2024-09-16T00:13:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 -- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102ef0 msgr2=0x7f29a819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102ef0 0x7f29a819ce10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29a000b970 tx=0x7f29a000be30 comp rx=0 tx=0).stop 2024-09-16T00:13:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 -- 172.21.15.78:0/3680328289 shutdown_connections 2024-09-16T00:13:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29940a3240 0x7f29940a5700 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29a8102ef0 0x7f29a819ce10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8102570 0x7f29a8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 --2- 172.21.15.78:0/3680328289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29a8100e20 0x7f29a8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 -- 172.21.15.78:0/3680328289 >> 172.21.15.78:0/3680328289 conn(0x7f29a80fc7d0 msgr2=0x7f29a8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 -- 172.21.15.78:0/3680328289 shutdown_connections 2024-09-16T00:13:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:18.240+0000 7f29ad8a8640 1 -- 172.21.15.78:0/3680328289 wait complete. 2024-09-16T00:13:18.996 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T00:13:19.240 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.752+0000 7ff9a1b89640 1 -- 172.21.15.78:0/3318662558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c106b80 msgr2=0x7ff99c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.752+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/3318662558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c106b80 0x7ff99c106f60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009900 tx=0x7ff98c02a410 comp rx=0 tx=0).stop 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.752+0000 7ff9a1b89640 1 -- 172.21.15.78:0/3318662558 shutdown_connections 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.752+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/3318662558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99c100a50 0x7ff99c10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.752+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/3318662558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff99c1000b0 0x7ff99c100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.752+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/3318662558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c106b80 0x7ff99c106f60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.752+0000 7ff9a1b89640 1 -- 172.21.15.78:0/3318662558 >> 172.21.15.78:0/3318662558 conn(0x7ff99c0fbee0 msgr2=0x7ff99c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.752+0000 7ff9a1b89640 1 -- 172.21.15.78:0/3318662558 shutdown_connections 2024-09-16T00:13:19.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 -- 172.21.15.78:0/3318662558 wait complete. 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 Processor -- start 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 -- start start 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c1000b0 0x7ff99c101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff99c100a50 0x7ff99c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99c106b80 0x7ff99c101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c112090 con 0x7ff99c1000b0 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c111f10 con 0x7ff99c106b80 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.753+0000 7ff9a1b89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c112210 con 0x7ff99c100a50 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c1000b0 0x7ff99c101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c1000b0 0x7ff99c101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51316/0 (socket says 172.21.15.78:51316) 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 -- 172.21.15.78:0/2945836155 learned_addr learned my addr 172.21.15.78:0/2945836155 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:19.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99affd640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff99c100a50 0x7ff99c1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99bfff640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99c106b80 0x7ff99c101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 -- 172.21.15.78:0/2945836155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff99c100a50 msgr2=0x7ff99c1017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff99c100a50 0x7ff99c1017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 -- 172.21.15.78:0/2945836155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99c106b80 msgr2=0x7ff99c101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99c106b80 0x7ff99c101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 -- 172.21.15.78:0/2945836155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c1a69f0 con 0x7ff99c1000b0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c1000b0 0x7ff99c101290 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff98c006010 tx=0x7ff98c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff998ff9640 1 -- 172.21.15.78:0/2945836155 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c03f070 con 0x7ff99c1000b0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff998ff9640 1 -- 172.21.15.78:0/2945836155 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c031c30 con 0x7ff99c1000b0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff9a1b89640 1 -- 172.21.15.78:0/2945836155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c1a6c80 con 0x7ff99c1000b0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff998ff9640 1 -- 172.21.15.78:0/2945836155 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c035030 con 0x7ff99c1000b0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.754+0000 7ff9a1b89640 1 -- 172.21.15.78:0/2945836155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c1a71b0 con 0x7ff99c1000b0 2024-09-16T00:13:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.756+0000 7ff998ff9640 1 -- 172.21.15.78:0/2945836155 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff98c042070 con 0x7ff99c1000b0 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff9a3f62740 1 Processor -- start 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff9a3f62740 1 -- start start 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e505db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e510670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 0x56015e512fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56015e46c090 con 0x56015e5059d0 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56015e413a70 con 0x56015e5062f0 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56015e468050 con 0x56015e510bb0 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff998ff9640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff9840a32c0 0x7ff9840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff99bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e505db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff99bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e505db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51324/0 (socket says 172.21.15.78:51324) 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff99bfff640 1 -- 172.21.15.78:0/3264334627 learned_addr learned my addr 172.21.15.78:0/3264334627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff99bfff640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff9840a32c0 0x7ff9840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff998ff9640 1 -- 172.21.15.78:0/2945836155 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff98c0f0300 con 0x7ff99c1000b0 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.757+0000 7ff99affd640 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 0x56015e512fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e510670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff99bfff640 1 -- 172.21.15.78:0/3264334627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 msgr2=0x56015e512fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff99bfff640 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 0x56015e512fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff99bfff640 1 -- 172.21.15.78:0/3264334627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 msgr2=0x56015e510670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff99bfff640 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e510670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff99bfff640 1 -- 172.21.15.78:0/3264334627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015e420130 con 0x56015e5059d0 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff99bfff640 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e505db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff994012d80 tx=0x7ff994010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b26a4eb3f909c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff99bfff640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff9840a32c0 0x7ff9840a5780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff994010c40 tx=0x7ff99400c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff981ffb640 1 -- 172.21.15.78:0/3264334627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994024070 con 0x56015e5059d0 2024-09-16T00:13:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff981ffb640 1 -- 172.21.15.78:0/3264334627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9940125a0 con 0x56015e5059d0 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.758+0000 7ff981ffb640 1 -- 172.21.15.78:0/3264334627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994005220 con 0x56015e5059d0 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3264334627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 msgr2=0x56015e505db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e505db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff994012d80 tx=0x7ff994010740 comp rx=0 tx=0).stop 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3264334627 shutdown_connections 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 0x56015e512fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e510670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/3264334627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e505db0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3264334627 >> 172.21.15.78:0/3264334627 conn(0x56015e501f60 msgr2=0x56015e502340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3264334627 shutdown_connections 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3264334627 wait complete. 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 -- start start 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.759+0000 7ff9a3f62740 1 Processor -- start 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5059d0 0x56015e5104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 0x56015e50d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e510bb0 0x56015e50dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56015e513780 con 0x56015e510bb0 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56015e426f20 con 0x56015e5059d0 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56015e513900 con 0x56015e5062f0 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff9a3f62740 10 client.0 _ll_register_callbacks cb 0x56015e3f0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e510bb0 0x56015e50dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e510bb0 0x56015e50dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51340/0 (socket says 172.21.15.78:51340) 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 0x56015e50d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 -- 172.21.15.78:0/2213380033 learned_addr learned my addr 172.21.15.78:0/2213380033 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5059d0 0x56015e5104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 -- 172.21.15.78:0/2213380033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 msgr2=0x56015e50d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 0x56015e50d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 -- 172.21.15.78:0/2213380033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5059d0 msgr2=0x56015e5104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5059d0 0x56015e5104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 -- 172.21.15.78:0/2213380033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015e1dd2b0 con 0x56015e510bb0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff99bfff640 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e510bb0 0x56015e50dd40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff994012fd0 tx=0x7ff99400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959c93ffdf012d02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff97affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff97affd640 1 -- 172.21.15.78:0/2213380033 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994024050 con 0x56015e510bb0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff97affd640 1 -- 172.21.15.78:0/2213380033 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff994005800 con 0x56015e510bb0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2213380033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9940125a0 con 0x56015e510bb0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.760+0000 7ff97affd640 1 -- 172.21.15.78:0/2213380033 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff994004a00 con 0x56015e510bb0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff9a3f62740 10 client.15243 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2213380033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56015e51f060 con 0x56015e510bb0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff97affd640 1 -- 172.21.15.78:0/2213380033 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9940125a0 con 0x56015e510bb0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff97affd640 1 -- 172.21.15.78:0/2213380033 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff99400f040 con 0x56015e510bb0 2024-09-16T00:13:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff9a3f62740 10 client.15243 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff9a3f62740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff9a3f62740 20 client.15243 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56015e5213e0 0x56015e522850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff9a3f62740 4 client.15243 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2213380033 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56015e479600 con 0x56015e5213e0 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.761+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56015e5213e0 0x56015e522850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.762+0000 7ff97affd640 10 client.15243 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.762+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56015e5213e0 0x56015e522850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f04ff2be6069e5f server_cookie=16531abafab3c39b in_seq=0 out_seq=0 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.762+0000 7ff97affd640 1 -- 172.21.15.78:0/2213380033 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56015e479600 con 0x56015e5213e0 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.762+0000 7ff97affd640 10 client.15243 handle_command_reply: tid=0 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.762+0000 7ff9a3f62740 1 client.15243 shutdown 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.762+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2213380033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e510bb0 msgr2=0x56015e50dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e510bb0 0x56015e50dd40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff994012fd0 tx=0x7ff99400cbf0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2213380033 shutdown_connections 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56015e5213e0 0x56015e522850 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e510bb0 0x56015e50dd40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 0x56015e50d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2213380033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5059d0 0x56015e5104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff97affd640 0 client.15243 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2213380033 >> 172.21.15.78:0/2213380033 conn(0x56015e501f60 msgr2=0x56015e507c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2213380033 shutdown_connections 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2213380033 wait complete. 2024-09-16T00:13:19.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.763+0000 7ff9a3f62740 20 client.15243 trim_cache size 0 max 16384 2024-09-16T00:13:19.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.780+0000 7ff9a3f62740 1 Processor -- start 2024-09-16T00:13:19.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.780+0000 7ff9a3f62740 1 -- start start 2024-09-16T00:13:19.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.780+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e520d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.780+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e522180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.780+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 0x56015e508d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.780+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56015e520b90 con 0x56015e5059d0 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.780+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56015e521260 con 0x56015e5062f0 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56015e521930 con 0x56015e510bb0 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e520d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e520d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51342/0 (socket says 172.21.15.78:51342) 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 -- 172.21.15.78:0/3292430411 learned_addr learned my addr 172.21.15.78:0/3292430411 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99affd640 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e522180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 -- 172.21.15.78:0/3292430411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 msgr2=0x56015e508d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 0x56015e508d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 -- 172.21.15.78:0/3292430411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 msgr2=0x56015e522180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:19.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e522180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 -- 172.21.15.78:0/3292430411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015e50e340 con 0x56015e5059d0 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99affd640 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e522180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e520d10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0b8a20 tx=0x7ff98c0ed970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d195db0744698e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff9817fa640 1 -- 172.21.15.78:0/3292430411 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c049040 con 0x56015e5059d0 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff9817fa640 1 -- 172.21.15.78:0/3292430411 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c0bc5c0 con 0x56015e5059d0 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.781+0000 7ff9817fa640 1 -- 172.21.15.78:0/3292430411 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0ba950 con 0x56015e5059d0 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3292430411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 msgr2=0x56015e520d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e520d10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0b8a20 tx=0x7ff98c0ed970 comp rx=0 tx=0).stop 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3292430411 shutdown_connections 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e510bb0 0x56015e508d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e5062f0 0x56015e522180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/3292430411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e520d10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3292430411 >> 172.21.15.78:0/3292430411 conn(0x56015e501f60 msgr2=0x56015e508610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3292430411 shutdown_connections 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 -- 172.21.15.78:0/3292430411 wait complete. 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.782+0000 7ff9a3f62740 1 -- start start 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff9a3f62740 1 Processor -- start 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e523f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 0x56015e524490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff9a3f62740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e510bb0 0x56015e589ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56015e5226d0 con 0x56015e5059d0 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56015e522000 con 0x56015e510bb0 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff9a3f62740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56015e522da0 con 0x56015e5062f0 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff9a3f62740 10 client.0 _ll_register_callbacks cb 0x7ff994012b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e523f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e523f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51348/0 (socket says 172.21.15.78:51348) 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 -- 172.21.15.78:0/2666239914 learned_addr learned my addr 172.21.15.78:0/2666239914 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99affd640 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 0x56015e524490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99bfff640 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e510bb0 0x56015e589ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 -- 172.21.15.78:0/2666239914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 msgr2=0x56015e524490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 0x56015e524490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 -- 172.21.15.78:0/2666239914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e510bb0 msgr2=0x56015e589ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e510bb0 0x56015e589ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 -- 172.21.15.78:0/2666239914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015e58a020 con 0x56015e5059d0 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff99b7fe640 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e523f50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0bc400 tx=0x7ff98c0ba660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad8fc876bf40b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.783+0000 7ff97affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff97affd640 1 -- 172.21.15.78:0/2666239914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0da060 con 0x56015e5059d0 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff97affd640 1 -- 172.21.15.78:0/2666239914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff98c03b4f0 con 0x56015e5059d0 2024-09-16T00:13:19.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2666239914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98c0bc5c0 con 0x56015e5059d0 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff97affd640 1 -- 172.21.15.78:0/2666239914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff98c049040 con 0x56015e5059d0 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 10 client.15249 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2666239914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56015e58b2d0 con 0x56015e5059d0 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff97affd640 1 -- 172.21.15.78:0/2666239914 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c0bc5c0 con 0x56015e5059d0 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff97affd640 1 -- 172.21.15.78:0/2666239914 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff98c0c4da0 con 0x56015e5059d0 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 10 client.15249 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 20 client.15249 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56015e58fca0 0x56015e58e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 4 client.15249 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.784+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2666239914 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "status"}) -- 0x56015e473b10 con 0x56015e58fca0 2024-09-16T00:13:19.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.785+0000 7ff99affd640 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56015e58fca0 0x56015e58e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:19.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.785+0000 7ff99affd640 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56015e58fca0 0x56015e58e9b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89c6ca998d2575bd server_cookie=ff4e7fea7829d14d in_seq=0 out_seq=0 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.785+0000 7ff97affd640 10 client.15249 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.785+0000 7ff97affd640 1 -- 172.21.15.78:0/2666239914 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56015e473b10 con 0x56015e58fca0 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.785+0000 7ff97affd640 10 client.15249 handle_command_reply: tid=0 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.785+0000 7ff9a3f62740 1 client.15249 shutdown 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.785+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2666239914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 msgr2=0x56015e523f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e523f50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0bc400 tx=0x7ff98c0ba660 comp rx=0 tx=0).stop 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2666239914 shutdown_connections 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56015e58fca0 0x56015e58e9b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56015e510bb0 0x56015e589ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56015e5062f0 0x56015e524490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 --2- 172.21.15.78:0/2666239914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56015e5059d0 0x56015e523f50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2666239914 >> 172.21.15.78:0/2666239914 conn(0x56015e501f60 msgr2=0x56015e5007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff97affd640 0 client.15249 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2666239914 shutdown_connections 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 1 -- 172.21.15.78:0/2666239914 wait complete. 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a3f62740 20 client.15249 trim_cache size 0 max 16384 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24445, 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:19.794 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:19.795 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:19.795 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 100.28808227099999, 2024-09-16T00:13:19.795 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:13:19.795 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:19.795 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:13:19.795 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 156.11440208900001 2024-09-16T00:13:19.795 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 -- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff9840a32c0 msgr2=0x7ff9840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff9840a32c0 0x7ff9840a5780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff994010c40 tx=0x7ff99400c000 comp rx=0 tx=0).stop 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 -- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c1000b0 msgr2=0x7ff99c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c1000b0 0x7ff99c101290 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff98c006010 tx=0x7ff98c003c10 comp rx=0 tx=0).stop 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 -- 172.21.15.78:0/2945836155 shutdown_connections 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff9840a32c0 0x7ff9840a5780 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff99c106b80 0x7ff99c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff99c100a50 0x7ff99c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 --2- 172.21.15.78:0/2945836155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff99c1000b0 0x7ff99c101290 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 -- 172.21.15.78:0/2945836155 >> 172.21.15.78:0/2945836155 conn(0x7ff99c0fbee0 msgr2=0x7ff99c0fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.786+0000 7ff9a1b89640 1 -- 172.21.15.78:0/2945836155 shutdown_connections 2024-09-16T00:13:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:19.787+0000 7ff9a1b89640 1 -- 172.21.15.78:0/2945836155 wait complete. 2024-09-16T00:13:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:19 smithi193 ceph-mon[35484]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:19 smithi148 ceph-mon[35599]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:19 smithi078 ceph-mon[31157]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:20.472 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:13:20.721 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 -- 172.21.15.78:0/447336372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68102ef0 msgr2=0x7faa6810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/447336372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68102ef0 0x7faa6810da40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faa60009ca0 tx=0x7faa6002a950 comp rx=0 tx=0).stop 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 -- 172.21.15.78:0/447336372 shutdown_connections 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/447336372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68102ef0 0x7faa6810da40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/447336372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa68102570 0x7faa681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/447336372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa68100e20 0x7faa68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 -- 172.21.15.78:0/447336372 >> 172.21.15.78:0/447336372 conn(0x7faa680fc7d0 msgr2=0x7faa680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 -- 172.21.15.78:0/447336372 shutdown_connections 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.184+0000 7faa6ddc9640 1 -- 172.21.15.78:0/447336372 wait complete. 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.185+0000 7faa6ddc9640 1 Processor -- start 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.185+0000 7faa6ddc9640 1 -- start start 2024-09-16T00:13:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.185+0000 7faa6ddc9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68100e20 0x7faa681985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.185+0000 7faa6ddc9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa68102570 0x7faa68198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.185+0000 7faa6ddc9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa68102ef0 0x7faa6819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.185+0000 7faa6ddc9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa68111020 con 0x7faa68100e20 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.185+0000 7faa6ddc9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faa68110ea0 con 0x7faa68102ef0 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.185+0000 7faa6ddc9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa681111a0 con 0x7faa68102570 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68100e20 0x7faa681985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68100e20 0x7faa681985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51360/0 (socket says 172.21.15.78:51360) 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 -- 172.21.15.78:0/2063700234 learned_addr learned my addr 172.21.15.78:0/2063700234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa67fff640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa68102ef0 0x7faa6819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa5effd640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa68102570 0x7faa68198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 -- 172.21.15.78:0/2063700234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa68102570 msgr2=0x7faa68198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa68102570 0x7faa68198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 -- 172.21.15.78:0/2063700234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa68102ef0 msgr2=0x7faa6819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa68102ef0 0x7faa6819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 -- 172.21.15.78:0/2063700234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6819d590 con 0x7faa68100e20 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa5effd640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa68102570 0x7faa68198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa67fff640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa68102ef0 0x7faa6819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa677fe640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68100e20 0x7faa681985e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa5800b4a0 tx=0x7faa5800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa657fa640 1 -- 172.21.15.78:0/2063700234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58014070 con 0x7faa68100e20 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa657fa640 1 -- 172.21.15.78:0/2063700234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa58010af0 con 0x7faa68100e20 2024-09-16T00:13:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa6ddc9640 1 -- 172.21.15.78:0/2063700234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6819d880 con 0x7faa68100e20 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa657fa640 1 -- 172.21.15.78:0/2063700234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58017d10 con 0x7faa68100e20 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.186+0000 7faa6ddc9640 1 -- 172.21.15.78:0/2063700234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa681a5160 con 0x7faa68100e20 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.188+0000 7faa657fa640 1 -- 172.21.15.78:0/2063700234 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7faa58017070 con 0x7faa68100e20 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa701a2740 1 Processor -- start 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa701a2740 1 -- start start 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa701a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 0x563743105db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa701a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x563743110670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa701a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x563743112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa701a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56374306c090 con 0x563743110bb0 2024-09-16T00:13:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa701a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563743013a70 con 0x5637431059d0 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa701a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563743068050 con 0x5637431062f0 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa5effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x563743112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa5effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x563743112fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51368/0 (socket says 172.21.15.78:51368) 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa5effd640 1 -- 172.21.15.78:0/3057279685 learned_addr learned my addr 172.21.15.78:0/3057279685 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa657fa640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa540a3240 0x7faa540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa677fe640 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x563743110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.189+0000 7faa67fff640 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 0x563743105db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa67fff640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa540a3240 0x7faa540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5effd640 1 -- 172.21.15.78:0/3057279685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 msgr2=0x563743110670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5effd640 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x563743110670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5effd640 1 -- 172.21.15.78:0/3057279685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 msgr2=0x563743105db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5effd640 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 0x563743105db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5effd640 1 -- 172.21.15.78:0/3057279685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563743020130 con 0x563743110bb0 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa677fe640 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x563743110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa657fa640 1 -- 172.21.15.78:0/2063700234 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faa580cf150 con 0x7faa68100e20 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5effd640 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x563743112fa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faa5000a9d0 tx=0x7faa5000ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6119baa83c2fcea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5dffb640 1 -- 172.21.15.78:0/3057279685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50015070 con 0x563743110bb0 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5dffb640 1 -- 172.21.15.78:0/3057279685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5000cc60 con 0x563743110bb0 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa67fff640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa540a3240 0x7faa540a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faa60009a60 tx=0x7faa600095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.190+0000 7faa5dffb640 1 -- 172.21.15.78:0/3057279685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50018600 con 0x563743110bb0 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 -- 172.21.15.78:0/3057279685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 msgr2=0x563743112fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x563743112fa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faa5000a9d0 tx=0x7faa5000ae90 comp rx=0 tx=0).stop 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 -- 172.21.15.78:0/3057279685 shutdown_connections 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x563743112fa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x563743110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 --2- 172.21.15.78:0/3057279685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 0x563743105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 -- 172.21.15.78:0/3057279685 >> 172.21.15.78:0/3057279685 conn(0x563743101f60 msgr2=0x563743102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 -- 172.21.15.78:0/3057279685 shutdown_connections 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 -- 172.21.15.78:0/3057279685 wait complete. 2024-09-16T00:13:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.191+0000 7faa701a2740 1 -- start start 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa701a2740 1 Processor -- start 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa701a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743110330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa701a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 0x56374310d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa701a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x56374310dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa701a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563743113780 con 0x5637431059d0 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa701a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563743026f20 con 0x5637431062f0 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa701a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563743113900 con 0x563743110bb0 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa701a2740 10 client.0 _ll_register_callbacks cb 0x563742ff0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743110330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743110330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51376/0 (socket says 172.21.15.78:51376) 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 -- 172.21.15.78:0/4108903567 learned_addr learned my addr 172.21.15.78:0/4108903567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa5effd640 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 0x56374310d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa67fff640 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x56374310dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 -- 172.21.15.78:0/4108903567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 msgr2=0x56374310dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x56374310dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 -- 172.21.15.78:0/4108903567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 msgr2=0x56374310d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 0x56374310d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 -- 172.21.15.78:0/4108903567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563742ddd2b0 con 0x5637431059d0 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa677fe640 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743110330 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faa5800be60 tx=0x7faa580092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0488f255666c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.192+0000 7faa46ffd640 1 -- 172.21.15.78:0/4108903567 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5801e070 con 0x5637431059d0 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa46ffd640 1 -- 172.21.15.78:0/4108903567 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faa58010ca0 con 0x5637431059d0 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 1 -- 172.21.15.78:0/4108903567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5000cc60 con 0x5637431059d0 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa46ffd640 1 -- 172.21.15.78:0/4108903567 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa5809b3b0 con 0x5637431059d0 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 10 client.15261 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 1 -- 172.21.15.78:0/4108903567 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56374311eff0 con 0x5637431059d0 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa46ffd640 1 -- 172.21.15.78:0/4108903567 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5000cc60 con 0x5637431059d0 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa46ffd640 1 -- 172.21.15.78:0/4108903567 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7faa580d10d0 con 0x5637431059d0 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 20 client.15261 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563743121580 0x5637431229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 4 client.15261 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa701a2740 1 -- 172.21.15.78:0/4108903567 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563743079600 con 0x563743121580 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.193+0000 7faa5effd640 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563743121580 0x5637431229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.194+0000 7faa5effd640 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563743121580 0x5637431229f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d5ad98dd9ce408 server_cookie=d68530482a387afe in_seq=0 out_seq=0 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.194+0000 7faa46ffd640 10 client.15261 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa46ffd640 1 -- 172.21.15.78:0/4108903567 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563743079600 con 0x563743121580 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa46ffd640 10 client.15261 handle_command_reply: tid=0 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 client.15261 shutdown 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 -- 172.21.15.78:0/4108903567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 msgr2=0x563743110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743110330 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faa5800be60 tx=0x7faa580092f0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 -- 172.21.15.78:0/4108903567 shutdown_connections 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563743121580 0x5637431229f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x56374310dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 0x56374310d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa46ffd640 0 client.15261 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 --2- 172.21.15.78:0/4108903567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743110330 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 -- 172.21.15.78:0/4108903567 >> 172.21.15.78:0/4108903567 conn(0x563743101f60 msgr2=0x563743107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 -- 172.21.15.78:0/4108903567 shutdown_connections 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 1 -- 172.21.15.78:0/4108903567 wait complete. 2024-09-16T00:13:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.195+0000 7faa701a2740 20 client.15261 trim_cache size 0 max 16384 2024-09-16T00:13:21.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa701a2740 1 Processor -- start 2024-09-16T00:13:21.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa701a2740 1 -- start start 2024-09-16T00:13:21.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa701a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 0x56374310cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa701a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x56374310d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa701a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x56374310db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa701a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563743120d30 con 0x563743110bb0 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa701a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563743121400 con 0x5637431059d0 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa701a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563743121ad0 con 0x5637431062f0 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x56374310db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x56374310db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51378/0 (socket says 172.21.15.78:51378) 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa67fff640 1 -- 172.21.15.78:0/2742930333 learned_addr learned my addr 172.21.15.78:0/2742930333 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.213+0000 7faa5effd640 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x56374310d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa677fe640 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 0x56374310cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa67fff640 1 -- 172.21.15.78:0/2742930333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 msgr2=0x56374310d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:21.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa67fff640 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x56374310d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa67fff640 1 -- 172.21.15.78:0/2742930333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 msgr2=0x56374310cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa67fff640 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 0x56374310cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa67fff640 1 -- 172.21.15.78:0/2742930333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637431106a0 con 0x563743110bb0 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa5effd640 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x56374310d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa67fff640 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x56374310db80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faa6002f4a0 tx=0x7faa6003b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b939023c36a2662c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa5d7fa640 1 -- 172.21.15.78:0/2742930333 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6004e020 con 0x563743110bb0 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa5d7fa640 1 -- 172.21.15.78:0/2742930333 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa60004520 con 0x563743110bb0 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.214+0000 7faa5d7fa640 1 -- 172.21.15.78:0/2742930333 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60034040 con 0x563743110bb0 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 -- 172.21.15.78:0/2742930333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 msgr2=0x56374310db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x56374310db80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faa6002f4a0 tx=0x7faa6003b070 comp rx=0 tx=0).stop 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 -- 172.21.15.78:0/2742930333 shutdown_connections 2024-09-16T00:13:21.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563743110bb0 0x56374310db80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637431062f0 0x56374310d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 --2- 172.21.15.78:0/2742930333 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431059d0 0x56374310cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 -- 172.21.15.78:0/2742930333 >> 172.21.15.78:0/2742930333 conn(0x563743101f60 msgr2=0x5637430fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 -- 172.21.15.78:0/2742930333 shutdown_connections 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 -- 172.21.15.78:0/2742930333 wait complete. 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 -- start start 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.215+0000 7faa701a2740 1 Processor -- start 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa701a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743125fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa701a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 0x563743126520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa701a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x563743189c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa701a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563743122870 con 0x5637431059d0 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa701a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5637431221a0 con 0x5637431062f0 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa701a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563743122f40 con 0x563743110bb0 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743125fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa701a2740 10 client.0 _ll_register_callbacks cb 0x7faa58097720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743125fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51394/0 (socket says 172.21.15.78:51394) 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 -- 172.21.15.78:0/3106378636 learned_addr learned my addr 172.21.15.78:0/3106378636 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa5effd640 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 0x563743126520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa67fff640 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x563743189c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 -- 172.21.15.78:0/3106378636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 msgr2=0x563743189c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x563743189c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 -- 172.21.15.78:0/3106378636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 msgr2=0x563743126520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:21.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 0x563743126520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 -- 172.21.15.78:0/3106378636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563743126e20 con 0x5637431059d0 2024-09-16T00:13:21.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa67fff640 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x563743189c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:21.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa677fe640 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743125fe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faa580d1490 tx=0x7faa58097650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d62b7410285b77 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:21.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:21.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa46ffd640 1 -- 172.21.15.78:0/3106378636 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5801e070 con 0x5637431059d0 2024-09-16T00:13:21.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa46ffd640 1 -- 172.21.15.78:0/3106378636 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faa58066230 con 0x5637431059d0 2024-09-16T00:13:21.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.216+0000 7faa701a2740 1 -- 172.21.15.78:0/3106378636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa60004520 con 0x5637431059d0 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa46ffd640 1 -- 172.21.15.78:0/3106378636 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa5809ad10 con 0x5637431059d0 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa701a2740 10 client.15267 fetch_fsmap learned FSMap version 28 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa701a2740 1 -- 172.21.15.78:0/3106378636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56374318b2f0 con 0x5637431059d0 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa46ffd640 1 -- 172.21.15.78:0/3106378636 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa60004520 con 0x5637431059d0 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa46ffd640 1 -- 172.21.15.78:0/3106378636 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7faa58097100 con 0x5637431059d0 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa701a2740 10 client.15267 fetch_fsmap finished waiting for FSMap version 28 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa701a2740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa701a2740 20 client.15267 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa701a2740 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56374318fd00 0x56374318ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa701a2740 4 client.15267 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:21.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa701a2740 1 -- 172.21.15.78:0/3106378636 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "status"}) -- 0x563743073b10 con 0x56374318fd00 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.217+0000 7faa5effd640 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56374318fd00 0x56374318ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa46ffd640 10 client.15267 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa5effd640 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56374318fd00 0x56374318ea10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d2310d93dc4460a server_cookie=2c49820c6accadd7 in_seq=0 out_seq=0 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa46ffd640 1 -- 172.21.15.78:0/3106378636 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563743073b10 con 0x56374318fd00 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa46ffd640 10 client.15267 handle_command_reply: tid=0 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 client.15267 shutdown 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 -- 172.21.15.78:0/3106378636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 msgr2=0x563743125fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743125fe0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faa580d1490 tx=0x7faa58097650 comp rx=0 tx=0).stop 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 -- 172.21.15.78:0/3106378636 shutdown_connections 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56374318fd00 0x56374318ea10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563743110bb0 0x563743189c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637431062f0 0x563743126520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 --2- 172.21.15.78:0/3106378636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637431059d0 0x563743125fe0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa46ffd640 0 client.15267 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:21.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 -- 172.21.15.78:0/3106378636 >> 172.21.15.78:0/3106378636 conn(0x563743101f60 msgr2=0x56374310d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:21.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 -- 172.21.15.78:0/3106378636 shutdown_connections 2024-09-16T00:13:21.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 1 -- 172.21.15.78:0/3106378636 wait complete. 2024-09-16T00:13:21.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.218+0000 7faa701a2740 20 client.15267 trim_cache size 0 max 16384 2024-09-16T00:13:21.233 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24470, 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 135.59276292300001, 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 28, 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 86, 2024-09-16T00:13:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 147.385996818 2024-09-16T00:13:21.235 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:21.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 -- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa540a3240 msgr2=0x7faa540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa540a3240 0x7faa540a5700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faa60009a60 tx=0x7faa600095c0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 -- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68100e20 msgr2=0x7faa681985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68100e20 0x7faa681985e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa5800b4a0 tx=0x7faa5800b960 comp rx=0 tx=0).stop 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 -- 172.21.15.78:0/2063700234 shutdown_connections 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa540a3240 0x7faa540a5700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa68102ef0 0x7faa6819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa68102570 0x7faa68198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 --2- 172.21.15.78:0/2063700234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa68100e20 0x7faa681985e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 -- 172.21.15.78:0/2063700234 >> 172.21.15.78:0/2063700234 conn(0x7faa680fc7d0 msgr2=0x7faa6806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 -- 172.21.15.78:0/2063700234 shutdown_connections 2024-09-16T00:13:21.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:21.219+0000 7faa6ddc9640 1 -- 172.21.15.78:0/2063700234 wait complete. 2024-09-16T00:13:21.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:21 smithi193 ceph-mon[35484]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:21 smithi148 ceph-mon[35599]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:21.971 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:21 smithi078 ceph-mon[31157]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:21.999 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T00:13:22.016 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T00:13:22.016 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T00:13:22.017 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T00:13:22.017 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T00:13:22.033 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T00:13:22.034 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-16T00:13:22.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:22.034 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-16T00:13:22.051 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-16T00:13:22.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:22.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:13:22.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:22.119 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-09-16T00:13:22.218 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-16T00:13:22.218 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:13:22.219 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-16T00:13:22.219 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-16T00:13:22.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:13:22.468 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1820129288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 msgr2=0x7f0aa410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1820129288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 0x7f0aa410eaa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009ca0 tx=0x7f0a9c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1820129288 shutdown_connections 2024-09-16T00:13:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1820129288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 0x7f0aa410eaa0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1820129288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aa41027a0 0x7f0aa41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1820129288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aa4101df0 0x7f0aa41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1820129288 >> 172.21.15.78:0/1820129288 conn(0x7f0aa40fd8b0 msgr2=0x7f0aa40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:23.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1820129288 shutdown_connections 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.989+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1820129288 wait complete. 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.990+0000 7f0aa8cfd640 1 Processor -- start 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.990+0000 7f0aa8cfd640 1 -- start start 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.990+0000 7f0aa8cfd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aa4101df0 0x7f0aa41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.990+0000 7f0aa8cfd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aa41027a0 0x7f0aa4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.990+0000 7f0aa8cfd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 0x7f0aa419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.990+0000 7f0aa8cfd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4111fa0 con 0x7f0aa4106f20 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.990+0000 7f0aa8cfd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4111e20 con 0x7f0aa41027a0 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.990+0000 7f0aa8cfd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4112120 con 0x7f0aa4101df0 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 0x7f0aa419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 0x7f0aa419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51410/0 (socket says 172.21.15.78:51410) 2024-09-16T00:13:23.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 -- 172.21.15.78:0/1207367908 learned_addr learned my addr 172.21.15.78:0/1207367908 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa1d9b640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aa41027a0 0x7f0aa4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa259c640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aa4101df0 0x7f0aa41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 -- 172.21.15.78:0/1207367908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aa4101df0 msgr2=0x7f0aa41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aa4101df0 0x7f0aa41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 -- 172.21.15.78:0/1207367908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aa41027a0 msgr2=0x7f0aa4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aa41027a0 0x7f0aa4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 -- 172.21.15.78:0/1207367908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa419d380 con 0x7f0aa4106f20 2024-09-16T00:13:23.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa1d9b640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aa41027a0 0x7f0aa4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa2d9d640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 0x7f0aa419cca0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c031a20 tx=0x7f0a9c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0a937fe640 1 -- 172.21.15.78:0/1207367908 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c003c30 con 0x7f0aa4106f20 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0a937fe640 1 -- 172.21.15.78:0/1207367908 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a9c003dd0 con 0x7f0aa4106f20 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa419d610 con 0x7f0aa4106f20 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0a937fe640 1 -- 172.21.15.78:0/1207367908 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c02fcf0 con 0x7f0aa4106f20 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.991+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa41a4f50 con 0x7f0aa4106f20 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.992+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa41021d0 con 0x7f0aa4106f20 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.993+0000 7f0a937fe640 1 -- 172.21.15.78:0/1207367908 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f0a9c004050 con 0x7f0aa4106f20 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.994+0000 7f0a937fe640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a8c0a32c0 0x7f0a8c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.994+0000 7f0aa259c640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a8c0a32c0 0x7f0a8c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.994+0000 7f0a937fe640 1 -- 172.21.15.78:0/1207367908 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0a9c0f11b0 con 0x7f0aa4106f20 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.995+0000 7f0aa259c640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a8c0a32c0 0x7f0a8c0a5780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0a94005fd0 tx=0x7f0a94005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:23.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:22.997+0000 7f0a937fe640 1 -- 172.21.15.78:0/1207367908 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a9c0bdc00 con 0x7f0aa4106f20 2024-09-16T00:13:23.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.198+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0aa410a160 con 0x7f0aa4106f20 2024-09-16T00:13:23.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.199+0000 7f0a937fe640 1 -- 172.21.15.78:0/1207367908 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f0a9c02fe90 con 0x7f0aa4106f20 2024-09-16T00:13:23.201 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T00:13:23.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a8c0a32c0 msgr2=0x7f0a8c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:23.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a8c0a32c0 0x7f0a8c0a5780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0a94005fd0 tx=0x7f0a94005850 comp rx=0 tx=0).stop 2024-09-16T00:13:23.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 msgr2=0x7f0aa419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 0x7f0aa419cca0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c031a20 tx=0x7f0a9c031a50 comp rx=0 tx=0).stop 2024-09-16T00:13:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 shutdown_connections 2024-09-16T00:13:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a8c0a32c0 0x7f0a8c0a5780 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0aa4106f20 0x7f0aa419cca0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0aa41027a0 0x7f0aa4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 --2- 172.21.15.78:0/1207367908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0aa4101df0 0x7f0aa41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:23.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.201+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 >> 172.21.15.78:0/1207367908 conn(0x7f0aa40fd8b0 msgr2=0x7f0aa40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:23.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.202+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 shutdown_connections 2024-09-16T00:13:23.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:23.202+0000 7f0aa8cfd640 1 -- 172.21.15.78:0/1207367908 wait complete. 2024-09-16T00:13:23.891 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:23 smithi078 ceph-mon[31157]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:13:23.891 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:23 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1207367908' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:13:23.919 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T00:13:23.920 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:13:23.920 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-16T00:13:23.920 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T00:13:23.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-16T00:13:23.920 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T00:13:23.920 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:13:23.920 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:13:23.920 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-16T00:13:23.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:23.921 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-16T00:13:23.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:23 smithi193 ceph-mon[35484]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:13:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:23 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1207367908' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:13:23.938 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42178sec preferred_lft 42178sec 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link noprefixroute 2024-09-16T00:13:23.939 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:13:23.940 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:13:23.940 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:13:23.940 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:13:23.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:13:23.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-09-16T00:13:23.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-09-16T00:13:23.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:13:23.940 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:13:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:23 smithi148 ceph-mon[35599]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-16T00:13:23.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:23 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1207367908' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:13:24.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:13:24.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:24.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:24.080 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:13:24.104 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-16T00:13:24.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:24.105 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-16T00:13:24.162 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-16T00:13:24.162 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:13:24.162 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T00:13:24.162 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T00:13:24.163 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:13:24.163 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:13:24.163 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:13:24.163 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T00:13:24.163 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T00:13:24.163 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T00:13:24.163 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:13:24.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:13:24.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:24.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:24.290 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-16T00:13:24.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:24.305 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-09-16T00:13:24.361 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:13:24.364 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:13:24.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-16T00:13:24.365 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-16T00:13:24.365 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:13:24.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:13:24.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:24.480 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-16T00:13:24.480 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:13:24.481 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:13:24.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-16T00:13:24.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:13:24.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-16T00:13:24.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-16T00:13:24.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-16T00:13:24.481 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:13:24.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:13:24.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:24.594 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:13:24.594 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-16T00:13:24.594 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-16T00:13:24.595 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:13:24.595 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-16T00:13:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:13:24.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:24.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:13:24.672 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /tmp/cephfs 2024-09-16T00:13:24.687 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-16T00:13:24.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:13:24.687 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /tmp/cephfs 2024-09-16T00:13:24.742 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-16T00:13:24.742 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-16T00:13:24.743 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/21dc2d71-c2a3-4518-91bc-47ab979ecd32 2024-09-16T00:13:24.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:24.743 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/21dc2d71-c2a3-4518-91bc-47ab979ecd32 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-09-16T00:13:24.804 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:24 smithi078 ceph-mon[31157]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:24.826 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-09-16T00:13:24.826 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-16T00:13:24.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:24 smithi193 ceph-mon[35484]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:24.946 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-09-16T00:13:24.947 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-16T00:13:24.947 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.148:3300,172.21.15.193:3300" 2024-09-16T00:13:24.947 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@eb73a744-73be-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/21dc2d71-c2a3-4518-91bc-47ab979ecd32 2024-09-16T00:13:24.947 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.78:3300/172.21.15.148:3300/172.21.15.193:3300" will pass to kernel 2024-09-16T00:13:24.949 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T00:13:24.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:13:24.950 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /tmp/cephfs 2024-09-16T00:13:24.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:24 smithi148 ceph-mon[35599]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:25.020 DEBUG:teuthology.orchestra.run.smithi078:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:13:25.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:13:25.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:25.118 INFO:teuthology.orchestra.run.smithi078.stdout:client24842 2024-09-16T00:13:25.118 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:13:25.118 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:13:25.136 DEBUG:teuthology.orchestra.run.smithi078:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:13:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:13:25.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:client24842 2024-09-16T00:13:25.231 DEBUG:teuthology.orchestra.run.smithi078:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:13:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:13:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:client24842 2024-09-16T00:13:25.328 DEBUG:teuthology.orchestra.run.smithi078:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/eb73a744-73be-11ef-bceb-c7b262605968.client24842/status') 2024-09-16T00:13:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:13:25.386 INFO:teuthology.orchestra.run.smithi078.stdout:instance: client.24842 (3)172.21.15.78:0/3217881942 2024-09-16T00:13:25.386 INFO:teuthology.orchestra.run.smithi078.stdout:blocklisted: false 2024-09-16T00:13:25.387 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records in 2024-09-16T00:13:25.387 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records out 2024-09-16T00:13:25.387 INFO:teuthology.orchestra.run.smithi078.stderr:71 bytes copied, 2.3043e-05 s, 3.1 MB/s 2024-09-16T00:13:25.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:13:25.394 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T00:13:25.411 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T00:13:25.411 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi078.front.sepia.ceph.com 2024-09-16T00:13:25.411 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-16T00:13:25.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.511+0000 7f0299598640 1 -- 172.21.15.78:0/1600397722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 msgr2=0x7f0294103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.511+0000 7f0299598640 1 --2- 172.21.15.78:0/1600397722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 0x7f0294103550 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0284009930 tx=0x7f028402a420 comp rx=0 tx=0).stop 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 -- 172.21.15.78:0/1600397722 shutdown_connections 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 --2- 172.21.15.78:0/1600397722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0294100580 0x7f029410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 --2- 172.21.15.78:0/1600397722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ffbc0 0x7f0294100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 --2- 172.21.15.78:0/1600397722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 0x7f0294103550 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 -- 172.21.15.78:0/1600397722 >> 172.21.15.78:0/1600397722 conn(0x7f02940fb970 msgr2=0x7f02940fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 -- 172.21.15.78:0/1600397722 shutdown_connections 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 -- 172.21.15.78:0/1600397722 wait complete. 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 Processor -- start 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 -- start start 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ffbc0 0x7f0294197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:25.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.512+0000 7f0299598640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0294100580 0x7f0294198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f0299598640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 0x7f029419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f0299598640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f029410d7f0 con 0x7f0294103150 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f0299598640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f029410d670 con 0x7f02940ffbc0 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f0299598640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f029410d970 con 0x7f0294100580 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 0x7f029419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 0x7f029419f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51440/0 (socket says 172.21.15.78:51440) 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 -- 172.21.15.78:0/3947880874 learned_addr learned my addr 172.21.15.78:0/3947880874 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f0292ffd640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ffbc0 0x7f0294197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02927fc640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0294100580 0x7f0294198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 -- 172.21.15.78:0/3947880874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0294100580 msgr2=0x7f0294198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0294100580 0x7f0294198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 -- 172.21.15.78:0/3947880874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ffbc0 msgr2=0x7f0294197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:25.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ffbc0 0x7f0294197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 -- 172.21.15.78:0/3947880874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029419fa40 con 0x7f0294103150 2024-09-16T00:13:25.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f02937fe640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 0x7f029419f3d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f028c00cad0 tx=0x7f028c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:25.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f027bfff640 1 -- 172.21.15.78:0/3947880874 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00ae20 con 0x7f0294103150 2024-09-16T00:13:25.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f027bfff640 1 -- 172.21.15.78:0/3947880874 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f028c005020 con 0x7f0294103150 2024-09-16T00:13:25.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029419fd30 con 0x7f0294103150 2024-09-16T00:13:25.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f027bfff640 1 -- 172.21.15.78:0/3947880874 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c00f6f0 con 0x7f0294103150 2024-09-16T00:13:25.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.513+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941a0220 con 0x7f0294103150 2024-09-16T00:13:25.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.514+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0294109090 con 0x7f0294103150 2024-09-16T00:13:25.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.515+0000 7f027bfff640 1 -- 172.21.15.78:0/3947880874 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f028c0051d0 con 0x7f0294103150 2024-09-16T00:13:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.516+0000 7f027bfff640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f027c0a32f0 0x7f027c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.516+0000 7f0292ffd640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f027c0a32f0 0x7f027c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.517+0000 7f027bfff640 1 -- 172.21.15.78:0/3947880874 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f028c0d0000 con 0x7f0294103150 2024-09-16T00:13:25.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.517+0000 7f0292ffd640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f027c0a32f0 0x7f027c0a57b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0284009900 tx=0x7f0284000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:25.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.521+0000 7f027bfff640 1 -- 172.21.15.78:0/3947880874 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f028c014030 con 0x7f0294103150 2024-09-16T00:13:25.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.726+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f029402d7c0 con 0x7f0294103150 2024-09-16T00:13:25.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.778+0000 7f027bfff640 1 -- 172.21.15.78:0/3947880874 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f028c09c9c0 con 0x7f0294103150 2024-09-16T00:13:25.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f027c0a32f0 msgr2=0x7f027c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:25.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f027c0a32f0 0x7f027c0a57b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0284009900 tx=0x7f0284000bc0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 msgr2=0x7f029419f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:25.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 0x7f029419f3d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f028c00cad0 tx=0x7f028c00cf90 comp rx=0 tx=0).stop 2024-09-16T00:13:25.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 shutdown_connections 2024-09-16T00:13:25.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f027c0a32f0 0x7f027c0a57b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0294103150 0x7f029419f3d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0294100580 0x7f0294198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 --2- 172.21.15.78:0/3947880874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f02940ffbc0 0x7f0294197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 >> 172.21.15.78:0/3947880874 conn(0x7f02940fb970 msgr2=0x7f0294106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:25.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 shutdown_connections 2024-09-16T00:13:25.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.780+0000 7f0299598640 1 -- 172.21.15.78:0/3947880874 wait complete. 2024-09-16T00:13:25.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-16T00:13:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 -- 172.21.15.78:0/3862228999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404102610 msgr2=0x7f6404108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 --2- 172.21.15.78:0/3862228999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404102610 0x7f6404108ea0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009ca0 tx=0x7f63fc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 -- 172.21.15.78:0/3862228999 shutdown_connections 2024-09-16T00:13:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 --2- 172.21.15.78:0/3862228999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404102610 0x7f6404108ea0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 --2- 172.21.15.78:0/3862228999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6404101c50 0x7f64041020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 --2- 172.21.15.78:0/3862228999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404100a60 0x7f6404100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 -- 172.21.15.78:0/3862228999 >> 172.21.15.78:0/3862228999 conn(0x7f64040fc1e0 msgr2=0x7f64040fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 -- 172.21.15.78:0/3862228999 shutdown_connections 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.895+0000 7f6408e42640 1 -- 172.21.15.78:0/3862228999 wait complete. 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6408e42640 1 Processor -- start 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6408e42640 1 -- start start 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6408e42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6404100a60 0x7f6404196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6408e42640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404101c50 0x7f64041971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6408e42640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404102610 0x7f640419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6408e42640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f640410c6c0 con 0x7f6404101c50 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6408e42640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f640410c540 con 0x7f6404100a60 2024-09-16T00:13:25.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6408e42640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f640410c840 con 0x7f6404102610 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6401d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404101c50 0x7f64041971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6401d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404101c50 0x7f64041971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51458/0 (socket says 172.21.15.78:51458) 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6401d9b640 1 -- 172.21.15.78:0/3964933066 learned_addr learned my addr 172.21.15.78:0/3964933066 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f640259c640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6404100a60 0x7f6404196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6402d9d640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404102610 0x7f640419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6401d9b640 1 -- 172.21.15.78:0/3964933066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404102610 msgr2=0x7f640419e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6401d9b640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404102610 0x7f640419e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6401d9b640 1 -- 172.21.15.78:0/3964933066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6404100a60 msgr2=0x7f6404196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6401d9b640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6404100a60 0x7f6404196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6401d9b640 1 -- 172.21.15.78:0/3964933066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640419e8c0 con 0x7f6404101c50 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f640259c640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6404100a60 0x7f6404196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.896+0000 7f6402d9d640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404102610 0x7f640419e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.897+0000 7f6401d9b640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404101c50 0x7f64041971c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f63f800d950 tx=0x7f63f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:25.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.897+0000 7f63f37fe640 1 -- 172.21.15.78:0/3964933066 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8017030 con 0x7f6404101c50 2024-09-16T00:13:25.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.897+0000 7f63f37fe640 1 -- 172.21.15.78:0/3964933066 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f8004020 con 0x7f6404101c50 2024-09-16T00:13:25.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.897+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640419ebb0 con 0x7f6404101c50 2024-09-16T00:13:25.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.897+0000 7f63f37fe640 1 -- 172.21.15.78:0/3964933066 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f80047a0 con 0x7f6404101c50 2024-09-16T00:13:25.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.897+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640419f100 con 0x7f6404101c50 2024-09-16T00:13:25.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.898+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6404107e00 con 0x7f6404101c50 2024-09-16T00:13:25.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.899+0000 7f63f37fe640 1 -- 172.21.15.78:0/3964933066 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f63f80041d0 con 0x7f6404101c50 2024-09-16T00:13:25.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.899+0000 7f63f37fe640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63ec0a32f0 0x7f63ec0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:25.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.900+0000 7f640259c640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63ec0a32f0 0x7f63ec0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:25.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.900+0000 7f63f37fe640 1 -- 172.21.15.78:0/3964933066 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f63f80cf730 con 0x7f6404101c50 2024-09-16T00:13:25.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.900+0000 7f640259c640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63ec0a32f0 0x7f63ec0a57b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005fd0 tx=0x7f63f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:25.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:25.904+0000 7f63f37fe640 1 -- 172.21.15.78:0/3964933066 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63f809c0d0 con 0x7f6404101c50 2024-09-16T00:13:26.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.096+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f64040626d0 con 0x7f63ec0a32f0 2024-09-16T00:13:26.102 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:25 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3947880874' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T00:13:26.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.116+0000 7f63f37fe640 1 -- 172.21.15.78:0/3964933066 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f64040626d0 con 0x7f63ec0a32f0 2024-09-16T00:13:26.118 INFO:teuthology.orchestra.run.smithi078.stdout:{} 2024-09-16T00:13:26.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63ec0a32f0 msgr2=0x7f63ec0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63ec0a32f0 0x7f63ec0a57b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005fd0 tx=0x7f63f4005850 comp rx=0 tx=0).stop 2024-09-16T00:13:26.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404101c50 msgr2=0x7f64041971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404101c50 0x7f64041971c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f63f800d950 tx=0x7f63f800de10 comp rx=0 tx=0).stop 2024-09-16T00:13:26.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 shutdown_connections 2024-09-16T00:13:26.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63ec0a32f0 0x7f63ec0a57b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404102610 0x7f640419e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6404101c50 0x7f64041971c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 --2- 172.21.15.78:0/3964933066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6404100a60 0x7f6404196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 >> 172.21.15.78:0/3964933066 conn(0x7f64040fc1e0 msgr2=0x7f6404104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:26.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 shutdown_connections 2024-09-16T00:13:26.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.118+0000 7f6408e42640 1 -- 172.21.15.78:0/3964933066 wait complete. 2024-09-16T00:13:26.130 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-16T00:13:26.144 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-16T00:13:26.155 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:13:26.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:25 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3947880874' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T00:13:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:25 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3947880874' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T00:13:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.253+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/10208120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c100c90 msgr2=0x7fd59c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.253+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/10208120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c100c90 0x7fd59c101110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd590009900 tx=0x7fd59002a410 comp rx=0 tx=0).stop 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.253+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/10208120 shutdown_connections 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.253+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/10208120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd59c101650 0x7fd59c108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.253+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/10208120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c100c90 0x7fd59c101110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.253+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/10208120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd59c102040 0x7fd59c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.253+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/10208120 >> 172.21.15.78:0/10208120 conn(0x7fd59c0fb950 msgr2=0x7fd59c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.253+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/10208120 shutdown_connections 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/10208120 wait complete. 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 Processor -- start 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 -- start start 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd59c100c90 0x7fd59c10b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c101650 0x7fd59c109a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd59c102040 0x7fd59c109f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c10d830 con 0x7fd59c101650 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c10d6b0 con 0x7fd59c102040 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.254+0000 7fd5a0c1e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c10d9b0 con 0x7fd59c100c90 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c101650 0x7fd59c109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c101650 0x7fd59c109a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58614/0 (socket says 172.21.15.78:58614) 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 -- 172.21.15.78:0/623618352 learned_addr learned my addr 172.21.15.78:0/623618352 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd59a59c640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd59c100c90 0x7fd59c10b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd59ad9d640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd59c102040 0x7fd59c109f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 -- 172.21.15.78:0/623618352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd59c100c90 msgr2=0x7fd59c10b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd59c100c90 0x7fd59c10b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 -- 172.21.15.78:0/623618352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd59c102040 msgr2=0x7fd59c109f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd59c102040 0x7fd59c109f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 -- 172.21.15.78:0/623618352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c10a7f0 con 0x7fd59c101650 2024-09-16T00:13:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd599d9b640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c101650 0x7fd59c109a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd590031610 tx=0x7fd5900043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd58b7fe640 1 -- 172.21.15.78:0/623618352 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59003f030 con 0x7fd59c101650 2024-09-16T00:13:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd58b7fe640 1 -- 172.21.15.78:0/623618352 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd590003ce0 con 0x7fd59c101650 2024-09-16T00:13:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59c1a1f70 con 0x7fd59c101650 2024-09-16T00:13:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd58b7fe640 1 -- 172.21.15.78:0/623618352 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590035070 con 0x7fd59c101650 2024-09-16T00:13:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.255+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59c1a2370 con 0x7fd59c101650 2024-09-16T00:13:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.256+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59c107ee0 con 0x7fd59c101650 2024-09-16T00:13:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.258+0000 7fd58b7fe640 1 -- 172.21.15.78:0/623618352 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fd590042030 con 0x7fd59c101650 2024-09-16T00:13:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.258+0000 7fd58b7fe640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd5840a3250 0x7fd5840a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.258+0000 7fd59a59c640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd5840a3250 0x7fd5840a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.259+0000 7fd58b7fe640 1 -- 172.21.15.78:0/623618352 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd5900f0210 con 0x7fd59c101650 2024-09-16T00:13:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.259+0000 7fd59a59c640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd5840a3250 0x7fd5840a5710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd58c005fd0 tx=0x7fd58c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.263+0000 7fd58b7fe640 1 -- 172.21.15.78:0/623618352 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5900bcbd0 con 0x7fd59c101650 2024-09-16T00:13:26.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.431+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd59c062ec0 con 0x7fd59c101650 2024-09-16T00:13:26.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.432+0000 7fd58b7fe640 1 -- 172.21.15.78:0/623618352 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24757 (secure 0 0 0) 0x7fd5900bc4d0 con 0x7fd59c101650 2024-09-16T00:13:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:26.435 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:13:04.600749+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:13:26.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd5840a3250 msgr2=0x7fd5840a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd5840a3250 0x7fd5840a5710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd58c005fd0 tx=0x7fd58c00b410 comp rx=0 tx=0).stop 2024-09-16T00:13:26.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c101650 msgr2=0x7fd59c109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c101650 0x7fd59c109a20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd590031610 tx=0x7fd5900043b0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 shutdown_connections 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd5840a3250 0x7fd5840a5710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd59c102040 0x7fd59c109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd59c101650 0x7fd59c109a20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 --2- 172.21.15.78:0/623618352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd59c100c90 0x7fd59c10b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.434+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 >> 172.21.15.78:0/623618352 conn(0x7fd59c0fb950 msgr2=0x7fd59c1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.435+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 shutdown_connections 2024-09-16T00:13:26.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.435+0000 7fd5a0c1e640 1 -- 172.21.15.78:0/623618352 wait complete. 2024-09-16T00:13:26.446 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:13:26.100+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:13:26.446 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:13:26.100+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:13:26.447 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:13:26.100+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:13:26.447 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:13:26.100+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:13:26.447 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:13:26.100+0000 7fe6005e7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:13:26.448 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:06:20.608203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:11:01.500267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:11:03.483225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T00:11:09.892287+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T00:13:26.449 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:13:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 -- 172.21.15.78:0/2578040738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c101bf0 msgr2=0x7f877c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 --2- 172.21.15.78:0/2578040738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c101bf0 0x7f877c102070 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8770009900 tx=0x7f877002a410 comp rx=0 tx=0).stop 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 -- 172.21.15.78:0/2578040738 shutdown_connections 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 --2- 172.21.15.78:0/2578040738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f877c1025b0 0x7f877c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 --2- 172.21.15.78:0/2578040738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c101bf0 0x7f877c102070 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 --2- 172.21.15.78:0/2578040738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f877c100a00 0x7f877c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 -- 172.21.15.78:0/2578040738 >> 172.21.15.78:0/2578040738 conn(0x7f877c0fc140 msgr2=0x7f877c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 -- 172.21.15.78:0/2578040738 shutdown_connections 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 -- 172.21.15.78:0/2578040738 wait complete. 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 Processor -- start 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.550+0000 7f8782047640 1 -- start start 2024-09-16T00:13:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8782047640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f877c100a00 0x7f877c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8782047640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f877c101bf0 0x7f877c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8782047640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c1025b0 0x7f877c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8782047640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f877c10d7d0 con 0x7f877c1025b0 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8782047640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f877c10d650 con 0x7f877c101bf0 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8782047640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f877c10d950 con 0x7f877c100a00 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c1025b0 0x7f877c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c1025b0 0x7f877c19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58628/0 (socket says 172.21.15.78:58628) 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 -- 172.21.15.78:0/3057090428 learned_addr learned my addr 172.21.15.78:0/3057090428 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877affd640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f877c101bf0 0x7f877c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877b7fe640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f877c100a00 0x7f877c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 -- 172.21.15.78:0/3057090428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f877c100a00 msgr2=0x7f877c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f877c100a00 0x7f877c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 -- 172.21.15.78:0/3057090428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f877c101bf0 msgr2=0x7f877c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f877c101bf0 0x7f877c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 -- 172.21.15.78:0/3057090428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c19fac0 con 0x7f877c1025b0 2024-09-16T00:13:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f877bfff640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c1025b0 0x7f877c19f450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f877400eaa0 tx=0x7f877400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8778ff9640 1 -- 172.21.15.78:0/3057090428 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877400cdb0 con 0x7f877c1025b0 2024-09-16T00:13:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8778ff9640 1 -- 172.21.15.78:0/3057090428 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774005020 con 0x7f877c1025b0 2024-09-16T00:13:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8778ff9640 1 -- 172.21.15.78:0/3057090428 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87740116b0 con 0x7f877c1025b0 2024-09-16T00:13:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877c19fdb0 con 0x7f877c1025b0 2024-09-16T00:13:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.551+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877c1a0270 con 0x7f877c1025b0 2024-09-16T00:13:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.552+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f877c107e90 con 0x7f877c1025b0 2024-09-16T00:13:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.553+0000 7f8778ff9640 1 -- 172.21.15.78:0/3057090428 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f8774005560 con 0x7f877c1025b0 2024-09-16T00:13:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.554+0000 7f8778ff9640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f87640a32d0 0x7f87640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.554+0000 7f877b7fe640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f87640a32d0 0x7f87640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.554+0000 7f8778ff9640 1 -- 172.21.15.78:0/3057090428 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f87740d0110 con 0x7f877c1025b0 2024-09-16T00:13:26.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.555+0000 7f877b7fe640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f87640a32d0 0x7f87640a5790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f876c005fd0 tx=0x7f876c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:26.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.557+0000 7f8778ff9640 1 -- 172.21.15.78:0/3057090428 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8774016030 con 0x7f877c1025b0 2024-09-16T00:13:26.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.729+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f877c02d7c0 con 0x7f877c1025b0 2024-09-16T00:13:26.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.729+0000 7f8778ff9640 1 -- 172.21.15.78:0/3057090428 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f877409c500 con 0x7f877c1025b0 2024-09-16T00:13:26.730 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-16T00:13:26.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f87640a32d0 msgr2=0x7f87640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f87640a32d0 0x7f87640a5790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f876c005fd0 tx=0x7f876c00b390 comp rx=0 tx=0).stop 2024-09-16T00:13:26.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c1025b0 msgr2=0x7f877c19f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c1025b0 0x7f877c19f450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f877400eaa0 tx=0x7f877400ef60 comp rx=0 tx=0).stop 2024-09-16T00:13:26.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 shutdown_connections 2024-09-16T00:13:26.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f87640a32d0 0x7f87640a5790 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f877c1025b0 0x7f877c19f450 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f877c101bf0 0x7f877c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 --2- 172.21.15.78:0/3057090428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f877c100a00 0x7f877c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 >> 172.21.15.78:0/3057090428 conn(0x7f877c0fc140 msgr2=0x7f877c105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:26.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 shutdown_connections 2024-09-16T00:13:26.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.731+0000 7f8782047640 1 -- 172.21.15.78:0/3057090428 wait complete. 2024-09-16T00:13:26.743 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:13:26.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 -- 172.21.15.78:0/1834906050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 msgr2=0x7fefc4109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 --2- 172.21.15.78:0/1834906050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 0x7fefc4109c80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009ca0 tx=0x7fefbc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 -- 172.21.15.78:0/1834906050 shutdown_connections 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 --2- 172.21.15.78:0/1834906050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 0x7fefc4109c80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 --2- 172.21.15.78:0/1834906050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4104ee0 0x7fefc41072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 --2- 172.21.15.78:0/1834906050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc41025b0 0x7fefc41049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 -- 172.21.15.78:0/1834906050 >> 172.21.15.78:0/1834906050 conn(0x7fefc40fc1b0 msgr2=0x7fefc40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 -- 172.21.15.78:0/1834906050 shutdown_connections 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.849+0000 7fefcadd6640 1 -- 172.21.15.78:0/1834906050 wait complete. 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefcadd6640 1 Processor -- start 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefcadd6640 1 -- start start 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefcadd6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc41025b0 0x7fefc419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefcadd6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4104ee0 0x7fefc419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefcadd6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 0x7fefc41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefcadd6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410d840 con 0x7fefc4107810 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefcadd6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410d6c0 con 0x7fefc41025b0 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefcadd6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410d9c0 con 0x7fefc4104ee0 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 0x7fefc41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 0x7fefc41a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58656/0 (socket says 172.21.15.78:58656) 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 -- 172.21.15.78:0/222997219 learned_addr learned my addr 172.21.15.78:0/222997219 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc8b72640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc41025b0 0x7fefc419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 -- 172.21.15.78:0/222997219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4104ee0 msgr2=0x7fefc419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4104ee0 0x7fefc419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 -- 172.21.15.78:0/222997219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc41025b0 msgr2=0x7fefc419a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc41025b0 0x7fefc419a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:26.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 -- 172.21.15.78:0/222997219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc41a1d60 con 0x7fefc4107810 2024-09-16T00:13:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc8b72640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc41025b0 0x7fefc419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.850+0000 7fefc9373640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 0x7fefc41a16f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009430 tx=0x7fefbc031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.851+0000 7fefc1ffb640 1 -- 172.21.15.78:0/222997219 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc003c30 con 0x7fefc4107810 2024-09-16T00:13:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.851+0000 7fefc1ffb640 1 -- 172.21.15.78:0/222997219 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefbc003dd0 con 0x7fefc4107810 2024-09-16T00:13:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.851+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc41a1ff0 con 0x7fefc4107810 2024-09-16T00:13:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.851+0000 7fefc1ffb640 1 -- 172.21.15.78:0/222997219 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc004840 con 0x7fefc4107810 2024-09-16T00:13:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.851+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41a2590 con 0x7fefc4107810 2024-09-16T00:13:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.852+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc4109300 con 0x7fefc4107810 2024-09-16T00:13:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.853+0000 7fefc1ffb640 1 -- 172.21.15.78:0/222997219 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fefbc02f790 con 0x7fefc4107810 2024-09-16T00:13:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.854+0000 7fefc1ffb640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefb40a32e0 0x7fefb40a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.854+0000 7fefc8b72640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefb40a32e0 0x7fefb40a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.854+0000 7fefc1ffb640 1 -- 172.21.15.78:0/222997219 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fefbc0f1130 con 0x7fefc4107810 2024-09-16T00:13:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.854+0000 7fefc8b72640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefb40a32e0 0x7fefb40a57a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fefb80058c0 tx=0x7fefb8005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:26.858+0000 7fefc1ffb640 1 -- 172.21.15.78:0/222997219 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefbc0bdbc0 con 0x7fefc4107810 2024-09-16T00:13:27.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3947880874' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T00:13:27.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:27.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-mon[31157]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:27.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-mon[31157]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-16T00:13:27.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/623618352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:27.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:26 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3057090428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:13:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.024+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fefc402da10 con 0x7fefc4107810 2024-09-16T00:13:27.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.025+0000 7fefc1ffb640 1 -- 172.21.15.78:0/222997219 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fefbc0bd4c0 con 0x7fefc4107810 2024-09-16T00:13:27.025 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-09-16T00:13:27.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefb40a32e0 msgr2=0x7fefb40a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefb40a32e0 0x7fefb40a57a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fefb80058c0 tx=0x7fefb8005ec0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 msgr2=0x7fefc41a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 0x7fefc41a16f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009430 tx=0x7fefbc031ec0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 shutdown_connections 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefb40a32e0 0x7fefb40a57a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4107810 0x7fefc41a16f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4104ee0 0x7fefc419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 --2- 172.21.15.78:0/222997219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc41025b0 0x7fefc419a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 >> 172.21.15.78:0/222997219 conn(0x7fefc40fc1b0 msgr2=0x7fefc4101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 shutdown_connections 2024-09-16T00:13:27.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.027+0000 7fefcadd6640 1 -- 172.21.15.78:0/222997219 wait complete. 2024-09-16T00:13:27.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:13:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 -- 172.21.15.78:0/1515401936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 msgr2=0x7f3c801049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1515401936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 0x7f3c801049a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009900 tx=0x7f3c7002a410 comp rx=0 tx=0).stop 2024-09-16T00:13:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 -- 172.21.15.78:0/1515401936 shutdown_connections 2024-09-16T00:13:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1515401936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c80107810 0x7f3c80109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1515401936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c80104ee0 0x7f3c801072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1515401936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 0x7f3c801049a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 -- 172.21.15.78:0/1515401936 >> 172.21.15.78:0/1515401936 conn(0x7f3c800fc1b0 msgr2=0x7f3c800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 -- 172.21.15.78:0/1515401936 shutdown_connections 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.142+0000 7f3c857c4640 1 -- 172.21.15.78:0/1515401936 wait complete. 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c857c4640 1 Processor -- start 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c857c4640 1 -- start start 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c857c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 0x7f3c80197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c857c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c80104ee0 0x7f3c80198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c857c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c80107810 0x7f3c8019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c857c4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8010d860 con 0x7f3c801025b0 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c857c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8010d6e0 con 0x7f3c80107810 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c857c4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8010d9e0 con 0x7f3c80104ee0 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 0x7f3c80197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:27.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 0x7f3c80197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58674/0 (socket says 172.21.15.78:58674) 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7effd640 1 -- 172.21.15.78:0/1229703235 learned_addr learned my addr 172.21.15.78:0/1229703235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7effd640 1 -- 172.21.15.78:0/1229703235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c80104ee0 msgr2=0x7f3c80198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7effd640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c80104ee0 0x7f3c80198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7effd640 1 -- 172.21.15.78:0/1229703235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c80107810 msgr2=0x7f3c8019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7f7fe640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c80107810 0x7f3c8019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7effd640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c80107810 0x7f3c8019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7effd640 1 -- 172.21.15.78:0/1229703235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8019fb40 con 0x7f3c801025b0 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.143+0000 7f3c7f7fe640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c80107810 0x7f3c8019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.144+0000 7f3c7effd640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 0x7f3c80197f20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3c70031610 tx=0x7f3c70004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.144+0000 7f3c67fff640 1 -- 172.21.15.78:0/1229703235 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7003f030 con 0x7f3c801025b0 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.144+0000 7f3c67fff640 1 -- 172.21.15.78:0/1229703235 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c70003ce0 con 0x7f3c801025b0 2024-09-16T00:13:27.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.144+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8019fdd0 con 0x7f3c801025b0 2024-09-16T00:13:27.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.144+0000 7f3c67fff640 1 -- 172.21.15.78:0/1229703235 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c70035070 con 0x7f3c801025b0 2024-09-16T00:13:27.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.144+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c801a02e0 con 0x7f3c801025b0 2024-09-16T00:13:27.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.145+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c800fdca0 con 0x7f3c801025b0 2024-09-16T00:13:27.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.146+0000 7f3c67fff640 1 -- 172.21.15.78:0/1229703235 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f3c70042030 con 0x7f3c801025b0 2024-09-16T00:13:27.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.147+0000 7f3c67fff640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c680a32e0 0x7f3c680a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:27.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.147+0000 7f3c7e7fc640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c680a32e0 0x7f3c680a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:27.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.147+0000 7f3c67fff640 1 -- 172.21.15.78:0/1229703235 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c700f04a0 con 0x7f3c801025b0 2024-09-16T00:13:27.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.147+0000 7f3c7e7fc640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c680a32e0 0x7f3c680a57a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c80199430 tx=0x7f3c74006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:27.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.151+0000 7f3c67fff640 1 -- 172.21.15.78:0/1229703235 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c700bcd00 con 0x7f3c801025b0 2024-09-16T00:13:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:26 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3947880874' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T00:13:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:26 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:26 smithi193 ceph-mon[35484]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:26 smithi193 ceph-mon[35484]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-16T00:13:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:26 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/623618352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:26 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3057090428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:13:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:26 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3947880874' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T00:13:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:26 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:26 smithi148 ceph-mon[35599]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:13:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:26 smithi148 ceph-mon[35599]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-16T00:13:27.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:26 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/623618352' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:27.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:26 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3057090428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:13:27.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.318+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3c8002da30 con 0x7f3c801025b0 2024-09-16T00:13:27.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.319+0000 7f3c67fff640 1 -- 172.21.15.78:0/1229703235 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f3c700bc600 con 0x7f3c801025b0 2024-09-16T00:13:27.320 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-09-16T00:13:27.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c680a32e0 msgr2=0x7f3c680a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c680a32e0 0x7f3c680a57a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3c80199430 tx=0x7f3c74006c90 comp rx=0 tx=0).stop 2024-09-16T00:13:27.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 msgr2=0x7f3c80197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 0x7f3c80197f20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3c70031610 tx=0x7f3c70004430 comp rx=0 tx=0).stop 2024-09-16T00:13:27.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 shutdown_connections 2024-09-16T00:13:27.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c680a32e0 0x7f3c680a57a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c80107810 0x7f3c8019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c80104ee0 0x7f3c80198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 --2- 172.21.15.78:0/1229703235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c801025b0 0x7f3c80197f20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 >> 172.21.15.78:0/1229703235 conn(0x7f3c800fc1b0 msgr2=0x7f3c80102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:27.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 shutdown_connections 2024-09-16T00:13:27.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.321+0000 7f3c857c4640 1 -- 172.21.15.78:0/1229703235 wait complete. 2024-09-16T00:13:27.333 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T00:13:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 -- 172.21.15.78:0/3771519114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 msgr2=0x7fd848109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/3771519114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 0x7fd848109c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd840009ca0 tx=0x7fd84002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 -- 172.21.15.78:0/3771519114 shutdown_connections 2024-09-16T00:13:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/3771519114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 0x7fd848109c80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/3771519114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd848104ee0 0x7fd8481072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/3771519114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481025b0 0x7fd8481049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 -- 172.21.15.78:0/3771519114 >> 172.21.15.78:0/3771519114 conn(0x7fd8480fc1b0 msgr2=0x7fd8480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 -- 172.21.15.78:0/3771519114 shutdown_connections 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 -- 172.21.15.78:0/3771519114 wait complete. 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.434+0000 7fd84c8a6640 1 Processor -- start 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84c8a6640 1 -- start start 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84c8a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481025b0 0x7fd848197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84c8a6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd848104ee0 0x7fd848198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84c8a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 0x7fd84819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84c8a6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd84810d7d0 con 0x7fd848107810 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84c8a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd84810d650 con 0x7fd8481025b0 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84c8a6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd84810d950 con 0x7fd848104ee0 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 0x7fd84819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:27.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 0x7fd84819f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58700/0 (socket says 172.21.15.78:58700) 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 -- 172.21.15.78:0/2071497993 learned_addr learned my addr 172.21.15.78:0/2071497993 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84659c640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481025b0 0x7fd848197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd83ffff640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd848104ee0 0x7fd848198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 -- 172.21.15.78:0/2071497993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd848104ee0 msgr2=0x7fd848198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd848104ee0 0x7fd848198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 -- 172.21.15.78:0/2071497993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481025b0 msgr2=0x7fd848197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481025b0 0x7fd848197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 -- 172.21.15.78:0/2071497993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84819fa50 con 0x7fd848107810 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd83ffff640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd848104ee0 0x7fd848198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd846d9d640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 0x7fd84819f3e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd840009430 tx=0x7fd840031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd83f7fe640 1 -- 172.21.15.78:0/2071497993 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840003c30 con 0x7fd848107810 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd83f7fe640 1 -- 172.21.15.78:0/2071497993 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd840003dd0 con 0x7fd848107810 2024-09-16T00:13:27.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84819fce0 con 0x7fd848107810 2024-09-16T00:13:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.435+0000 7fd83f7fe640 1 -- 172.21.15.78:0/2071497993 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840004840 con 0x7fd848107810 2024-09-16T00:13:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.436+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8481a0220 con 0x7fd848107810 2024-09-16T00:13:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.437+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8480fdca0 con 0x7fd848107810 2024-09-16T00:13:27.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.438+0000 7fd83f7fe640 1 -- 172.21.15.78:0/2071497993 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fd84002f790 con 0x7fd848107810 2024-09-16T00:13:27.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.439+0000 7fd83f7fe640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8340a3360 0x7fd8340a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:27.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.439+0000 7fd84659c640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8340a3360 0x7fd8340a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:27.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.439+0000 7fd83f7fe640 1 -- 172.21.15.78:0/2071497993 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd8400f10e0 con 0x7fd848107810 2024-09-16T00:13:27.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.439+0000 7fd84659c640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8340a3360 0x7fd8340a5820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd838005e10 tx=0x7fd838005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:27.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.443+0000 7fd83f7fe640 1 -- 172.21.15.78:0/2071497993 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8400bda50 con 0x7fd848107810 2024-09-16T00:13:27.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.608+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd848062e90 con 0x7fd848107810 2024-09-16T00:13:27.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.609+0000 7fd83f7fe640 1 -- 172.21.15.78:0/2071497993 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fd8400bd350 con 0x7fd848107810 2024-09-16T00:13:27.609 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-09-16T00:13:27.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8340a3360 msgr2=0x7fd8340a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8340a3360 0x7fd8340a5820 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd838005e10 tx=0x7fd838005da0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 msgr2=0x7fd84819f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 0x7fd84819f3e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd840009430 tx=0x7fd840031ec0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 shutdown_connections 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8340a3360 0x7fd8340a5820 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd848107810 0x7fd84819f3e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd848104ee0 0x7fd848198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 --2- 172.21.15.78:0/2071497993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd8481025b0 0x7fd848197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 >> 172.21.15.78:0/2071497993 conn(0x7fd8480fc1b0 msgr2=0x7fd848102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 shutdown_connections 2024-09-16T00:13:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:27.611+0000 7fd84c8a6640 1 -- 172.21.15.78:0/2071497993 wait complete. 2024-09-16T00:13:27.622 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-16T00:13:27.623 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:13:27.870 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:27.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:27 smithi078 ceph-mon[31157]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T00:13:27.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:27 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/222997219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:13:27.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:27 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1229703235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:13:27.992 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:27 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2071497993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:13:28.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:27 smithi193 ceph-mon[35484]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T00:13:28.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:27 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/222997219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:13:28.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:27 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1229703235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:13:28.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:27 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2071497993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:13:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:27 smithi148 ceph-mon[35599]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T00:13:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:27 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/222997219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:13:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:27 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1229703235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:13:28.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:27 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2071497993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:13:28.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 -- 172.21.15.78:0/3488114360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070102ef0 msgr2=0x7f007010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:28.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 --2- 172.21.15.78:0/3488114360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070102ef0 0x7f007010da40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0068009ca0 tx=0x7f006802a950 comp rx=0 tx=0).stop 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 -- 172.21.15.78:0/3488114360 shutdown_connections 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 --2- 172.21.15.78:0/3488114360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070102ef0 0x7f007010da40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 --2- 172.21.15.78:0/3488114360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070102570 0x7f00701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 --2- 172.21.15.78:0/3488114360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0070100e20 0x7f0070101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 -- 172.21.15.78:0/3488114360 >> 172.21.15.78:0/3488114360 conn(0x7f00700fc7d0 msgr2=0x7f00700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 -- 172.21.15.78:0/3488114360 shutdown_connections 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.344+0000 7f0075896640 1 -- 172.21.15.78:0/3488114360 wait complete. 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f0075896640 1 Processor -- start 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f0075896640 1 -- start start 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f0075896640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070100e20 0x7f0070197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f0075896640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0070102570 0x7f0070197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:28.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f0075896640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070102ef0 0x7f007019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f0075896640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0070110fa0 con 0x7f0070100e20 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f0075896640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0070110e20 con 0x7f0070102ef0 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f0075896640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0070111120 con 0x7f0070102570 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.345+0000 7f006effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070100e20 0x7f0070197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070100e20 0x7f0070197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58716/0 (socket says 172.21.15.78:58716) 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006effd640 1 -- 172.21.15.78:0/1088435539 learned_addr learned my addr 172.21.15.78:0/1088435539 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006e7fc640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0070102570 0x7f0070197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006f7fe640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070102ef0 0x7f007019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006effd640 1 -- 172.21.15.78:0/1088435539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0070102570 msgr2=0x7f0070197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006effd640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0070102570 0x7f0070197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006effd640 1 -- 172.21.15.78:0/1088435539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070102ef0 msgr2=0x7f007019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006effd640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070102ef0 0x7f007019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006effd640 1 -- 172.21.15.78:0/1088435539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007019c2e0 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006f7fe640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070102ef0 0x7f007019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f006effd640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070100e20 0x7f0070197330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f006000b4a0 tx=0x7f006000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f0074894640 1 -- 172.21.15.78:0/1088435539 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060014070 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007019c5d0 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f0074894640 1 -- 172.21.15.78:0/1088435539 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060010af0 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f0074894640 1 -- 172.21.15.78:0/1088435539 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060017d10 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.346+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00701a3eb0 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.347+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0070101200 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.348+0000 7f0074894640 1 -- 172.21.15.78:0/1088435539 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f0060017070 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.349+0000 7f0074894640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f00580a32d0 0x7f00580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.349+0000 7f0074894640 1 -- 172.21.15.78:0/1088435539 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f00600cf170 con 0x7f0070100e20 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.349+0000 7f006e7fc640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f00580a32d0 0x7f00580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:28.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.349+0000 7f006e7fc640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f00580a32d0 0x7f00580a5790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0064005e10 tx=0x7f0064005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:28.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.353+0000 7f0074894640 1 -- 172.21.15.78:0/1088435539 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f006009bae0 con 0x7f0070100e20 2024-09-16T00:13:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.556+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00701a40e0 con 0x7f0070100e20 2024-09-16T00:13:28.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.557+0000 7f0074894640 1 -- 172.21.15.78:0/1088435539 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7f006009b3e0 con 0x7f0070100e20 2024-09-16T00:13:28.558 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:28.558 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-16T00:13:25:778566+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:25.728293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:13:28.559 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f00580a32d0 msgr2=0x7f00580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f00580a32d0 0x7f00580a5790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0064005e10 tx=0x7f0064005da0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070100e20 msgr2=0x7f0070197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070100e20 0x7f0070197330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f006000b4a0 tx=0x7f006000b960 comp rx=0 tx=0).stop 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 shutdown_connections 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f00580a32d0 0x7f00580a5790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0070102ef0 0x7f007019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0070102570 0x7f0070197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 --2- 172.21.15.78:0/1088435539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0070100e20 0x7f0070197330 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:28.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 >> 172.21.15.78:0/1088435539 conn(0x7f00700fc7d0 msgr2=0x7f0070106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 shutdown_connections 2024-09-16T00:13:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:28.559+0000 7f0075896640 1 -- 172.21.15.78:0/1088435539 wait complete. 2024-09-16T00:13:28.927 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:28 smithi078 ceph-mon[31157]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T00:13:28.928 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:28 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1088435539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:29.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:28 smithi193 ceph-mon[35484]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T00:13:29.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:28 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1088435539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:29.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:28 smithi148 ceph-mon[35599]: pgmap v363: 129 pgs: 129 active+clean; 647 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-16T00:13:29.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:28 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1088435539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:29.247 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-16T00:13:29.247 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-16T00:13:29.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 -- 172.21.15.78:0/3844746836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 msgr2=0x7ff454108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 --2- 172.21.15.78:0/3844746836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 0x7ff454108e70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009ca0 tx=0x7ff44c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 -- 172.21.15.78:0/3844746836 shutdown_connections 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 --2- 172.21.15.78:0/3844746836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 0x7ff454108e70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 --2- 172.21.15.78:0/3844746836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff454101c20 0x7ff4541020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 --2- 172.21.15.78:0/3844746836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454100a30 0x7ff454100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 -- 172.21.15.78:0/3844746836 >> 172.21.15.78:0/3844746836 conn(0x7ff4540fc190 msgr2=0x7ff4540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 -- 172.21.15.78:0/3844746836 shutdown_connections 2024-09-16T00:13:29.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 -- 172.21.15.78:0/3844746836 wait complete. 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.349+0000 7ff459230640 1 Processor -- start 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff459230640 1 -- start start 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff459230640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff454100a30 0x7ff454197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff459230640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101c20 0x7ff454198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff459230640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 0x7ff45419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff459230640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff45410c690 con 0x7ff4541025e0 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff459230640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff45410c510 con 0x7ff454100a30 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff459230640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff45410c810 con 0x7ff454101c20 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff45359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 0x7ff45419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff45359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 0x7ff45419f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58734/0 (socket says 172.21.15.78:58734) 2024-09-16T00:13:29.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff45359e640 1 -- 172.21.15.78:0/944967423 learned_addr learned my addr 172.21.15.78:0/944967423 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff452d9d640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff454100a30 0x7ff454197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff45359e640 1 -- 172.21.15.78:0/944967423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101c20 msgr2=0x7ff454198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff44a59c640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101c20 0x7ff454198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff45359e640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101c20 0x7ff454198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff45359e640 1 -- 172.21.15.78:0/944967423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff454100a30 msgr2=0x7ff454197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff45359e640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff454100a30 0x7ff454197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.350+0000 7ff45359e640 1 -- 172.21.15.78:0/944967423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45419f950 con 0x7ff4541025e0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.351+0000 7ff45359e640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 0x7ff45419f2b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff44c031ec0 tx=0x7ff44c031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.351+0000 7ff450d99640 1 -- 172.21.15.78:0/944967423 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c003c30 con 0x7ff4541025e0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.351+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45419fbe0 con 0x7ff4541025e0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.351+0000 7ff450d99640 1 -- 172.21.15.78:0/944967423 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44c003dd0 con 0x7ff4541025e0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.351+0000 7ff450d99640 1 -- 172.21.15.78:0/944967423 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c004770 con 0x7ff4541025e0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.351+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4541a0180 con 0x7ff4541025e0 2024-09-16T00:13:29.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.352+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4541a0350 con 0x7ff4541025e0 2024-09-16T00:13:29.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.353+0000 7ff450d99640 1 -- 172.21.15.78:0/944967423 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7ff44c02f790 con 0x7ff4541025e0 2024-09-16T00:13:29.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.354+0000 7ff450d99640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff4400a32e0 0x7ff4400a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:29.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.354+0000 7ff452d9d640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff4400a32e0 0x7ff4400a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:29.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.354+0000 7ff450d99640 1 -- 172.21.15.78:0/944967423 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff44c0f1330 con 0x7ff4541025e0 2024-09-16T00:13:29.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.354+0000 7ff452d9d640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff4400a32e0 0x7ff4400a57a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4440058c0 tx=0x7ff444005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:29.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.358+0000 7ff450d99640 1 -- 172.21.15.78:0/944967423 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff44c0bdd40 con 0x7ff4541025e0 2024-09-16T00:13:29.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.521+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4541a0520 con 0x7ff4400a32e0 2024-09-16T00:13:29.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.529+0000 7ff450d99640 1 -- 172.21.15.78:0/944967423 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340526 (secure 0 0 0) 0x7ff4541a0520 con 0x7ff4400a32e0 2024-09-16T00:13:29.534 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:29.537 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:13:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff4400a32e0 msgr2=0x7ff4400a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff4400a32e0 0x7ff4400a57a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff4440058c0 tx=0x7ff444005850 comp rx=0 tx=0).stop 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 msgr2=0x7ff45419f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 0x7ff45419f2b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff44c031ec0 tx=0x7ff44c031ef0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 shutdown_connections 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff4400a32e0 0x7ff4400a57a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4541025e0 0x7ff45419f2b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101c20 0x7ff454198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 --2- 172.21.15.78:0/944967423 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff454100a30 0x7ff454197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 >> 172.21.15.78:0/944967423 conn(0x7ff4540fc190 msgr2=0x7ff4540fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 shutdown_connections 2024-09-16T00:13:29.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.533+0000 7ff459230640 1 -- 172.21.15.78:0/944967423 wait complete. 2024-09-16T00:13:29.545 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-16T00:13:28.430524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662281,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1195,"num_read_kb":722,"num_write":452,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359472,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582352,"statfs":{"total":1151940427776,"available":1151572328448,"internally_reserved":0,"allocated":14680064,"data_stored":8130884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1009,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003061"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:23.683882+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:13:23.683882+0000","last_peered":"2024-09-16T00:13:23.683882+0000","last_clean":"2024-09-16T00:13:23.683882+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:13:23.683882+0000","last_undegraded":"2024-09-16T00:13:23.683882+0000","last_fullsized":"2024-09-16T00:13:23.683882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610803+0000","last_change":"2024-09-16T00:12:04.132674+0000","last_active":"2024-09-16T00:13:04.610803+0000","last_peered":"2024-09-16T00:13:04.610803+0000","last_clean":"2024-09-16T00:13:04.610803+0000","last_became_active":"2024-09-16T00:12:04.132108+0000","last_became_peered":"2024-09-16T00:12:04.132108+0000","last_unstale":"2024-09-16T00:13:04.610803+0000","last_undegraded":"2024-09-16T00:13:04.610803+0000","last_fullsized":"2024-09-16T00:13:04.610803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:10.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664962+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:13:04.664962+0000","last_peered":"2024-09-16T00:13:04.664962+0000","last_clean":"2024-09-16T00:13:04.664962+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:13:04.664962+0000","last_undegraded":"2024-09-16T00:13:04.664962+0000","last_fullsized":"2024-09-16T00:13:04.664962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663575+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:13:04.663575+0000","last_peered":"2024-09-16T00:13:04.663575+0000","last_clean":"2024-09-16T00:13:04.663575+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:13:04.663575+0000","last_undegraded":"2024-09-16T00:13:04.663575+0000","last_fullsized":"2024-09-16T00:13:04.663575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611038+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:13:04.611038+0000","last_peered":"2024-09-16T00:13:04.611038+0000","last_clean":"2024-09-16T00:13:04.611038+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:13:04.611038+0000","last_undegraded":"2024-09-16T00:13:04.611038+0000","last_fullsized":"2024-09-16T00:13:04.611038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664541+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:13:04.664541+0000","last_peered":"2024-09-16T00:13:04.664541+0000","last_clean":"2024-09-16T00:13:04.664541+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:13:04.664541+0000","last_undegraded":"2024-09-16T00:13:04.664541+0000","last_fullsized":"2024-09-16T00:13:04.664541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605363+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:13:04.605363+0000","last_peered":"2024-09-16T00:13:04.605363+0000","last_clean":"2024-09-16T00:13:04.605363+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:13:04.605363+0000","last_undegraded":"2024-09-16T00:13:04.605363+0000","last_fullsized":"2024-09-16T00:13:04.605363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611375+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:13:04.611375+0000","last_peered":"2024-09-16T00:13:04.611375+0000","last_clean":"2024-09-16T00:13:04.611375+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:13:04.611375+0000","last_undegraded":"2024-09-16T00:13:04.611375+0000","last_fullsized":"2024-09-16T00:13:04.611375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664351+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:13:04.664351+0000","last_peered":"2024-09-16T00:13:04.664351+0000","last_clean":"2024-09-16T00:13:04.664351+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:13:04.664351+0000","last_undegraded":"2024-09-16T00:13:04.664351+0000","last_fullsized":"2024-09-16T00:13:04.664351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605600+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:13:04.605600+0000","last_peered":"2024-09-16T00:13:04.605600+0000","last_clean":"2024-09-16T00:13:04.605600+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:13:04.605600+0000","last_undegraded":"2024-09-16T00:13:04.605600+0000","last_fullsized":"2024-09-16T00:13:04.605600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611103+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:13:04.611103+0000","last_peered":"2024-09-16T00:13:04.611103+0000","last_clean":"2024-09-16T00:13:04.611103+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:13:04.611103+0000","last_undegraded":"2024-09-16T00:13:04.611103+0000","last_fullsized":"2024-09-16T00:13:04.611103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649281+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:13:04.649281+0000","last_peered":"2024-09-16T00:13:04.649281+0000","last_clean":"2024-09-16T00:13:04.649281+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:13:04.649281+0000","last_undegraded":"2024-09-16T00:13:04.649281+0000","last_fullsized":"2024-09-16T00:13:04.649281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664252+0000","last_change":"2024-09-16T00:12:04.033464+0000","last_active":"2024-09-16T00:13:04.664252+0000","last_peered":"2024-09-16T00:13:04.664252+0000","last_clean":"2024-09-16T00:13:04.664252+0000","last_became_active":"2024-09-16T00:12:04.028916+0000","last_became_peered":"2024-09-16T00:12:04.028916+0000","last_unstale":"2024-09-16T00:13:04.664252+0000","last_undegraded":"2024-09-16T00:13:04.664252+0000","last_fullsized":"2024-09-16T00:13:04.664252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:47.359979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611584+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:13:04.611584+0000","last_peered":"2024-09-16T00:13:04.611584+0000","last_clean":"2024-09-16T00:13:04.611584+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:13:04.611584+0000","last_undegraded":"2024-09-16T00:13:04.611584+0000","last_fullsized":"2024-09-16T00:13:04.611584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664652+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:13:04.664652+0000","last_peered":"2024-09-16T00:13:04.664652+0000","last_clean":"2024-09-16T00:13:04.664652+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:13:04.664652+0000","last_undegraded":"2024-09-16T00:13:04.664652+0000","last_fullsized":"2024-09-16T00:13:04.664652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610969+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:13:04.610969+0000","last_peered":"2024-09-16T00:13:04.610969+0000","last_clean":"2024-09-16T00:13:04.610969+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:13:04.610969+0000","last_undegraded":"2024-09-16T00:13:04.610969+0000","last_fullsized":"2024-09-16T00:13:04.610969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611391+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:13:04.611391+0000","last_peered":"2024-09-16T00:13:04.611391+0000","last_clean":"2024-09-16T00:13:04.611391+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:13:04.611391+0000","last_undegraded":"2024-09-16T00:13:04.611391+0000","last_fullsized":"2024-09-16T00:13:04.611391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663782+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:13:04.663782+0000","last_peered":"2024-09-16T00:13:04.663782+0000","last_clean":"2024-09-16T00:13:04.663782+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:13:04.663782+0000","last_undegraded":"2024-09-16T00:13:04.663782+0000","last_fullsized":"2024-09-16T00:13:04.663782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664498+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:13:04.664498+0000","last_peered":"2024-09-16T00:13:04.664498+0000","last_clean":"2024-09-16T00:13:04.664498+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:13:04.664498+0000","last_undegraded":"2024-09-16T00:13:04.664498+0000","last_fullsized":"2024-09-16T00:13:04.664498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609329+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:13:04.609329+0000","last_peered":"2024-09-16T00:13:04.609329+0000","last_clean":"2024-09-16T00:13:04.609329+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:13:04.609329+0000","last_undegraded":"2024-09-16T00:13:04.609329+0000","last_fullsized":"2024-09-16T00:13:04.609329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605426+0000","last_change":"2024-09-16T00:12:04.034002+0000","last_active":"2024-09-16T00:13:04.605426+0000","last_peered":"2024-09-16T00:13:04.605426+0000","last_clean":"2024-09-16T00:13:04.605426+0000","last_became_active":"2024-09-16T00:12:04.030959+0000","last_became_peered":"2024-09-16T00:12:04.030959+0000","last_unstale":"2024-09-16T00:13:04.605426+0000","last_undegraded":"2024-09-16T00:13:04.605426+0000","last_fullsized":"2024-09-16T00:13:04.605426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:47.730692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611324+0000","last_change":"2024-09-16T00:12:04.029906+0000","last_active":"2024-09-16T00:13:04.611324+0000","last_peered":"2024-09-16T00:13:04.611324+0000","last_clean":"2024-09-16T00:13:04.611324+0000","last_became_active":"2024-09-16T00:12:04.029259+0000","last_became_peered":"2024-09-16T00:12:04.029259+0000","last_unstale":"2024-09-16T00:13:04.611324+0000","last_undegraded":"2024-09-16T00:13:04.611324+0000","last_fullsized":"2024-09-16T00:13:04.611324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:23.399762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,1],"acting":[4,8,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663413+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:13:04.663413+0000","last_peered":"2024-09-16T00:13:04.663413+0000","last_clean":"2024-09-16T00:13:04.663413+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:13:04.663413+0000","last_undegraded":"2024-09-16T00:13:04.663413+0000","last_fullsized":"2024-09-16T00:13:04.663413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611856+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:13:04.611856+0000","last_peered":"2024-09-16T00:13:04.611856+0000","last_clean":"2024-09-16T00:13:04.611856+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:13:04.611856+0000","last_undegraded":"2024-09-16T00:13:04.611856+0000","last_fullsized":"2024-09-16T00:13:04.611856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610909+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:13:04.610909+0000","last_peered":"2024-09-16T00:13:04.610909+0000","last_clean":"2024-09-16T00:13:04.610909+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:13:04.610909+0000","last_undegraded":"2024-09-16T00:13:04.610909+0000","last_fullsized":"2024-09-16T00:13:04.610909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.665046+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:13:04.665046+0000","last_peered":"2024-09-16T00:13:04.665046+0000","last_clean":"2024-09-16T00:13:04.665046+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:13:04.665046+0000","last_undegraded":"2024-09-16T00:13:04.665046+0000","last_fullsized":"2024-09-16T00:13:04.665046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'19","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609370+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:13:04.609370+0000","last_peered":"2024-09-16T00:13:04.609370+0000","last_clean":"2024-09-16T00:13:04.609370+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:13:04.609370+0000","last_undegraded":"2024-09-16T00:13:04.609370+0000","last_fullsized":"2024-09-16T00:13:04.609370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664101+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:13:04.664101+0000","last_peered":"2024-09-16T00:13:04.664101+0000","last_clean":"2024-09-16T00:13:04.664101+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:13:04.664101+0000","last_undegraded":"2024-09-16T00:13:04.664101+0000","last_fullsized":"2024-09-16T00:13:04.664101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606866+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:13:04.606866+0000","last_peered":"2024-09-16T00:13:04.606866+0000","last_clean":"2024-09-16T00:13:04.606866+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:13:04.606866+0000","last_undegraded":"2024-09-16T00:13:04.606866+0000","last_fullsized":"2024-09-16T00:13:04.606866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664634+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:13:04.664634+0000","last_peered":"2024-09-16T00:13:04.664634+0000","last_clean":"2024-09-16T00:13:04.664634+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:13:04.664634+0000","last_undegraded":"2024-09-16T00:13:04.664634+0000","last_fullsized":"2024-09-16T00:13:04.664634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664409+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:13:04.664409+0000","last_peered":"2024-09-16T00:13:04.664409+0000","last_clean":"2024-09-16T00:13:04.664409+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:13:04.664409+0000","last_undegraded":"2024-09-16T00:13:04.664409+0000","last_fullsized":"2024-09-16T00:13:04.664409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664046+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:13:04.664046+0000","last_peered":"2024-09-16T00:13:04.664046+0000","last_clean":"2024-09-16T00:13:04.664046+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:13:04.664046+0000","last_undegraded":"2024-09-16T00:13:04.664046+0000","last_fullsized":"2024-09-16T00:13:04.664046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609377+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:13:04.609377+0000","last_peered":"2024-09-16T00:13:04.609377+0000","last_clean":"2024-09-16T00:13:04.609377+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:13:04.609377+0000","last_undegraded":"2024-09-16T00:13:04.609377+0000","last_fullsized":"2024-09-16T00:13:04.609377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664159+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:13:04.664159+0000","last_peered":"2024-09-16T00:13:04.664159+0000","last_clean":"2024-09-16T00:13:04.664159+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:13:04.664159+0000","last_undegraded":"2024-09-16T00:13:04.664159+0000","last_fullsized":"2024-09-16T00:13:04.664159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664057+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:13:04.664057+0000","last_peered":"2024-09-16T00:13:04.664057+0000","last_clean":"2024-09-16T00:13:04.664057+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:13:04.664057+0000","last_undegraded":"2024-09-16T00:13:04.664057+0000","last_fullsized":"2024-09-16T00:13:04.664057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:21.681566+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:13:21.681566+0000","last_peered":"2024-09-16T00:13:21.681566+0000","last_clean":"2024-09-16T00:13:21.681566+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:13:21.681566+0000","last_undegraded":"2024-09-16T00:13:21.681566+0000","last_fullsized":"2024-09-16T00:13:21.681566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606730+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:13:04.606730+0000","last_peered":"2024-09-16T00:13:04.606730+0000","last_clean":"2024-09-16T00:13:04.606730+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:13:04.606730+0000","last_undegraded":"2024-09-16T00:13:04.606730+0000","last_fullsized":"2024-09-16T00:13:04.606730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605309+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:13:04.605309+0000","last_peered":"2024-09-16T00:13:04.605309+0000","last_clean":"2024-09-16T00:13:04.605309+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:13:04.605309+0000","last_undegraded":"2024-09-16T00:13:04.605309+0000","last_fullsized":"2024-09-16T00:13:04.605309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:26.730658+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:13:26.730658+0000","last_peered":"2024-09-16T00:13:26.730658+0000","last_clean":"2024-09-16T00:13:26.730658+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:13:26.730658+0000","last_undegraded":"2024-09-16T00:13:26.730658+0000","last_fullsized":"2024-09-16T00:13:26.730658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606979+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:13:04.606979+0000","last_peered":"2024-09-16T00:13:04.606979+0000","last_clean":"2024-09-16T00:13:04.606979+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:13:04.606979+0000","last_undegraded":"2024-09-16T00:13:04.606979+0000","last_fullsized":"2024-09-16T00:13:04.606979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606488+0000","last_change":"2024-09-16T00:12:04.027859+0000","last_active":"2024-09-16T00:13:04.606488+0000","last_peered":"2024-09-16T00:13:04.606488+0000","last_clean":"2024-09-16T00:13:04.606488+0000","last_became_active":"2024-09-16T00:12:04.027506+0000","last_became_peered":"2024-09-16T00:12:04.027506+0000","last_unstale":"2024-09-16T00:13:04.606488+0000","last_undegraded":"2024-09-16T00:13:04.606488+0000","last_fullsized":"2024-09-16T00:13:04.606488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:34.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649981+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:13:04.649981+0000","last_peered":"2024-09-16T00:13:04.649981+0000","last_clean":"2024-09-16T00:13:04.649981+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:13:04.649981+0000","last_undegraded":"2024-09-16T00:13:04.649981+0000","last_fullsized":"2024-09-16T00:13:04.649981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605678+0000","last_change":"2024-09-16T00:12:04.029471+0000","last_active":"2024-09-16T00:13:04.605678+0000","last_peered":"2024-09-16T00:13:04.605678+0000","last_clean":"2024-09-16T00:13:04.605678+0000","last_became_active":"2024-09-16T00:12:04.029054+0000","last_became_peered":"2024-09-16T00:12:04.029054+0000","last_unstale":"2024-09-16T00:13:04.605678+0000","last_undegraded":"2024-09-16T00:13:04.605678+0000","last_fullsized":"2024-09-16T00:13:04.605678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:52.054218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664553+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:13:04.664553+0000","last_peered":"2024-09-16T00:13:04.664553+0000","last_clean":"2024-09-16T00:13:04.664553+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:13:04.664553+0000","last_undegraded":"2024-09-16T00:13:04.664553+0000","last_fullsized":"2024-09-16T00:13:04.664553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611832+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:13:04.611832+0000","last_peered":"2024-09-16T00:13:04.611832+0000","last_clean":"2024-09-16T00:13:04.611832+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:13:04.611832+0000","last_undegraded":"2024-09-16T00:13:04.611832+0000","last_fullsized":"2024-09-16T00:13:04.611832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649917+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:13:04.649917+0000","last_peered":"2024-09-16T00:13:04.649917+0000","last_clean":"2024-09-16T00:13:04.649917+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:13:04.649917+0000","last_undegraded":"2024-09-16T00:13:04.649917+0000","last_fullsized":"2024-09-16T00:13:04.649917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650098+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:13:04.650098+0000","last_peered":"2024-09-16T00:13:04.650098+0000","last_clean":"2024-09-16T00:13:04.650098+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:13:04.650098+0000","last_undegraded":"2024-09-16T00:13:04.650098+0000","last_fullsized":"2024-09-16T00:13:04.650098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649329+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:13:04.649329+0000","last_peered":"2024-09-16T00:13:04.649329+0000","last_clean":"2024-09-16T00:13:04.649329+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:13:04.649329+0000","last_undegraded":"2024-09-16T00:13:04.649329+0000","last_fullsized":"2024-09-16T00:13:04.649329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:23.694358+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:13:23.694358+0000","last_peered":"2024-09-16T00:13:23.694358+0000","last_clean":"2024-09-16T00:13:23.694358+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:13:23.694358+0000","last_undegraded":"2024-09-16T00:13:23.694358+0000","last_fullsized":"2024-09-16T00:13:23.694358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605291+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:13:04.605291+0000","last_peered":"2024-09-16T00:13:04.605291+0000","last_clean":"2024-09-16T00:13:04.605291+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:13:04.605291+0000","last_undegraded":"2024-09-16T00:13:04.605291+0000","last_fullsized":"2024-09-16T00:13:04.605291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649345+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:13:04.649345+0000","last_peered":"2024-09-16T00:13:04.649345+0000","last_clean":"2024-09-16T00:13:04.649345+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:13:04.649345+0000","last_undegraded":"2024-09-16T00:13:04.649345+0000","last_fullsized":"2024-09-16T00:13:04.649345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605828+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:13:04.605828+0000","last_peered":"2024-09-16T00:13:04.605828+0000","last_clean":"2024-09-16T00:13:04.605828+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:13:04.605828+0000","last_undegraded":"2024-09-16T00:13:04.605828+0000","last_fullsized":"2024-09-16T00:13:04.605828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663922+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:13:04.663922+0000","last_peered":"2024-09-16T00:13:04.663922+0000","last_clean":"2024-09-16T00:13:04.663922+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:13:04.663922+0000","last_undegraded":"2024-09-16T00:13:04.663922+0000","last_fullsized":"2024-09-16T00:13:04.663922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650076+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:13:04.650076+0000","last_peered":"2024-09-16T00:13:04.650076+0000","last_clean":"2024-09-16T00:13:04.650076+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:13:04.650076+0000","last_undegraded":"2024-09-16T00:13:04.650076+0000","last_fullsized":"2024-09-16T00:13:04.650076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.607017+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:13:04.607017+0000","last_peered":"2024-09-16T00:13:04.607017+0000","last_clean":"2024-09-16T00:13:04.607017+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:13:04.607017+0000","last_undegraded":"2024-09-16T00:13:04.607017+0000","last_fullsized":"2024-09-16T00:13:04.607017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605023+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:13:04.605023+0000","last_peered":"2024-09-16T00:13:04.605023+0000","last_clean":"2024-09-16T00:13:04.605023+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:13:04.605023+0000","last_undegraded":"2024-09-16T00:13:04.605023+0000","last_fullsized":"2024-09-16T00:13:04.605023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663799+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:13:04.663799+0000","last_peered":"2024-09-16T00:13:04.663799+0000","last_clean":"2024-09-16T00:13:04.663799+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:13:04.663799+0000","last_undegraded":"2024-09-16T00:13:04.663799+0000","last_fullsized":"2024-09-16T00:13:04.663799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664979+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:13:04.664979+0000","last_peered":"2024-09-16T00:13:04.664979+0000","last_clean":"2024-09-16T00:13:04.664979+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:13:04.664979+0000","last_undegraded":"2024-09-16T00:13:04.664979+0000","last_fullsized":"2024-09-16T00:13:04.664979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605103+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:13:04.605103+0000","last_peered":"2024-09-16T00:13:04.605103+0000","last_clean":"2024-09-16T00:13:04.605103+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:13:04.605103+0000","last_undegraded":"2024-09-16T00:13:04.605103+0000","last_fullsized":"2024-09-16T00:13:04.605103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605218+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:13:04.605218+0000","last_peered":"2024-09-16T00:13:04.605218+0000","last_clean":"2024-09-16T00:13:04.605218+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:13:04.605218+0000","last_undegraded":"2024-09-16T00:13:04.605218+0000","last_fullsized":"2024-09-16T00:13:04.605218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664161+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:13:04.664161+0000","last_peered":"2024-09-16T00:13:04.664161+0000","last_clean":"2024-09-16T00:13:04.664161+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:13:04.664161+0000","last_undegraded":"2024-09-16T00:13:04.664161+0000","last_fullsized":"2024-09-16T00:13:04.664161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606964+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:13:04.606964+0000","last_peered":"2024-09-16T00:13:04.606964+0000","last_clean":"2024-09-16T00:13:04.606964+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:13:04.606964+0000","last_undegraded":"2024-09-16T00:13:04.606964+0000","last_fullsized":"2024-09-16T00:13:04.606964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649207+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:13:04.649207+0000","last_peered":"2024-09-16T00:13:04.649207+0000","last_clean":"2024-09-16T00:13:04.649207+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:13:04.649207+0000","last_undegraded":"2024-09-16T00:13:04.649207+0000","last_fullsized":"2024-09-16T00:13:04.649207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:26.731685+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:13:26.731685+0000","last_peered":"2024-09-16T00:13:26.731685+0000","last_clean":"2024-09-16T00:13:26.731685+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:13:26.731685+0000","last_undegraded":"2024-09-16T00:13:26.731685+0000","last_fullsized":"2024-09-16T00:13:26.731685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609415+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:13:04.609415+0000","last_peered":"2024-09-16T00:13:04.609415+0000","last_clean":"2024-09-16T00:13:04.609415+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:13:04.609415+0000","last_undegraded":"2024-09-16T00:13:04.609415+0000","last_fullsized":"2024-09-16T00:13:04.609415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649611+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:13:04.649611+0000","last_peered":"2024-09-16T00:13:04.649611+0000","last_clean":"2024-09-16T00:13:04.649611+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:13:04.649611+0000","last_undegraded":"2024-09-16T00:13:04.649611+0000","last_fullsized":"2024-09-16T00:13:04.649611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605440+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:13:04.605440+0000","last_peered":"2024-09-16T00:13:04.605440+0000","last_clean":"2024-09-16T00:13:04.605440+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:13:04.605440+0000","last_undegraded":"2024-09-16T00:13:04.605440+0000","last_fullsized":"2024-09-16T00:13:04.605440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664557+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:13:04.664557+0000","last_peered":"2024-09-16T00:13:04.664557+0000","last_clean":"2024-09-16T00:13:04.664557+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:13:04.664557+0000","last_undegraded":"2024-09-16T00:13:04.664557+0000","last_fullsized":"2024-09-16T00:13:04.664557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606631+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:13:04.606631+0000","last_peered":"2024-09-16T00:13:04.606631+0000","last_clean":"2024-09-16T00:13:04.606631+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:13:04.606631+0000","last_undegraded":"2024-09-16T00:13:04.606631+0000","last_fullsized":"2024-09-16T00:13:04.606631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611195+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:13:04.611195+0000","last_peered":"2024-09-16T00:13:04.611195+0000","last_clean":"2024-09-16T00:13:04.611195+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:13:04.611195+0000","last_undegraded":"2024-09-16T00:13:04.611195+0000","last_fullsized":"2024-09-16T00:13:04.611195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605540+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:13:04.605540+0000","last_peered":"2024-09-16T00:13:04.605540+0000","last_clean":"2024-09-16T00:13:04.605540+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:13:04.605540+0000","last_undegraded":"2024-09-16T00:13:04.605540+0000","last_fullsized":"2024-09-16T00:13:04.605540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664372+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:13:04.664372+0000","last_peered":"2024-09-16T00:13:04.664372+0000","last_clean":"2024-09-16T00:13:04.664372+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:13:04.664372+0000","last_undegraded":"2024-09-16T00:13:04.664372+0000","last_fullsized":"2024-09-16T00:13:04.664372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":293,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.017020+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:13:25.017020+0000","last_peered":"2024-09-16T00:13:25.017020+0000","last_clean":"2024-09-16T00:13:25.017020+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:13:25.017020+0000","last_undegraded":"2024-09-16T00:13:25.017020+0000","last_fullsized":"2024-09-16T00:13:25.017020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63488,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":168,"num_write":42,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609072+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:13:04.609072+0000","last_peered":"2024-09-16T00:13:04.609072+0000","last_clean":"2024-09-16T00:13:04.609072+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:13:04.609072+0000","last_undegraded":"2024-09-16T00:13:04.609072+0000","last_fullsized":"2024-09-16T00:13:04.609072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605601+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:13:04.605601+0000","last_peered":"2024-09-16T00:13:04.605601+0000","last_clean":"2024-09-16T00:13:04.605601+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:13:04.605601+0000","last_undegraded":"2024-09-16T00:13:04.605601+0000","last_fullsized":"2024-09-16T00:13:04.605601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605741+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:13:04.605741+0000","last_peered":"2024-09-16T00:13:04.605741+0000","last_clean":"2024-09-16T00:13:04.605741+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:13:04.605741+0000","last_undegraded":"2024-09-16T00:13:04.605741+0000","last_fullsized":"2024-09-16T00:13:04.605741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611591+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:13:04.611591+0000","last_peered":"2024-09-16T00:13:04.611591+0000","last_clean":"2024-09-16T00:13:04.611591+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:13:04.611591+0000","last_undegraded":"2024-09-16T00:13:04.611591+0000","last_fullsized":"2024-09-16T00:13:04.611591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663373+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:13:04.663373+0000","last_peered":"2024-09-16T00:13:04.663373+0000","last_clean":"2024-09-16T00:13:04.663373+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:13:04.663373+0000","last_undegraded":"2024-09-16T00:13:04.663373+0000","last_fullsized":"2024-09-16T00:13:04.663373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649999+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:13:04.649999+0000","last_peered":"2024-09-16T00:13:04.649999+0000","last_clean":"2024-09-16T00:13:04.649999+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:13:04.649999+0000","last_undegraded":"2024-09-16T00:13:04.649999+0000","last_fullsized":"2024-09-16T00:13:04.649999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.608894+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:13:04.608894+0000","last_peered":"2024-09-16T00:13:04.608894+0000","last_clean":"2024-09-16T00:13:04.608894+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:13:04.608894+0000","last_undegraded":"2024-09-16T00:13:04.608894+0000","last_fullsized":"2024-09-16T00:13:04.608894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605695+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:13:04.605695+0000","last_peered":"2024-09-16T00:13:04.605695+0000","last_clean":"2024-09-16T00:13:04.605695+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:13:04.605695+0000","last_undegraded":"2024-09-16T00:13:04.605695+0000","last_fullsized":"2024-09-16T00:13:04.605695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":248,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:24.695271+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:13:24.695271+0000","last_peered":"2024-09-16T00:13:24.695271+0000","last_clean":"2024-09-16T00:13:24.695271+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:13:24.695271+0000","last_undegraded":"2024-09-16T00:13:24.695271+0000","last_fullsized":"2024-09-16T00:13:24.695271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664350+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:13:04.664350+0000","last_peered":"2024-09-16T00:13:04.664350+0000","last_clean":"2024-09-16T00:13:04.664350+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:13:04.664350+0000","last_undegraded":"2024-09-16T00:13:04.664350+0000","last_fullsized":"2024-09-16T00:13:04.664350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663978+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:13:04.663978+0000","last_peered":"2024-09-16T00:13:04.663978+0000","last_clean":"2024-09-16T00:13:04.663978+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:13:04.663978+0000","last_undegraded":"2024-09-16T00:13:04.663978+0000","last_fullsized":"2024-09-16T00:13:04.663978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609007+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:13:04.609007+0000","last_peered":"2024-09-16T00:13:04.609007+0000","last_clean":"2024-09-16T00:13:04.609007+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:13:04.609007+0000","last_undegraded":"2024-09-16T00:13:04.609007+0000","last_fullsized":"2024-09-16T00:13:04.609007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605461+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:13:04.605461+0000","last_peered":"2024-09-16T00:13:04.605461+0000","last_clean":"2024-09-16T00:13:04.605461+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:13:04.605461+0000","last_undegraded":"2024-09-16T00:13:04.605461+0000","last_fullsized":"2024-09-16T00:13:04.605461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664904+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:13:04.664904+0000","last_peered":"2024-09-16T00:13:04.664904+0000","last_clean":"2024-09-16T00:13:04.664904+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:13:04.664904+0000","last_undegraded":"2024-09-16T00:13:04.664904+0000","last_fullsized":"2024-09-16T00:13:04.664904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663317+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:13:04.663317+0000","last_peered":"2024-09-16T00:13:04.663317+0000","last_clean":"2024-09-16T00:13:04.663317+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:13:04.663317+0000","last_undegraded":"2024-09-16T00:13:04.663317+0000","last_fullsized":"2024-09-16T00:13:04.663317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606517+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:13:04.606517+0000","last_peered":"2024-09-16T00:13:04.606517+0000","last_clean":"2024-09-16T00:13:04.606517+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:13:04.606517+0000","last_undegraded":"2024-09-16T00:13:04.606517+0000","last_fullsized":"2024-09-16T00:13:04.606517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664780+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:13:04.664780+0000","last_peered":"2024-09-16T00:13:04.664780+0000","last_clean":"2024-09-16T00:13:04.664780+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:13:04.664780+0000","last_undegraded":"2024-09-16T00:13:04.664780+0000","last_fullsized":"2024-09-16T00:13:04.664780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605384+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:13:04.605384+0000","last_peered":"2024-09-16T00:13:04.605384+0000","last_clean":"2024-09-16T00:13:04.605384+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:13:04.605384+0000","last_undegraded":"2024-09-16T00:13:04.605384+0000","last_fullsized":"2024-09-16T00:13:04.605384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610913+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:13:04.610913+0000","last_peered":"2024-09-16T00:13:04.610913+0000","last_clean":"2024-09-16T00:13:04.610913+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:13:04.610913+0000","last_undegraded":"2024-09-16T00:13:04.610913+0000","last_fullsized":"2024-09-16T00:13:04.610913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.697247+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:13:25.697247+0000","last_peered":"2024-09-16T00:13:25.697247+0000","last_clean":"2024-09-16T00:13:25.697247+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:13:25.697247+0000","last_undegraded":"2024-09-16T00:13:25.697247+0000","last_fullsized":"2024-09-16T00:13:25.697247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663709+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:13:04.663709+0000","last_peered":"2024-09-16T00:13:04.663709+0000","last_clean":"2024-09-16T00:13:04.663709+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:13:04.663709+0000","last_undegraded":"2024-09-16T00:13:04.663709+0000","last_fullsized":"2024-09-16T00:13:04.663709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611136+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:13:04.611136+0000","last_peered":"2024-09-16T00:13:04.611136+0000","last_clean":"2024-09-16T00:13:04.611136+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:13:04.611136+0000","last_undegraded":"2024-09-16T00:13:04.611136+0000","last_fullsized":"2024-09-16T00:13:04.611136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663783+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:13:04.663783+0000","last_peered":"2024-09-16T00:13:04.663783+0000","last_clean":"2024-09-16T00:13:04.663783+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:13:04.663783+0000","last_undegraded":"2024-09-16T00:13:04.663783+0000","last_fullsized":"2024-09-16T00:13:04.663783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611426+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:13:04.611426+0000","last_peered":"2024-09-16T00:13:04.611426+0000","last_clean":"2024-09-16T00:13:04.611426+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:13:04.611426+0000","last_undegraded":"2024-09-16T00:13:04.611426+0000","last_fullsized":"2024-09-16T00:13:04.611426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610934+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:13:04.610934+0000","last_peered":"2024-09-16T00:13:04.610934+0000","last_clean":"2024-09-16T00:13:04.610934+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:13:04.610934+0000","last_undegraded":"2024-09-16T00:13:04.610934+0000","last_fullsized":"2024-09-16T00:13:04.610934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663979+0000","last_change":"2024-09-16T00:12:04.029280+0000","last_active":"2024-09-16T00:13:04.663979+0000","last_peered":"2024-09-16T00:13:04.663979+0000","last_clean":"2024-09-16T00:13:04.663979+0000","last_became_active":"2024-09-16T00:12:04.028973+0000","last_became_peered":"2024-09-16T00:12:04.028973+0000","last_unstale":"2024-09-16T00:13:04.663979+0000","last_undegraded":"2024-09-16T00:13:04.663979+0000","last_fullsized":"2024-09-16T00:13:04.663979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606846+0000","last_change":"2024-09-16T00:12:04.028319+0000","last_active":"2024-09-16T00:13:04.606846+0000","last_peered":"2024-09-16T00:13:04.606846+0000","last_clean":"2024-09-16T00:13:04.606846+0000","last_became_active":"2024-09-16T00:12:04.027985+0000","last_became_peered":"2024-09-16T00:12:04.027985+0000","last_unstale":"2024-09-16T00:13:04.606846+0000","last_undegraded":"2024-09-16T00:13:04.606846+0000","last_fullsized":"2024-09-16T00:13:04.606846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.119428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663790+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:13:04.663790+0000","last_peered":"2024-09-16T00:13:04.663790+0000","last_clean":"2024-09-16T00:13:04.663790+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:13:04.663790+0000","last_undegraded":"2024-09-16T00:13:04.663790+0000","last_fullsized":"2024-09-16T00:13:04.663790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611369+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:13:04.611369+0000","last_peered":"2024-09-16T00:13:04.611369+0000","last_clean":"2024-09-16T00:13:04.611369+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:13:04.611369+0000","last_undegraded":"2024-09-16T00:13:04.611369+0000","last_fullsized":"2024-09-16T00:13:04.611369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.616864+0000","last_change":"2024-09-16T00:13:04.612598+0000","last_active":"2024-09-16T00:13:04.616864+0000","last_peered":"2024-09-16T00:13:04.616864+0000","last_clean":"2024-09-16T00:13:04.616864+0000","last_became_active":"2024-09-16T00:13:04.612310+0000","last_became_peered":"2024-09-16T00:13:04.612310+0000","last_unstale":"2024-09-16T00:13:04.616864+0000","last_undegraded":"2024-09-16T00:13:04.616864+0000","last_fullsized":"2024-09-16T00:13:04.616864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:59.381944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605240+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:13:04.605240+0000","last_peered":"2024-09-16T00:13:04.605240+0000","last_clean":"2024-09-16T00:13:04.605240+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:13:04.605240+0000","last_undegraded":"2024-09-16T00:13:04.605240+0000","last_fullsized":"2024-09-16T00:13:04.605240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663870+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:13:04.663870+0000","last_peered":"2024-09-16T00:13:04.663870+0000","last_clean":"2024-09-16T00:13:04.663870+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:13:04.663870+0000","last_undegraded":"2024-09-16T00:13:04.663870+0000","last_fullsized":"2024-09-16T00:13:04.663870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605535+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:13:04.605535+0000","last_peered":"2024-09-16T00:13:04.605535+0000","last_clean":"2024-09-16T00:13:04.605535+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:13:04.605535+0000","last_undegraded":"2024-09-16T00:13:04.605535+0000","last_fullsized":"2024-09-16T00:13:04.605535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606766+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:13:04.606766+0000","last_peered":"2024-09-16T00:13:04.606766+0000","last_clean":"2024-09-16T00:13:04.606766+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:13:04.606766+0000","last_undegraded":"2024-09-16T00:13:04.606766+0000","last_fullsized":"2024-09-16T00:13:04.606766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663880+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:13:04.663880+0000","last_peered":"2024-09-16T00:13:04.663880+0000","last_clean":"2024-09-16T00:13:04.663880+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:13:04.663880+0000","last_undegraded":"2024-09-16T00:13:04.663880+0000","last_fullsized":"2024-09-16T00:13:04.663880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611528+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:13:04.611528+0000","last_peered":"2024-09-16T00:13:04.611528+0000","last_clean":"2024-09-16T00:13:04.611528+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:13:04.611528+0000","last_undegraded":"2024-09-16T00:13:04.611528+0000","last_fullsized":"2024-09-16T00:13:04.611528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609303+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:13:04.609303+0000","last_peered":"2024-09-16T00:13:04.609303+0000","last_clean":"2024-09-16T00:13:04.609303+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:13:04.609303+0000","last_undegraded":"2024-09-16T00:13:04.609303+0000","last_fullsized":"2024-09-16T00:13:04.609303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663938+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:13:04.663938+0000","last_peered":"2024-09-16T00:13:04.663938+0000","last_clean":"2024-09-16T00:13:04.663938+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:13:04.663938+0000","last_undegraded":"2024-09-16T00:13:04.663938+0000","last_fullsized":"2024-09-16T00:13:04.663938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605284+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:13:04.605284+0000","last_peered":"2024-09-16T00:13:04.605284+0000","last_clean":"2024-09-16T00:13:04.605284+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:13:04.605284+0000","last_undegraded":"2024-09-16T00:13:04.605284+0000","last_fullsized":"2024-09-16T00:13:04.605284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663375+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:13:04.663375+0000","last_peered":"2024-09-16T00:13:04.663375+0000","last_clean":"2024-09-16T00:13:04.663375+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:13:04.663375+0000","last_undegraded":"2024-09-16T00:13:04.663375+0000","last_fullsized":"2024-09-16T00:13:04.663375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663627+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:13:04.663627+0000","last_peered":"2024-09-16T00:13:04.663627+0000","last_clean":"2024-09-16T00:13:04.663627+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:13:04.663627+0000","last_undegraded":"2024-09-16T00:13:04.663627+0000","last_fullsized":"2024-09-16T00:13:04.663627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606689+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:13:04.606689+0000","last_peered":"2024-09-16T00:13:04.606689+0000","last_clean":"2024-09-16T00:13:04.606689+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:13:04.606689+0000","last_undegraded":"2024-09-16T00:13:04.606689+0000","last_fullsized":"2024-09-16T00:13:04.606689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611636+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:13:04.611636+0000","last_peered":"2024-09-16T00:13:04.611636+0000","last_clean":"2024-09-16T00:13:04.611636+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:13:04.611636+0000","last_undegraded":"2024-09-16T00:13:04.611636+0000","last_fullsized":"2024-09-16T00:13:04.611636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649291+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:13:04.649291+0000","last_peered":"2024-09-16T00:13:04.649291+0000","last_clean":"2024-09-16T00:13:04.649291+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:13:04.649291+0000","last_undegraded":"2024-09-16T00:13:04.649291+0000","last_fullsized":"2024-09-16T00:13:04.649291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664002+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:13:04.664002+0000","last_peered":"2024-09-16T00:13:04.664002+0000","last_clean":"2024-09-16T00:13:04.664002+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:13:04.664002+0000","last_undegraded":"2024-09-16T00:13:04.664002+0000","last_fullsized":"2024-09-16T00:13:04.664002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605780+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:13:04.605780+0000","last_peered":"2024-09-16T00:13:04.605780+0000","last_clean":"2024-09-16T00:13:04.605780+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:13:04.605780+0000","last_undegraded":"2024-09-16T00:13:04.605780+0000","last_fullsized":"2024-09-16T00:13:04.605780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663843+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:13:04.663843+0000","last_peered":"2024-09-16T00:13:04.663843+0000","last_clean":"2024-09-16T00:13:04.663843+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:13:04.663843+0000","last_undegraded":"2024-09-16T00:13:04.663843+0000","last_fullsized":"2024-09-16T00:13:04.663843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649554+0000","last_change":"2024-09-16T00:12:04.028342+0000","last_active":"2024-09-16T00:13:04.649554+0000","last_peered":"2024-09-16T00:13:04.649554+0000","last_clean":"2024-09-16T00:13:04.649554+0000","last_became_active":"2024-09-16T00:12:04.027889+0000","last_became_peered":"2024-09-16T00:12:04.027889+0000","last_unstale":"2024-09-16T00:13:04.649554+0000","last_undegraded":"2024-09-16T00:13:04.649554+0000","last_fullsized":"2024-09-16T00:13:04.649554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:47.840428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663964+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:13:04.663964+0000","last_peered":"2024-09-16T00:13:04.663964+0000","last_clean":"2024-09-16T00:13:04.663964+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:13:04.663964+0000","last_undegraded":"2024-09-16T00:13:04.663964+0000","last_fullsized":"2024-09-16T00:13:04.663964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663878+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:13:04.663878+0000","last_peered":"2024-09-16T00:13:04.663878+0000","last_clean":"2024-09-16T00:13:04.663878+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:13:04.663878+0000","last_undegraded":"2024-09-16T00:13:04.663878+0000","last_fullsized":"2024-09-16T00:13:04.663878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611689+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:13:04.611689+0000","last_peered":"2024-09-16T00:13:04.611689+0000","last_clean":"2024-09-16T00:13:04.611689+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:13:04.611689+0000","last_undegraded":"2024-09-16T00:13:04.611689+0000","last_fullsized":"2024-09-16T00:13:04.611689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664072+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:13:04.664072+0000","last_peered":"2024-09-16T00:13:04.664072+0000","last_clean":"2024-09-16T00:13:04.664072+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:13:04.664072+0000","last_undegraded":"2024-09-16T00:13:04.664072+0000","last_fullsized":"2024-09-16T00:13:04.664072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664019+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:13:04.664019+0000","last_peered":"2024-09-16T00:13:04.664019+0000","last_clean":"2024-09-16T00:13:04.664019+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:13:04.664019+0000","last_undegraded":"2024-09-16T00:13:04.664019+0000","last_fullsized":"2024-09-16T00:13:04.664019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71679,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1069,"num_read_kb":613,"num_write":269,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":515263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1736704,"data_stored":1165567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972405,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1069056,"data_stored":518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201338,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":515037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1658880,"data_stored":1111677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593796,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36972,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708180,"statfs":{"total":95995035648,"available":95957176320,"internally_reserved":0,"allocated":1667072,"data_stored":1107033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":515164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":509871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1130496,"data_stored":575435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1110016,"data_stored":574373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":512153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:13:29.552 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:13:29.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 -- 172.21.15.78:0/3730308511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c1036e0 msgr2=0x7f6f1c109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 --2- 172.21.15.78:0/3730308511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c1036e0 0x7f6f1c109f70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009ca0 tx=0x7f6f1402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 -- 172.21.15.78:0/3730308511 shutdown_connections 2024-09-16T00:13:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 --2- 172.21.15.78:0/3730308511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c1036e0 0x7f6f1c109f70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 --2- 172.21.15.78:0/3730308511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f1c102d20 0x7f6f1c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 --2- 172.21.15.78:0/3730308511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f1c101b30 0x7f6f1c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 -- 172.21.15.78:0/3730308511 >> 172.21.15.78:0/3730308511 conn(0x7f6f1c0fd270 msgr2=0x7f6f1c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 -- 172.21.15.78:0/3730308511 shutdown_connections 2024-09-16T00:13:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.684+0000 7f6f20e27640 1 -- 172.21.15.78:0/3730308511 wait complete. 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f20e27640 1 Processor -- start 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f20e27640 1 -- start start 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f20e27640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f1c101b30 0x7f6f1c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f20e27640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c102d20 0x7f6f1c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f20e27640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f1c1036e0 0x7f6f1c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f20e27640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10d790 con 0x7f6f1c102d20 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f20e27640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10d610 con 0x7f6f1c1036e0 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f20e27640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c10d910 con 0x7f6f1c101b30 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c102d20 0x7f6f1c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c102d20 0x7f6f1c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58756/0 (socket says 172.21.15.78:58756) 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 -- 172.21.15.78:0/2306856319 learned_addr learned my addr 172.21.15.78:0/2306856319 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f1a59c640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f1c101b30 0x7f6f1c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f1ad9d640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f1c1036e0 0x7f6f1c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 -- 172.21.15.78:0/2306856319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f1c101b30 msgr2=0x7f6f1c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f1c101b30 0x7f6f1c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 -- 172.21.15.78:0/2306856319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f1c1036e0 msgr2=0x7f6f1c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f1c1036e0 0x7f6f1c19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 -- 172.21.15.78:0/2306856319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c19fa10 con 0x7f6f1c102d20 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f1a59c640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f1c101b30 0x7f6f1c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.685+0000 7f6f19d9b640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c102d20 0x7f6f1c198330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000d950 tx=0x7f6f1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.686+0000 7f6f0b7fe640 1 -- 172.21.15.78:0/2306856319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10017070 con 0x7f6f1c102d20 2024-09-16T00:13:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.686+0000 7f6f0b7fe640 1 -- 172.21.15.78:0/2306856319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f10009c30 con 0x7f6f1c102d20 2024-09-16T00:13:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.686+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c19fd00 con 0x7f6f1c102d20 2024-09-16T00:13:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.686+0000 7f6f0b7fe640 1 -- 172.21.15.78:0/2306856319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000f6b0 con 0x7f6f1c102d20 2024-09-16T00:13:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.686+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1a01f0 con 0x7f6f1c102d20 2024-09-16T00:13:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.687+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c108f20 con 0x7f6f1c102d20 2024-09-16T00:13:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.688+0000 7f6f0b7fe640 1 -- 172.21.15.78:0/2306856319 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f6f100109d0 con 0x7f6f1c102d20 2024-09-16T00:13:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.689+0000 7f6f0b7fe640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f040a32e0 0x7f6f040a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.689+0000 7f6f0b7fe640 1 -- 172.21.15.78:0/2306856319 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6f100cfb00 con 0x7f6f1c102d20 2024-09-16T00:13:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.689+0000 7f6f1a59c640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f040a32e0 0x7f6f040a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.689+0000 7f6f1a59c640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f040a32e0 0x7f6f040a57a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0058c0 tx=0x7f6f0c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.693+0000 7f6f0b7fe640 1 -- 172.21.15.78:0/2306856319 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f10009de0 con 0x7f6f1c102d20 2024-09-16T00:13:29.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.858+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f1c062630 con 0x7f6f040a32e0 2024-09-16T00:13:29.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.865+0000 7f6f0b7fe640 1 -- 172.21.15.78:0/2306856319 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340526 (secure 0 0 0) 0x7f6f1c062630 con 0x7f6f040a32e0 2024-09-16T00:13:29.869 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:29.871 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:13:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.868+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f040a32e0 msgr2=0x7f6f040a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.868+0000 7f6f20e27640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f040a32e0 0x7f6f040a57a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0058c0 tx=0x7f6f0c005850 comp rx=0 tx=0).stop 2024-09-16T00:13:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.868+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c102d20 msgr2=0x7f6f1c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.868+0000 7f6f20e27640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c102d20 0x7f6f1c198330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000d950 tx=0x7f6f1000de10 comp rx=0 tx=0).stop 2024-09-16T00:13:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.869+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 shutdown_connections 2024-09-16T00:13:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.869+0000 7f6f20e27640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f040a32e0 0x7f6f040a57a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.869+0000 7f6f20e27640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f1c1036e0 0x7f6f1c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.869+0000 7f6f20e27640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f1c102d20 0x7f6f1c198330 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.869+0000 7f6f20e27640 1 --2- 172.21.15.78:0/2306856319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f1c101b30 0x7f6f1c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.869+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 >> 172.21.15.78:0/2306856319 conn(0x7f6f1c0fd270 msgr2=0x7f6f1c1060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.869+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 shutdown_connections 2024-09-16T00:13:29.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:29.869+0000 7f6f20e27640 1 -- 172.21.15.78:0/2306856319 wait complete. 2024-09-16T00:13:29.880 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-16T00:13:28.430524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662281,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1195,"num_read_kb":722,"num_write":452,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359472,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582352,"statfs":{"total":1151940427776,"available":1151572328448,"internally_reserved":0,"allocated":14680064,"data_stored":8130884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1009,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003061"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:23.683882+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:13:23.683882+0000","last_peered":"2024-09-16T00:13:23.683882+0000","last_clean":"2024-09-16T00:13:23.683882+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:13:23.683882+0000","last_undegraded":"2024-09-16T00:13:23.683882+0000","last_fullsized":"2024-09-16T00:13:23.683882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610803+0000","last_change":"2024-09-16T00:12:04.132674+0000","last_active":"2024-09-16T00:13:04.610803+0000","last_peered":"2024-09-16T00:13:04.610803+0000","last_clean":"2024-09-16T00:13:04.610803+0000","last_became_active":"2024-09-16T00:12:04.132108+0000","last_became_peered":"2024-09-16T00:12:04.132108+0000","last_unstale":"2024-09-16T00:13:04.610803+0000","last_undegraded":"2024-09-16T00:13:04.610803+0000","last_fullsized":"2024-09-16T00:13:04.610803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:10.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664962+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:13:04.664962+0000","last_peered":"2024-09-16T00:13:04.664962+0000","last_clean":"2024-09-16T00:13:04.664962+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:13:04.664962+0000","last_undegraded":"2024-09-16T00:13:04.664962+0000","last_fullsized":"2024-09-16T00:13:04.664962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663575+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:13:04.663575+0000","last_peered":"2024-09-16T00:13:04.663575+0000","last_clean":"2024-09-16T00:13:04.663575+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:13:04.663575+0000","last_undegraded":"2024-09-16T00:13:04.663575+0000","last_fullsized":"2024-09-16T00:13:04.663575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611038+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:13:04.611038+0000","last_peered":"2024-09-16T00:13:04.611038+0000","last_clean":"2024-09-16T00:13:04.611038+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:13:04.611038+0000","last_undegraded":"2024-09-16T00:13:04.611038+0000","last_fullsized":"2024-09-16T00:13:04.611038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664541+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:13:04.664541+0000","last_peered":"2024-09-16T00:13:04.664541+0000","last_clean":"2024-09-16T00:13:04.664541+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:13:04.664541+0000","last_undegraded":"2024-09-16T00:13:04.664541+0000","last_fullsized":"2024-09-16T00:13:04.664541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605363+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:13:04.605363+0000","last_peered":"2024-09-16T00:13:04.605363+0000","last_clean":"2024-09-16T00:13:04.605363+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:13:04.605363+0000","last_undegraded":"2024-09-16T00:13:04.605363+0000","last_fullsized":"2024-09-16T00:13:04.605363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611375+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:13:04.611375+0000","last_peered":"2024-09-16T00:13:04.611375+0000","last_clean":"2024-09-16T00:13:04.611375+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:13:04.611375+0000","last_undegraded":"2024-09-16T00:13:04.611375+0000","last_fullsized":"2024-09-16T00:13:04.611375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664351+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:13:04.664351+0000","last_peered":"2024-09-16T00:13:04.664351+0000","last_clean":"2024-09-16T00:13:04.664351+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:13:04.664351+0000","last_undegraded":"2024-09-16T00:13:04.664351+0000","last_fullsized":"2024-09-16T00:13:04.664351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605600+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:13:04.605600+0000","last_peered":"2024-09-16T00:13:04.605600+0000","last_clean":"2024-09-16T00:13:04.605600+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:13:04.605600+0000","last_undegraded":"2024-09-16T00:13:04.605600+0000","last_fullsized":"2024-09-16T00:13:04.605600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611103+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:13:04.611103+0000","last_peered":"2024-09-16T00:13:04.611103+0000","last_clean":"2024-09-16T00:13:04.611103+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:13:04.611103+0000","last_undegraded":"2024-09-16T00:13:04.611103+0000","last_fullsized":"2024-09-16T00:13:04.611103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649281+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:13:04.649281+0000","last_peered":"2024-09-16T00:13:04.649281+0000","last_clean":"2024-09-16T00:13:04.649281+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:13:04.649281+0000","last_undegraded":"2024-09-16T00:13:04.649281+0000","last_fullsized":"2024-09-16T00:13:04.649281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664252+0000","last_change":"2024-09-16T00:12:04.033464+0000","last_active":"2024-09-16T00:13:04.664252+0000","last_peered":"2024-09-16T00:13:04.664252+0000","last_clean":"2024-09-16T00:13:04.664252+0000","last_became_active":"2024-09-16T00:12:04.028916+0000","last_became_peered":"2024-09-16T00:12:04.028916+0000","last_unstale":"2024-09-16T00:13:04.664252+0000","last_undegraded":"2024-09-16T00:13:04.664252+0000","last_fullsized":"2024-09-16T00:13:04.664252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:47.359979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611584+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:13:04.611584+0000","last_peered":"2024-09-16T00:13:04.611584+0000","last_clean":"2024-09-16T00:13:04.611584+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:13:04.611584+0000","last_undegraded":"2024-09-16T00:13:04.611584+0000","last_fullsized":"2024-09-16T00:13:04.611584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664652+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:13:04.664652+0000","last_peered":"2024-09-16T00:13:04.664652+0000","last_clean":"2024-09-16T00:13:04.664652+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:13:04.664652+0000","last_undegraded":"2024-09-16T00:13:04.664652+0000","last_fullsized":"2024-09-16T00:13:04.664652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610969+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:13:04.610969+0000","last_peered":"2024-09-16T00:13:04.610969+0000","last_clean":"2024-09-16T00:13:04.610969+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:13:04.610969+0000","last_undegraded":"2024-09-16T00:13:04.610969+0000","last_fullsized":"2024-09-16T00:13:04.610969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611391+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:13:04.611391+0000","last_peered":"2024-09-16T00:13:04.611391+0000","last_clean":"2024-09-16T00:13:04.611391+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:13:04.611391+0000","last_undegraded":"2024-09-16T00:13:04.611391+0000","last_fullsized":"2024-09-16T00:13:04.611391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663782+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:13:04.663782+0000","last_peered":"2024-09-16T00:13:04.663782+0000","last_clean":"2024-09-16T00:13:04.663782+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:13:04.663782+0000","last_undegraded":"2024-09-16T00:13:04.663782+0000","last_fullsized":"2024-09-16T00:13:04.663782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664498+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:13:04.664498+0000","last_peered":"2024-09-16T00:13:04.664498+0000","last_clean":"2024-09-16T00:13:04.664498+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:13:04.664498+0000","last_undegraded":"2024-09-16T00:13:04.664498+0000","last_fullsized":"2024-09-16T00:13:04.664498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609329+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:13:04.609329+0000","last_peered":"2024-09-16T00:13:04.609329+0000","last_clean":"2024-09-16T00:13:04.609329+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:13:04.609329+0000","last_undegraded":"2024-09-16T00:13:04.609329+0000","last_fullsized":"2024-09-16T00:13:04.609329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605426+0000","last_change":"2024-09-16T00:12:04.034002+0000","last_active":"2024-09-16T00:13:04.605426+0000","last_peered":"2024-09-16T00:13:04.605426+0000","last_clean":"2024-09-16T00:13:04.605426+0000","last_became_active":"2024-09-16T00:12:04.030959+0000","last_became_peered":"2024-09-16T00:12:04.030959+0000","last_unstale":"2024-09-16T00:13:04.605426+0000","last_undegraded":"2024-09-16T00:13:04.605426+0000","last_fullsized":"2024-09-16T00:13:04.605426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:47.730692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611324+0000","last_change":"2024-09-16T00:12:04.029906+0000","last_active":"2024-09-16T00:13:04.611324+0000","last_peered":"2024-09-16T00:13:04.611324+0000","last_clean":"2024-09-16T00:13:04.611324+0000","last_became_active":"2024-09-16T00:12:04.029259+0000","last_became_peered":"2024-09-16T00:12:04.029259+0000","last_unstale":"2024-09-16T00:13:04.611324+0000","last_undegraded":"2024-09-16T00:13:04.611324+0000","last_fullsized":"2024-09-16T00:13:04.611324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:23.399762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,1],"acting":[4,8,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663413+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:13:04.663413+0000","last_peered":"2024-09-16T00:13:04.663413+0000","last_clean":"2024-09-16T00:13:04.663413+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:13:04.663413+0000","last_undegraded":"2024-09-16T00:13:04.663413+0000","last_fullsized":"2024-09-16T00:13:04.663413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611856+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:13:04.611856+0000","last_peered":"2024-09-16T00:13:04.611856+0000","last_clean":"2024-09-16T00:13:04.611856+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:13:04.611856+0000","last_undegraded":"2024-09-16T00:13:04.611856+0000","last_fullsized":"2024-09-16T00:13:04.611856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610909+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:13:04.610909+0000","last_peered":"2024-09-16T00:13:04.610909+0000","last_clean":"2024-09-16T00:13:04.610909+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:13:04.610909+0000","last_undegraded":"2024-09-16T00:13:04.610909+0000","last_fullsized":"2024-09-16T00:13:04.610909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.665046+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:13:04.665046+0000","last_peered":"2024-09-16T00:13:04.665046+0000","last_clean":"2024-09-16T00:13:04.665046+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:13:04.665046+0000","last_undegraded":"2024-09-16T00:13:04.665046+0000","last_fullsized":"2024-09-16T00:13:04.665046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'19","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609370+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:13:04.609370+0000","last_peered":"2024-09-16T00:13:04.609370+0000","last_clean":"2024-09-16T00:13:04.609370+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:13:04.609370+0000","last_undegraded":"2024-09-16T00:13:04.609370+0000","last_fullsized":"2024-09-16T00:13:04.609370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664101+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:13:04.664101+0000","last_peered":"2024-09-16T00:13:04.664101+0000","last_clean":"2024-09-16T00:13:04.664101+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:13:04.664101+0000","last_undegraded":"2024-09-16T00:13:04.664101+0000","last_fullsized":"2024-09-16T00:13:04.664101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606866+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:13:04.606866+0000","last_peered":"2024-09-16T00:13:04.606866+0000","last_clean":"2024-09-16T00:13:04.606866+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:13:04.606866+0000","last_undegraded":"2024-09-16T00:13:04.606866+0000","last_fullsized":"2024-09-16T00:13:04.606866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664634+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:13:04.664634+0000","last_peered":"2024-09-16T00:13:04.664634+0000","last_clean":"2024-09-16T00:13:04.664634+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:13:04.664634+0000","last_undegraded":"2024-09-16T00:13:04.664634+0000","last_fullsized":"2024-09-16T00:13:04.664634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664409+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:13:04.664409+0000","last_peered":"2024-09-16T00:13:04.664409+0000","last_clean":"2024-09-16T00:13:04.664409+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:13:04.664409+0000","last_undegraded":"2024-09-16T00:13:04.664409+0000","last_fullsized":"2024-09-16T00:13:04.664409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664046+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:13:04.664046+0000","last_peered":"2024-09-16T00:13:04.664046+0000","last_clean":"2024-09-16T00:13:04.664046+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:13:04.664046+0000","last_undegraded":"2024-09-16T00:13:04.664046+0000","last_fullsized":"2024-09-16T00:13:04.664046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609377+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:13:04.609377+0000","last_peered":"2024-09-16T00:13:04.609377+0000","last_clean":"2024-09-16T00:13:04.609377+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:13:04.609377+0000","last_undegraded":"2024-09-16T00:13:04.609377+0000","last_fullsized":"2024-09-16T00:13:04.609377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664159+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:13:04.664159+0000","last_peered":"2024-09-16T00:13:04.664159+0000","last_clean":"2024-09-16T00:13:04.664159+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:13:04.664159+0000","last_undegraded":"2024-09-16T00:13:04.664159+0000","last_fullsized":"2024-09-16T00:13:04.664159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664057+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:13:04.664057+0000","last_peered":"2024-09-16T00:13:04.664057+0000","last_clean":"2024-09-16T00:13:04.664057+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:13:04.664057+0000","last_undegraded":"2024-09-16T00:13:04.664057+0000","last_fullsized":"2024-09-16T00:13:04.664057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:21.681566+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:13:21.681566+0000","last_peered":"2024-09-16T00:13:21.681566+0000","last_clean":"2024-09-16T00:13:21.681566+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:13:21.681566+0000","last_undegraded":"2024-09-16T00:13:21.681566+0000","last_fullsized":"2024-09-16T00:13:21.681566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606730+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:13:04.606730+0000","last_peered":"2024-09-16T00:13:04.606730+0000","last_clean":"2024-09-16T00:13:04.606730+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:13:04.606730+0000","last_undegraded":"2024-09-16T00:13:04.606730+0000","last_fullsized":"2024-09-16T00:13:04.606730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605309+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:13:04.605309+0000","last_peered":"2024-09-16T00:13:04.605309+0000","last_clean":"2024-09-16T00:13:04.605309+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:13:04.605309+0000","last_undegraded":"2024-09-16T00:13:04.605309+0000","last_fullsized":"2024-09-16T00:13:04.605309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:26.730658+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:13:26.730658+0000","last_peered":"2024-09-16T00:13:26.730658+0000","last_clean":"2024-09-16T00:13:26.730658+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:13:26.730658+0000","last_undegraded":"2024-09-16T00:13:26.730658+0000","last_fullsized":"2024-09-16T00:13:26.730658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606979+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:13:04.606979+0000","last_peered":"2024-09-16T00:13:04.606979+0000","last_clean":"2024-09-16T00:13:04.606979+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:13:04.606979+0000","last_undegraded":"2024-09-16T00:13:04.606979+0000","last_fullsized":"2024-09-16T00:13:04.606979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606488+0000","last_change":"2024-09-16T00:12:04.027859+0000","last_active":"2024-09-16T00:13:04.606488+0000","last_peered":"2024-09-16T00:13:04.606488+0000","last_clean":"2024-09-16T00:13:04.606488+0000","last_became_active":"2024-09-16T00:12:04.027506+0000","last_became_peered":"2024-09-16T00:12:04.027506+0000","last_unstale":"2024-09-16T00:13:04.606488+0000","last_undegraded":"2024-09-16T00:13:04.606488+0000","last_fullsized":"2024-09-16T00:13:04.606488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:34.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649981+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:13:04.649981+0000","last_peered":"2024-09-16T00:13:04.649981+0000","last_clean":"2024-09-16T00:13:04.649981+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:13:04.649981+0000","last_undegraded":"2024-09-16T00:13:04.649981+0000","last_fullsized":"2024-09-16T00:13:04.649981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605678+0000","last_change":"2024-09-16T00:12:04.029471+0000","last_active":"2024-09-16T00:13:04.605678+0000","last_peered":"2024-09-16T00:13:04.605678+0000","last_clean":"2024-09-16T00:13:04.605678+0000","last_became_active":"2024-09-16T00:12:04.029054+0000","last_became_peered":"2024-09-16T00:12:04.029054+0000","last_unstale":"2024-09-16T00:13:04.605678+0000","last_undegraded":"2024-09-16T00:13:04.605678+0000","last_fullsized":"2024-09-16T00:13:04.605678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:52.054218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664553+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:13:04.664553+0000","last_peered":"2024-09-16T00:13:04.664553+0000","last_clean":"2024-09-16T00:13:04.664553+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:13:04.664553+0000","last_undegraded":"2024-09-16T00:13:04.664553+0000","last_fullsized":"2024-09-16T00:13:04.664553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611832+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:13:04.611832+0000","last_peered":"2024-09-16T00:13:04.611832+0000","last_clean":"2024-09-16T00:13:04.611832+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:13:04.611832+0000","last_undegraded":"2024-09-16T00:13:04.611832+0000","last_fullsized":"2024-09-16T00:13:04.611832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649917+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:13:04.649917+0000","last_peered":"2024-09-16T00:13:04.649917+0000","last_clean":"2024-09-16T00:13:04.649917+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:13:04.649917+0000","last_undegraded":"2024-09-16T00:13:04.649917+0000","last_fullsized":"2024-09-16T00:13:04.649917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650098+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:13:04.650098+0000","last_peered":"2024-09-16T00:13:04.650098+0000","last_clean":"2024-09-16T00:13:04.650098+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:13:04.650098+0000","last_undegraded":"2024-09-16T00:13:04.650098+0000","last_fullsized":"2024-09-16T00:13:04.650098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649329+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:13:04.649329+0000","last_peered":"2024-09-16T00:13:04.649329+0000","last_clean":"2024-09-16T00:13:04.649329+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:13:04.649329+0000","last_undegraded":"2024-09-16T00:13:04.649329+0000","last_fullsized":"2024-09-16T00:13:04.649329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:23.694358+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:13:23.694358+0000","last_peered":"2024-09-16T00:13:23.694358+0000","last_clean":"2024-09-16T00:13:23.694358+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:13:23.694358+0000","last_undegraded":"2024-09-16T00:13:23.694358+0000","last_fullsized":"2024-09-16T00:13:23.694358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605291+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:13:04.605291+0000","last_peered":"2024-09-16T00:13:04.605291+0000","last_clean":"2024-09-16T00:13:04.605291+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:13:04.605291+0000","last_undegraded":"2024-09-16T00:13:04.605291+0000","last_fullsized":"2024-09-16T00:13:04.605291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649345+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:13:04.649345+0000","last_peered":"2024-09-16T00:13:04.649345+0000","last_clean":"2024-09-16T00:13:04.649345+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:13:04.649345+0000","last_undegraded":"2024-09-16T00:13:04.649345+0000","last_fullsized":"2024-09-16T00:13:04.649345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605828+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:13:04.605828+0000","last_peered":"2024-09-16T00:13:04.605828+0000","last_clean":"2024-09-16T00:13:04.605828+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:13:04.605828+0000","last_undegraded":"2024-09-16T00:13:04.605828+0000","last_fullsized":"2024-09-16T00:13:04.605828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663922+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:13:04.663922+0000","last_peered":"2024-09-16T00:13:04.663922+0000","last_clean":"2024-09-16T00:13:04.663922+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:13:04.663922+0000","last_undegraded":"2024-09-16T00:13:04.663922+0000","last_fullsized":"2024-09-16T00:13:04.663922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650076+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:13:04.650076+0000","last_peered":"2024-09-16T00:13:04.650076+0000","last_clean":"2024-09-16T00:13:04.650076+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:13:04.650076+0000","last_undegraded":"2024-09-16T00:13:04.650076+0000","last_fullsized":"2024-09-16T00:13:04.650076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.607017+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:13:04.607017+0000","last_peered":"2024-09-16T00:13:04.607017+0000","last_clean":"2024-09-16T00:13:04.607017+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:13:04.607017+0000","last_undegraded":"2024-09-16T00:13:04.607017+0000","last_fullsized":"2024-09-16T00:13:04.607017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605023+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:13:04.605023+0000","last_peered":"2024-09-16T00:13:04.605023+0000","last_clean":"2024-09-16T00:13:04.605023+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:13:04.605023+0000","last_undegraded":"2024-09-16T00:13:04.605023+0000","last_fullsized":"2024-09-16T00:13:04.605023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663799+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:13:04.663799+0000","last_peered":"2024-09-16T00:13:04.663799+0000","last_clean":"2024-09-16T00:13:04.663799+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:13:04.663799+0000","last_undegraded":"2024-09-16T00:13:04.663799+0000","last_fullsized":"2024-09-16T00:13:04.663799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664979+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:13:04.664979+0000","last_peered":"2024-09-16T00:13:04.664979+0000","last_clean":"2024-09-16T00:13:04.664979+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:13:04.664979+0000","last_undegraded":"2024-09-16T00:13:04.664979+0000","last_fullsized":"2024-09-16T00:13:04.664979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605103+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:13:04.605103+0000","last_peered":"2024-09-16T00:13:04.605103+0000","last_clean":"2024-09-16T00:13:04.605103+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:13:04.605103+0000","last_undegraded":"2024-09-16T00:13:04.605103+0000","last_fullsized":"2024-09-16T00:13:04.605103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605218+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:13:04.605218+0000","last_peered":"2024-09-16T00:13:04.605218+0000","last_clean":"2024-09-16T00:13:04.605218+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:13:04.605218+0000","last_undegraded":"2024-09-16T00:13:04.605218+0000","last_fullsized":"2024-09-16T00:13:04.605218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664161+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:13:04.664161+0000","last_peered":"2024-09-16T00:13:04.664161+0000","last_clean":"2024-09-16T00:13:04.664161+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:13:04.664161+0000","last_undegraded":"2024-09-16T00:13:04.664161+0000","last_fullsized":"2024-09-16T00:13:04.664161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606964+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:13:04.606964+0000","last_peered":"2024-09-16T00:13:04.606964+0000","last_clean":"2024-09-16T00:13:04.606964+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:13:04.606964+0000","last_undegraded":"2024-09-16T00:13:04.606964+0000","last_fullsized":"2024-09-16T00:13:04.606964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649207+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:13:04.649207+0000","last_peered":"2024-09-16T00:13:04.649207+0000","last_clean":"2024-09-16T00:13:04.649207+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:13:04.649207+0000","last_undegraded":"2024-09-16T00:13:04.649207+0000","last_fullsized":"2024-09-16T00:13:04.649207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:26.731685+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:13:26.731685+0000","last_peered":"2024-09-16T00:13:26.731685+0000","last_clean":"2024-09-16T00:13:26.731685+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:13:26.731685+0000","last_undegraded":"2024-09-16T00:13:26.731685+0000","last_fullsized":"2024-09-16T00:13:26.731685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609415+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:13:04.609415+0000","last_peered":"2024-09-16T00:13:04.609415+0000","last_clean":"2024-09-16T00:13:04.609415+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:13:04.609415+0000","last_undegraded":"2024-09-16T00:13:04.609415+0000","last_fullsized":"2024-09-16T00:13:04.609415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649611+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:13:04.649611+0000","last_peered":"2024-09-16T00:13:04.649611+0000","last_clean":"2024-09-16T00:13:04.649611+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:13:04.649611+0000","last_undegraded":"2024-09-16T00:13:04.649611+0000","last_fullsized":"2024-09-16T00:13:04.649611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605440+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:13:04.605440+0000","last_peered":"2024-09-16T00:13:04.605440+0000","last_clean":"2024-09-16T00:13:04.605440+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:13:04.605440+0000","last_undegraded":"2024-09-16T00:13:04.605440+0000","last_fullsized":"2024-09-16T00:13:04.605440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664557+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:13:04.664557+0000","last_peered":"2024-09-16T00:13:04.664557+0000","last_clean":"2024-09-16T00:13:04.664557+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:13:04.664557+0000","last_undegraded":"2024-09-16T00:13:04.664557+0000","last_fullsized":"2024-09-16T00:13:04.664557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606631+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:13:04.606631+0000","last_peered":"2024-09-16T00:13:04.606631+0000","last_clean":"2024-09-16T00:13:04.606631+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:13:04.606631+0000","last_undegraded":"2024-09-16T00:13:04.606631+0000","last_fullsized":"2024-09-16T00:13:04.606631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611195+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:13:04.611195+0000","last_peered":"2024-09-16T00:13:04.611195+0000","last_clean":"2024-09-16T00:13:04.611195+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:13:04.611195+0000","last_undegraded":"2024-09-16T00:13:04.611195+0000","last_fullsized":"2024-09-16T00:13:04.611195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605540+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:13:04.605540+0000","last_peered":"2024-09-16T00:13:04.605540+0000","last_clean":"2024-09-16T00:13:04.605540+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:13:04.605540+0000","last_undegraded":"2024-09-16T00:13:04.605540+0000","last_fullsized":"2024-09-16T00:13:04.605540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664372+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:13:04.664372+0000","last_peered":"2024-09-16T00:13:04.664372+0000","last_clean":"2024-09-16T00:13:04.664372+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:13:04.664372+0000","last_undegraded":"2024-09-16T00:13:04.664372+0000","last_fullsized":"2024-09-16T00:13:04.664372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":293,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.017020+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:13:25.017020+0000","last_peered":"2024-09-16T00:13:25.017020+0000","last_clean":"2024-09-16T00:13:25.017020+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:13:25.017020+0000","last_undegraded":"2024-09-16T00:13:25.017020+0000","last_fullsized":"2024-09-16T00:13:25.017020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63488,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":168,"num_write":42,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609072+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:13:04.609072+0000","last_peered":"2024-09-16T00:13:04.609072+0000","last_clean":"2024-09-16T00:13:04.609072+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:13:04.609072+0000","last_undegraded":"2024-09-16T00:13:04.609072+0000","last_fullsized":"2024-09-16T00:13:04.609072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605601+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:13:04.605601+0000","last_peered":"2024-09-16T00:13:04.605601+0000","last_clean":"2024-09-16T00:13:04.605601+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:13:04.605601+0000","last_undegraded":"2024-09-16T00:13:04.605601+0000","last_fullsized":"2024-09-16T00:13:04.605601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605741+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:13:04.605741+0000","last_peered":"2024-09-16T00:13:04.605741+0000","last_clean":"2024-09-16T00:13:04.605741+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:13:04.605741+0000","last_undegraded":"2024-09-16T00:13:04.605741+0000","last_fullsized":"2024-09-16T00:13:04.605741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611591+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:13:04.611591+0000","last_peered":"2024-09-16T00:13:04.611591+0000","last_clean":"2024-09-16T00:13:04.611591+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:13:04.611591+0000","last_undegraded":"2024-09-16T00:13:04.611591+0000","last_fullsized":"2024-09-16T00:13:04.611591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663373+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:13:04.663373+0000","last_peered":"2024-09-16T00:13:04.663373+0000","last_clean":"2024-09-16T00:13:04.663373+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:13:04.663373+0000","last_undegraded":"2024-09-16T00:13:04.663373+0000","last_fullsized":"2024-09-16T00:13:04.663373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649999+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:13:04.649999+0000","last_peered":"2024-09-16T00:13:04.649999+0000","last_clean":"2024-09-16T00:13:04.649999+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:13:04.649999+0000","last_undegraded":"2024-09-16T00:13:04.649999+0000","last_fullsized":"2024-09-16T00:13:04.649999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.608894+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:13:04.608894+0000","last_peered":"2024-09-16T00:13:04.608894+0000","last_clean":"2024-09-16T00:13:04.608894+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:13:04.608894+0000","last_undegraded":"2024-09-16T00:13:04.608894+0000","last_fullsized":"2024-09-16T00:13:04.608894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605695+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:13:04.605695+0000","last_peered":"2024-09-16T00:13:04.605695+0000","last_clean":"2024-09-16T00:13:04.605695+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:13:04.605695+0000","last_undegraded":"2024-09-16T00:13:04.605695+0000","last_fullsized":"2024-09-16T00:13:04.605695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":248,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:24.695271+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:13:24.695271+0000","last_peered":"2024-09-16T00:13:24.695271+0000","last_clean":"2024-09-16T00:13:24.695271+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:13:24.695271+0000","last_undegraded":"2024-09-16T00:13:24.695271+0000","last_fullsized":"2024-09-16T00:13:24.695271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664350+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:13:04.664350+0000","last_peered":"2024-09-16T00:13:04.664350+0000","last_clean":"2024-09-16T00:13:04.664350+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:13:04.664350+0000","last_undegraded":"2024-09-16T00:13:04.664350+0000","last_fullsized":"2024-09-16T00:13:04.664350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663978+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:13:04.663978+0000","last_peered":"2024-09-16T00:13:04.663978+0000","last_clean":"2024-09-16T00:13:04.663978+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:13:04.663978+0000","last_undegraded":"2024-09-16T00:13:04.663978+0000","last_fullsized":"2024-09-16T00:13:04.663978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609007+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:13:04.609007+0000","last_peered":"2024-09-16T00:13:04.609007+0000","last_clean":"2024-09-16T00:13:04.609007+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:13:04.609007+0000","last_undegraded":"2024-09-16T00:13:04.609007+0000","last_fullsized":"2024-09-16T00:13:04.609007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605461+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:13:04.605461+0000","last_peered":"2024-09-16T00:13:04.605461+0000","last_clean":"2024-09-16T00:13:04.605461+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:13:04.605461+0000","last_undegraded":"2024-09-16T00:13:04.605461+0000","last_fullsized":"2024-09-16T00:13:04.605461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664904+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:13:04.664904+0000","last_peered":"2024-09-16T00:13:04.664904+0000","last_clean":"2024-09-16T00:13:04.664904+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:13:04.664904+0000","last_undegraded":"2024-09-16T00:13:04.664904+0000","last_fullsized":"2024-09-16T00:13:04.664904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663317+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:13:04.663317+0000","last_peered":"2024-09-16T00:13:04.663317+0000","last_clean":"2024-09-16T00:13:04.663317+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:13:04.663317+0000","last_undegraded":"2024-09-16T00:13:04.663317+0000","last_fullsized":"2024-09-16T00:13:04.663317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606517+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:13:04.606517+0000","last_peered":"2024-09-16T00:13:04.606517+0000","last_clean":"2024-09-16T00:13:04.606517+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:13:04.606517+0000","last_undegraded":"2024-09-16T00:13:04.606517+0000","last_fullsized":"2024-09-16T00:13:04.606517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664780+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:13:04.664780+0000","last_peered":"2024-09-16T00:13:04.664780+0000","last_clean":"2024-09-16T00:13:04.664780+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:13:04.664780+0000","last_undegraded":"2024-09-16T00:13:04.664780+0000","last_fullsized":"2024-09-16T00:13:04.664780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605384+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:13:04.605384+0000","last_peered":"2024-09-16T00:13:04.605384+0000","last_clean":"2024-09-16T00:13:04.605384+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:13:04.605384+0000","last_undegraded":"2024-09-16T00:13:04.605384+0000","last_fullsized":"2024-09-16T00:13:04.605384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610913+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:13:04.610913+0000","last_peered":"2024-09-16T00:13:04.610913+0000","last_clean":"2024-09-16T00:13:04.610913+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:13:04.610913+0000","last_undegraded":"2024-09-16T00:13:04.610913+0000","last_fullsized":"2024-09-16T00:13:04.610913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.697247+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:13:25.697247+0000","last_peered":"2024-09-16T00:13:25.697247+0000","last_clean":"2024-09-16T00:13:25.697247+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:13:25.697247+0000","last_undegraded":"2024-09-16T00:13:25.697247+0000","last_fullsized":"2024-09-16T00:13:25.697247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663709+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:13:04.663709+0000","last_peered":"2024-09-16T00:13:04.663709+0000","last_clean":"2024-09-16T00:13:04.663709+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:13:04.663709+0000","last_undegraded":"2024-09-16T00:13:04.663709+0000","last_fullsized":"2024-09-16T00:13:04.663709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611136+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:13:04.611136+0000","last_peered":"2024-09-16T00:13:04.611136+0000","last_clean":"2024-09-16T00:13:04.611136+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:13:04.611136+0000","last_undegraded":"2024-09-16T00:13:04.611136+0000","last_fullsized":"2024-09-16T00:13:04.611136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663783+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:13:04.663783+0000","last_peered":"2024-09-16T00:13:04.663783+0000","last_clean":"2024-09-16T00:13:04.663783+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:13:04.663783+0000","last_undegraded":"2024-09-16T00:13:04.663783+0000","last_fullsized":"2024-09-16T00:13:04.663783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611426+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:13:04.611426+0000","last_peered":"2024-09-16T00:13:04.611426+0000","last_clean":"2024-09-16T00:13:04.611426+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:13:04.611426+0000","last_undegraded":"2024-09-16T00:13:04.611426+0000","last_fullsized":"2024-09-16T00:13:04.611426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610934+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:13:04.610934+0000","last_peered":"2024-09-16T00:13:04.610934+0000","last_clean":"2024-09-16T00:13:04.610934+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:13:04.610934+0000","last_undegraded":"2024-09-16T00:13:04.610934+0000","last_fullsized":"2024-09-16T00:13:04.610934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663979+0000","last_change":"2024-09-16T00:12:04.029280+0000","last_active":"2024-09-16T00:13:04.663979+0000","last_peered":"2024-09-16T00:13:04.663979+0000","last_clean":"2024-09-16T00:13:04.663979+0000","last_became_active":"2024-09-16T00:12:04.028973+0000","last_became_peered":"2024-09-16T00:12:04.028973+0000","last_unstale":"2024-09-16T00:13:04.663979+0000","last_undegraded":"2024-09-16T00:13:04.663979+0000","last_fullsized":"2024-09-16T00:13:04.663979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606846+0000","last_change":"2024-09-16T00:12:04.028319+0000","last_active":"2024-09-16T00:13:04.606846+0000","last_peered":"2024-09-16T00:13:04.606846+0000","last_clean":"2024-09-16T00:13:04.606846+0000","last_became_active":"2024-09-16T00:12:04.027985+0000","last_became_peered":"2024-09-16T00:12:04.027985+0000","last_unstale":"2024-09-16T00:13:04.606846+0000","last_undegraded":"2024-09-16T00:13:04.606846+0000","last_fullsized":"2024-09-16T00:13:04.606846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.119428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663790+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:13:04.663790+0000","last_peered":"2024-09-16T00:13:04.663790+0000","last_clean":"2024-09-16T00:13:04.663790+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:13:04.663790+0000","last_undegraded":"2024-09-16T00:13:04.663790+0000","last_fullsized":"2024-09-16T00:13:04.663790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611369+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:13:04.611369+0000","last_peered":"2024-09-16T00:13:04.611369+0000","last_clean":"2024-09-16T00:13:04.611369+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:13:04.611369+0000","last_undegraded":"2024-09-16T00:13:04.611369+0000","last_fullsized":"2024-09-16T00:13:04.611369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.616864+0000","last_change":"2024-09-16T00:13:04.612598+0000","last_active":"2024-09-16T00:13:04.616864+0000","last_peered":"2024-09-16T00:13:04.616864+0000","last_clean":"2024-09-16T00:13:04.616864+0000","last_became_active":"2024-09-16T00:13:04.612310+0000","last_became_peered":"2024-09-16T00:13:04.612310+0000","last_unstale":"2024-09-16T00:13:04.616864+0000","last_undegraded":"2024-09-16T00:13:04.616864+0000","last_fullsized":"2024-09-16T00:13:04.616864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:59.381944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605240+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:13:04.605240+0000","last_peered":"2024-09-16T00:13:04.605240+0000","last_clean":"2024-09-16T00:13:04.605240+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:13:04.605240+0000","last_undegraded":"2024-09-16T00:13:04.605240+0000","last_fullsized":"2024-09-16T00:13:04.605240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663870+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:13:04.663870+0000","last_peered":"2024-09-16T00:13:04.663870+0000","last_clean":"2024-09-16T00:13:04.663870+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:13:04.663870+0000","last_undegraded":"2024-09-16T00:13:04.663870+0000","last_fullsized":"2024-09-16T00:13:04.663870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605535+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:13:04.605535+0000","last_peered":"2024-09-16T00:13:04.605535+0000","last_clean":"2024-09-16T00:13:04.605535+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:13:04.605535+0000","last_undegraded":"2024-09-16T00:13:04.605535+0000","last_fullsized":"2024-09-16T00:13:04.605535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606766+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:13:04.606766+0000","last_peered":"2024-09-16T00:13:04.606766+0000","last_clean":"2024-09-16T00:13:04.606766+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:13:04.606766+0000","last_undegraded":"2024-09-16T00:13:04.606766+0000","last_fullsized":"2024-09-16T00:13:04.606766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663880+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:13:04.663880+0000","last_peered":"2024-09-16T00:13:04.663880+0000","last_clean":"2024-09-16T00:13:04.663880+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:13:04.663880+0000","last_undegraded":"2024-09-16T00:13:04.663880+0000","last_fullsized":"2024-09-16T00:13:04.663880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611528+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:13:04.611528+0000","last_peered":"2024-09-16T00:13:04.611528+0000","last_clean":"2024-09-16T00:13:04.611528+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:13:04.611528+0000","last_undegraded":"2024-09-16T00:13:04.611528+0000","last_fullsized":"2024-09-16T00:13:04.611528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609303+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:13:04.609303+0000","last_peered":"2024-09-16T00:13:04.609303+0000","last_clean":"2024-09-16T00:13:04.609303+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:13:04.609303+0000","last_undegraded":"2024-09-16T00:13:04.609303+0000","last_fullsized":"2024-09-16T00:13:04.609303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663938+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:13:04.663938+0000","last_peered":"2024-09-16T00:13:04.663938+0000","last_clean":"2024-09-16T00:13:04.663938+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:13:04.663938+0000","last_undegraded":"2024-09-16T00:13:04.663938+0000","last_fullsized":"2024-09-16T00:13:04.663938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605284+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:13:04.605284+0000","last_peered":"2024-09-16T00:13:04.605284+0000","last_clean":"2024-09-16T00:13:04.605284+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:13:04.605284+0000","last_undegraded":"2024-09-16T00:13:04.605284+0000","last_fullsized":"2024-09-16T00:13:04.605284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663375+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:13:04.663375+0000","last_peered":"2024-09-16T00:13:04.663375+0000","last_clean":"2024-09-16T00:13:04.663375+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:13:04.663375+0000","last_undegraded":"2024-09-16T00:13:04.663375+0000","last_fullsized":"2024-09-16T00:13:04.663375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663627+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:13:04.663627+0000","last_peered":"2024-09-16T00:13:04.663627+0000","last_clean":"2024-09-16T00:13:04.663627+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:13:04.663627+0000","last_undegraded":"2024-09-16T00:13:04.663627+0000","last_fullsized":"2024-09-16T00:13:04.663627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606689+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:13:04.606689+0000","last_peered":"2024-09-16T00:13:04.606689+0000","last_clean":"2024-09-16T00:13:04.606689+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:13:04.606689+0000","last_undegraded":"2024-09-16T00:13:04.606689+0000","last_fullsized":"2024-09-16T00:13:04.606689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611636+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:13:04.611636+0000","last_peered":"2024-09-16T00:13:04.611636+0000","last_clean":"2024-09-16T00:13:04.611636+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:13:04.611636+0000","last_undegraded":"2024-09-16T00:13:04.611636+0000","last_fullsized":"2024-09-16T00:13:04.611636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649291+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:13:04.649291+0000","last_peered":"2024-09-16T00:13:04.649291+0000","last_clean":"2024-09-16T00:13:04.649291+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:13:04.649291+0000","last_undegraded":"2024-09-16T00:13:04.649291+0000","last_fullsized":"2024-09-16T00:13:04.649291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664002+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:13:04.664002+0000","last_peered":"2024-09-16T00:13:04.664002+0000","last_clean":"2024-09-16T00:13:04.664002+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:13:04.664002+0000","last_undegraded":"2024-09-16T00:13:04.664002+0000","last_fullsized":"2024-09-16T00:13:04.664002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605780+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:13:04.605780+0000","last_peered":"2024-09-16T00:13:04.605780+0000","last_clean":"2024-09-16T00:13:04.605780+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:13:04.605780+0000","last_undegraded":"2024-09-16T00:13:04.605780+0000","last_fullsized":"2024-09-16T00:13:04.605780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663843+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:13:04.663843+0000","last_peered":"2024-09-16T00:13:04.663843+0000","last_clean":"2024-09-16T00:13:04.663843+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:13:04.663843+0000","last_undegraded":"2024-09-16T00:13:04.663843+0000","last_fullsized":"2024-09-16T00:13:04.663843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649554+0000","last_change":"2024-09-16T00:12:04.028342+0000","last_active":"2024-09-16T00:13:04.649554+0000","last_peered":"2024-09-16T00:13:04.649554+0000","last_clean":"2024-09-16T00:13:04.649554+0000","last_became_active":"2024-09-16T00:12:04.027889+0000","last_became_peered":"2024-09-16T00:12:04.027889+0000","last_unstale":"2024-09-16T00:13:04.649554+0000","last_undegraded":"2024-09-16T00:13:04.649554+0000","last_fullsized":"2024-09-16T00:13:04.649554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:47.840428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663964+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:13:04.663964+0000","last_peered":"2024-09-16T00:13:04.663964+0000","last_clean":"2024-09-16T00:13:04.663964+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:13:04.663964+0000","last_undegraded":"2024-09-16T00:13:04.663964+0000","last_fullsized":"2024-09-16T00:13:04.663964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663878+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:13:04.663878+0000","last_peered":"2024-09-16T00:13:04.663878+0000","last_clean":"2024-09-16T00:13:04.663878+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:13:04.663878+0000","last_undegraded":"2024-09-16T00:13:04.663878+0000","last_fullsized":"2024-09-16T00:13:04.663878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611689+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:13:04.611689+0000","last_peered":"2024-09-16T00:13:04.611689+0000","last_clean":"2024-09-16T00:13:04.611689+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:13:04.611689+0000","last_undegraded":"2024-09-16T00:13:04.611689+0000","last_fullsized":"2024-09-16T00:13:04.611689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664072+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:13:04.664072+0000","last_peered":"2024-09-16T00:13:04.664072+0000","last_clean":"2024-09-16T00:13:04.664072+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:13:04.664072+0000","last_undegraded":"2024-09-16T00:13:04.664072+0000","last_fullsized":"2024-09-16T00:13:04.664072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664019+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:13:04.664019+0000","last_peered":"2024-09-16T00:13:04.664019+0000","last_clean":"2024-09-16T00:13:04.664019+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:13:04.664019+0000","last_undegraded":"2024-09-16T00:13:04.664019+0000","last_fullsized":"2024-09-16T00:13:04.664019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71679,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1069,"num_read_kb":613,"num_write":269,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":515263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1736704,"data_stored":1165567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972405,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1069056,"data_stored":518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201338,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":515037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1658880,"data_stored":1111677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593796,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36972,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708180,"statfs":{"total":95995035648,"available":95957176320,"internally_reserved":0,"allocated":1667072,"data_stored":1107033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":515164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":509871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1130496,"data_stored":575435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1110016,"data_stored":574373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":512153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:13:29.886 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-16T00:13:29.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:13:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.018+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1985205649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c100a10 msgr2=0x7f3d5c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.018+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1985205649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c100a10 0x7f3d5c100e10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009900 tx=0x7f3d5402a410 comp rx=0 tx=0).stop 2024-09-16T00:13:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1985205649 shutdown_connections 2024-09-16T00:13:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1985205649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c1025c0 0x7f3d5c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1985205649 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c101c00 0x7f3d5c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1985205649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c100a10 0x7f3d5c100e10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1985205649 >> 172.21.15.78:0/1985205649 conn(0x7f3d5c0fc170 msgr2=0x7f3d5c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:30.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1985205649 shutdown_connections 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1985205649 wait complete. 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 Processor -- start 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 -- start start 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c100a10 0x7f3d5c196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c101c00 0x7f3d5c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c1025c0 0x7f3d5c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c10c690 con 0x7f3d5c101c00 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c10c510 con 0x7f3d5c100a10 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d62ad2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c10c810 con 0x7f3d5c1025c0 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c101c00 0x7f3d5c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c101c00 0x7f3d5c1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58780/0 (socket says 172.21.15.78:58780) 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.019+0000 7f3d5bfff640 1 -- 172.21.15.78:0/1651906867 learned_addr learned my addr 172.21.15.78:0/1651906867 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:30.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d6106f640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c1025c0 0x7f3d5c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d6086e640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c100a10 0x7f3d5c196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d5bfff640 1 -- 172.21.15.78:0/1651906867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c1025c0 msgr2=0x7f3d5c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d5bfff640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c1025c0 0x7f3d5c19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d5bfff640 1 -- 172.21.15.78:0/1651906867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c100a10 msgr2=0x7f3d5c196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d5bfff640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c100a10 0x7f3d5c196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d5bfff640 1 -- 172.21.15.78:0/1651906867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c19e990 con 0x7f3d5c101c00 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d6106f640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c1025c0 0x7f3d5c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d6086e640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c100a10 0x7f3d5c196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d5bfff640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c101c00 0x7f3d5c1972b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00db80 tx=0x7f3d4c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d59ffb640 1 -- 172.21.15.78:0/1651906867 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c017070 con 0x7f3d5c101c00 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d59ffb640 1 -- 172.21.15.78:0/1651906867 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d4c003b70 con 0x7f3d5c101c00 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5c19ec80 con 0x7f3d5c101c00 2024-09-16T00:13:30.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d59ffb640 1 -- 172.21.15.78:0/1651906867 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0106c0 con 0x7f3d5c101c00 2024-09-16T00:13:30.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.020+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5c19f1d0 con 0x7f3d5c101c00 2024-09-16T00:13:30.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.021+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d5c107e50 con 0x7f3d5c101c00 2024-09-16T00:13:30.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.022+0000 7f3d59ffb640 1 -- 172.21.15.78:0/1651906867 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f3d4c004020 con 0x7f3d5c101c00 2024-09-16T00:13:30.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.023+0000 7f3d59ffb640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3d480a32e0 0x7f3d480a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:30.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.023+0000 7f3d6086e640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3d480a32e0 0x7f3d480a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:30.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.023+0000 7f3d59ffb640 1 -- 172.21.15.78:0/1651906867 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3d4c0cfd40 con 0x7f3d5c101c00 2024-09-16T00:13:30.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.023+0000 7f3d6086e640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3d480a32e0 0x7f3d480a57a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3d54000c00 tx=0x7f3d54031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:30.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.027+0000 7f3d59ffb640 1 -- 172.21.15.78:0/1651906867 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d4c020070 con 0x7f3d5c101c00 2024-09-16T00:13:30.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:29 smithi193 ceph-mon[35484]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:30.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.193+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d5c062630 con 0x7f3d480a32e0 2024-09-16T00:13:30.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.200+0000 7f3d59ffb640 1 -- 172.21.15.78:0/1651906867 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340526 (secure 0 0 0) 0x7f3d5c062630 con 0x7f3d480a32e0 2024-09-16T00:13:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:30.207 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:13:30.209 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:29 smithi148 ceph-mon[35599]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3d480a32e0 msgr2=0x7f3d480a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3d480a32e0 0x7f3d480a57a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3d54000c00 tx=0x7f3d54031400 comp rx=0 tx=0).stop 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c101c00 msgr2=0x7f3d5c1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c101c00 0x7f3d5c1972b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00db80 tx=0x7f3d4c00b630 comp rx=0 tx=0).stop 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 shutdown_connections 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3d480a32e0 0x7f3d480a57a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d5c1025c0 0x7f3d5c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d5c101c00 0x7f3d5c1972b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 --2- 172.21.15.78:0/1651906867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c100a10 0x7f3d5c196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 >> 172.21.15.78:0/1651906867 conn(0x7f3d5c0fc170 msgr2=0x7f3d5c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 shutdown_connections 2024-09-16T00:13:30.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:30.204+0000 7f3d62ad2640 1 -- 172.21.15.78:0/1651906867 wait complete. 2024-09-16T00:13:30.215 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:29 smithi078 ceph-mon[31157]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:30.217 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-16T00:13:28.430524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662281,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1195,"num_read_kb":722,"num_write":452,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359472,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582352,"statfs":{"total":1151940427776,"available":1151572328448,"internally_reserved":0,"allocated":14680064,"data_stored":8130884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1009,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003061"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:23.683882+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:13:23.683882+0000","last_peered":"2024-09-16T00:13:23.683882+0000","last_clean":"2024-09-16T00:13:23.683882+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:13:23.683882+0000","last_undegraded":"2024-09-16T00:13:23.683882+0000","last_fullsized":"2024-09-16T00:13:23.683882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610803+0000","last_change":"2024-09-16T00:12:04.132674+0000","last_active":"2024-09-16T00:13:04.610803+0000","last_peered":"2024-09-16T00:13:04.610803+0000","last_clean":"2024-09-16T00:13:04.610803+0000","last_became_active":"2024-09-16T00:12:04.132108+0000","last_became_peered":"2024-09-16T00:12:04.132108+0000","last_unstale":"2024-09-16T00:13:04.610803+0000","last_undegraded":"2024-09-16T00:13:04.610803+0000","last_fullsized":"2024-09-16T00:13:04.610803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:10.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664962+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:13:04.664962+0000","last_peered":"2024-09-16T00:13:04.664962+0000","last_clean":"2024-09-16T00:13:04.664962+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:13:04.664962+0000","last_undegraded":"2024-09-16T00:13:04.664962+0000","last_fullsized":"2024-09-16T00:13:04.664962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663575+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:13:04.663575+0000","last_peered":"2024-09-16T00:13:04.663575+0000","last_clean":"2024-09-16T00:13:04.663575+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:13:04.663575+0000","last_undegraded":"2024-09-16T00:13:04.663575+0000","last_fullsized":"2024-09-16T00:13:04.663575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611038+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:13:04.611038+0000","last_peered":"2024-09-16T00:13:04.611038+0000","last_clean":"2024-09-16T00:13:04.611038+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:13:04.611038+0000","last_undegraded":"2024-09-16T00:13:04.611038+0000","last_fullsized":"2024-09-16T00:13:04.611038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664541+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:13:04.664541+0000","last_peered":"2024-09-16T00:13:04.664541+0000","last_clean":"2024-09-16T00:13:04.664541+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:13:04.664541+0000","last_undegraded":"2024-09-16T00:13:04.664541+0000","last_fullsized":"2024-09-16T00:13:04.664541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605363+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:13:04.605363+0000","last_peered":"2024-09-16T00:13:04.605363+0000","last_clean":"2024-09-16T00:13:04.605363+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:13:04.605363+0000","last_undegraded":"2024-09-16T00:13:04.605363+0000","last_fullsized":"2024-09-16T00:13:04.605363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611375+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:13:04.611375+0000","last_peered":"2024-09-16T00:13:04.611375+0000","last_clean":"2024-09-16T00:13:04.611375+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:13:04.611375+0000","last_undegraded":"2024-09-16T00:13:04.611375+0000","last_fullsized":"2024-09-16T00:13:04.611375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664351+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:13:04.664351+0000","last_peered":"2024-09-16T00:13:04.664351+0000","last_clean":"2024-09-16T00:13:04.664351+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:13:04.664351+0000","last_undegraded":"2024-09-16T00:13:04.664351+0000","last_fullsized":"2024-09-16T00:13:04.664351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605600+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:13:04.605600+0000","last_peered":"2024-09-16T00:13:04.605600+0000","last_clean":"2024-09-16T00:13:04.605600+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:13:04.605600+0000","last_undegraded":"2024-09-16T00:13:04.605600+0000","last_fullsized":"2024-09-16T00:13:04.605600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611103+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:13:04.611103+0000","last_peered":"2024-09-16T00:13:04.611103+0000","last_clean":"2024-09-16T00:13:04.611103+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:13:04.611103+0000","last_undegraded":"2024-09-16T00:13:04.611103+0000","last_fullsized":"2024-09-16T00:13:04.611103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649281+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:13:04.649281+0000","last_peered":"2024-09-16T00:13:04.649281+0000","last_clean":"2024-09-16T00:13:04.649281+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:13:04.649281+0000","last_undegraded":"2024-09-16T00:13:04.649281+0000","last_fullsized":"2024-09-16T00:13:04.649281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664252+0000","last_change":"2024-09-16T00:12:04.033464+0000","last_active":"2024-09-16T00:13:04.664252+0000","last_peered":"2024-09-16T00:13:04.664252+0000","last_clean":"2024-09-16T00:13:04.664252+0000","last_became_active":"2024-09-16T00:12:04.028916+0000","last_became_peered":"2024-09-16T00:12:04.028916+0000","last_unstale":"2024-09-16T00:13:04.664252+0000","last_undegraded":"2024-09-16T00:13:04.664252+0000","last_fullsized":"2024-09-16T00:13:04.664252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:47.359979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611584+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:13:04.611584+0000","last_peered":"2024-09-16T00:13:04.611584+0000","last_clean":"2024-09-16T00:13:04.611584+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:13:04.611584+0000","last_undegraded":"2024-09-16T00:13:04.611584+0000","last_fullsized":"2024-09-16T00:13:04.611584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664652+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:13:04.664652+0000","last_peered":"2024-09-16T00:13:04.664652+0000","last_clean":"2024-09-16T00:13:04.664652+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:13:04.664652+0000","last_undegraded":"2024-09-16T00:13:04.664652+0000","last_fullsized":"2024-09-16T00:13:04.664652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610969+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:13:04.610969+0000","last_peered":"2024-09-16T00:13:04.610969+0000","last_clean":"2024-09-16T00:13:04.610969+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:13:04.610969+0000","last_undegraded":"2024-09-16T00:13:04.610969+0000","last_fullsized":"2024-09-16T00:13:04.610969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611391+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:13:04.611391+0000","last_peered":"2024-09-16T00:13:04.611391+0000","last_clean":"2024-09-16T00:13:04.611391+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:13:04.611391+0000","last_undegraded":"2024-09-16T00:13:04.611391+0000","last_fullsized":"2024-09-16T00:13:04.611391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663782+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:13:04.663782+0000","last_peered":"2024-09-16T00:13:04.663782+0000","last_clean":"2024-09-16T00:13:04.663782+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:13:04.663782+0000","last_undegraded":"2024-09-16T00:13:04.663782+0000","last_fullsized":"2024-09-16T00:13:04.663782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664498+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:13:04.664498+0000","last_peered":"2024-09-16T00:13:04.664498+0000","last_clean":"2024-09-16T00:13:04.664498+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:13:04.664498+0000","last_undegraded":"2024-09-16T00:13:04.664498+0000","last_fullsized":"2024-09-16T00:13:04.664498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609329+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:13:04.609329+0000","last_peered":"2024-09-16T00:13:04.609329+0000","last_clean":"2024-09-16T00:13:04.609329+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:13:04.609329+0000","last_undegraded":"2024-09-16T00:13:04.609329+0000","last_fullsized":"2024-09-16T00:13:04.609329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605426+0000","last_change":"2024-09-16T00:12:04.034002+0000","last_active":"2024-09-16T00:13:04.605426+0000","last_peered":"2024-09-16T00:13:04.605426+0000","last_clean":"2024-09-16T00:13:04.605426+0000","last_became_active":"2024-09-16T00:12:04.030959+0000","last_became_peered":"2024-09-16T00:12:04.030959+0000","last_unstale":"2024-09-16T00:13:04.605426+0000","last_undegraded":"2024-09-16T00:13:04.605426+0000","last_fullsized":"2024-09-16T00:13:04.605426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:47.730692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611324+0000","last_change":"2024-09-16T00:12:04.029906+0000","last_active":"2024-09-16T00:13:04.611324+0000","last_peered":"2024-09-16T00:13:04.611324+0000","last_clean":"2024-09-16T00:13:04.611324+0000","last_became_active":"2024-09-16T00:12:04.029259+0000","last_became_peered":"2024-09-16T00:12:04.029259+0000","last_unstale":"2024-09-16T00:13:04.611324+0000","last_undegraded":"2024-09-16T00:13:04.611324+0000","last_fullsized":"2024-09-16T00:13:04.611324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:23.399762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,1],"acting":[4,8,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663413+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:13:04.663413+0000","last_peered":"2024-09-16T00:13:04.663413+0000","last_clean":"2024-09-16T00:13:04.663413+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:13:04.663413+0000","last_undegraded":"2024-09-16T00:13:04.663413+0000","last_fullsized":"2024-09-16T00:13:04.663413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611856+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:13:04.611856+0000","last_peered":"2024-09-16T00:13:04.611856+0000","last_clean":"2024-09-16T00:13:04.611856+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:13:04.611856+0000","last_undegraded":"2024-09-16T00:13:04.611856+0000","last_fullsized":"2024-09-16T00:13:04.611856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610909+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:13:04.610909+0000","last_peered":"2024-09-16T00:13:04.610909+0000","last_clean":"2024-09-16T00:13:04.610909+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:13:04.610909+0000","last_undegraded":"2024-09-16T00:13:04.610909+0000","last_fullsized":"2024-09-16T00:13:04.610909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.665046+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:13:04.665046+0000","last_peered":"2024-09-16T00:13:04.665046+0000","last_clean":"2024-09-16T00:13:04.665046+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:13:04.665046+0000","last_undegraded":"2024-09-16T00:13:04.665046+0000","last_fullsized":"2024-09-16T00:13:04.665046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'19","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609370+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:13:04.609370+0000","last_peered":"2024-09-16T00:13:04.609370+0000","last_clean":"2024-09-16T00:13:04.609370+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:13:04.609370+0000","last_undegraded":"2024-09-16T00:13:04.609370+0000","last_fullsized":"2024-09-16T00:13:04.609370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664101+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:13:04.664101+0000","last_peered":"2024-09-16T00:13:04.664101+0000","last_clean":"2024-09-16T00:13:04.664101+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:13:04.664101+0000","last_undegraded":"2024-09-16T00:13:04.664101+0000","last_fullsized":"2024-09-16T00:13:04.664101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606866+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:13:04.606866+0000","last_peered":"2024-09-16T00:13:04.606866+0000","last_clean":"2024-09-16T00:13:04.606866+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:13:04.606866+0000","last_undegraded":"2024-09-16T00:13:04.606866+0000","last_fullsized":"2024-09-16T00:13:04.606866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664634+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:13:04.664634+0000","last_peered":"2024-09-16T00:13:04.664634+0000","last_clean":"2024-09-16T00:13:04.664634+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:13:04.664634+0000","last_undegraded":"2024-09-16T00:13:04.664634+0000","last_fullsized":"2024-09-16T00:13:04.664634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664409+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:13:04.664409+0000","last_peered":"2024-09-16T00:13:04.664409+0000","last_clean":"2024-09-16T00:13:04.664409+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:13:04.664409+0000","last_undegraded":"2024-09-16T00:13:04.664409+0000","last_fullsized":"2024-09-16T00:13:04.664409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664046+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:13:04.664046+0000","last_peered":"2024-09-16T00:13:04.664046+0000","last_clean":"2024-09-16T00:13:04.664046+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:13:04.664046+0000","last_undegraded":"2024-09-16T00:13:04.664046+0000","last_fullsized":"2024-09-16T00:13:04.664046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609377+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:13:04.609377+0000","last_peered":"2024-09-16T00:13:04.609377+0000","last_clean":"2024-09-16T00:13:04.609377+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:13:04.609377+0000","last_undegraded":"2024-09-16T00:13:04.609377+0000","last_fullsized":"2024-09-16T00:13:04.609377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664159+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:13:04.664159+0000","last_peered":"2024-09-16T00:13:04.664159+0000","last_clean":"2024-09-16T00:13:04.664159+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:13:04.664159+0000","last_undegraded":"2024-09-16T00:13:04.664159+0000","last_fullsized":"2024-09-16T00:13:04.664159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664057+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:13:04.664057+0000","last_peered":"2024-09-16T00:13:04.664057+0000","last_clean":"2024-09-16T00:13:04.664057+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:13:04.664057+0000","last_undegraded":"2024-09-16T00:13:04.664057+0000","last_fullsized":"2024-09-16T00:13:04.664057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:21.681566+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:13:21.681566+0000","last_peered":"2024-09-16T00:13:21.681566+0000","last_clean":"2024-09-16T00:13:21.681566+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:13:21.681566+0000","last_undegraded":"2024-09-16T00:13:21.681566+0000","last_fullsized":"2024-09-16T00:13:21.681566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606730+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:13:04.606730+0000","last_peered":"2024-09-16T00:13:04.606730+0000","last_clean":"2024-09-16T00:13:04.606730+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:13:04.606730+0000","last_undegraded":"2024-09-16T00:13:04.606730+0000","last_fullsized":"2024-09-16T00:13:04.606730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605309+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:13:04.605309+0000","last_peered":"2024-09-16T00:13:04.605309+0000","last_clean":"2024-09-16T00:13:04.605309+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:13:04.605309+0000","last_undegraded":"2024-09-16T00:13:04.605309+0000","last_fullsized":"2024-09-16T00:13:04.605309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:26.730658+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:13:26.730658+0000","last_peered":"2024-09-16T00:13:26.730658+0000","last_clean":"2024-09-16T00:13:26.730658+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:13:26.730658+0000","last_undegraded":"2024-09-16T00:13:26.730658+0000","last_fullsized":"2024-09-16T00:13:26.730658+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606979+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:13:04.606979+0000","last_peered":"2024-09-16T00:13:04.606979+0000","last_clean":"2024-09-16T00:13:04.606979+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:13:04.606979+0000","last_undegraded":"2024-09-16T00:13:04.606979+0000","last_fullsized":"2024-09-16T00:13:04.606979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606488+0000","last_change":"2024-09-16T00:12:04.027859+0000","last_active":"2024-09-16T00:13:04.606488+0000","last_peered":"2024-09-16T00:13:04.606488+0000","last_clean":"2024-09-16T00:13:04.606488+0000","last_became_active":"2024-09-16T00:12:04.027506+0000","last_became_peered":"2024-09-16T00:12:04.027506+0000","last_unstale":"2024-09-16T00:13:04.606488+0000","last_undegraded":"2024-09-16T00:13:04.606488+0000","last_fullsized":"2024-09-16T00:13:04.606488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:34.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649981+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:13:04.649981+0000","last_peered":"2024-09-16T00:13:04.649981+0000","last_clean":"2024-09-16T00:13:04.649981+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:13:04.649981+0000","last_undegraded":"2024-09-16T00:13:04.649981+0000","last_fullsized":"2024-09-16T00:13:04.649981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605678+0000","last_change":"2024-09-16T00:12:04.029471+0000","last_active":"2024-09-16T00:13:04.605678+0000","last_peered":"2024-09-16T00:13:04.605678+0000","last_clean":"2024-09-16T00:13:04.605678+0000","last_became_active":"2024-09-16T00:12:04.029054+0000","last_became_peered":"2024-09-16T00:12:04.029054+0000","last_unstale":"2024-09-16T00:13:04.605678+0000","last_undegraded":"2024-09-16T00:13:04.605678+0000","last_fullsized":"2024-09-16T00:13:04.605678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:52.054218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664553+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:13:04.664553+0000","last_peered":"2024-09-16T00:13:04.664553+0000","last_clean":"2024-09-16T00:13:04.664553+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:13:04.664553+0000","last_undegraded":"2024-09-16T00:13:04.664553+0000","last_fullsized":"2024-09-16T00:13:04.664553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611832+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:13:04.611832+0000","last_peered":"2024-09-16T00:13:04.611832+0000","last_clean":"2024-09-16T00:13:04.611832+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:13:04.611832+0000","last_undegraded":"2024-09-16T00:13:04.611832+0000","last_fullsized":"2024-09-16T00:13:04.611832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649917+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:13:04.649917+0000","last_peered":"2024-09-16T00:13:04.649917+0000","last_clean":"2024-09-16T00:13:04.649917+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:13:04.649917+0000","last_undegraded":"2024-09-16T00:13:04.649917+0000","last_fullsized":"2024-09-16T00:13:04.649917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650098+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:13:04.650098+0000","last_peered":"2024-09-16T00:13:04.650098+0000","last_clean":"2024-09-16T00:13:04.650098+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:13:04.650098+0000","last_undegraded":"2024-09-16T00:13:04.650098+0000","last_fullsized":"2024-09-16T00:13:04.650098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649329+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:13:04.649329+0000","last_peered":"2024-09-16T00:13:04.649329+0000","last_clean":"2024-09-16T00:13:04.649329+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:13:04.649329+0000","last_undegraded":"2024-09-16T00:13:04.649329+0000","last_fullsized":"2024-09-16T00:13:04.649329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:23.694358+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:13:23.694358+0000","last_peered":"2024-09-16T00:13:23.694358+0000","last_clean":"2024-09-16T00:13:23.694358+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:13:23.694358+0000","last_undegraded":"2024-09-16T00:13:23.694358+0000","last_fullsized":"2024-09-16T00:13:23.694358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605291+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:13:04.605291+0000","last_peered":"2024-09-16T00:13:04.605291+0000","last_clean":"2024-09-16T00:13:04.605291+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:13:04.605291+0000","last_undegraded":"2024-09-16T00:13:04.605291+0000","last_fullsized":"2024-09-16T00:13:04.605291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649345+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:13:04.649345+0000","last_peered":"2024-09-16T00:13:04.649345+0000","last_clean":"2024-09-16T00:13:04.649345+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:13:04.649345+0000","last_undegraded":"2024-09-16T00:13:04.649345+0000","last_fullsized":"2024-09-16T00:13:04.649345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605828+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:13:04.605828+0000","last_peered":"2024-09-16T00:13:04.605828+0000","last_clean":"2024-09-16T00:13:04.605828+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:13:04.605828+0000","last_undegraded":"2024-09-16T00:13:04.605828+0000","last_fullsized":"2024-09-16T00:13:04.605828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663922+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:13:04.663922+0000","last_peered":"2024-09-16T00:13:04.663922+0000","last_clean":"2024-09-16T00:13:04.663922+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:13:04.663922+0000","last_undegraded":"2024-09-16T00:13:04.663922+0000","last_fullsized":"2024-09-16T00:13:04.663922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650076+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:13:04.650076+0000","last_peered":"2024-09-16T00:13:04.650076+0000","last_clean":"2024-09-16T00:13:04.650076+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:13:04.650076+0000","last_undegraded":"2024-09-16T00:13:04.650076+0000","last_fullsized":"2024-09-16T00:13:04.650076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.607017+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:13:04.607017+0000","last_peered":"2024-09-16T00:13:04.607017+0000","last_clean":"2024-09-16T00:13:04.607017+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:13:04.607017+0000","last_undegraded":"2024-09-16T00:13:04.607017+0000","last_fullsized":"2024-09-16T00:13:04.607017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605023+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:13:04.605023+0000","last_peered":"2024-09-16T00:13:04.605023+0000","last_clean":"2024-09-16T00:13:04.605023+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:13:04.605023+0000","last_undegraded":"2024-09-16T00:13:04.605023+0000","last_fullsized":"2024-09-16T00:13:04.605023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663799+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:13:04.663799+0000","last_peered":"2024-09-16T00:13:04.663799+0000","last_clean":"2024-09-16T00:13:04.663799+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:13:04.663799+0000","last_undegraded":"2024-09-16T00:13:04.663799+0000","last_fullsized":"2024-09-16T00:13:04.663799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664979+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:13:04.664979+0000","last_peered":"2024-09-16T00:13:04.664979+0000","last_clean":"2024-09-16T00:13:04.664979+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:13:04.664979+0000","last_undegraded":"2024-09-16T00:13:04.664979+0000","last_fullsized":"2024-09-16T00:13:04.664979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605103+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:13:04.605103+0000","last_peered":"2024-09-16T00:13:04.605103+0000","last_clean":"2024-09-16T00:13:04.605103+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:13:04.605103+0000","last_undegraded":"2024-09-16T00:13:04.605103+0000","last_fullsized":"2024-09-16T00:13:04.605103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605218+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:13:04.605218+0000","last_peered":"2024-09-16T00:13:04.605218+0000","last_clean":"2024-09-16T00:13:04.605218+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:13:04.605218+0000","last_undegraded":"2024-09-16T00:13:04.605218+0000","last_fullsized":"2024-09-16T00:13:04.605218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664161+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:13:04.664161+0000","last_peered":"2024-09-16T00:13:04.664161+0000","last_clean":"2024-09-16T00:13:04.664161+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:13:04.664161+0000","last_undegraded":"2024-09-16T00:13:04.664161+0000","last_fullsized":"2024-09-16T00:13:04.664161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606964+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:13:04.606964+0000","last_peered":"2024-09-16T00:13:04.606964+0000","last_clean":"2024-09-16T00:13:04.606964+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:13:04.606964+0000","last_undegraded":"2024-09-16T00:13:04.606964+0000","last_fullsized":"2024-09-16T00:13:04.606964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649207+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:13:04.649207+0000","last_peered":"2024-09-16T00:13:04.649207+0000","last_clean":"2024-09-16T00:13:04.649207+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:13:04.649207+0000","last_undegraded":"2024-09-16T00:13:04.649207+0000","last_fullsized":"2024-09-16T00:13:04.649207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:26.731685+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:13:26.731685+0000","last_peered":"2024-09-16T00:13:26.731685+0000","last_clean":"2024-09-16T00:13:26.731685+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:13:26.731685+0000","last_undegraded":"2024-09-16T00:13:26.731685+0000","last_fullsized":"2024-09-16T00:13:26.731685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609415+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:13:04.609415+0000","last_peered":"2024-09-16T00:13:04.609415+0000","last_clean":"2024-09-16T00:13:04.609415+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:13:04.609415+0000","last_undegraded":"2024-09-16T00:13:04.609415+0000","last_fullsized":"2024-09-16T00:13:04.609415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649611+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:13:04.649611+0000","last_peered":"2024-09-16T00:13:04.649611+0000","last_clean":"2024-09-16T00:13:04.649611+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:13:04.649611+0000","last_undegraded":"2024-09-16T00:13:04.649611+0000","last_fullsized":"2024-09-16T00:13:04.649611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605440+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:13:04.605440+0000","last_peered":"2024-09-16T00:13:04.605440+0000","last_clean":"2024-09-16T00:13:04.605440+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:13:04.605440+0000","last_undegraded":"2024-09-16T00:13:04.605440+0000","last_fullsized":"2024-09-16T00:13:04.605440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664557+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:13:04.664557+0000","last_peered":"2024-09-16T00:13:04.664557+0000","last_clean":"2024-09-16T00:13:04.664557+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:13:04.664557+0000","last_undegraded":"2024-09-16T00:13:04.664557+0000","last_fullsized":"2024-09-16T00:13:04.664557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606631+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:13:04.606631+0000","last_peered":"2024-09-16T00:13:04.606631+0000","last_clean":"2024-09-16T00:13:04.606631+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:13:04.606631+0000","last_undegraded":"2024-09-16T00:13:04.606631+0000","last_fullsized":"2024-09-16T00:13:04.606631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611195+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:13:04.611195+0000","last_peered":"2024-09-16T00:13:04.611195+0000","last_clean":"2024-09-16T00:13:04.611195+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:13:04.611195+0000","last_undegraded":"2024-09-16T00:13:04.611195+0000","last_fullsized":"2024-09-16T00:13:04.611195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605540+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:13:04.605540+0000","last_peered":"2024-09-16T00:13:04.605540+0000","last_clean":"2024-09-16T00:13:04.605540+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:13:04.605540+0000","last_undegraded":"2024-09-16T00:13:04.605540+0000","last_fullsized":"2024-09-16T00:13:04.605540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664372+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:13:04.664372+0000","last_peered":"2024-09-16T00:13:04.664372+0000","last_clean":"2024-09-16T00:13:04.664372+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:13:04.664372+0000","last_undegraded":"2024-09-16T00:13:04.664372+0000","last_fullsized":"2024-09-16T00:13:04.664372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":293,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.017020+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:13:25.017020+0000","last_peered":"2024-09-16T00:13:25.017020+0000","last_clean":"2024-09-16T00:13:25.017020+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:13:25.017020+0000","last_undegraded":"2024-09-16T00:13:25.017020+0000","last_fullsized":"2024-09-16T00:13:25.017020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63488,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":168,"num_write":42,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609072+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:13:04.609072+0000","last_peered":"2024-09-16T00:13:04.609072+0000","last_clean":"2024-09-16T00:13:04.609072+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:13:04.609072+0000","last_undegraded":"2024-09-16T00:13:04.609072+0000","last_fullsized":"2024-09-16T00:13:04.609072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605601+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:13:04.605601+0000","last_peered":"2024-09-16T00:13:04.605601+0000","last_clean":"2024-09-16T00:13:04.605601+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:13:04.605601+0000","last_undegraded":"2024-09-16T00:13:04.605601+0000","last_fullsized":"2024-09-16T00:13:04.605601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605741+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:13:04.605741+0000","last_peered":"2024-09-16T00:13:04.605741+0000","last_clean":"2024-09-16T00:13:04.605741+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:13:04.605741+0000","last_undegraded":"2024-09-16T00:13:04.605741+0000","last_fullsized":"2024-09-16T00:13:04.605741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611591+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:13:04.611591+0000","last_peered":"2024-09-16T00:13:04.611591+0000","last_clean":"2024-09-16T00:13:04.611591+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:13:04.611591+0000","last_undegraded":"2024-09-16T00:13:04.611591+0000","last_fullsized":"2024-09-16T00:13:04.611591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663373+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:13:04.663373+0000","last_peered":"2024-09-16T00:13:04.663373+0000","last_clean":"2024-09-16T00:13:04.663373+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:13:04.663373+0000","last_undegraded":"2024-09-16T00:13:04.663373+0000","last_fullsized":"2024-09-16T00:13:04.663373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649999+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:13:04.649999+0000","last_peered":"2024-09-16T00:13:04.649999+0000","last_clean":"2024-09-16T00:13:04.649999+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:13:04.649999+0000","last_undegraded":"2024-09-16T00:13:04.649999+0000","last_fullsized":"2024-09-16T00:13:04.649999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.608894+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:13:04.608894+0000","last_peered":"2024-09-16T00:13:04.608894+0000","last_clean":"2024-09-16T00:13:04.608894+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:13:04.608894+0000","last_undegraded":"2024-09-16T00:13:04.608894+0000","last_fullsized":"2024-09-16T00:13:04.608894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605695+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:13:04.605695+0000","last_peered":"2024-09-16T00:13:04.605695+0000","last_clean":"2024-09-16T00:13:04.605695+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:13:04.605695+0000","last_undegraded":"2024-09-16T00:13:04.605695+0000","last_fullsized":"2024-09-16T00:13:04.605695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":248,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:24.695271+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:13:24.695271+0000","last_peered":"2024-09-16T00:13:24.695271+0000","last_clean":"2024-09-16T00:13:24.695271+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:13:24.695271+0000","last_undegraded":"2024-09-16T00:13:24.695271+0000","last_fullsized":"2024-09-16T00:13:24.695271+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664350+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:13:04.664350+0000","last_peered":"2024-09-16T00:13:04.664350+0000","last_clean":"2024-09-16T00:13:04.664350+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:13:04.664350+0000","last_undegraded":"2024-09-16T00:13:04.664350+0000","last_fullsized":"2024-09-16T00:13:04.664350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663978+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:13:04.663978+0000","last_peered":"2024-09-16T00:13:04.663978+0000","last_clean":"2024-09-16T00:13:04.663978+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:13:04.663978+0000","last_undegraded":"2024-09-16T00:13:04.663978+0000","last_fullsized":"2024-09-16T00:13:04.663978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609007+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:13:04.609007+0000","last_peered":"2024-09-16T00:13:04.609007+0000","last_clean":"2024-09-16T00:13:04.609007+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:13:04.609007+0000","last_undegraded":"2024-09-16T00:13:04.609007+0000","last_fullsized":"2024-09-16T00:13:04.609007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605461+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:13:04.605461+0000","last_peered":"2024-09-16T00:13:04.605461+0000","last_clean":"2024-09-16T00:13:04.605461+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:13:04.605461+0000","last_undegraded":"2024-09-16T00:13:04.605461+0000","last_fullsized":"2024-09-16T00:13:04.605461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664904+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:13:04.664904+0000","last_peered":"2024-09-16T00:13:04.664904+0000","last_clean":"2024-09-16T00:13:04.664904+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:13:04.664904+0000","last_undegraded":"2024-09-16T00:13:04.664904+0000","last_fullsized":"2024-09-16T00:13:04.664904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663317+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:13:04.663317+0000","last_peered":"2024-09-16T00:13:04.663317+0000","last_clean":"2024-09-16T00:13:04.663317+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:13:04.663317+0000","last_undegraded":"2024-09-16T00:13:04.663317+0000","last_fullsized":"2024-09-16T00:13:04.663317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606517+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:13:04.606517+0000","last_peered":"2024-09-16T00:13:04.606517+0000","last_clean":"2024-09-16T00:13:04.606517+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:13:04.606517+0000","last_undegraded":"2024-09-16T00:13:04.606517+0000","last_fullsized":"2024-09-16T00:13:04.606517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664780+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:13:04.664780+0000","last_peered":"2024-09-16T00:13:04.664780+0000","last_clean":"2024-09-16T00:13:04.664780+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:13:04.664780+0000","last_undegraded":"2024-09-16T00:13:04.664780+0000","last_fullsized":"2024-09-16T00:13:04.664780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605384+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:13:04.605384+0000","last_peered":"2024-09-16T00:13:04.605384+0000","last_clean":"2024-09-16T00:13:04.605384+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:13:04.605384+0000","last_undegraded":"2024-09-16T00:13:04.605384+0000","last_fullsized":"2024-09-16T00:13:04.605384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610913+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:13:04.610913+0000","last_peered":"2024-09-16T00:13:04.610913+0000","last_clean":"2024-09-16T00:13:04.610913+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:13:04.610913+0000","last_undegraded":"2024-09-16T00:13:04.610913+0000","last_fullsized":"2024-09-16T00:13:04.610913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.697247+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:13:25.697247+0000","last_peered":"2024-09-16T00:13:25.697247+0000","last_clean":"2024-09-16T00:13:25.697247+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:13:25.697247+0000","last_undegraded":"2024-09-16T00:13:25.697247+0000","last_fullsized":"2024-09-16T00:13:25.697247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663709+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:13:04.663709+0000","last_peered":"2024-09-16T00:13:04.663709+0000","last_clean":"2024-09-16T00:13:04.663709+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:13:04.663709+0000","last_undegraded":"2024-09-16T00:13:04.663709+0000","last_fullsized":"2024-09-16T00:13:04.663709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611136+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:13:04.611136+0000","last_peered":"2024-09-16T00:13:04.611136+0000","last_clean":"2024-09-16T00:13:04.611136+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:13:04.611136+0000","last_undegraded":"2024-09-16T00:13:04.611136+0000","last_fullsized":"2024-09-16T00:13:04.611136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663783+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:13:04.663783+0000","last_peered":"2024-09-16T00:13:04.663783+0000","last_clean":"2024-09-16T00:13:04.663783+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:13:04.663783+0000","last_undegraded":"2024-09-16T00:13:04.663783+0000","last_fullsized":"2024-09-16T00:13:04.663783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611426+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:13:04.611426+0000","last_peered":"2024-09-16T00:13:04.611426+0000","last_clean":"2024-09-16T00:13:04.611426+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:13:04.611426+0000","last_undegraded":"2024-09-16T00:13:04.611426+0000","last_fullsized":"2024-09-16T00:13:04.611426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610934+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:13:04.610934+0000","last_peered":"2024-09-16T00:13:04.610934+0000","last_clean":"2024-09-16T00:13:04.610934+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:13:04.610934+0000","last_undegraded":"2024-09-16T00:13:04.610934+0000","last_fullsized":"2024-09-16T00:13:04.610934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663979+0000","last_change":"2024-09-16T00:12:04.029280+0000","last_active":"2024-09-16T00:13:04.663979+0000","last_peered":"2024-09-16T00:13:04.663979+0000","last_clean":"2024-09-16T00:13:04.663979+0000","last_became_active":"2024-09-16T00:12:04.028973+0000","last_became_peered":"2024-09-16T00:12:04.028973+0000","last_unstale":"2024-09-16T00:13:04.663979+0000","last_undegraded":"2024-09-16T00:13:04.663979+0000","last_fullsized":"2024-09-16T00:13:04.663979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606846+0000","last_change":"2024-09-16T00:12:04.028319+0000","last_active":"2024-09-16T00:13:04.606846+0000","last_peered":"2024-09-16T00:13:04.606846+0000","last_clean":"2024-09-16T00:13:04.606846+0000","last_became_active":"2024-09-16T00:12:04.027985+0000","last_became_peered":"2024-09-16T00:12:04.027985+0000","last_unstale":"2024-09-16T00:13:04.606846+0000","last_undegraded":"2024-09-16T00:13:04.606846+0000","last_fullsized":"2024-09-16T00:13:04.606846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.119428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663790+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:13:04.663790+0000","last_peered":"2024-09-16T00:13:04.663790+0000","last_clean":"2024-09-16T00:13:04.663790+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:13:04.663790+0000","last_undegraded":"2024-09-16T00:13:04.663790+0000","last_fullsized":"2024-09-16T00:13:04.663790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611369+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:13:04.611369+0000","last_peered":"2024-09-16T00:13:04.611369+0000","last_clean":"2024-09-16T00:13:04.611369+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:13:04.611369+0000","last_undegraded":"2024-09-16T00:13:04.611369+0000","last_fullsized":"2024-09-16T00:13:04.611369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.616864+0000","last_change":"2024-09-16T00:13:04.612598+0000","last_active":"2024-09-16T00:13:04.616864+0000","last_peered":"2024-09-16T00:13:04.616864+0000","last_clean":"2024-09-16T00:13:04.616864+0000","last_became_active":"2024-09-16T00:13:04.612310+0000","last_became_peered":"2024-09-16T00:13:04.612310+0000","last_unstale":"2024-09-16T00:13:04.616864+0000","last_undegraded":"2024-09-16T00:13:04.616864+0000","last_fullsized":"2024-09-16T00:13:04.616864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:59.381944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605240+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:13:04.605240+0000","last_peered":"2024-09-16T00:13:04.605240+0000","last_clean":"2024-09-16T00:13:04.605240+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:13:04.605240+0000","last_undegraded":"2024-09-16T00:13:04.605240+0000","last_fullsized":"2024-09-16T00:13:04.605240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663870+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:13:04.663870+0000","last_peered":"2024-09-16T00:13:04.663870+0000","last_clean":"2024-09-16T00:13:04.663870+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:13:04.663870+0000","last_undegraded":"2024-09-16T00:13:04.663870+0000","last_fullsized":"2024-09-16T00:13:04.663870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605535+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:13:04.605535+0000","last_peered":"2024-09-16T00:13:04.605535+0000","last_clean":"2024-09-16T00:13:04.605535+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:13:04.605535+0000","last_undegraded":"2024-09-16T00:13:04.605535+0000","last_fullsized":"2024-09-16T00:13:04.605535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606766+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:13:04.606766+0000","last_peered":"2024-09-16T00:13:04.606766+0000","last_clean":"2024-09-16T00:13:04.606766+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:13:04.606766+0000","last_undegraded":"2024-09-16T00:13:04.606766+0000","last_fullsized":"2024-09-16T00:13:04.606766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663880+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:13:04.663880+0000","last_peered":"2024-09-16T00:13:04.663880+0000","last_clean":"2024-09-16T00:13:04.663880+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:13:04.663880+0000","last_undegraded":"2024-09-16T00:13:04.663880+0000","last_fullsized":"2024-09-16T00:13:04.663880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611528+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:13:04.611528+0000","last_peered":"2024-09-16T00:13:04.611528+0000","last_clean":"2024-09-16T00:13:04.611528+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:13:04.611528+0000","last_undegraded":"2024-09-16T00:13:04.611528+0000","last_fullsized":"2024-09-16T00:13:04.611528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609303+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:13:04.609303+0000","last_peered":"2024-09-16T00:13:04.609303+0000","last_clean":"2024-09-16T00:13:04.609303+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:13:04.609303+0000","last_undegraded":"2024-09-16T00:13:04.609303+0000","last_fullsized":"2024-09-16T00:13:04.609303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663938+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:13:04.663938+0000","last_peered":"2024-09-16T00:13:04.663938+0000","last_clean":"2024-09-16T00:13:04.663938+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:13:04.663938+0000","last_undegraded":"2024-09-16T00:13:04.663938+0000","last_fullsized":"2024-09-16T00:13:04.663938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605284+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:13:04.605284+0000","last_peered":"2024-09-16T00:13:04.605284+0000","last_clean":"2024-09-16T00:13:04.605284+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:13:04.605284+0000","last_undegraded":"2024-09-16T00:13:04.605284+0000","last_fullsized":"2024-09-16T00:13:04.605284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663375+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:13:04.663375+0000","last_peered":"2024-09-16T00:13:04.663375+0000","last_clean":"2024-09-16T00:13:04.663375+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:13:04.663375+0000","last_undegraded":"2024-09-16T00:13:04.663375+0000","last_fullsized":"2024-09-16T00:13:04.663375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663627+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:13:04.663627+0000","last_peered":"2024-09-16T00:13:04.663627+0000","last_clean":"2024-09-16T00:13:04.663627+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:13:04.663627+0000","last_undegraded":"2024-09-16T00:13:04.663627+0000","last_fullsized":"2024-09-16T00:13:04.663627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606689+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:13:04.606689+0000","last_peered":"2024-09-16T00:13:04.606689+0000","last_clean":"2024-09-16T00:13:04.606689+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:13:04.606689+0000","last_undegraded":"2024-09-16T00:13:04.606689+0000","last_fullsized":"2024-09-16T00:13:04.606689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611636+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:13:04.611636+0000","last_peered":"2024-09-16T00:13:04.611636+0000","last_clean":"2024-09-16T00:13:04.611636+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:13:04.611636+0000","last_undegraded":"2024-09-16T00:13:04.611636+0000","last_fullsized":"2024-09-16T00:13:04.611636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649291+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:13:04.649291+0000","last_peered":"2024-09-16T00:13:04.649291+0000","last_clean":"2024-09-16T00:13:04.649291+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:13:04.649291+0000","last_undegraded":"2024-09-16T00:13:04.649291+0000","last_fullsized":"2024-09-16T00:13:04.649291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664002+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:13:04.664002+0000","last_peered":"2024-09-16T00:13:04.664002+0000","last_clean":"2024-09-16T00:13:04.664002+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:13:04.664002+0000","last_undegraded":"2024-09-16T00:13:04.664002+0000","last_fullsized":"2024-09-16T00:13:04.664002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605780+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:13:04.605780+0000","last_peered":"2024-09-16T00:13:04.605780+0000","last_clean":"2024-09-16T00:13:04.605780+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:13:04.605780+0000","last_undegraded":"2024-09-16T00:13:04.605780+0000","last_fullsized":"2024-09-16T00:13:04.605780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663843+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:13:04.663843+0000","last_peered":"2024-09-16T00:13:04.663843+0000","last_clean":"2024-09-16T00:13:04.663843+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:13:04.663843+0000","last_undegraded":"2024-09-16T00:13:04.663843+0000","last_fullsized":"2024-09-16T00:13:04.663843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649554+0000","last_change":"2024-09-16T00:12:04.028342+0000","last_active":"2024-09-16T00:13:04.649554+0000","last_peered":"2024-09-16T00:13:04.649554+0000","last_clean":"2024-09-16T00:13:04.649554+0000","last_became_active":"2024-09-16T00:12:04.027889+0000","last_became_peered":"2024-09-16T00:12:04.027889+0000","last_unstale":"2024-09-16T00:13:04.649554+0000","last_undegraded":"2024-09-16T00:13:04.649554+0000","last_fullsized":"2024-09-16T00:13:04.649554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:47.840428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663964+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:13:04.663964+0000","last_peered":"2024-09-16T00:13:04.663964+0000","last_clean":"2024-09-16T00:13:04.663964+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:13:04.663964+0000","last_undegraded":"2024-09-16T00:13:04.663964+0000","last_fullsized":"2024-09-16T00:13:04.663964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663878+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:13:04.663878+0000","last_peered":"2024-09-16T00:13:04.663878+0000","last_clean":"2024-09-16T00:13:04.663878+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:13:04.663878+0000","last_undegraded":"2024-09-16T00:13:04.663878+0000","last_fullsized":"2024-09-16T00:13:04.663878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611689+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:13:04.611689+0000","last_peered":"2024-09-16T00:13:04.611689+0000","last_clean":"2024-09-16T00:13:04.611689+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:13:04.611689+0000","last_undegraded":"2024-09-16T00:13:04.611689+0000","last_fullsized":"2024-09-16T00:13:04.611689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664072+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:13:04.664072+0000","last_peered":"2024-09-16T00:13:04.664072+0000","last_clean":"2024-09-16T00:13:04.664072+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:13:04.664072+0000","last_undegraded":"2024-09-16T00:13:04.664072+0000","last_fullsized":"2024-09-16T00:13:04.664072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664019+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:13:04.664019+0000","last_peered":"2024-09-16T00:13:04.664019+0000","last_clean":"2024-09-16T00:13:04.664019+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:13:04.664019+0000","last_undegraded":"2024-09-16T00:13:04.664019+0000","last_fullsized":"2024-09-16T00:13:04.664019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71679,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1069,"num_read_kb":613,"num_write":269,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":515263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1736704,"data_stored":1165567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972405,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1069056,"data_stored":518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201338,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":515037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1658880,"data_stored":1111677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593796,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36972,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708180,"statfs":{"total":95995035648,"available":95957176320,"internally_reserved":0,"allocated":1667072,"data_stored":1107033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":515164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":509871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1130496,"data_stored":575435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1110016,"data_stored":574373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":512153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:13:30.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:13:30.503 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:30.966 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:30 smithi078 ceph-mon[31157]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:30.966 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:30 smithi078 ceph-mon[31157]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:30.966 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:30 smithi078 ceph-mon[31157]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T00:13:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 -- 172.21.15.78:0/56291163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 msgr2=0x7f6b4c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 --2- 172.21.15.78:0/56291163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 0x7f6b4c100ed0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009930 tx=0x7f6b4002a420 comp rx=0 tx=0).stop 2024-09-16T00:13:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 -- 172.21.15.78:0/56291163 shutdown_connections 2024-09-16T00:13:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 --2- 172.21.15.78:0/56291163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b4c101410 0x7f6b4c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 --2- 172.21.15.78:0/56291163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 0x7f6b4c100ed0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 --2- 172.21.15.78:0/56291163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b4c106a70 0x7f6b4c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 -- 172.21.15.78:0/56291163 >> 172.21.15.78:0/56291163 conn(0x7f6b4c0fc7d0 msgr2=0x7f6b4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 -- 172.21.15.78:0/56291163 shutdown_connections 2024-09-16T00:13:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.044+0000 7f6b51947640 1 -- 172.21.15.78:0/56291163 wait complete. 2024-09-16T00:13:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b51947640 1 Processor -- start 2024-09-16T00:13:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b51947640 1 -- start start 2024-09-16T00:13:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b51947640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 0x7f6b4c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b51947640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b4c101410 0x7f6b4c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b51947640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b4c106a70 0x7f6b4c1a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b51947640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c110e90 con 0x7f6b4c100a70 2024-09-16T00:13:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b51947640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c110d10 con 0x7f6b4c106a70 2024-09-16T00:13:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b51947640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c111010 con 0x7f6b4c101410 2024-09-16T00:13:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 0x7f6b4c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b4affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 0x7f6b4c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58806/0 (socket says 172.21.15.78:58806) 2024-09-16T00:13:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b4affd640 1 -- 172.21.15.78:0/3733168246 learned_addr learned my addr 172.21.15.78:0/3733168246 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b4a7fc640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b4c101410 0x7f6b4c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.045+0000 7f6b4b7fe640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b4c106a70 0x7f6b4c1a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b4affd640 1 -- 172.21.15.78:0/3733168246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b4c101410 msgr2=0x7f6b4c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b4affd640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b4c101410 0x7f6b4c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b4affd640 1 -- 172.21.15.78:0/3733168246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b4c106a70 msgr2=0x7f6b4c1a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b4affd640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b4c106a70 0x7f6b4c1a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b4affd640 1 -- 172.21.15.78:0/3733168246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c1a4920 con 0x7f6b4c100a70 2024-09-16T00:13:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b4a7fc640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b4c101410 0x7f6b4c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b4affd640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 0x7f6b4c199360 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c890 tx=0x7f6b3c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b50945640 1 -- 172.21.15.78:0/3733168246 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004220 con 0x7f6b4c100a70 2024-09-16T00:13:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b50945640 1 -- 172.21.15.78:0/3733168246 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3c0043c0 con 0x7f6b4c100a70 2024-09-16T00:13:31.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c1a4af0 con 0x7f6b4c100a70 2024-09-16T00:13:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b50945640 1 -- 172.21.15.78:0/3733168246 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c0049e0 con 0x7f6b4c100a70 2024-09-16T00:13:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.046+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c1a4fb0 con 0x7f6b4c100a70 2024-09-16T00:13:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.047+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b4c100ed0 con 0x7f6b4c100a70 2024-09-16T00:13:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.048+0000 7f6b50945640 1 -- 172.21.15.78:0/3733168246 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f6b3c0079b0 con 0x7f6b4c100a70 2024-09-16T00:13:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.049+0000 7f6b50945640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b340a32d0 0x7f6b340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.049+0000 7f6b4a7fc640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b340a32d0 0x7f6b340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.049+0000 7f6b50945640 1 -- 172.21.15.78:0/3733168246 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6b3c0cff10 con 0x7f6b4c100a70 2024-09-16T00:13:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.049+0000 7f6b4a7fc640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b340a32d0 0x7f6b340a5790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6b40000c00 tx=0x7f6b40005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.053+0000 7f6b50945640 1 -- 172.21.15.78:0/3733168246 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b3c014030 con 0x7f6b4c100a70 2024-09-16T00:13:31.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:30 smithi193 ceph-mon[35484]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:31.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:30 smithi193 ceph-mon[35484]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:31.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:30 smithi193 ceph-mon[35484]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T00:13:31.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:30 smithi148 ceph-mon[35599]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:31.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:30 smithi148 ceph-mon[35599]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:31.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:30 smithi148 ceph-mon[35599]: pgmap v364: 129 pgs: 129 active+clean; 647 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T00:13:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.254+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b4c109000 con 0x7f6b4c100a70 2024-09-16T00:13:31.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.254+0000 7f6b50945640 1 -- 172.21.15.78:0/3733168246 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7f6b3c09c770 con 0x7f6b4c100a70 2024-09-16T00:13:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:31.255 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-16T00:13:25:778566+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:25.728293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:13:31.256 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-16T00:13:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b340a32d0 msgr2=0x7f6b340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b340a32d0 0x7f6b340a5790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6b40000c00 tx=0x7f6b40005a20 comp rx=0 tx=0).stop 2024-09-16T00:13:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 msgr2=0x7f6b4c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 0x7f6b4c199360 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c00c890 tx=0x7f6b3c00cd50 comp rx=0 tx=0).stop 2024-09-16T00:13:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 shutdown_connections 2024-09-16T00:13:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b340a32d0 0x7f6b340a5790 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b4c106a70 0x7f6b4c1a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b4c101410 0x7f6b4c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.256+0000 7f6b51947640 1 --2- 172.21.15.78:0/3733168246 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b4c100a70 0x7f6b4c199360 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.257+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 >> 172.21.15.78:0/3733168246 conn(0x7f6b4c0fc7d0 msgr2=0x7f6b4c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.257+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 shutdown_connections 2024-09-16T00:13:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:31.257+0000 7f6b51947640 1 -- 172.21.15.78:0/3733168246 wait complete. 2024-09-16T00:13:31.996 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:31 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3733168246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:32.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:13:32.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:31 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3733168246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:31 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3733168246' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:32.271 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 -- 172.21.15.78:0/4205620457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438102790 msgr2=0x7f54381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 --2- 172.21.15.78:0/4205620457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438102790 0x7f54381067e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f542c009900 tx=0x7f542c02a410 comp rx=0 tx=0).stop 2024-09-16T00:13:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 -- 172.21.15.78:0/4205620457 shutdown_connections 2024-09-16T00:13:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 --2- 172.21.15.78:0/4205620457 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5438106f10 0x7f543810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 --2- 172.21.15.78:0/4205620457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438102790 0x7f54381067e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 --2- 172.21.15.78:0/4205620457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5438101de0 0x7f54381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 -- 172.21.15.78:0/4205620457 >> 172.21.15.78:0/4205620457 conn(0x7f54380fd8c0 msgr2=0x7f54380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 -- 172.21.15.78:0/4205620457 shutdown_connections 2024-09-16T00:13:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.763+0000 7f543c841640 1 -- 172.21.15.78:0/4205620457 wait complete. 2024-09-16T00:13:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543c841640 1 Processor -- start 2024-09-16T00:13:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543c841640 1 -- start start 2024-09-16T00:13:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543c841640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5438101de0 0x7f54381984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543c841640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5438102790 0x7f54381989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543c841640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438106f10 0x7f543819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543c841640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5438111fb0 con 0x7f5438106f10 2024-09-16T00:13:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543c841640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5438111e30 con 0x7f5438101de0 2024-09-16T00:13:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543c841640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5438112130 con 0x7f5438102790 2024-09-16T00:13:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5436d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438106f10 0x7f543819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5436d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438106f10 0x7f543819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58832/0 (socket says 172.21.15.78:58832) 2024-09-16T00:13:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5436d9d640 1 -- 172.21.15.78:0/375148038 learned_addr learned my addr 172.21.15.78:0/375148038 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5435d9b640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5438102790 0x7f54381989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543659c640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5438101de0 0x7f54381984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5436d9d640 1 -- 172.21.15.78:0/375148038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5438102790 msgr2=0x7f54381989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5436d9d640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5438102790 0x7f54381989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5436d9d640 1 -- 172.21.15.78:0/375148038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5438101de0 msgr2=0x7f54381984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5436d9d640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5438101de0 0x7f54381984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f5436d9d640 1 -- 172.21.15.78:0/375148038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543819d460 con 0x7f5438106f10 2024-09-16T00:13:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.764+0000 7f543659c640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5438101de0 0x7f54381984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.765+0000 7f5436d9d640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438106f10 0x7f543819cd80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f543000b970 tx=0x7f543000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.765+0000 7f54277fe640 1 -- 172.21.15.78:0/375148038 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430019070 con 0x7f5438106f10 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.765+0000 7f54277fe640 1 -- 172.21.15.78:0/375148038 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f543000cb40 con 0x7f5438106f10 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.765+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543819d750 con 0x7f5438106f10 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.765+0000 7f54277fe640 1 -- 172.21.15.78:0/375148038 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430007650 con 0x7f5438106f10 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.765+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54381a5030 con 0x7f5438106f10 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.766+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54381021c0 con 0x7f5438106f10 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.767+0000 7f54277fe640 1 -- 172.21.15.78:0/375148038 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f543001c070 con 0x7f5438106f10 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.768+0000 7f54277fe640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54200a3350 0x7f54200a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.768+0000 7f54277fe640 1 -- 172.21.15.78:0/375148038 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f54300cf6e0 con 0x7f5438106f10 2024-09-16T00:13:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.768+0000 7f543659c640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54200a3350 0x7f54200a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.768+0000 7f543659c640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54200a3350 0x7f54200a5810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5428009b40 tx=0x7f5428005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:32.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.771+0000 7f54277fe640 1 -- 172.21.15.78:0/375148038 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f543009ba00 con 0x7f5438106f10 2024-09-16T00:13:32.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.940+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f543810a1b0 con 0x7f5438106f10 2024-09-16T00:13:32.941 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:32 smithi078 ceph-mon[31157]: pgmap v365: 129 pgs: 129 active+clean; 654 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-16T00:13:32.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.941+0000 7f54277fe640 1 -- 172.21.15.78:0/375148038 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24757 (secure 0 0 0) 0x7f543009fd00 con 0x7f5438106f10 2024-09-16T00:13:32.943 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:32.943 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:13:04.600749+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:13:32.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.943+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54200a3350 msgr2=0x7f54200a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.943+0000 7f543c841640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54200a3350 0x7f54200a5810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5428009b40 tx=0x7f5428005e50 comp rx=0 tx=0).stop 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.943+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438106f10 msgr2=0x7f543819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.943+0000 7f543c841640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438106f10 0x7f543819cd80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f543000b970 tx=0x7f543000be30 comp rx=0 tx=0).stop 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.944+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 shutdown_connections 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.944+0000 7f543c841640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54200a3350 0x7f54200a5810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.944+0000 7f543c841640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5438106f10 0x7f543819cd80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.944+0000 7f543c841640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5438102790 0x7f54381989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.944+0000 7f543c841640 1 --2- 172.21.15.78:0/375148038 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5438101de0 0x7f54381984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.944+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 >> 172.21.15.78:0/375148038 conn(0x7f54380fd8c0 msgr2=0x7f54380feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.944+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 shutdown_connections 2024-09-16T00:13:32.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:32.944+0000 7f543c841640 1 -- 172.21.15.78:0/375148038 wait complete. 2024-09-16T00:13:33.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:32 smithi193 ceph-mon[35484]: pgmap v365: 129 pgs: 129 active+clean; 654 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-16T00:13:33.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:32 smithi148 ceph-mon[35599]: pgmap v365: 129 pgs: 129 active+clean; 654 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-16T00:13:33.636 DEBUG:tasks.fwd_scrub:Yielding 2024-09-16T00:13:33.637 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-16T00:13:33.653 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:13:33.654 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:13:33.654 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-16T00:13:33.654 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:13:33.654 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:13:33.918 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:33.918 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:33 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/375148038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:33 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/375148038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:33 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/375148038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 -- 172.21.15.78:0/1406554224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418100a80 msgr2=0x7fb418100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 --2- 172.21.15.78:0/1406554224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418100a80 0x7fb418100ee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009930 tx=0x7fb40c02a420 comp rx=0 tx=0).stop 2024-09-16T00:13:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 -- 172.21.15.78:0/1406554224 shutdown_connections 2024-09-16T00:13:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 --2- 172.21.15.78:0/1406554224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418101420 0x7fb41810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 --2- 172.21.15.78:0/1406554224 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418100a80 0x7fb418100ee0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 --2- 172.21.15.78:0/1406554224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb418106a80 0x7fb418106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 -- 172.21.15.78:0/1406554224 >> 172.21.15.78:0/1406554224 conn(0x7fb4180fc7a0 msgr2=0x7fb4180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 -- 172.21.15.78:0/1406554224 shutdown_connections 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.426+0000 7fb41d02d640 1 -- 172.21.15.78:0/1406554224 wait complete. 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41d02d640 1 Processor -- start 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41d02d640 1 -- start start 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41d02d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb418100a80 0x7fb4181973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41d02d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418101420 0x7fb418197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41d02d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418106a80 0x7fb41819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41d02d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb418110ea0 con 0x7fb418106a80 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41d02d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb418110d20 con 0x7fb418101420 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41d02d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb418111020 con 0x7fb418100a80 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418106a80 0x7fb41819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418106a80 0x7fb41819bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58858/0 (socket says 172.21.15.78:58858) 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.427+0000 7fb41759e640 1 -- 172.21.15.78:0/2857136453 learned_addr learned my addr 172.21.15.78:0/2857136453 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb416d9d640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb418100a80 0x7fb4181973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41659c640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418101420 0x7fb418197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41759e640 1 -- 172.21.15.78:0/2857136453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb418100a80 msgr2=0x7fb4181973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41759e640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb418100a80 0x7fb4181973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41759e640 1 -- 172.21.15.78:0/2857136453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418101420 msgr2=0x7fb418197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41759e640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418101420 0x7fb418197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41759e640 1 -- 172.21.15.78:0/2857136453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41819c390 con 0x7fb418106a80 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41759e640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418106a80 0x7fb41819bcb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb41000eaa0 tx=0x7fb41000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb3fffff640 1 -- 172.21.15.78:0/2857136453 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41000cdb0 con 0x7fb418106a80 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41d02d640 1 -- 172.21.15.78:0/2857136453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41819c680 con 0x7fb418106a80 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb3fffff640 1 -- 172.21.15.78:0/2857136453 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb410005020 con 0x7fb418106a80 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb3fffff640 1 -- 172.21.15.78:0/2857136453 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4100116b0 con 0x7fb418106a80 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.428+0000 7fb41d02d640 1 -- 172.21.15.78:0/2857136453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a3f60 con 0x7fb418106a80 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.430+0000 7fb3fffff640 1 -- 172.21.15.78:0/2857136453 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fb410005560 con 0x7fb418106a80 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.430+0000 7fb41f406740 1 Processor -- start 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb41f406740 1 -- start start 2024-09-16T00:13:34.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb3fffff640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb4000a32d0 0x7fb4000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb41f406740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d25f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb41f406740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d26480 0x558827d30800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb41f406740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 0x558827d33130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb41f406740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558827c8b280 con 0x558827d25b60 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb41f406740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558827c32b10 con 0x558827d30d40 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb41f406740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558827c87240 con 0x558827d26480 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d25f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d25f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58870/0 (socket says 172.21.15.78:58870) 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb416d9d640 1 -- 172.21.15.78:0/4282461759 learned_addr learned my addr 172.21.15.78:0/4282461759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb3fffff640 1 -- 172.21.15.78:0/2857136453 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb4100d0130 con 0x7fb418106a80 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb41659c640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb4000a32d0 0x7fb4000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb416d9d640 1 -- 172.21.15.78:0/4282461759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d26480 msgr2=0x558827d30800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:34.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb416d9d640 1 --2- 172.21.15.78:0/4282461759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d26480 0x558827d30800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb416d9d640 1 -- 172.21.15.78:0/4282461759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 msgr2=0x558827d33130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:34.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb416d9d640 1 --2- 172.21.15.78:0/4282461759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 0x558827d33130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.431+0000 7fb416d9d640 1 -- 172.21.15.78:0/4282461759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558827c3f1f0 con 0x558827d25b60 2024-09-16T00:13:34.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.432+0000 7fb416d9d640 1 --2- 172.21.15.78:0/4282461759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d25f40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b820 tx=0x7fb40800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a032feda6729ea87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:34.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.432+0000 7fb41659c640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb4000a32d0 0x7fb4000a5790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb40c000c00 tx=0x7fb40c032db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:34.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.432+0000 7fb3fd7fa640 1 -- 172.21.15.78:0/4282461759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800d950 con 0x558827d25b60 2024-09-16T00:13:34.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.432+0000 7fb3fd7fa640 1 -- 172.21.15.78:0/4282461759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408019070 con 0x558827d25b60 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.432+0000 7fb3fd7fa640 1 -- 172.21.15.78:0/4282461759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080123e0 con 0x558827d25b60 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- 172.21.15.78:0/4282461759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 msgr2=0x558827d25f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 --2- 172.21.15.78:0/4282461759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d25f40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b820 tx=0x7fb40800bce0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- 172.21.15.78:0/4282461759 shutdown_connections 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 --2- 172.21.15.78:0/4282461759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 0x558827d33130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 --2- 172.21.15.78:0/4282461759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d26480 0x558827d30800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 --2- 172.21.15.78:0/4282461759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d25f40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- 172.21.15.78:0/4282461759 >> 172.21.15.78:0/4282461759 conn(0x558827d220f0 msgr2=0x558827d224d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- 172.21.15.78:0/4282461759 shutdown_connections 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- 172.21.15.78:0/4282461759 wait complete. 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- start start 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 Processor -- start 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d2c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 0x558827d2c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 0x558827d40cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558827d33910 con 0x558827d25b60 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558827c45f10 con 0x558827d26480 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.433+0000 7fb41f406740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558827d33a90 con 0x558827d30d40 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d2c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb41f406740 10 client.0 _ll_register_callbacks cb 0x558827c0ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d2c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58878/0 (socket says 172.21.15.78:58878) 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 -- 172.21.15.78:0/4145261433 learned_addr learned my addr 172.21.15.78:0/4145261433 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb41659c640 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 0x558827d2c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb41759e640 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 0x558827d40cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 -- 172.21.15.78:0/4145261433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 msgr2=0x558827d40cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:34.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 0x558827d40cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 -- 172.21.15.78:0/4145261433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 msgr2=0x558827d2c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:34.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 0x558827d2c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 -- 172.21.15.78:0/4145261433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588279fc6a0 con 0x558827d25b60 2024-09-16T00:13:34.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb416d9d640 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d2c2c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb408014ba0 tx=0x7fb408014660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b927e5606feb7a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:34.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb3faff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb3faff5640 1 -- 172.21.15.78:0/4145261433 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408023070 con 0x558827d25b60 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb3faff5640 1 -- 172.21.15.78:0/4145261433 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb40800dd60 con 0x558827d25b60 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb41f406740 1 -- 172.21.15.78:0/4145261433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb408019070 con 0x558827d25b60 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb3faff5640 1 -- 172.21.15.78:0/4145261433 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb40802f050 con 0x558827d25b60 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb41f406740 10 client.15369 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb41f406740 1 -- 172.21.15.78:0/4145261433 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558827d438f0 con 0x558827d25b60 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.434+0000 7fb3faff5640 1 -- 172.21.15.78:0/4145261433 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408019070 con 0x558827d25b60 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb3faff5640 1 -- 172.21.15.78:0/4145261433 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb408013070 con 0x558827d25b60 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb41f406740 10 client.15369 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb41f406740 10 client.15369 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb41f406740 20 client.15369 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:34.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb41f406740 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558827d44c40 0x558827d47030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb41f406740 4 client.15369 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb41f406740 1 -- 172.21.15.78:0/4145261433 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558827c987f0 con 0x558827d44c40 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb41659c640 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558827d44c40 0x558827d47030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb41659c640 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558827d44c40 0x558827d47030 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba4ecf0c2ece335 server_cookie=812bcdddc1042a91 in_seq=0 out_seq=0 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.435+0000 7fb3faff5640 10 client.15369 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb3faff5640 1 -- 172.21.15.78:0/4145261433 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558827c987f0 con 0x558827d44c40 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb3faff5640 10 client.15369 handle_command_reply: tid=0 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb41f406740 1 client.15369 shutdown 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb41f406740 1 -- 172.21.15.78:0/4145261433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 msgr2=0x558827d2c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb41f406740 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d2c2c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb408014ba0 tx=0x7fb408014660 comp rx=0 tx=0).stop 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb41f406740 1 -- 172.21.15.78:0/4145261433 shutdown_connections 2024-09-16T00:13:34.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb41f406740 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558827d44c40 0x558827d47030 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb41f406740 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 0x558827d40cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb41f406740 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 0x558827d2c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb41f406740 1 --2- 172.21.15.78:0/4145261433 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d2c2c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.436+0000 7fb3faff5640 0 client.15369 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:34.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.437+0000 7fb41f406740 1 -- 172.21.15.78:0/4145261433 >> 172.21.15.78:0/4145261433 conn(0x558827d220f0 msgr2=0x558827d21270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:34.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.437+0000 7fb41f406740 1 -- 172.21.15.78:0/4145261433 shutdown_connections 2024-09-16T00:13:34.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.437+0000 7fb41f406740 1 -- 172.21.15.78:0/4145261433 wait complete. 2024-09-16T00:13:34.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.437+0000 7fb41f406740 20 client.15369 trim_cache size 0 max 16384 2024-09-16T00:13:34.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41f406740 1 Processor -- start 2024-09-16T00:13:34.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41f406740 1 -- start start 2024-09-16T00:13:34.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41f406740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d31940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41f406740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 0x558827d2b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41f406740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 0x558827d34cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41f406740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181aca20 con 0x558827d25b60 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41f406740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181ae560 con 0x558827d26480 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41f406740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181ad0f0 con 0x558827d30d40 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d31940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d31940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58884/0 (socket says 172.21.15.78:58884) 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 -- 172.21.15.78:0/1234642015 learned_addr learned my addr 172.21.15.78:0/1234642015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41759e640 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 0x558827d34cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb41659c640 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 0x558827d2b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 -- 172.21.15.78:0/1234642015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 msgr2=0x558827d34cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:34.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 0x558827d34cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 -- 172.21.15.78:0/1234642015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 msgr2=0x558827d2b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 0x558827d2b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 -- 172.21.15.78:0/1234642015 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558827d1d830 con 0x558827d25b60 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.454+0000 7fb416d9d640 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d31940 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b390 tx=0x7fb40801d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2987e9d48df202d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb3fcff9640 1 -- 172.21.15.78:0/1234642015 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40800a070 con 0x558827d25b60 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb3fcff9640 1 -- 172.21.15.78:0/1234642015 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408019070 con 0x558827d25b60 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb3fcff9640 1 -- 172.21.15.78:0/1234642015 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408023050 con 0x558827d25b60 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 -- 172.21.15.78:0/1234642015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 msgr2=0x558827d31940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d31940 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b390 tx=0x7fb40801d7c0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 -- 172.21.15.78:0/1234642015 shutdown_connections 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d30d40 0x558827d34cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d26480 0x558827d2b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 --2- 172.21.15.78:0/1234642015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d25b60 0x558827d31940 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 -- 172.21.15.78:0/1234642015 >> 172.21.15.78:0/1234642015 conn(0x558827d220f0 msgr2=0x558827d2a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 -- 172.21.15.78:0/1234642015 shutdown_connections 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.455+0000 7fb41f406740 1 -- 172.21.15.78:0/1234642015 wait complete. 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 1 -- start start 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 1 Processor -- start 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d25b60 0x558827d3ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d26480 0x558827d3b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 0x558827d3f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181ade90 con 0x558827d26480 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181ad7c0 con 0x558827d30d40 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181aaa00 con 0x558827d25b60 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41f406740 10 client.0 _ll_register_callbacks cb 0x7fb40800b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d26480 0x558827d3b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d26480 0x558827d3b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58886/0 (socket says 172.21.15.78:58886) 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41659c640 1 -- 172.21.15.78:0/422778582 learned_addr learned my addr 172.21.15.78:0/422778582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb416d9d640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d25b60 0x558827d3ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41759e640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 0x558827d3f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41659c640 1 -- 172.21.15.78:0/422778582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d25b60 msgr2=0x558827d3ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41659c640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d25b60 0x558827d3ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41659c640 1 -- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 msgr2=0x558827d3f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41659c640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 0x558827d3f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41659c640 1 -- 172.21.15.78:0/422778582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558827d3b920 con 0x558827d26480 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb41759e640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 0x558827d3f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.456+0000 7fb416d9d640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d25b60 0x558827d3ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41659c640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d26480 0x558827d3b090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0096c0 tx=0x7fb40c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93e49406b7ff959 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb3faff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb3faff5640 1 -- 172.21.15.78:0/422778582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c035050 con 0x558827d26480 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb3faff5640 1 -- 172.21.15.78:0/422778582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb40c004310 con 0x558827d26480 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 1 -- 172.21.15.78:0/422778582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb408019070 con 0x558827d26480 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb3faff5640 1 -- 172.21.15.78:0/422778582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb40c05e050 con 0x558827d26480 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 10 client.15375 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:34.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 1 -- 172.21.15.78:0/422778582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558827dafc10 con 0x558827d26480 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb3faff5640 1 -- 172.21.15.78:0/422778582 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408019070 con 0x558827d26480 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb3faff5640 1 -- 172.21.15.78:0/422778582 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb40c059030 con 0x558827d26480 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 10 client.15375 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 10 client.15375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 20 client.15375 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558827db03c0 0x558827db27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 4 client.15375 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.457+0000 7fb41f406740 1 -- 172.21.15.78:0/422778582 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558827d33520 con 0x558827db03c0 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.458+0000 7fb416d9d640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558827db03c0 0x558827db27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.458+0000 7fb416d9d640 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558827db03c0 0x558827db27e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428165bff079b2ab server_cookie=19e5e52fa0bc2c18 in_seq=0 out_seq=0 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.458+0000 7fb3faff5640 10 client.15375 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb3faff5640 1 -- 172.21.15.78:0/422778582 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558827d33520 con 0x558827db03c0 2024-09-16T00:13:34.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb3faff5640 10 client.15375 handle_command_reply: tid=0 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 client.15375 shutdown 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 -- 172.21.15.78:0/422778582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d26480 msgr2=0x558827d3b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d26480 0x558827d3b090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0096c0 tx=0x7fb40c009260 comp rx=0 tx=0).stop 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 -- 172.21.15.78:0/422778582 shutdown_connections 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558827db03c0 0x558827db27e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558827d30d40 0x558827d3f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558827d26480 0x558827d3b090 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 --2- 172.21.15.78:0/422778582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558827d25b60 0x558827d3ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb41f406740 1 -- 172.21.15.78:0/422778582 >> 172.21.15.78:0/422778582 conn(0x558827d220f0 msgr2=0x558827d2b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.459+0000 7fb3faff5640 0 client.15375 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41f406740 1 -- 172.21.15.78:0/422778582 shutdown_connections 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41f406740 1 -- 172.21.15.78:0/422778582 wait complete. 2024-09-16T00:13:34.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41f406740 20 client.15375 trim_cache size 0 max 16384 2024-09-16T00:13:34.478 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:34.478 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:13:34.478 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "185c8017-a15d-4587-8211-8782e5acd86e", 2024-09-16T00:13:34.478 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:13:34.478 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 -- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb4000a32d0 msgr2=0x7fb4000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb4000a32d0 0x7fb4000a5790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb40c000c00 tx=0x7fb40c032db0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 -- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418106a80 msgr2=0x7fb41819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418106a80 0x7fb41819bcb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb41000eaa0 tx=0x7fb41000ef60 comp rx=0 tx=0).stop 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 -- 172.21.15.78:0/2857136453 shutdown_connections 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb4000a32d0 0x7fb4000a5790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb418106a80 0x7fb41819bcb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb418101420 0x7fb418197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 --2- 172.21.15.78:0/2857136453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb418100a80 0x7fb4181973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:34.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 -- 172.21.15.78:0/2857136453 >> 172.21.15.78:0/2857136453 conn(0x7fb4180fc7a0 msgr2=0x7fb4180fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:34.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 -- 172.21.15.78:0/2857136453 shutdown_connections 2024-09-16T00:13:34.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:34.460+0000 7fb41d02d640 1 -- 172.21.15.78:0/2857136453 wait complete. 2024-09-16T00:13:35.095 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:34 smithi078 ceph-mon[31157]: pgmap v366: 129 pgs: 129 active+clean; 654 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:13:35.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:34 smithi193 ceph-mon[35484]: pgmap v366: 129 pgs: 129 active+clean; 654 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:13:35.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:34 smithi148 ceph-mon[35599]: pgmap v366: 129 pgs: 129 active+clean; 654 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:13:35.256 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:13:35.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 -- 172.21.15.78:0/2406437531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c102790 msgr2=0x7f639c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:35.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 --2- 172.21.15.78:0/2406437531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c102790 0x7f639c1067e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6384009900 tx=0x7f638402a410 comp rx=0 tx=0).stop 2024-09-16T00:13:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 -- 172.21.15.78:0/2406437531 shutdown_connections 2024-09-16T00:13:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 --2- 172.21.15.78:0/2406437531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f639c106f10 0x7f639c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 --2- 172.21.15.78:0/2406437531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c102790 0x7f639c1067e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 --2- 172.21.15.78:0/2406437531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f639c101de0 0x7f639c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 -- 172.21.15.78:0/2406437531 >> 172.21.15.78:0/2406437531 conn(0x7f639c0fd8c0 msgr2=0x7f639c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 -- 172.21.15.78:0/2406437531 shutdown_connections 2024-09-16T00:13:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.718+0000 7f63a08eb640 1 -- 172.21.15.78:0/2406437531 wait complete. 2024-09-16T00:13:35.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f63a08eb640 1 Processor -- start 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f63a08eb640 1 -- start start 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f63a08eb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c101de0 0x7f639c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f63a08eb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f639c102790 0x7f639c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f63a08eb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f639c106f10 0x7f639c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f63a08eb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f639c111f30 con 0x7f639c101de0 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f63a08eb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f639c111db0 con 0x7f639c106f10 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f63a08eb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f639c1120b0 con 0x7f639c102790 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c101de0 0x7f639c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c101de0 0x7f639c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58900/0 (socket says 172.21.15.78:58900) 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639a59c640 1 -- 172.21.15.78:0/805307986 learned_addr learned my addr 172.21.15.78:0/805307986 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f6393fff640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f639c102790 0x7f639c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:35.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639ad9d640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f639c106f10 0x7f639c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639a59c640 1 -- 172.21.15.78:0/805307986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f639c102790 msgr2=0x7f639c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639a59c640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f639c102790 0x7f639c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639a59c640 1 -- 172.21.15.78:0/805307986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f639c106f10 msgr2=0x7f639c19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639a59c640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f639c106f10 0x7f639c19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639a59c640 1 -- 172.21.15.78:0/805307986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639c19d330 con 0x7f639c101de0 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f6393fff640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f639c102790 0x7f639c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.719+0000 7f639ad9d640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f639c106f10 0x7f639c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.720+0000 7f639a59c640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c101de0 0x7f639c198380 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c920 tx=0x7f638c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.720+0000 7f63937fe640 1 -- 172.21.15.78:0/805307986 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c007c70 con 0x7f639c101de0 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.720+0000 7f63937fe640 1 -- 172.21.15.78:0/805307986 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638c007e10 con 0x7f639c101de0 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.720+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639c19d620 con 0x7f639c101de0 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.720+0000 7f63937fe640 1 -- 172.21.15.78:0/805307986 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c018890 con 0x7f639c101de0 2024-09-16T00:13:35.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.720+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639c1a4f00 con 0x7f639c101de0 2024-09-16T00:13:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.721+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639c1021c0 con 0x7f639c101de0 2024-09-16T00:13:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.722+0000 7f63937fe640 1 -- 172.21.15.78:0/805307986 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f638c018a70 con 0x7f639c101de0 2024-09-16T00:13:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.722+0000 7f63937fe640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63880a32d0 0x7f63880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.723+0000 7f6393fff640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63880a32d0 0x7f63880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.723+0000 7f63937fe640 1 -- 172.21.15.78:0/805307986 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f638c0d0000 con 0x7f639c101de0 2024-09-16T00:13:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.723+0000 7f6393fff640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63880a32d0 0x7f63880a5790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f639c0fdce0 tx=0x7f6384005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:35.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.727+0000 7f63937fe640 1 -- 172.21.15.78:0/805307986 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f638c014030 con 0x7f639c101de0 2024-09-16T00:13:35.795 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.901+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f639c10a160 con 0x7f63880a32d0 2024-09-16T00:13:35.902 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:35 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:13:35.902 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:35 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:13:35.902 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:35 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:13:35.902 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:35 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:13:35.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.909+0000 7f63937fe640 1 -- 172.21.15.78:0/805307986 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340528 (secure 0 0 0) 0x7f63840315b0 con 0x7f63880a32d0 2024-09-16T00:13:35.915 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:35.918 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.912+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63880a32d0 msgr2=0x7f63880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.912+0000 7f63a08eb640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63880a32d0 0x7f63880a5790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f639c0fdce0 tx=0x7f6384005a20 comp rx=0 tx=0).stop 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.912+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c101de0 msgr2=0x7f639c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.912+0000 7f63a08eb640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c101de0 0x7f639c198380 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c920 tx=0x7f638c00cde0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.913+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 shutdown_connections 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.913+0000 7f63a08eb640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f63880a32d0 0x7f63880a5790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.913+0000 7f63a08eb640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f639c106f10 0x7f639c19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.913+0000 7f63a08eb640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f639c102790 0x7f639c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.913+0000 7f63a08eb640 1 --2- 172.21.15.78:0/805307986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f639c101de0 0x7f639c198380 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.913+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 >> 172.21.15.78:0/805307986 conn(0x7f639c0fd8c0 msgr2=0x7f639c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.913+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 shutdown_connections 2024-09-16T00:13:35.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:35.913+0000 7f63a08eb640 1 -- 172.21.15.78:0/805307986 wait complete. 2024-09-16T00:13:36.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:35 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:13:36.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:35 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:13:36.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:35 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:13:36.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:35 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:13:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:35 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:13:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:35 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:13:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:35 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:13:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:35 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:13:36.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 -- 172.21.15.78:0/2870155884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 msgr2=0x7fb21c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 --2- 172.21.15.78:0/2870155884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 0x7fb21c1029b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009900 tx=0x7fb20c02a480 comp rx=0 tx=0).stop 2024-09-16T00:13:36.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 -- 172.21.15.78:0/2870155884 shutdown_connections 2024-09-16T00:13:36.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 --2- 172.21.15.78:0/2870155884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb21c102ef0 0x7fb21c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 --2- 172.21.15.78:0/2870155884 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 0x7fb21c1029b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 --2- 172.21.15.78:0/2870155884 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb21c100e20 0x7fb21c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 -- 172.21.15.78:0/2870155884 >> 172.21.15.78:0/2870155884 conn(0x7fb21c0fc7d0 msgr2=0x7fb21c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 -- 172.21.15.78:0/2870155884 shutdown_connections 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 -- 172.21.15.78:0/2870155884 wait complete. 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.509+0000 7fb223f83640 1 Processor -- start 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb223f83640 1 -- start start 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb223f83640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb21c100e20 0x7fb21c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb223f83640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 0x7fb21c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb223f83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb21c102ef0 0x7fb21c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb223f83640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c110fa0 con 0x7fb21c102570 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb223f83640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c110e20 con 0x7fb21c100e20 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb223f83640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c111120 con 0x7fb21c102ef0 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 0x7fb21c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 0x7fb21c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53948/0 (socket says 172.21.15.78:53948) 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 -- 172.21.15.78:0/4131194174 learned_addr learned my addr 172.21.15.78:0/4131194174 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb221d1f640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb21c100e20 0x7fb21c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb222520640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb21c102ef0 0x7fb21c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 -- 172.21.15.78:0/4131194174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb21c102ef0 msgr2=0x7fb21c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb21c102ef0 0x7fb21c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 -- 172.21.15.78:0/4131194174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb21c100e20 msgr2=0x7fb21c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb21c100e20 0x7fb21c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 -- 172.21.15.78:0/4131194174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21c19d4e0 con 0x7fb21c102570 2024-09-16T00:13:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb222520640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb21c102ef0 0x7fb21c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.510+0000 7fb22151e640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 0x7fb21c198a70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb20c006010 tx=0x7fb20c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.511+0000 7fb212ffd640 1 -- 172.21.15.78:0/4131194174 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c03f070 con 0x7fb21c102570 2024-09-16T00:13:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.511+0000 7fb223f83640 1 -- 172.21.15.78:0/4131194174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21c19d770 con 0x7fb21c102570 2024-09-16T00:13:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.511+0000 7fb212ffd640 1 -- 172.21.15.78:0/4131194174 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb20c031c30 con 0x7fb21c102570 2024-09-16T00:13:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.511+0000 7fb212ffd640 1 -- 172.21.15.78:0/4131194174 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c035030 con 0x7fb21c102570 2024-09-16T00:13:36.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.511+0000 7fb223f83640 1 -- 172.21.15.78:0/4131194174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21c1a50b0 con 0x7fb21c102570 2024-09-16T00:13:36.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.513+0000 7fb212ffd640 1 -- 172.21.15.78:0/4131194174 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fb20c042070 con 0x7fb21c102570 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.513+0000 7fb22635c740 1 Processor -- start 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.513+0000 7fb22635c740 1 -- start start 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb22635c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb22635c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb22635c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6d2680 0x55bcfd6d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb22635c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bcfd62d2e0 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb22635c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bcfd5d4ce0 con 0x55bcfd6c74a0 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb22635c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bcfd6292a0 con 0x55bcfd6d2680 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6d2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53954/0 (socket says 172.21.15.78:53954) 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb22151e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6d2680 0x55bcfd6d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb222520640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 -- 172.21.15.78:0/1566945332 learned_addr learned my addr 172.21.15.78:0/1566945332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb212ffd640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 -- 172.21.15.78:0/1566945332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6d2680 msgr2=0x55bcfd6d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 --2- 172.21.15.78:0/1566945332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6d2680 0x55bcfd6d4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 -- 172.21.15.78:0/1566945332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 msgr2=0x55bcfd6c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 --2- 172.21.15.78:0/1566945332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 -- 172.21.15.78:0/1566945332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcfd5b9440 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb222520640 1 --2- 172.21.15.78:0/1566945332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb212ffd640 1 -- 172.21.15.78:0/4131194174 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb20c0f0670 con 0x7fb21c102570 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb222520640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.514+0000 7fb221d1f640 1 --2- 172.21.15.78:0/1566945332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6d2140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb21800bd10 tx=0x7fb2180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac696c67c91e53e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb207fff640 1 -- 172.21.15.78:0/1566945332 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218004030 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb207fff640 1 -- 172.21.15.78:0/1566945332 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2180041d0 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb207fff640 1 -- 172.21.15.78:0/1566945332 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2180117e0 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb22635c740 1 -- 172.21.15.78:0/1566945332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 msgr2=0x55bcfd6d2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb22635c740 1 --2- 172.21.15.78:0/1566945332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6d2140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb21800bd10 tx=0x7fb2180107a0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb222520640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb2080a32d0 0x7fb2080a5790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb21400a3c0 tx=0x7fb214004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb22635c740 1 -- 172.21.15.78:0/1566945332 shutdown_connections 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb22635c740 1 --2- 172.21.15.78:0/1566945332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6d2680 0x55bcfd6d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb22635c740 1 --2- 172.21.15.78:0/1566945332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6d2140 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb22635c740 1 --2- 172.21.15.78:0/1566945332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.515+0000 7fb22635c740 1 -- 172.21.15.78:0/1566945332 >> 172.21.15.78:0/1566945332 conn(0x55bcfd6c3a30 msgr2=0x55bcfd6c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 -- 172.21.15.78:0/1566945332 shutdown_connections 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 -- 172.21.15.78:0/1566945332 wait complete. 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 -- start start 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 Processor -- start 2024-09-16T00:13:36.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6d2680 0x55bcfd6cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bcfd6d9660 con 0x55bcfd6d2680 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bcfd5e8050 con 0x55bcfd6c74a0 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.516+0000 7fb22635c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bcfd6d97e0 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb22635c740 10 client.0 _ll_register_callbacks cb 0x55bcfd628160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6d2680 0x55bcfd6cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6d2680 0x55bcfd6cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53964/0 (socket says 172.21.15.78:53964) 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 -- 172.21.15.78:0/1374389204 learned_addr learned my addr 172.21.15.78:0/1374389204 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb221d1f640 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb22151e640 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 -- 172.21.15.78:0/1374389204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c7dc0 msgr2=0x55bcfd6cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:36.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 -- 172.21.15.78:0/1374389204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 msgr2=0x55bcfd6d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6d1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 -- 172.21.15.78:0/1374389204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcfd39e3c0 con 0x55bcfd6d2680 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb222520640 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6d2680 0x55bcfd6cf780 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb214011b70 tx=0x7fb214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daebab2be0ba6fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb2057fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb2057fa640 1 -- 172.21.15.78:0/1374389204 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214022020 con 0x55bcfd6d2680 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb2057fa640 1 -- 172.21.15.78:0/1374389204 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb21401e450 con 0x55bcfd6d2680 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb22635c740 1 -- 172.21.15.78:0/1374389204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2180041d0 con 0x55bcfd6d2680 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb2057fa640 1 -- 172.21.15.78:0/1374389204 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb21402d050 con 0x55bcfd6d2680 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb22635c740 10 client.15393 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.517+0000 7fb22635c740 1 -- 172.21.15.78:0/1374389204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bcfd6e5040 con 0x55bcfd6d2680 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb2057fa640 1 -- 172.21.15.78:0/1374389204 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2180041d0 con 0x55bcfd6d2680 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb2057fa640 1 -- 172.21.15.78:0/1374389204 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb21401d710 con 0x55bcfd6d2680 2024-09-16T00:13:36.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb22635c740 10 client.15393 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb22635c740 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb22635c740 20 client.15393 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb22635c740 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55bcfd6e6500 0x55bcfd6e8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb22635c740 4 client.15393 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb22635c740 1 -- 172.21.15.78:0/1374389204 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcfd63a850 con 0x55bcfd6e6500 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.518+0000 7fb221d1f640 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55bcfd6e6500 0x55bcfd6e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.519+0000 7fb221d1f640 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55bcfd6e6500 0x55bcfd6e8910 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb1d55e019a3903 server_cookie=4318f1a528de941e in_seq=0 out_seq=0 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.519+0000 7fb2057fa640 10 client.15393 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.519+0000 7fb2057fa640 1 -- 172.21.15.78:0/1374389204 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcfd63a850 con 0x55bcfd6e6500 2024-09-16T00:13:36.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.519+0000 7fb2057fa640 10 client.15393 handle_command_reply: tid=0 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.519+0000 7fb22635c740 1 client.15393 shutdown 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.519+0000 7fb22635c740 1 -- 172.21.15.78:0/1374389204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6d2680 msgr2=0x55bcfd6cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.519+0000 7fb22635c740 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6d2680 0x55bcfd6cf780 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb214011b70 tx=0x7fb214004a40 comp rx=0 tx=0).stop 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 1 -- 172.21.15.78:0/1374389204 shutdown_connections 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55bcfd6e6500 0x55bcfd6e8910 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6d2680 0x55bcfd6cf780 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb2057fa640 0 client.15393 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 1 --2- 172.21.15.78:0/1374389204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6d1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 1 -- 172.21.15.78:0/1374389204 >> 172.21.15.78:0/1374389204 conn(0x55bcfd6c3a30 msgr2=0x55bcfd6c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 1 -- 172.21.15.78:0/1374389204 shutdown_connections 2024-09-16T00:13:36.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 1 -- 172.21.15.78:0/1374389204 wait complete. 2024-09-16T00:13:36.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.520+0000 7fb22635c740 20 client.15393 trim_cache size 0 max 16384 2024-09-16T00:13:36.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.536+0000 7fb22635c740 1 Processor -- start 2024-09-16T00:13:36.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.536+0000 7fb22635c740 1 -- start start 2024-09-16T00:13:36.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb22635c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6e28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb22635c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb22635c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb22635c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c1acae0 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb22635c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c1ae620 con 0x55bcfd6c3a30 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb22635c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c1ad1b0 con 0x55bcfd6c74a0 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6ee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53980/0 (socket says 172.21.15.78:53980) 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 -- 172.21.15.78:0/1714947391 learned_addr learned my addr 172.21.15.78:0/1714947391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb221d1f640 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6e28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb22151e640 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 -- 172.21.15.78:0/1714947391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c74a0 msgr2=0x55bcfd6ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:36.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 -- 172.21.15.78:0/1714947391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c3a30 msgr2=0x55bcfd6e28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6e28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 -- 172.21.15.78:0/1714947391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcfd6c8330 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb221d1f640 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6e28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb222520640 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6ee1e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb2140070b0 tx=0x7fb214007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fabef254335df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb2077fe640 1 -- 172.21.15.78:0/1714947391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214022040 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb2077fe640 1 -- 172.21.15.78:0/1714947391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21401d890 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.537+0000 7fb2077fe640 1 -- 172.21.15.78:0/1714947391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21401e950 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 -- 172.21.15.78:0/1714947391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 msgr2=0x55bcfd6ee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6ee1e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb2140070b0 tx=0x7fb214007eb0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 -- 172.21.15.78:0/1714947391 shutdown_connections 2024-09-16T00:13:36.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6ee1e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 --2- 172.21.15.78:0/1714947391 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6e28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 -- 172.21.15.78:0/1714947391 >> 172.21.15.78:0/1714947391 conn(0x55bcfd6cf8e0 msgr2=0x55bcfd6ca030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:36.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 -- 172.21.15.78:0/1714947391 shutdown_connections 2024-09-16T00:13:36.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 -- 172.21.15.78:0/1714947391 wait complete. 2024-09-16T00:13:36.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 -- start start 2024-09-16T00:13:36.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.538+0000 7fb22635c740 1 Processor -- start 2024-09-16T00:13:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22635c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6df320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22635c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6da650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22635c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22635c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c1adf50 con 0x55bcfd6c74a0 2024-09-16T00:13:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22635c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c1ad880 con 0x55bcfd6c7dc0 2024-09-16T00:13:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22635c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb21c1aaac0 con 0x55bcfd6c3a30 2024-09-16T00:13:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22635c740 10 client.0 _ll_register_callbacks cb 0x7fb214007cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6da650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53982/0 (socket says 172.21.15.78:53982) 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 -- 172.21.15.78:0/790345732 learned_addr learned my addr 172.21.15.78:0/790345732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb222520640 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6dad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb221d1f640 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 -- 172.21.15.78:0/790345732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c3a30 msgr2=0x55bcfd6df320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6df320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 -- 172.21.15.78:0/790345732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c7dc0 msgr2=0x55bcfd6dad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6dad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 -- 172.21.15.78:0/790345732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcfd6c72d0 con 0x55bcfd6c74a0 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb22151e640 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6da650 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0edcf0 tx=0x7fb20c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd20ada0644f27ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.539+0000 7fb2057fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb2057fa640 1 -- 172.21.15.78:0/790345732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c049040 con 0x55bcfd6c74a0 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb2057fa640 1 -- 172.21.15.78:0/790345732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb20c0b8e40 con 0x55bcfd6c74a0 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 1 -- 172.21.15.78:0/790345732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb21401d890 con 0x55bcfd6c74a0 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb2057fa640 1 -- 172.21.15.78:0/790345732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb20c0bc130 con 0x55bcfd6c74a0 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 10 client.15399 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 1 -- 172.21.15.78:0/790345732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bcfd7511f0 con 0x55bcfd6c74a0 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb2057fa640 1 -- 172.21.15.78:0/790345732 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21401d890 con 0x55bcfd6c74a0 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb2057fa640 1 -- 172.21.15.78:0/790345732 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb20c0bca30 con 0x55bcfd6c74a0 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 10 client.15399 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 20 client.15399 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55bcfd6d2680 0x55bcfd753a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 4 client.15399 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:13:36.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.540+0000 7fb22635c740 1 -- 172.21.15.78:0/790345732 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bcfd634d60 con 0x55bcfd6d2680 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb221d1f640 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55bcfd6d2680 0x55bcfd753a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb2057fa640 10 client.15399 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb221d1f640 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55bcfd6d2680 0x55bcfd753a00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e205e09257db0e server_cookie=339508102c353cd5 in_seq=0 out_seq=0 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb2057fa640 1 -- 172.21.15.78:0/790345732 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bcfd634d60 con 0x55bcfd6d2680 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb2057fa640 10 client.15399 handle_command_reply: tid=0 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 client.15399 shutdown 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 -- 172.21.15.78:0/790345732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c74a0 msgr2=0x55bcfd6da650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6da650 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb20c0edcf0 tx=0x7fb20c005c90 comp rx=0 tx=0).stop 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 -- 172.21.15.78:0/790345732 shutdown_connections 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55bcfd6d2680 0x55bcfd753a00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bcfd6c7dc0 0x55bcfd6dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bcfd6c74a0 0x55bcfd6da650 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb2057fa640 0 client.15399 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 --2- 172.21.15.78:0/790345732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bcfd6c3a30 0x55bcfd6df320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.541+0000 7fb22635c740 1 -- 172.21.15.78:0/790345732 >> 172.21.15.78:0/790345732 conn(0x55bcfd6cf8e0 msgr2=0x55bcfd6ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb22635c740 1 -- 172.21.15.78:0/790345732 shutdown_connections 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb22635c740 1 -- 172.21.15.78:0/790345732 wait complete. 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb22635c740 20 client.15399 trim_cache size 0 max 16384 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:13:36.549 INFO:teuthology.orchestra.run.smithi078.stdout: "185c8017-a15d-4587-8211-8782e5acd86e": { 2024-09-16T00:13:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:13:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "185c8017-a15d-4587-8211-8782e5acd86e", 2024-09-16T00:13:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:13:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:13:36.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:13:36.550 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 -- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb2080a32d0 msgr2=0x7fb2080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb2080a32d0 0x7fb2080a5790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb21400a3c0 tx=0x7fb214004b50 comp rx=0 tx=0).stop 2024-09-16T00:13:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 -- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 msgr2=0x7fb21c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 0x7fb21c198a70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb20c006010 tx=0x7fb20c003c10 comp rx=0 tx=0).stop 2024-09-16T00:13:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 -- 172.21.15.78:0/4131194174 shutdown_connections 2024-09-16T00:13:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb2080a32d0 0x7fb2080a5790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb21c102ef0 0x7fb21c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb21c102570 0x7fb21c198a70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 --2- 172.21.15.78:0/4131194174 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb21c100e20 0x7fb21c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 -- 172.21.15.78:0/4131194174 >> 172.21.15.78:0/4131194174 conn(0x7fb21c0fc7d0 msgr2=0x7fb21c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 -- 172.21.15.78:0/4131194174 shutdown_connections 2024-09-16T00:13:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:36.542+0000 7fb223f83640 1 -- 172.21.15.78:0/4131194174 wait complete. 2024-09-16T00:13:37.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:36 smithi193 ceph-mon[35484]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:37.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:36 smithi193 ceph-mon[35484]: pgmap v367: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-16T00:13:37.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:36 smithi078 ceph-mon[31157]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:37.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:36 smithi078 ceph-mon[31157]: pgmap v367: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-16T00:13:37.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:36 smithi148 ceph-mon[35599]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:37.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:36 smithi148 ceph-mon[35599]: pgmap v367: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-16T00:13:37.319 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-16T00:13:34.432156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669901,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1254,"num_read_kb":757,"num_write":464,"num_write_kb":3109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359460,"kb_used_data":14360,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582364,"statfs":{"total":1151940427776,"available":1151572340736,"internally_reserved":0,"allocated":14704640,"data_stored":8155061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8629,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":67,"num_write":18,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003213"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:28.691167+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:13:28.691167+0000","last_peered":"2024-09-16T00:13:28.691167+0000","last_clean":"2024-09-16T00:13:28.691167+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:13:28.691167+0000","last_undegraded":"2024-09-16T00:13:28.691167+0000","last_fullsized":"2024-09-16T00:13:28.691167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610803+0000","last_change":"2024-09-16T00:12:04.132674+0000","last_active":"2024-09-16T00:13:04.610803+0000","last_peered":"2024-09-16T00:13:04.610803+0000","last_clean":"2024-09-16T00:13:04.610803+0000","last_became_active":"2024-09-16T00:12:04.132108+0000","last_became_peered":"2024-09-16T00:12:04.132108+0000","last_unstale":"2024-09-16T00:13:04.610803+0000","last_undegraded":"2024-09-16T00:13:04.610803+0000","last_fullsized":"2024-09-16T00:13:04.610803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:10.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664962+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:13:04.664962+0000","last_peered":"2024-09-16T00:13:04.664962+0000","last_clean":"2024-09-16T00:13:04.664962+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:13:04.664962+0000","last_undegraded":"2024-09-16T00:13:04.664962+0000","last_fullsized":"2024-09-16T00:13:04.664962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663575+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:13:04.663575+0000","last_peered":"2024-09-16T00:13:04.663575+0000","last_clean":"2024-09-16T00:13:04.663575+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:13:04.663575+0000","last_undegraded":"2024-09-16T00:13:04.663575+0000","last_fullsized":"2024-09-16T00:13:04.663575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611038+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:13:04.611038+0000","last_peered":"2024-09-16T00:13:04.611038+0000","last_clean":"2024-09-16T00:13:04.611038+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:13:04.611038+0000","last_undegraded":"2024-09-16T00:13:04.611038+0000","last_fullsized":"2024-09-16T00:13:04.611038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664541+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:13:04.664541+0000","last_peered":"2024-09-16T00:13:04.664541+0000","last_clean":"2024-09-16T00:13:04.664541+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:13:04.664541+0000","last_undegraded":"2024-09-16T00:13:04.664541+0000","last_fullsized":"2024-09-16T00:13:04.664541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605363+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:13:04.605363+0000","last_peered":"2024-09-16T00:13:04.605363+0000","last_clean":"2024-09-16T00:13:04.605363+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:13:04.605363+0000","last_undegraded":"2024-09-16T00:13:04.605363+0000","last_fullsized":"2024-09-16T00:13:04.605363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611375+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:13:04.611375+0000","last_peered":"2024-09-16T00:13:04.611375+0000","last_clean":"2024-09-16T00:13:04.611375+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:13:04.611375+0000","last_undegraded":"2024-09-16T00:13:04.611375+0000","last_fullsized":"2024-09-16T00:13:04.611375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664351+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:13:04.664351+0000","last_peered":"2024-09-16T00:13:04.664351+0000","last_clean":"2024-09-16T00:13:04.664351+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:13:04.664351+0000","last_undegraded":"2024-09-16T00:13:04.664351+0000","last_fullsized":"2024-09-16T00:13:04.664351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605600+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:13:04.605600+0000","last_peered":"2024-09-16T00:13:04.605600+0000","last_clean":"2024-09-16T00:13:04.605600+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:13:04.605600+0000","last_undegraded":"2024-09-16T00:13:04.605600+0000","last_fullsized":"2024-09-16T00:13:04.605600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611103+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:13:04.611103+0000","last_peered":"2024-09-16T00:13:04.611103+0000","last_clean":"2024-09-16T00:13:04.611103+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:13:04.611103+0000","last_undegraded":"2024-09-16T00:13:04.611103+0000","last_fullsized":"2024-09-16T00:13:04.611103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649281+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:13:04.649281+0000","last_peered":"2024-09-16T00:13:04.649281+0000","last_clean":"2024-09-16T00:13:04.649281+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:13:04.649281+0000","last_undegraded":"2024-09-16T00:13:04.649281+0000","last_fullsized":"2024-09-16T00:13:04.649281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664252+0000","last_change":"2024-09-16T00:12:04.033464+0000","last_active":"2024-09-16T00:13:04.664252+0000","last_peered":"2024-09-16T00:13:04.664252+0000","last_clean":"2024-09-16T00:13:04.664252+0000","last_became_active":"2024-09-16T00:12:04.028916+0000","last_became_peered":"2024-09-16T00:12:04.028916+0000","last_unstale":"2024-09-16T00:13:04.664252+0000","last_undegraded":"2024-09-16T00:13:04.664252+0000","last_fullsized":"2024-09-16T00:13:04.664252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:47.359979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611584+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:13:04.611584+0000","last_peered":"2024-09-16T00:13:04.611584+0000","last_clean":"2024-09-16T00:13:04.611584+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:13:04.611584+0000","last_undegraded":"2024-09-16T00:13:04.611584+0000","last_fullsized":"2024-09-16T00:13:04.611584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664652+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:13:04.664652+0000","last_peered":"2024-09-16T00:13:04.664652+0000","last_clean":"2024-09-16T00:13:04.664652+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:13:04.664652+0000","last_undegraded":"2024-09-16T00:13:04.664652+0000","last_fullsized":"2024-09-16T00:13:04.664652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:15.431074+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:13:15.431074+0000","last_peered":"2024-09-16T00:13:15.431074+0000","last_clean":"2024-09-16T00:13:15.431074+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:13:15.431074+0000","last_undegraded":"2024-09-16T00:13:15.431074+0000","last_fullsized":"2024-09-16T00:13:15.431074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611391+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:13:04.611391+0000","last_peered":"2024-09-16T00:13:04.611391+0000","last_clean":"2024-09-16T00:13:04.611391+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:13:04.611391+0000","last_undegraded":"2024-09-16T00:13:04.611391+0000","last_fullsized":"2024-09-16T00:13:04.611391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663782+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:13:04.663782+0000","last_peered":"2024-09-16T00:13:04.663782+0000","last_clean":"2024-09-16T00:13:04.663782+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:13:04.663782+0000","last_undegraded":"2024-09-16T00:13:04.663782+0000","last_fullsized":"2024-09-16T00:13:04.663782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664498+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:13:04.664498+0000","last_peered":"2024-09-16T00:13:04.664498+0000","last_clean":"2024-09-16T00:13:04.664498+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:13:04.664498+0000","last_undegraded":"2024-09-16T00:13:04.664498+0000","last_fullsized":"2024-09-16T00:13:04.664498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609374+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:13:04.609374+0000","last_peered":"2024-09-16T00:13:04.609374+0000","last_clean":"2024-09-16T00:13:04.609374+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:13:04.609374+0000","last_undegraded":"2024-09-16T00:13:04.609374+0000","last_fullsized":"2024-09-16T00:13:04.609374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605426+0000","last_change":"2024-09-16T00:12:04.034002+0000","last_active":"2024-09-16T00:13:04.605426+0000","last_peered":"2024-09-16T00:13:04.605426+0000","last_clean":"2024-09-16T00:13:04.605426+0000","last_became_active":"2024-09-16T00:12:04.030959+0000","last_became_peered":"2024-09-16T00:12:04.030959+0000","last_unstale":"2024-09-16T00:13:04.605426+0000","last_undegraded":"2024-09-16T00:13:04.605426+0000","last_fullsized":"2024-09-16T00:13:04.605426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:47.730692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611324+0000","last_change":"2024-09-16T00:12:04.029906+0000","last_active":"2024-09-16T00:13:04.611324+0000","last_peered":"2024-09-16T00:13:04.611324+0000","last_clean":"2024-09-16T00:13:04.611324+0000","last_became_active":"2024-09-16T00:12:04.029259+0000","last_became_peered":"2024-09-16T00:12:04.029259+0000","last_unstale":"2024-09-16T00:13:04.611324+0000","last_undegraded":"2024-09-16T00:13:04.611324+0000","last_fullsized":"2024-09-16T00:13:04.611324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:23.399762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,1],"acting":[4,8,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663413+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:13:04.663413+0000","last_peered":"2024-09-16T00:13:04.663413+0000","last_clean":"2024-09-16T00:13:04.663413+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:13:04.663413+0000","last_undegraded":"2024-09-16T00:13:04.663413+0000","last_fullsized":"2024-09-16T00:13:04.663413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611856+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:13:04.611856+0000","last_peered":"2024-09-16T00:13:04.611856+0000","last_clean":"2024-09-16T00:13:04.611856+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:13:04.611856+0000","last_undegraded":"2024-09-16T00:13:04.611856+0000","last_fullsized":"2024-09-16T00:13:04.611856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610909+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:13:04.610909+0000","last_peered":"2024-09-16T00:13:04.610909+0000","last_clean":"2024-09-16T00:13:04.610909+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:13:04.610909+0000","last_undegraded":"2024-09-16T00:13:04.610909+0000","last_fullsized":"2024-09-16T00:13:04.610909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.665046+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:13:04.665046+0000","last_peered":"2024-09-16T00:13:04.665046+0000","last_clean":"2024-09-16T00:13:04.665046+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:13:04.665046+0000","last_undegraded":"2024-09-16T00:13:04.665046+0000","last_fullsized":"2024-09-16T00:13:04.665046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'21","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.652660+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:13:25.652660+0000","last_peered":"2024-09-16T00:13:25.652660+0000","last_clean":"2024-09-16T00:13:25.652660+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:13:25.652660+0000","last_undegraded":"2024-09-16T00:13:25.652660+0000","last_fullsized":"2024-09-16T00:13:25.652660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664101+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:13:04.664101+0000","last_peered":"2024-09-16T00:13:04.664101+0000","last_clean":"2024-09-16T00:13:04.664101+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:13:04.664101+0000","last_undegraded":"2024-09-16T00:13:04.664101+0000","last_fullsized":"2024-09-16T00:13:04.664101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606866+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:13:04.606866+0000","last_peered":"2024-09-16T00:13:04.606866+0000","last_clean":"2024-09-16T00:13:04.606866+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:13:04.606866+0000","last_undegraded":"2024-09-16T00:13:04.606866+0000","last_fullsized":"2024-09-16T00:13:04.606866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664634+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:13:04.664634+0000","last_peered":"2024-09-16T00:13:04.664634+0000","last_clean":"2024-09-16T00:13:04.664634+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:13:04.664634+0000","last_undegraded":"2024-09-16T00:13:04.664634+0000","last_fullsized":"2024-09-16T00:13:04.664634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664409+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:13:04.664409+0000","last_peered":"2024-09-16T00:13:04.664409+0000","last_clean":"2024-09-16T00:13:04.664409+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:13:04.664409+0000","last_undegraded":"2024-09-16T00:13:04.664409+0000","last_fullsized":"2024-09-16T00:13:04.664409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664046+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:13:04.664046+0000","last_peered":"2024-09-16T00:13:04.664046+0000","last_clean":"2024-09-16T00:13:04.664046+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:13:04.664046+0000","last_undegraded":"2024-09-16T00:13:04.664046+0000","last_fullsized":"2024-09-16T00:13:04.664046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609377+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:13:04.609377+0000","last_peered":"2024-09-16T00:13:04.609377+0000","last_clean":"2024-09-16T00:13:04.609377+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:13:04.609377+0000","last_undegraded":"2024-09-16T00:13:04.609377+0000","last_fullsized":"2024-09-16T00:13:04.609377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664159+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:13:04.664159+0000","last_peered":"2024-09-16T00:13:04.664159+0000","last_clean":"2024-09-16T00:13:04.664159+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:13:04.664159+0000","last_undegraded":"2024-09-16T00:13:04.664159+0000","last_fullsized":"2024-09-16T00:13:04.664159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664057+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:13:04.664057+0000","last_peered":"2024-09-16T00:13:04.664057+0000","last_clean":"2024-09-16T00:13:04.664057+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:13:04.664057+0000","last_undegraded":"2024-09-16T00:13:04.664057+0000","last_fullsized":"2024-09-16T00:13:04.664057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:31.694904+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:13:31.694904+0000","last_peered":"2024-09-16T00:13:31.694904+0000","last_clean":"2024-09-16T00:13:31.694904+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:13:31.694904+0000","last_undegraded":"2024-09-16T00:13:31.694904+0000","last_fullsized":"2024-09-16T00:13:31.694904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606730+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:13:04.606730+0000","last_peered":"2024-09-16T00:13:04.606730+0000","last_clean":"2024-09-16T00:13:04.606730+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:13:04.606730+0000","last_undegraded":"2024-09-16T00:13:04.606730+0000","last_fullsized":"2024-09-16T00:13:04.606730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605309+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:13:04.605309+0000","last_peered":"2024-09-16T00:13:04.605309+0000","last_clean":"2024-09-16T00:13:04.605309+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:13:04.605309+0000","last_undegraded":"2024-09-16T00:13:04.605309+0000","last_fullsized":"2024-09-16T00:13:04.605309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:31.738965+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:13:31.738965+0000","last_peered":"2024-09-16T00:13:31.738965+0000","last_clean":"2024-09-16T00:13:31.738965+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:13:31.738965+0000","last_undegraded":"2024-09-16T00:13:31.738965+0000","last_fullsized":"2024-09-16T00:13:31.738965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606979+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:13:04.606979+0000","last_peered":"2024-09-16T00:13:04.606979+0000","last_clean":"2024-09-16T00:13:04.606979+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:13:04.606979+0000","last_undegraded":"2024-09-16T00:13:04.606979+0000","last_fullsized":"2024-09-16T00:13:04.606979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606488+0000","last_change":"2024-09-16T00:12:04.027859+0000","last_active":"2024-09-16T00:13:04.606488+0000","last_peered":"2024-09-16T00:13:04.606488+0000","last_clean":"2024-09-16T00:13:04.606488+0000","last_became_active":"2024-09-16T00:12:04.027506+0000","last_became_peered":"2024-09-16T00:12:04.027506+0000","last_unstale":"2024-09-16T00:13:04.606488+0000","last_undegraded":"2024-09-16T00:13:04.606488+0000","last_fullsized":"2024-09-16T00:13:04.606488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:34.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649981+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:13:04.649981+0000","last_peered":"2024-09-16T00:13:04.649981+0000","last_clean":"2024-09-16T00:13:04.649981+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:13:04.649981+0000","last_undegraded":"2024-09-16T00:13:04.649981+0000","last_fullsized":"2024-09-16T00:13:04.649981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605678+0000","last_change":"2024-09-16T00:12:04.029471+0000","last_active":"2024-09-16T00:13:04.605678+0000","last_peered":"2024-09-16T00:13:04.605678+0000","last_clean":"2024-09-16T00:13:04.605678+0000","last_became_active":"2024-09-16T00:12:04.029054+0000","last_became_peered":"2024-09-16T00:12:04.029054+0000","last_unstale":"2024-09-16T00:13:04.605678+0000","last_undegraded":"2024-09-16T00:13:04.605678+0000","last_fullsized":"2024-09-16T00:13:04.605678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:52.054218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664553+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:13:04.664553+0000","last_peered":"2024-09-16T00:13:04.664553+0000","last_clean":"2024-09-16T00:13:04.664553+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:13:04.664553+0000","last_undegraded":"2024-09-16T00:13:04.664553+0000","last_fullsized":"2024-09-16T00:13:04.664553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611832+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:13:04.611832+0000","last_peered":"2024-09-16T00:13:04.611832+0000","last_clean":"2024-09-16T00:13:04.611832+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:13:04.611832+0000","last_undegraded":"2024-09-16T00:13:04.611832+0000","last_fullsized":"2024-09-16T00:13:04.611832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649917+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:13:04.649917+0000","last_peered":"2024-09-16T00:13:04.649917+0000","last_clean":"2024-09-16T00:13:04.649917+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:13:04.649917+0000","last_undegraded":"2024-09-16T00:13:04.649917+0000","last_fullsized":"2024-09-16T00:13:04.649917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650098+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:13:04.650098+0000","last_peered":"2024-09-16T00:13:04.650098+0000","last_clean":"2024-09-16T00:13:04.650098+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:13:04.650098+0000","last_undegraded":"2024-09-16T00:13:04.650098+0000","last_fullsized":"2024-09-16T00:13:04.650098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649329+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:13:04.649329+0000","last_peered":"2024-09-16T00:13:04.649329+0000","last_clean":"2024-09-16T00:13:04.649329+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:13:04.649329+0000","last_undegraded":"2024-09-16T00:13:04.649329+0000","last_fullsized":"2024-09-16T00:13:04.649329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:28.701680+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:13:28.701680+0000","last_peered":"2024-09-16T00:13:28.701680+0000","last_clean":"2024-09-16T00:13:28.701680+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:13:28.701680+0000","last_undegraded":"2024-09-16T00:13:28.701680+0000","last_fullsized":"2024-09-16T00:13:28.701680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605291+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:13:04.605291+0000","last_peered":"2024-09-16T00:13:04.605291+0000","last_clean":"2024-09-16T00:13:04.605291+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:13:04.605291+0000","last_undegraded":"2024-09-16T00:13:04.605291+0000","last_fullsized":"2024-09-16T00:13:04.605291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649345+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:13:04.649345+0000","last_peered":"2024-09-16T00:13:04.649345+0000","last_clean":"2024-09-16T00:13:04.649345+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:13:04.649345+0000","last_undegraded":"2024-09-16T00:13:04.649345+0000","last_fullsized":"2024-09-16T00:13:04.649345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605828+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:13:04.605828+0000","last_peered":"2024-09-16T00:13:04.605828+0000","last_clean":"2024-09-16T00:13:04.605828+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:13:04.605828+0000","last_undegraded":"2024-09-16T00:13:04.605828+0000","last_fullsized":"2024-09-16T00:13:04.605828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663968+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:13:04.663968+0000","last_peered":"2024-09-16T00:13:04.663968+0000","last_clean":"2024-09-16T00:13:04.663968+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:13:04.663968+0000","last_undegraded":"2024-09-16T00:13:04.663968+0000","last_fullsized":"2024-09-16T00:13:04.663968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650076+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:13:04.650076+0000","last_peered":"2024-09-16T00:13:04.650076+0000","last_clean":"2024-09-16T00:13:04.650076+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:13:04.650076+0000","last_undegraded":"2024-09-16T00:13:04.650076+0000","last_fullsized":"2024-09-16T00:13:04.650076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.607017+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:13:04.607017+0000","last_peered":"2024-09-16T00:13:04.607017+0000","last_clean":"2024-09-16T00:13:04.607017+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:13:04.607017+0000","last_undegraded":"2024-09-16T00:13:04.607017+0000","last_fullsized":"2024-09-16T00:13:04.607017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605023+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:13:04.605023+0000","last_peered":"2024-09-16T00:13:04.605023+0000","last_clean":"2024-09-16T00:13:04.605023+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:13:04.605023+0000","last_undegraded":"2024-09-16T00:13:04.605023+0000","last_fullsized":"2024-09-16T00:13:04.605023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663799+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:13:04.663799+0000","last_peered":"2024-09-16T00:13:04.663799+0000","last_clean":"2024-09-16T00:13:04.663799+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:13:04.663799+0000","last_undegraded":"2024-09-16T00:13:04.663799+0000","last_fullsized":"2024-09-16T00:13:04.663799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:19.532889+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:13:19.532889+0000","last_peered":"2024-09-16T00:13:19.532889+0000","last_clean":"2024-09-16T00:13:19.532889+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:13:19.532889+0000","last_undegraded":"2024-09-16T00:13:19.532889+0000","last_fullsized":"2024-09-16T00:13:19.532889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605103+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:13:04.605103+0000","last_peered":"2024-09-16T00:13:04.605103+0000","last_clean":"2024-09-16T00:13:04.605103+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:13:04.605103+0000","last_undegraded":"2024-09-16T00:13:04.605103+0000","last_fullsized":"2024-09-16T00:13:04.605103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605218+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:13:04.605218+0000","last_peered":"2024-09-16T00:13:04.605218+0000","last_clean":"2024-09-16T00:13:04.605218+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:13:04.605218+0000","last_undegraded":"2024-09-16T00:13:04.605218+0000","last_fullsized":"2024-09-16T00:13:04.605218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664161+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:13:04.664161+0000","last_peered":"2024-09-16T00:13:04.664161+0000","last_clean":"2024-09-16T00:13:04.664161+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:13:04.664161+0000","last_undegraded":"2024-09-16T00:13:04.664161+0000","last_fullsized":"2024-09-16T00:13:04.664161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606964+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:13:04.606964+0000","last_peered":"2024-09-16T00:13:04.606964+0000","last_clean":"2024-09-16T00:13:04.606964+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:13:04.606964+0000","last_undegraded":"2024-09-16T00:13:04.606964+0000","last_fullsized":"2024-09-16T00:13:04.606964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649323+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:13:04.649323+0000","last_peered":"2024-09-16T00:13:04.649323+0000","last_clean":"2024-09-16T00:13:04.649323+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:13:04.649323+0000","last_undegraded":"2024-09-16T00:13:04.649323+0000","last_fullsized":"2024-09-16T00:13:04.649323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:31.739511+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:13:31.739511+0000","last_peered":"2024-09-16T00:13:31.739511+0000","last_clean":"2024-09-16T00:13:31.739511+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:13:31.739511+0000","last_undegraded":"2024-09-16T00:13:31.739511+0000","last_fullsized":"2024-09-16T00:13:31.739511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609415+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:13:04.609415+0000","last_peered":"2024-09-16T00:13:04.609415+0000","last_clean":"2024-09-16T00:13:04.609415+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:13:04.609415+0000","last_undegraded":"2024-09-16T00:13:04.609415+0000","last_fullsized":"2024-09-16T00:13:04.609415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649661+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:13:04.649661+0000","last_peered":"2024-09-16T00:13:04.649661+0000","last_clean":"2024-09-16T00:13:04.649661+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:13:04.649661+0000","last_undegraded":"2024-09-16T00:13:04.649661+0000","last_fullsized":"2024-09-16T00:13:04.649661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605440+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:13:04.605440+0000","last_peered":"2024-09-16T00:13:04.605440+0000","last_clean":"2024-09-16T00:13:04.605440+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:13:04.605440+0000","last_undegraded":"2024-09-16T00:13:04.605440+0000","last_fullsized":"2024-09-16T00:13:04.605440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664603+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:13:04.664603+0000","last_peered":"2024-09-16T00:13:04.664603+0000","last_clean":"2024-09-16T00:13:04.664603+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:13:04.664603+0000","last_undegraded":"2024-09-16T00:13:04.664603+0000","last_fullsized":"2024-09-16T00:13:04.664603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606631+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:13:04.606631+0000","last_peered":"2024-09-16T00:13:04.606631+0000","last_clean":"2024-09-16T00:13:04.606631+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:13:04.606631+0000","last_undegraded":"2024-09-16T00:13:04.606631+0000","last_fullsized":"2024-09-16T00:13:04.606631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611195+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:13:04.611195+0000","last_peered":"2024-09-16T00:13:04.611195+0000","last_clean":"2024-09-16T00:13:04.611195+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:13:04.611195+0000","last_undegraded":"2024-09-16T00:13:04.611195+0000","last_fullsized":"2024-09-16T00:13:04.611195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605540+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:13:04.605540+0000","last_peered":"2024-09-16T00:13:04.605540+0000","last_clean":"2024-09-16T00:13:04.605540+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:13:04.605540+0000","last_undegraded":"2024-09-16T00:13:04.605540+0000","last_fullsized":"2024-09-16T00:13:04.605540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664419+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:13:04.664419+0000","last_peered":"2024-09-16T00:13:04.664419+0000","last_clean":"2024-09-16T00:13:04.664419+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:13:04.664419+0000","last_undegraded":"2024-09-16T00:13:04.664419+0000","last_fullsized":"2024-09-16T00:13:04.664419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'42","reported_seq":308,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:29.702921+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:13:29.702921+0000","last_peered":"2024-09-16T00:13:29.702921+0000","last_clean":"2024-09-16T00:13:29.702921+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:13:29.702921+0000","last_undegraded":"2024-09-16T00:13:29.702921+0000","last_fullsized":"2024-09-16T00:13:29.702921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70715,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":227,"num_read_kb":182,"num_write":44,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609072+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:13:04.609072+0000","last_peered":"2024-09-16T00:13:04.609072+0000","last_clean":"2024-09-16T00:13:04.609072+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:13:04.609072+0000","last_undegraded":"2024-09-16T00:13:04.609072+0000","last_fullsized":"2024-09-16T00:13:04.609072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605601+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:13:04.605601+0000","last_peered":"2024-09-16T00:13:04.605601+0000","last_clean":"2024-09-16T00:13:04.605601+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:13:04.605601+0000","last_undegraded":"2024-09-16T00:13:04.605601+0000","last_fullsized":"2024-09-16T00:13:04.605601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605792+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:13:04.605792+0000","last_peered":"2024-09-16T00:13:04.605792+0000","last_clean":"2024-09-16T00:13:04.605792+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:13:04.605792+0000","last_undegraded":"2024-09-16T00:13:04.605792+0000","last_fullsized":"2024-09-16T00:13:04.605792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611640+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:13:04.611640+0000","last_peered":"2024-09-16T00:13:04.611640+0000","last_clean":"2024-09-16T00:13:04.611640+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:13:04.611640+0000","last_undegraded":"2024-09-16T00:13:04.611640+0000","last_fullsized":"2024-09-16T00:13:04.611640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663456+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:13:04.663456+0000","last_peered":"2024-09-16T00:13:04.663456+0000","last_clean":"2024-09-16T00:13:04.663456+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:13:04.663456+0000","last_undegraded":"2024-09-16T00:13:04.663456+0000","last_fullsized":"2024-09-16T00:13:04.663456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650055+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:13:04.650055+0000","last_peered":"2024-09-16T00:13:04.650055+0000","last_clean":"2024-09-16T00:13:04.650055+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:13:04.650055+0000","last_undegraded":"2024-09-16T00:13:04.650055+0000","last_fullsized":"2024-09-16T00:13:04.650055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.608894+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:13:04.608894+0000","last_peered":"2024-09-16T00:13:04.608894+0000","last_clean":"2024-09-16T00:13:04.608894+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:13:04.608894+0000","last_undegraded":"2024-09-16T00:13:04.608894+0000","last_fullsized":"2024-09-16T00:13:04.608894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605753+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:13:04.605753+0000","last_peered":"2024-09-16T00:13:04.605753+0000","last_clean":"2024-09-16T00:13:04.605753+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:13:04.605753+0000","last_undegraded":"2024-09-16T00:13:04.605753+0000","last_fullsized":"2024-09-16T00:13:04.605753+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":259,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:29.702699+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:13:29.702699+0000","last_peered":"2024-09-16T00:13:29.702699+0000","last_clean":"2024-09-16T00:13:29.702699+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:13:29.702699+0000","last_undegraded":"2024-09-16T00:13:29.702699+0000","last_fullsized":"2024-09-16T00:13:29.702699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664396+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:13:04.664396+0000","last_peered":"2024-09-16T00:13:04.664396+0000","last_clean":"2024-09-16T00:13:04.664396+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:13:04.664396+0000","last_undegraded":"2024-09-16T00:13:04.664396+0000","last_fullsized":"2024-09-16T00:13:04.664396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663978+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:13:04.663978+0000","last_peered":"2024-09-16T00:13:04.663978+0000","last_clean":"2024-09-16T00:13:04.663978+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:13:04.663978+0000","last_undegraded":"2024-09-16T00:13:04.663978+0000","last_fullsized":"2024-09-16T00:13:04.663978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609055+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:13:04.609055+0000","last_peered":"2024-09-16T00:13:04.609055+0000","last_clean":"2024-09-16T00:13:04.609055+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:13:04.609055+0000","last_undegraded":"2024-09-16T00:13:04.609055+0000","last_fullsized":"2024-09-16T00:13:04.609055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605507+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:13:04.605507+0000","last_peered":"2024-09-16T00:13:04.605507+0000","last_clean":"2024-09-16T00:13:04.605507+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:13:04.605507+0000","last_undegraded":"2024-09-16T00:13:04.605507+0000","last_fullsized":"2024-09-16T00:13:04.605507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664904+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:13:04.664904+0000","last_peered":"2024-09-16T00:13:04.664904+0000","last_clean":"2024-09-16T00:13:04.664904+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:13:04.664904+0000","last_undegraded":"2024-09-16T00:13:04.664904+0000","last_fullsized":"2024-09-16T00:13:04.664904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663729+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:13:04.663729+0000","last_peered":"2024-09-16T00:13:04.663729+0000","last_clean":"2024-09-16T00:13:04.663729+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:13:04.663729+0000","last_undegraded":"2024-09-16T00:13:04.663729+0000","last_fullsized":"2024-09-16T00:13:04.663729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606517+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:13:04.606517+0000","last_peered":"2024-09-16T00:13:04.606517+0000","last_clean":"2024-09-16T00:13:04.606517+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:13:04.606517+0000","last_undegraded":"2024-09-16T00:13:04.606517+0000","last_fullsized":"2024-09-16T00:13:04.606517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664780+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:13:04.664780+0000","last_peered":"2024-09-16T00:13:04.664780+0000","last_clean":"2024-09-16T00:13:04.664780+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:13:04.664780+0000","last_undegraded":"2024-09-16T00:13:04.664780+0000","last_fullsized":"2024-09-16T00:13:04.664780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605384+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:13:04.605384+0000","last_peered":"2024-09-16T00:13:04.605384+0000","last_clean":"2024-09-16T00:13:04.605384+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:13:04.605384+0000","last_undegraded":"2024-09-16T00:13:04.605384+0000","last_fullsized":"2024-09-16T00:13:04.605384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610913+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:13:04.610913+0000","last_peered":"2024-09-16T00:13:04.610913+0000","last_clean":"2024-09-16T00:13:04.610913+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:13:04.610913+0000","last_undegraded":"2024-09-16T00:13:04.610913+0000","last_fullsized":"2024-09-16T00:13:04.610913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:29.703460+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:13:29.703460+0000","last_peered":"2024-09-16T00:13:29.703460+0000","last_clean":"2024-09-16T00:13:29.703460+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:13:29.703460+0000","last_undegraded":"2024-09-16T00:13:29.703460+0000","last_fullsized":"2024-09-16T00:13:29.703460+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663709+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:13:04.663709+0000","last_peered":"2024-09-16T00:13:04.663709+0000","last_clean":"2024-09-16T00:13:04.663709+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:13:04.663709+0000","last_undegraded":"2024-09-16T00:13:04.663709+0000","last_fullsized":"2024-09-16T00:13:04.663709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611136+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:13:04.611136+0000","last_peered":"2024-09-16T00:13:04.611136+0000","last_clean":"2024-09-16T00:13:04.611136+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:13:04.611136+0000","last_undegraded":"2024-09-16T00:13:04.611136+0000","last_fullsized":"2024-09-16T00:13:04.611136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663783+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:13:04.663783+0000","last_peered":"2024-09-16T00:13:04.663783+0000","last_clean":"2024-09-16T00:13:04.663783+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:13:04.663783+0000","last_undegraded":"2024-09-16T00:13:04.663783+0000","last_fullsized":"2024-09-16T00:13:04.663783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611426+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:13:04.611426+0000","last_peered":"2024-09-16T00:13:04.611426+0000","last_clean":"2024-09-16T00:13:04.611426+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:13:04.611426+0000","last_undegraded":"2024-09-16T00:13:04.611426+0000","last_fullsized":"2024-09-16T00:13:04.611426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610934+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:13:04.610934+0000","last_peered":"2024-09-16T00:13:04.610934+0000","last_clean":"2024-09-16T00:13:04.610934+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:13:04.610934+0000","last_undegraded":"2024-09-16T00:13:04.610934+0000","last_fullsized":"2024-09-16T00:13:04.610934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663979+0000","last_change":"2024-09-16T00:12:04.029280+0000","last_active":"2024-09-16T00:13:04.663979+0000","last_peered":"2024-09-16T00:13:04.663979+0000","last_clean":"2024-09-16T00:13:04.663979+0000","last_became_active":"2024-09-16T00:12:04.028973+0000","last_became_peered":"2024-09-16T00:12:04.028973+0000","last_unstale":"2024-09-16T00:13:04.663979+0000","last_undegraded":"2024-09-16T00:13:04.663979+0000","last_fullsized":"2024-09-16T00:13:04.663979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606846+0000","last_change":"2024-09-16T00:12:04.028319+0000","last_active":"2024-09-16T00:13:04.606846+0000","last_peered":"2024-09-16T00:13:04.606846+0000","last_clean":"2024-09-16T00:13:04.606846+0000","last_became_active":"2024-09-16T00:12:04.027985+0000","last_became_peered":"2024-09-16T00:12:04.027985+0000","last_unstale":"2024-09-16T00:13:04.606846+0000","last_undegraded":"2024-09-16T00:13:04.606846+0000","last_fullsized":"2024-09-16T00:13:04.606846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.119428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663790+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:13:04.663790+0000","last_peered":"2024-09-16T00:13:04.663790+0000","last_clean":"2024-09-16T00:13:04.663790+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:13:04.663790+0000","last_undegraded":"2024-09-16T00:13:04.663790+0000","last_fullsized":"2024-09-16T00:13:04.663790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611369+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:13:04.611369+0000","last_peered":"2024-09-16T00:13:04.611369+0000","last_clean":"2024-09-16T00:13:04.611369+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:13:04.611369+0000","last_undegraded":"2024-09-16T00:13:04.611369+0000","last_fullsized":"2024-09-16T00:13:04.611369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.616864+0000","last_change":"2024-09-16T00:13:04.612598+0000","last_active":"2024-09-16T00:13:04.616864+0000","last_peered":"2024-09-16T00:13:04.616864+0000","last_clean":"2024-09-16T00:13:04.616864+0000","last_became_active":"2024-09-16T00:13:04.612310+0000","last_became_peered":"2024-09-16T00:13:04.612310+0000","last_unstale":"2024-09-16T00:13:04.616864+0000","last_undegraded":"2024-09-16T00:13:04.616864+0000","last_fullsized":"2024-09-16T00:13:04.616864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:59.381944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605240+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:13:04.605240+0000","last_peered":"2024-09-16T00:13:04.605240+0000","last_clean":"2024-09-16T00:13:04.605240+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:13:04.605240+0000","last_undegraded":"2024-09-16T00:13:04.605240+0000","last_fullsized":"2024-09-16T00:13:04.605240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663870+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:13:04.663870+0000","last_peered":"2024-09-16T00:13:04.663870+0000","last_clean":"2024-09-16T00:13:04.663870+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:13:04.663870+0000","last_undegraded":"2024-09-16T00:13:04.663870+0000","last_fullsized":"2024-09-16T00:13:04.663870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605535+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:13:04.605535+0000","last_peered":"2024-09-16T00:13:04.605535+0000","last_clean":"2024-09-16T00:13:04.605535+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:13:04.605535+0000","last_undegraded":"2024-09-16T00:13:04.605535+0000","last_fullsized":"2024-09-16T00:13:04.605535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606766+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:13:04.606766+0000","last_peered":"2024-09-16T00:13:04.606766+0000","last_clean":"2024-09-16T00:13:04.606766+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:13:04.606766+0000","last_undegraded":"2024-09-16T00:13:04.606766+0000","last_fullsized":"2024-09-16T00:13:04.606766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663880+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:13:04.663880+0000","last_peered":"2024-09-16T00:13:04.663880+0000","last_clean":"2024-09-16T00:13:04.663880+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:13:04.663880+0000","last_undegraded":"2024-09-16T00:13:04.663880+0000","last_fullsized":"2024-09-16T00:13:04.663880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611528+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:13:04.611528+0000","last_peered":"2024-09-16T00:13:04.611528+0000","last_clean":"2024-09-16T00:13:04.611528+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:13:04.611528+0000","last_undegraded":"2024-09-16T00:13:04.611528+0000","last_fullsized":"2024-09-16T00:13:04.611528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609303+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:13:04.609303+0000","last_peered":"2024-09-16T00:13:04.609303+0000","last_clean":"2024-09-16T00:13:04.609303+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:13:04.609303+0000","last_undegraded":"2024-09-16T00:13:04.609303+0000","last_fullsized":"2024-09-16T00:13:04.609303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663938+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:13:04.663938+0000","last_peered":"2024-09-16T00:13:04.663938+0000","last_clean":"2024-09-16T00:13:04.663938+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:13:04.663938+0000","last_undegraded":"2024-09-16T00:13:04.663938+0000","last_fullsized":"2024-09-16T00:13:04.663938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605284+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:13:04.605284+0000","last_peered":"2024-09-16T00:13:04.605284+0000","last_clean":"2024-09-16T00:13:04.605284+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:13:04.605284+0000","last_undegraded":"2024-09-16T00:13:04.605284+0000","last_fullsized":"2024-09-16T00:13:04.605284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663375+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:13:04.663375+0000","last_peered":"2024-09-16T00:13:04.663375+0000","last_clean":"2024-09-16T00:13:04.663375+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:13:04.663375+0000","last_undegraded":"2024-09-16T00:13:04.663375+0000","last_fullsized":"2024-09-16T00:13:04.663375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663627+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:13:04.663627+0000","last_peered":"2024-09-16T00:13:04.663627+0000","last_clean":"2024-09-16T00:13:04.663627+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:13:04.663627+0000","last_undegraded":"2024-09-16T00:13:04.663627+0000","last_fullsized":"2024-09-16T00:13:04.663627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606741+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:13:04.606741+0000","last_peered":"2024-09-16T00:13:04.606741+0000","last_clean":"2024-09-16T00:13:04.606741+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:13:04.606741+0000","last_undegraded":"2024-09-16T00:13:04.606741+0000","last_fullsized":"2024-09-16T00:13:04.606741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611683+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:13:04.611683+0000","last_peered":"2024-09-16T00:13:04.611683+0000","last_clean":"2024-09-16T00:13:04.611683+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:13:04.611683+0000","last_undegraded":"2024-09-16T00:13:04.611683+0000","last_fullsized":"2024-09-16T00:13:04.611683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649341+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:13:04.649341+0000","last_peered":"2024-09-16T00:13:04.649341+0000","last_clean":"2024-09-16T00:13:04.649341+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:13:04.649341+0000","last_undegraded":"2024-09-16T00:13:04.649341+0000","last_fullsized":"2024-09-16T00:13:04.649341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605811+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:13:04.605811+0000","last_peered":"2024-09-16T00:13:04.605811+0000","last_clean":"2024-09-16T00:13:04.605811+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:13:04.605811+0000","last_undegraded":"2024-09-16T00:13:04.605811+0000","last_fullsized":"2024-09-16T00:13:04.605811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663843+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:13:04.663843+0000","last_peered":"2024-09-16T00:13:04.663843+0000","last_clean":"2024-09-16T00:13:04.663843+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:13:04.663843+0000","last_undegraded":"2024-09-16T00:13:04.663843+0000","last_fullsized":"2024-09-16T00:13:04.663843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649554+0000","last_change":"2024-09-16T00:12:04.028342+0000","last_active":"2024-09-16T00:13:04.649554+0000","last_peered":"2024-09-16T00:13:04.649554+0000","last_clean":"2024-09-16T00:13:04.649554+0000","last_became_active":"2024-09-16T00:12:04.027889+0000","last_became_peered":"2024-09-16T00:12:04.027889+0000","last_unstale":"2024-09-16T00:13:04.649554+0000","last_undegraded":"2024-09-16T00:13:04.649554+0000","last_fullsized":"2024-09-16T00:13:04.649554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:47.840428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663964+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:13:04.663964+0000","last_peered":"2024-09-16T00:13:04.663964+0000","last_clean":"2024-09-16T00:13:04.663964+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:13:04.663964+0000","last_undegraded":"2024-09-16T00:13:04.663964+0000","last_fullsized":"2024-09-16T00:13:04.663964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663878+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:13:04.663878+0000","last_peered":"2024-09-16T00:13:04.663878+0000","last_clean":"2024-09-16T00:13:04.663878+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:13:04.663878+0000","last_undegraded":"2024-09-16T00:13:04.663878+0000","last_fullsized":"2024-09-16T00:13:04.663878+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611689+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:13:04.611689+0000","last_peered":"2024-09-16T00:13:04.611689+0000","last_clean":"2024-09-16T00:13:04.611689+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:13:04.611689+0000","last_undegraded":"2024-09-16T00:13:04.611689+0000","last_fullsized":"2024-09-16T00:13:04.611689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664072+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:13:04.664072+0000","last_peered":"2024-09-16T00:13:04.664072+0000","last_clean":"2024-09-16T00:13:04.664072+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:13:04.664072+0000","last_undegraded":"2024-09-16T00:13:04.664072+0000","last_fullsized":"2024-09-16T00:13:04.664072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664019+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:13:04.664019+0000","last_peered":"2024-09-16T00:13:04.664019+0000","last_clean":"2024-09-16T00:13:04.664019+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:13:04.664019+0000","last_undegraded":"2024-09-16T00:13:04.664019+0000","last_fullsized":"2024-09-16T00:13:04.664019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79299,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1128,"num_read_kb":648,"num_write":281,"num_write_kb":169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":237897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":239,"ondisk_log_size":239,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579948,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":515263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1744896,"data_stored":1173187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201339,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430272,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1112771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593797,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822730,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36972,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708180,"statfs":{"total":95995035648,"available":95957176320,"internally_reserved":0,"allocated":1667072,"data_stored":1107033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986256,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":515164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215189,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":509871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":582970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":582301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":512153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":72860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:13:37.325 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:13:37.723 INFO:tasks.cephfs.filesystem:scrub status for tag:185c8017-a15d-4587-8211-8782e5acd86e - {'path': '/', 'tag': '185c8017-a15d-4587-8211-8782e5acd86e', 'options': 'recursive,force'} 2024-09-16T00:13:37.805 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 -- 172.21.15.78:0/3576745438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 msgr2=0x7f1d9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/3576745438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 0x7f1d9c10ea90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a410 comp rx=0 tx=0).stop 2024-09-16T00:13:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 -- 172.21.15.78:0/3576745438 shutdown_connections 2024-09-16T00:13:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/3576745438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 0x7f1d9c10ea90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/3576745438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c102790 0x7f1d9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/3576745438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d9c101de0 0x7f1d9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 -- 172.21.15.78:0/3576745438 >> 172.21.15.78:0/3576745438 conn(0x7f1d9c0fd8c0 msgr2=0x7f1d9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:38.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 -- 172.21.15.78:0/3576745438 shutdown_connections 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.330+0000 7f1da1ba8640 1 -- 172.21.15.78:0/3576745438 wait complete. 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1da1ba8640 1 Processor -- start 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1da1ba8640 1 -- start start 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1da1ba8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d9c101de0 0x7f1d9c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1da1ba8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c102790 0x7f1d9c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1da1ba8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1da1ba8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c111f70 con 0x7f1d9c106f10 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1da1ba8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c111df0 con 0x7f1d9c101de0 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1da1ba8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1120f0 con 0x7f1d9c102790 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1d9bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:38.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1d9bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54000/0 (socket says 172.21.15.78:54000) 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1d9bfff640 1 -- 172.21.15.78:0/1152409177 learned_addr learned my addr 172.21.15.78:0/1152409177 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.331+0000 7f1d9affd640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c102790 0x7f1d9c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d9b7fe640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d9c101de0 0x7f1d9c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d9bfff640 1 -- 172.21.15.78:0/1152409177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c102790 msgr2=0x7f1d9c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d9bfff640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c102790 0x7f1d9c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d9bfff640 1 -- 172.21.15.78:0/1152409177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d9c101de0 msgr2=0x7f1d9c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d9bfff640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d9c101de0 0x7f1d9c19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d9bfff640 1 -- 172.21.15.78:0/1152409177 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c19f500 con 0x7f1d9c106f10 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d9bfff640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19ee20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005ab0 tx=0x7f1d940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d98ff9640 1 -- 172.21.15.78:0/1152409177 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9403f030 con 0x7f1d9c106f10 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d98ff9640 1 -- 172.21.15.78:0/1152409177 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d94003ce0 con 0x7f1d9c106f10 2024-09-16T00:13:38.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c19f790 con 0x7f1d9c106f10 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1d98ff9640 1 -- 172.21.15.78:0/1152409177 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94035070 con 0x7f1d9c106f10 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.332+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c1a70d0 con 0x7f1d9c106f10 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.333+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d9c1032c0 con 0x7f1d9c106f10 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.334+0000 7f1d98ff9640 1 -- 172.21.15.78:0/1152409177 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f1d94042030 con 0x7f1d9c106f10 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.335+0000 7f1d98ff9640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1d840a32d0 0x7f1d840a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.335+0000 7f1d9b7fe640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1d840a32d0 0x7f1d840a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.335+0000 7f1d98ff9640 1 -- 172.21.15.78:0/1152409177 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1d940f06c0 con 0x7f1d9c106f10 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.335+0000 7f1d9b7fe640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1d840a32d0 0x7f1d840a5790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c008d20 tx=0x7f1d8c0082b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:38.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.339+0000 7f1d98ff9640 1 -- 172.21.15.78:0/1152409177 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d940bcf20 con 0x7f1d9c106f10 2024-09-16T00:13:38.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.501+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d9c075430 con 0x7f1d840a32d0 2024-09-16T00:13:38.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.509+0000 7f1d98ff9640 1 -- 172.21.15.78:0/1152409177 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340528 (secure 0 0 0) 0x7f1d9c075430 con 0x7f1d840a32d0 2024-09-16T00:13:38.514 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:38.515 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:13:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.512+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1d840a32d0 msgr2=0x7f1d840a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.512+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1d840a32d0 0x7f1d840a5790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c008d20 tx=0x7f1d8c0082b0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.512+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 msgr2=0x7f1d9c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.512+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19ee20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005ab0 tx=0x7f1d940042f0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.513+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 shutdown_connections 2024-09-16T00:13:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.513+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1d840a32d0 0x7f1d840a5790 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.513+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d9c106f10 0x7f1d9c19ee20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.513+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d9c102790 0x7f1d9c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.513+0000 7f1da1ba8640 1 --2- 172.21.15.78:0/1152409177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d9c101de0 0x7f1d9c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.513+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 >> 172.21.15.78:0/1152409177 conn(0x7f1d9c0fd8c0 msgr2=0x7f1d9c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.513+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 shutdown_connections 2024-09-16T00:13:38.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:38.513+0000 7f1da1ba8640 1 -- 172.21.15.78:0/1152409177 wait complete. 2024-09-16T00:13:39.197 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-16T00:13:38.433120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":671039,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1302,"num_read_kb":786,"num_write":465,"num_write_kb":3111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":879,"ondisk_log_size":879,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359460,"kb_used_data":14360,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582364,"statfs":{"total":1151940427776,"available":1151572340736,"internally_reserved":0,"allocated":14704640,"data_stored":8158475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9459,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":71,"num_write":16,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003066"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:34.460100+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:13:34.460100+0000","last_peered":"2024-09-16T00:13:34.460100+0000","last_clean":"2024-09-16T00:13:34.460100+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:13:34.460100+0000","last_undegraded":"2024-09-16T00:13:34.460100+0000","last_fullsized":"2024-09-16T00:13:34.460100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610803+0000","last_change":"2024-09-16T00:12:04.132674+0000","last_active":"2024-09-16T00:13:04.610803+0000","last_peered":"2024-09-16T00:13:04.610803+0000","last_clean":"2024-09-16T00:13:04.610803+0000","last_became_active":"2024-09-16T00:12:04.132108+0000","last_became_peered":"2024-09-16T00:12:04.132108+0000","last_unstale":"2024-09-16T00:13:04.610803+0000","last_undegraded":"2024-09-16T00:13:04.610803+0000","last_fullsized":"2024-09-16T00:13:04.610803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:10.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664962+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:13:04.664962+0000","last_peered":"2024-09-16T00:13:04.664962+0000","last_clean":"2024-09-16T00:13:04.664962+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:13:04.664962+0000","last_undegraded":"2024-09-16T00:13:04.664962+0000","last_fullsized":"2024-09-16T00:13:04.664962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663575+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:13:04.663575+0000","last_peered":"2024-09-16T00:13:04.663575+0000","last_clean":"2024-09-16T00:13:04.663575+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:13:04.663575+0000","last_undegraded":"2024-09-16T00:13:04.663575+0000","last_fullsized":"2024-09-16T00:13:04.663575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611137+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:13:04.611137+0000","last_peered":"2024-09-16T00:13:04.611137+0000","last_clean":"2024-09-16T00:13:04.611137+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:13:04.611137+0000","last_undegraded":"2024-09-16T00:13:04.611137+0000","last_fullsized":"2024-09-16T00:13:04.611137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664592+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:13:04.664592+0000","last_peered":"2024-09-16T00:13:04.664592+0000","last_clean":"2024-09-16T00:13:04.664592+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:13:04.664592+0000","last_undegraded":"2024-09-16T00:13:04.664592+0000","last_fullsized":"2024-09-16T00:13:04.664592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605487+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:13:04.605487+0000","last_peered":"2024-09-16T00:13:04.605487+0000","last_clean":"2024-09-16T00:13:04.605487+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:13:04.605487+0000","last_undegraded":"2024-09-16T00:13:04.605487+0000","last_fullsized":"2024-09-16T00:13:04.605487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611375+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:13:04.611375+0000","last_peered":"2024-09-16T00:13:04.611375+0000","last_clean":"2024-09-16T00:13:04.611375+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:13:04.611375+0000","last_undegraded":"2024-09-16T00:13:04.611375+0000","last_fullsized":"2024-09-16T00:13:04.611375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664399+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:13:04.664399+0000","last_peered":"2024-09-16T00:13:04.664399+0000","last_clean":"2024-09-16T00:13:04.664399+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:13:04.664399+0000","last_undegraded":"2024-09-16T00:13:04.664399+0000","last_fullsized":"2024-09-16T00:13:04.664399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605665+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:13:04.605665+0000","last_peered":"2024-09-16T00:13:04.605665+0000","last_clean":"2024-09-16T00:13:04.605665+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:13:04.605665+0000","last_undegraded":"2024-09-16T00:13:04.605665+0000","last_fullsized":"2024-09-16T00:13:04.605665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611103+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:13:04.611103+0000","last_peered":"2024-09-16T00:13:04.611103+0000","last_clean":"2024-09-16T00:13:04.611103+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:13:04.611103+0000","last_undegraded":"2024-09-16T00:13:04.611103+0000","last_fullsized":"2024-09-16T00:13:04.611103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649281+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:13:04.649281+0000","last_peered":"2024-09-16T00:13:04.649281+0000","last_clean":"2024-09-16T00:13:04.649281+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:13:04.649281+0000","last_undegraded":"2024-09-16T00:13:04.649281+0000","last_fullsized":"2024-09-16T00:13:04.649281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664252+0000","last_change":"2024-09-16T00:12:04.033464+0000","last_active":"2024-09-16T00:13:04.664252+0000","last_peered":"2024-09-16T00:13:04.664252+0000","last_clean":"2024-09-16T00:13:04.664252+0000","last_became_active":"2024-09-16T00:12:04.028916+0000","last_became_peered":"2024-09-16T00:12:04.028916+0000","last_unstale":"2024-09-16T00:13:04.664252+0000","last_undegraded":"2024-09-16T00:13:04.664252+0000","last_fullsized":"2024-09-16T00:13:04.664252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:47.359979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611584+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:13:04.611584+0000","last_peered":"2024-09-16T00:13:04.611584+0000","last_clean":"2024-09-16T00:13:04.611584+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:13:04.611584+0000","last_undegraded":"2024-09-16T00:13:04.611584+0000","last_fullsized":"2024-09-16T00:13:04.611584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664652+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:13:04.664652+0000","last_peered":"2024-09-16T00:13:04.664652+0000","last_clean":"2024-09-16T00:13:04.664652+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:13:04.664652+0000","last_undegraded":"2024-09-16T00:13:04.664652+0000","last_fullsized":"2024-09-16T00:13:04.664652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:15.431074+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:13:15.431074+0000","last_peered":"2024-09-16T00:13:15.431074+0000","last_clean":"2024-09-16T00:13:15.431074+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:13:15.431074+0000","last_undegraded":"2024-09-16T00:13:15.431074+0000","last_fullsized":"2024-09-16T00:13:15.431074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611391+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:13:04.611391+0000","last_peered":"2024-09-16T00:13:04.611391+0000","last_clean":"2024-09-16T00:13:04.611391+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:13:04.611391+0000","last_undegraded":"2024-09-16T00:13:04.611391+0000","last_fullsized":"2024-09-16T00:13:04.611391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663782+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:13:04.663782+0000","last_peered":"2024-09-16T00:13:04.663782+0000","last_clean":"2024-09-16T00:13:04.663782+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:13:04.663782+0000","last_undegraded":"2024-09-16T00:13:04.663782+0000","last_fullsized":"2024-09-16T00:13:04.663782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664498+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:13:04.664498+0000","last_peered":"2024-09-16T00:13:04.664498+0000","last_clean":"2024-09-16T00:13:04.664498+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:13:04.664498+0000","last_undegraded":"2024-09-16T00:13:04.664498+0000","last_fullsized":"2024-09-16T00:13:04.664498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609374+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:13:04.609374+0000","last_peered":"2024-09-16T00:13:04.609374+0000","last_clean":"2024-09-16T00:13:04.609374+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:13:04.609374+0000","last_undegraded":"2024-09-16T00:13:04.609374+0000","last_fullsized":"2024-09-16T00:13:04.609374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605426+0000","last_change":"2024-09-16T00:12:04.034002+0000","last_active":"2024-09-16T00:13:04.605426+0000","last_peered":"2024-09-16T00:13:04.605426+0000","last_clean":"2024-09-16T00:13:04.605426+0000","last_became_active":"2024-09-16T00:12:04.030959+0000","last_became_peered":"2024-09-16T00:12:04.030959+0000","last_unstale":"2024-09-16T00:13:04.605426+0000","last_undegraded":"2024-09-16T00:13:04.605426+0000","last_fullsized":"2024-09-16T00:13:04.605426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:47.730692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611324+0000","last_change":"2024-09-16T00:12:04.029906+0000","last_active":"2024-09-16T00:13:04.611324+0000","last_peered":"2024-09-16T00:13:04.611324+0000","last_clean":"2024-09-16T00:13:04.611324+0000","last_became_active":"2024-09-16T00:12:04.029259+0000","last_became_peered":"2024-09-16T00:12:04.029259+0000","last_unstale":"2024-09-16T00:13:04.611324+0000","last_undegraded":"2024-09-16T00:13:04.611324+0000","last_fullsized":"2024-09-16T00:13:04.611324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:23.399762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,1],"acting":[4,8,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663413+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:13:04.663413+0000","last_peered":"2024-09-16T00:13:04.663413+0000","last_clean":"2024-09-16T00:13:04.663413+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:13:04.663413+0000","last_undegraded":"2024-09-16T00:13:04.663413+0000","last_fullsized":"2024-09-16T00:13:04.663413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611856+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:13:04.611856+0000","last_peered":"2024-09-16T00:13:04.611856+0000","last_clean":"2024-09-16T00:13:04.611856+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:13:04.611856+0000","last_undegraded":"2024-09-16T00:13:04.611856+0000","last_fullsized":"2024-09-16T00:13:04.611856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610909+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:13:04.610909+0000","last_peered":"2024-09-16T00:13:04.610909+0000","last_clean":"2024-09-16T00:13:04.610909+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:13:04.610909+0000","last_undegraded":"2024-09-16T00:13:04.610909+0000","last_fullsized":"2024-09-16T00:13:04.610909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.665046+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:13:04.665046+0000","last_peered":"2024-09-16T00:13:04.665046+0000","last_clean":"2024-09-16T00:13:04.665046+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:13:04.665046+0000","last_undegraded":"2024-09-16T00:13:04.665046+0000","last_fullsized":"2024-09-16T00:13:04.665046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'21","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.652660+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:13:25.652660+0000","last_peered":"2024-09-16T00:13:25.652660+0000","last_clean":"2024-09-16T00:13:25.652660+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:13:25.652660+0000","last_undegraded":"2024-09-16T00:13:25.652660+0000","last_fullsized":"2024-09-16T00:13:25.652660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664101+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:13:04.664101+0000","last_peered":"2024-09-16T00:13:04.664101+0000","last_clean":"2024-09-16T00:13:04.664101+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:13:04.664101+0000","last_undegraded":"2024-09-16T00:13:04.664101+0000","last_fullsized":"2024-09-16T00:13:04.664101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606866+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:13:04.606866+0000","last_peered":"2024-09-16T00:13:04.606866+0000","last_clean":"2024-09-16T00:13:04.606866+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:13:04.606866+0000","last_undegraded":"2024-09-16T00:13:04.606866+0000","last_fullsized":"2024-09-16T00:13:04.606866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664634+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:13:04.664634+0000","last_peered":"2024-09-16T00:13:04.664634+0000","last_clean":"2024-09-16T00:13:04.664634+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:13:04.664634+0000","last_undegraded":"2024-09-16T00:13:04.664634+0000","last_fullsized":"2024-09-16T00:13:04.664634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664409+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:13:04.664409+0000","last_peered":"2024-09-16T00:13:04.664409+0000","last_clean":"2024-09-16T00:13:04.664409+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:13:04.664409+0000","last_undegraded":"2024-09-16T00:13:04.664409+0000","last_fullsized":"2024-09-16T00:13:04.664409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664046+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:13:04.664046+0000","last_peered":"2024-09-16T00:13:04.664046+0000","last_clean":"2024-09-16T00:13:04.664046+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:13:04.664046+0000","last_undegraded":"2024-09-16T00:13:04.664046+0000","last_fullsized":"2024-09-16T00:13:04.664046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609377+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:13:04.609377+0000","last_peered":"2024-09-16T00:13:04.609377+0000","last_clean":"2024-09-16T00:13:04.609377+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:13:04.609377+0000","last_undegraded":"2024-09-16T00:13:04.609377+0000","last_fullsized":"2024-09-16T00:13:04.609377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664159+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:13:04.664159+0000","last_peered":"2024-09-16T00:13:04.664159+0000","last_clean":"2024-09-16T00:13:04.664159+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:13:04.664159+0000","last_undegraded":"2024-09-16T00:13:04.664159+0000","last_fullsized":"2024-09-16T00:13:04.664159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664057+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:13:04.664057+0000","last_peered":"2024-09-16T00:13:04.664057+0000","last_clean":"2024-09-16T00:13:04.664057+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:13:04.664057+0000","last_undegraded":"2024-09-16T00:13:04.664057+0000","last_fullsized":"2024-09-16T00:13:04.664057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:31.694904+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:13:31.694904+0000","last_peered":"2024-09-16T00:13:31.694904+0000","last_clean":"2024-09-16T00:13:31.694904+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:13:31.694904+0000","last_undegraded":"2024-09-16T00:13:31.694904+0000","last_fullsized":"2024-09-16T00:13:31.694904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606730+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:13:04.606730+0000","last_peered":"2024-09-16T00:13:04.606730+0000","last_clean":"2024-09-16T00:13:04.606730+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:13:04.606730+0000","last_undegraded":"2024-09-16T00:13:04.606730+0000","last_fullsized":"2024-09-16T00:13:04.606730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605309+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:13:04.605309+0000","last_peered":"2024-09-16T00:13:04.605309+0000","last_clean":"2024-09-16T00:13:04.605309+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:13:04.605309+0000","last_undegraded":"2024-09-16T00:13:04.605309+0000","last_fullsized":"2024-09-16T00:13:04.605309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:36.746691+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:13:36.746691+0000","last_peered":"2024-09-16T00:13:36.746691+0000","last_clean":"2024-09-16T00:13:36.746691+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:13:36.746691+0000","last_undegraded":"2024-09-16T00:13:36.746691+0000","last_fullsized":"2024-09-16T00:13:36.746691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606979+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:13:04.606979+0000","last_peered":"2024-09-16T00:13:04.606979+0000","last_clean":"2024-09-16T00:13:04.606979+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:13:04.606979+0000","last_undegraded":"2024-09-16T00:13:04.606979+0000","last_fullsized":"2024-09-16T00:13:04.606979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606488+0000","last_change":"2024-09-16T00:12:04.027859+0000","last_active":"2024-09-16T00:13:04.606488+0000","last_peered":"2024-09-16T00:13:04.606488+0000","last_clean":"2024-09-16T00:13:04.606488+0000","last_became_active":"2024-09-16T00:12:04.027506+0000","last_became_peered":"2024-09-16T00:12:04.027506+0000","last_unstale":"2024-09-16T00:13:04.606488+0000","last_undegraded":"2024-09-16T00:13:04.606488+0000","last_fullsized":"2024-09-16T00:13:04.606488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:34.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649981+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:13:04.649981+0000","last_peered":"2024-09-16T00:13:04.649981+0000","last_clean":"2024-09-16T00:13:04.649981+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:13:04.649981+0000","last_undegraded":"2024-09-16T00:13:04.649981+0000","last_fullsized":"2024-09-16T00:13:04.649981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605678+0000","last_change":"2024-09-16T00:12:04.029471+0000","last_active":"2024-09-16T00:13:04.605678+0000","last_peered":"2024-09-16T00:13:04.605678+0000","last_clean":"2024-09-16T00:13:04.605678+0000","last_became_active":"2024-09-16T00:12:04.029054+0000","last_became_peered":"2024-09-16T00:12:04.029054+0000","last_unstale":"2024-09-16T00:13:04.605678+0000","last_undegraded":"2024-09-16T00:13:04.605678+0000","last_fullsized":"2024-09-16T00:13:04.605678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:52.054218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664553+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:13:04.664553+0000","last_peered":"2024-09-16T00:13:04.664553+0000","last_clean":"2024-09-16T00:13:04.664553+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:13:04.664553+0000","last_undegraded":"2024-09-16T00:13:04.664553+0000","last_fullsized":"2024-09-16T00:13:04.664553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611832+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:13:04.611832+0000","last_peered":"2024-09-16T00:13:04.611832+0000","last_clean":"2024-09-16T00:13:04.611832+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:13:04.611832+0000","last_undegraded":"2024-09-16T00:13:04.611832+0000","last_fullsized":"2024-09-16T00:13:04.611832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649917+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:13:04.649917+0000","last_peered":"2024-09-16T00:13:04.649917+0000","last_clean":"2024-09-16T00:13:04.649917+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:13:04.649917+0000","last_undegraded":"2024-09-16T00:13:04.649917+0000","last_fullsized":"2024-09-16T00:13:04.649917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650098+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:13:04.650098+0000","last_peered":"2024-09-16T00:13:04.650098+0000","last_clean":"2024-09-16T00:13:04.650098+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:13:04.650098+0000","last_undegraded":"2024-09-16T00:13:04.650098+0000","last_fullsized":"2024-09-16T00:13:04.650098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649329+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:13:04.649329+0000","last_peered":"2024-09-16T00:13:04.649329+0000","last_clean":"2024-09-16T00:13:04.649329+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:13:04.649329+0000","last_undegraded":"2024-09-16T00:13:04.649329+0000","last_fullsized":"2024-09-16T00:13:04.649329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:33.709720+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:13:33.709720+0000","last_peered":"2024-09-16T00:13:33.709720+0000","last_clean":"2024-09-16T00:13:33.709720+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:13:33.709720+0000","last_undegraded":"2024-09-16T00:13:33.709720+0000","last_fullsized":"2024-09-16T00:13:33.709720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605291+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:13:04.605291+0000","last_peered":"2024-09-16T00:13:04.605291+0000","last_clean":"2024-09-16T00:13:04.605291+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:13:04.605291+0000","last_undegraded":"2024-09-16T00:13:04.605291+0000","last_fullsized":"2024-09-16T00:13:04.605291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649345+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:13:04.649345+0000","last_peered":"2024-09-16T00:13:04.649345+0000","last_clean":"2024-09-16T00:13:04.649345+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:13:04.649345+0000","last_undegraded":"2024-09-16T00:13:04.649345+0000","last_fullsized":"2024-09-16T00:13:04.649345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605828+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:13:04.605828+0000","last_peered":"2024-09-16T00:13:04.605828+0000","last_clean":"2024-09-16T00:13:04.605828+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:13:04.605828+0000","last_undegraded":"2024-09-16T00:13:04.605828+0000","last_fullsized":"2024-09-16T00:13:04.605828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663968+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:13:04.663968+0000","last_peered":"2024-09-16T00:13:04.663968+0000","last_clean":"2024-09-16T00:13:04.663968+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:13:04.663968+0000","last_undegraded":"2024-09-16T00:13:04.663968+0000","last_fullsized":"2024-09-16T00:13:04.663968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650076+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:13:04.650076+0000","last_peered":"2024-09-16T00:13:04.650076+0000","last_clean":"2024-09-16T00:13:04.650076+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:13:04.650076+0000","last_undegraded":"2024-09-16T00:13:04.650076+0000","last_fullsized":"2024-09-16T00:13:04.650076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.607017+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:13:04.607017+0000","last_peered":"2024-09-16T00:13:04.607017+0000","last_clean":"2024-09-16T00:13:04.607017+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:13:04.607017+0000","last_undegraded":"2024-09-16T00:13:04.607017+0000","last_fullsized":"2024-09-16T00:13:04.607017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605023+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:13:04.605023+0000","last_peered":"2024-09-16T00:13:04.605023+0000","last_clean":"2024-09-16T00:13:04.605023+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:13:04.605023+0000","last_undegraded":"2024-09-16T00:13:04.605023+0000","last_fullsized":"2024-09-16T00:13:04.605023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663799+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:13:04.663799+0000","last_peered":"2024-09-16T00:13:04.663799+0000","last_clean":"2024-09-16T00:13:04.663799+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:13:04.663799+0000","last_undegraded":"2024-09-16T00:13:04.663799+0000","last_fullsized":"2024-09-16T00:13:04.663799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:19.532889+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:13:19.532889+0000","last_peered":"2024-09-16T00:13:19.532889+0000","last_clean":"2024-09-16T00:13:19.532889+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:13:19.532889+0000","last_undegraded":"2024-09-16T00:13:19.532889+0000","last_fullsized":"2024-09-16T00:13:19.532889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605103+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:13:04.605103+0000","last_peered":"2024-09-16T00:13:04.605103+0000","last_clean":"2024-09-16T00:13:04.605103+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:13:04.605103+0000","last_undegraded":"2024-09-16T00:13:04.605103+0000","last_fullsized":"2024-09-16T00:13:04.605103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605218+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:13:04.605218+0000","last_peered":"2024-09-16T00:13:04.605218+0000","last_clean":"2024-09-16T00:13:04.605218+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:13:04.605218+0000","last_undegraded":"2024-09-16T00:13:04.605218+0000","last_fullsized":"2024-09-16T00:13:04.605218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664161+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:13:04.664161+0000","last_peered":"2024-09-16T00:13:04.664161+0000","last_clean":"2024-09-16T00:13:04.664161+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:13:04.664161+0000","last_undegraded":"2024-09-16T00:13:04.664161+0000","last_fullsized":"2024-09-16T00:13:04.664161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.607004+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:13:04.607004+0000","last_peered":"2024-09-16T00:13:04.607004+0000","last_clean":"2024-09-16T00:13:04.607004+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:13:04.607004+0000","last_undegraded":"2024-09-16T00:13:04.607004+0000","last_fullsized":"2024-09-16T00:13:04.607004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649323+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:13:04.649323+0000","last_peered":"2024-09-16T00:13:04.649323+0000","last_clean":"2024-09-16T00:13:04.649323+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:13:04.649323+0000","last_undegraded":"2024-09-16T00:13:04.649323+0000","last_fullsized":"2024-09-16T00:13:04.649323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:36.747287+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:13:36.747287+0000","last_peered":"2024-09-16T00:13:36.747287+0000","last_clean":"2024-09-16T00:13:36.747287+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:13:36.747287+0000","last_undegraded":"2024-09-16T00:13:36.747287+0000","last_fullsized":"2024-09-16T00:13:36.747287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609415+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:13:04.609415+0000","last_peered":"2024-09-16T00:13:04.609415+0000","last_clean":"2024-09-16T00:13:04.609415+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:13:04.609415+0000","last_undegraded":"2024-09-16T00:13:04.609415+0000","last_fullsized":"2024-09-16T00:13:04.609415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649661+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:13:04.649661+0000","last_peered":"2024-09-16T00:13:04.649661+0000","last_clean":"2024-09-16T00:13:04.649661+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:13:04.649661+0000","last_undegraded":"2024-09-16T00:13:04.649661+0000","last_fullsized":"2024-09-16T00:13:04.649661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605440+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:13:04.605440+0000","last_peered":"2024-09-16T00:13:04.605440+0000","last_clean":"2024-09-16T00:13:04.605440+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:13:04.605440+0000","last_undegraded":"2024-09-16T00:13:04.605440+0000","last_fullsized":"2024-09-16T00:13:04.605440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664603+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:13:04.664603+0000","last_peered":"2024-09-16T00:13:04.664603+0000","last_clean":"2024-09-16T00:13:04.664603+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:13:04.664603+0000","last_undegraded":"2024-09-16T00:13:04.664603+0000","last_fullsized":"2024-09-16T00:13:04.664603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606631+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:13:04.606631+0000","last_peered":"2024-09-16T00:13:04.606631+0000","last_clean":"2024-09-16T00:13:04.606631+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:13:04.606631+0000","last_undegraded":"2024-09-16T00:13:04.606631+0000","last_fullsized":"2024-09-16T00:13:04.606631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611195+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:13:04.611195+0000","last_peered":"2024-09-16T00:13:04.611195+0000","last_clean":"2024-09-16T00:13:04.611195+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:13:04.611195+0000","last_undegraded":"2024-09-16T00:13:04.611195+0000","last_fullsized":"2024-09-16T00:13:04.611195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605618+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:13:04.605618+0000","last_peered":"2024-09-16T00:13:04.605618+0000","last_clean":"2024-09-16T00:13:04.605618+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:13:04.605618+0000","last_undegraded":"2024-09-16T00:13:04.605618+0000","last_fullsized":"2024-09-16T00:13:04.605618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664419+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:13:04.664419+0000","last_peered":"2024-09-16T00:13:04.664419+0000","last_clean":"2024-09-16T00:13:04.664419+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:13:04.664419+0000","last_undegraded":"2024-09-16T00:13:04.664419+0000","last_fullsized":"2024-09-16T00:13:04.664419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'43","reported_seq":321,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:34.711137+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:13:34.711137+0000","last_peered":"2024-09-16T00:13:34.711137+0000","last_clean":"2024-09-16T00:13:34.711137+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:13:34.711137+0000","last_undegraded":"2024-09-16T00:13:34.711137+0000","last_fullsized":"2024-09-16T00:13:34.711137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71853,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":190,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609191+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:13:04.609191+0000","last_peered":"2024-09-16T00:13:04.609191+0000","last_clean":"2024-09-16T00:13:04.609191+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:13:04.609191+0000","last_undegraded":"2024-09-16T00:13:04.609191+0000","last_fullsized":"2024-09-16T00:13:04.609191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605601+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:13:04.605601+0000","last_peered":"2024-09-16T00:13:04.605601+0000","last_clean":"2024-09-16T00:13:04.605601+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:13:04.605601+0000","last_undegraded":"2024-09-16T00:13:04.605601+0000","last_fullsized":"2024-09-16T00:13:04.605601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605792+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:13:04.605792+0000","last_peered":"2024-09-16T00:13:04.605792+0000","last_clean":"2024-09-16T00:13:04.605792+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:13:04.605792+0000","last_undegraded":"2024-09-16T00:13:04.605792+0000","last_fullsized":"2024-09-16T00:13:04.605792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611640+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:13:04.611640+0000","last_peered":"2024-09-16T00:13:04.611640+0000","last_clean":"2024-09-16T00:13:04.611640+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:13:04.611640+0000","last_undegraded":"2024-09-16T00:13:04.611640+0000","last_fullsized":"2024-09-16T00:13:04.611640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663456+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:13:04.663456+0000","last_peered":"2024-09-16T00:13:04.663456+0000","last_clean":"2024-09-16T00:13:04.663456+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:13:04.663456+0000","last_undegraded":"2024-09-16T00:13:04.663456+0000","last_fullsized":"2024-09-16T00:13:04.663456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650055+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:13:04.650055+0000","last_peered":"2024-09-16T00:13:04.650055+0000","last_clean":"2024-09-16T00:13:04.650055+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:13:04.650055+0000","last_undegraded":"2024-09-16T00:13:04.650055+0000","last_fullsized":"2024-09-16T00:13:04.650055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.608952+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:13:04.608952+0000","last_peered":"2024-09-16T00:13:04.608952+0000","last_clean":"2024-09-16T00:13:04.608952+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:13:04.608952+0000","last_undegraded":"2024-09-16T00:13:04.608952+0000","last_fullsized":"2024-09-16T00:13:04.608952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605753+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:13:04.605753+0000","last_peered":"2024-09-16T00:13:04.605753+0000","last_clean":"2024-09-16T00:13:04.605753+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:13:04.605753+0000","last_undegraded":"2024-09-16T00:13:04.605753+0000","last_fullsized":"2024-09-16T00:13:04.605753+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":269,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:34.710559+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:13:34.710559+0000","last_peered":"2024-09-16T00:13:34.710559+0000","last_clean":"2024-09-16T00:13:34.710559+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:13:34.710559+0000","last_undegraded":"2024-09-16T00:13:34.710559+0000","last_fullsized":"2024-09-16T00:13:34.710559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664396+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:13:04.664396+0000","last_peered":"2024-09-16T00:13:04.664396+0000","last_clean":"2024-09-16T00:13:04.664396+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:13:04.664396+0000","last_undegraded":"2024-09-16T00:13:04.664396+0000","last_fullsized":"2024-09-16T00:13:04.664396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663978+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:13:04.663978+0000","last_peered":"2024-09-16T00:13:04.663978+0000","last_clean":"2024-09-16T00:13:04.663978+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:13:04.663978+0000","last_undegraded":"2024-09-16T00:13:04.663978+0000","last_fullsized":"2024-09-16T00:13:04.663978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609055+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:13:04.609055+0000","last_peered":"2024-09-16T00:13:04.609055+0000","last_clean":"2024-09-16T00:13:04.609055+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:13:04.609055+0000","last_undegraded":"2024-09-16T00:13:04.609055+0000","last_fullsized":"2024-09-16T00:13:04.609055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605507+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:13:04.605507+0000","last_peered":"2024-09-16T00:13:04.605507+0000","last_clean":"2024-09-16T00:13:04.605507+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:13:04.605507+0000","last_undegraded":"2024-09-16T00:13:04.605507+0000","last_fullsized":"2024-09-16T00:13:04.605507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664904+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:13:04.664904+0000","last_peered":"2024-09-16T00:13:04.664904+0000","last_clean":"2024-09-16T00:13:04.664904+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:13:04.664904+0000","last_undegraded":"2024-09-16T00:13:04.664904+0000","last_fullsized":"2024-09-16T00:13:04.664904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663729+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:13:04.663729+0000","last_peered":"2024-09-16T00:13:04.663729+0000","last_clean":"2024-09-16T00:13:04.663729+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:13:04.663729+0000","last_undegraded":"2024-09-16T00:13:04.663729+0000","last_fullsized":"2024-09-16T00:13:04.663729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606517+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:13:04.606517+0000","last_peered":"2024-09-16T00:13:04.606517+0000","last_clean":"2024-09-16T00:13:04.606517+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:13:04.606517+0000","last_undegraded":"2024-09-16T00:13:04.606517+0000","last_fullsized":"2024-09-16T00:13:04.606517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664822+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:13:04.664822+0000","last_peered":"2024-09-16T00:13:04.664822+0000","last_clean":"2024-09-16T00:13:04.664822+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:13:04.664822+0000","last_undegraded":"2024-09-16T00:13:04.664822+0000","last_fullsized":"2024-09-16T00:13:04.664822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605506+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:13:04.605506+0000","last_peered":"2024-09-16T00:13:04.605506+0000","last_clean":"2024-09-16T00:13:04.605506+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:13:04.605506+0000","last_undegraded":"2024-09-16T00:13:04.605506+0000","last_fullsized":"2024-09-16T00:13:04.605506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610913+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:13:04.610913+0000","last_peered":"2024-09-16T00:13:04.610913+0000","last_clean":"2024-09-16T00:13:04.610913+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:13:04.610913+0000","last_undegraded":"2024-09-16T00:13:04.610913+0000","last_fullsized":"2024-09-16T00:13:04.610913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:35.712969+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:13:35.712969+0000","last_peered":"2024-09-16T00:13:35.712969+0000","last_clean":"2024-09-16T00:13:35.712969+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:13:35.712969+0000","last_undegraded":"2024-09-16T00:13:35.712969+0000","last_fullsized":"2024-09-16T00:13:35.712969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663780+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:13:04.663780+0000","last_peered":"2024-09-16T00:13:04.663780+0000","last_clean":"2024-09-16T00:13:04.663780+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:13:04.663780+0000","last_undegraded":"2024-09-16T00:13:04.663780+0000","last_fullsized":"2024-09-16T00:13:04.663780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611136+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:13:04.611136+0000","last_peered":"2024-09-16T00:13:04.611136+0000","last_clean":"2024-09-16T00:13:04.611136+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:13:04.611136+0000","last_undegraded":"2024-09-16T00:13:04.611136+0000","last_fullsized":"2024-09-16T00:13:04.611136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663783+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:13:04.663783+0000","last_peered":"2024-09-16T00:13:04.663783+0000","last_clean":"2024-09-16T00:13:04.663783+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:13:04.663783+0000","last_undegraded":"2024-09-16T00:13:04.663783+0000","last_fullsized":"2024-09-16T00:13:04.663783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611426+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:13:04.611426+0000","last_peered":"2024-09-16T00:13:04.611426+0000","last_clean":"2024-09-16T00:13:04.611426+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:13:04.611426+0000","last_undegraded":"2024-09-16T00:13:04.611426+0000","last_fullsized":"2024-09-16T00:13:04.611426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610934+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:13:04.610934+0000","last_peered":"2024-09-16T00:13:04.610934+0000","last_clean":"2024-09-16T00:13:04.610934+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:13:04.610934+0000","last_undegraded":"2024-09-16T00:13:04.610934+0000","last_fullsized":"2024-09-16T00:13:04.610934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663979+0000","last_change":"2024-09-16T00:12:04.029280+0000","last_active":"2024-09-16T00:13:04.663979+0000","last_peered":"2024-09-16T00:13:04.663979+0000","last_clean":"2024-09-16T00:13:04.663979+0000","last_became_active":"2024-09-16T00:12:04.028973+0000","last_became_peered":"2024-09-16T00:12:04.028973+0000","last_unstale":"2024-09-16T00:13:04.663979+0000","last_undegraded":"2024-09-16T00:13:04.663979+0000","last_fullsized":"2024-09-16T00:13:04.663979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606846+0000","last_change":"2024-09-16T00:12:04.028319+0000","last_active":"2024-09-16T00:13:04.606846+0000","last_peered":"2024-09-16T00:13:04.606846+0000","last_clean":"2024-09-16T00:13:04.606846+0000","last_became_active":"2024-09-16T00:12:04.027985+0000","last_became_peered":"2024-09-16T00:12:04.027985+0000","last_unstale":"2024-09-16T00:13:04.606846+0000","last_undegraded":"2024-09-16T00:13:04.606846+0000","last_fullsized":"2024-09-16T00:13:04.606846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.119428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663790+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:13:04.663790+0000","last_peered":"2024-09-16T00:13:04.663790+0000","last_clean":"2024-09-16T00:13:04.663790+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:13:04.663790+0000","last_undegraded":"2024-09-16T00:13:04.663790+0000","last_fullsized":"2024-09-16T00:13:04.663790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611369+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:13:04.611369+0000","last_peered":"2024-09-16T00:13:04.611369+0000","last_clean":"2024-09-16T00:13:04.611369+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:13:04.611369+0000","last_undegraded":"2024-09-16T00:13:04.611369+0000","last_fullsized":"2024-09-16T00:13:04.611369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.616864+0000","last_change":"2024-09-16T00:13:04.612598+0000","last_active":"2024-09-16T00:13:04.616864+0000","last_peered":"2024-09-16T00:13:04.616864+0000","last_clean":"2024-09-16T00:13:04.616864+0000","last_became_active":"2024-09-16T00:13:04.612310+0000","last_became_peered":"2024-09-16T00:13:04.612310+0000","last_unstale":"2024-09-16T00:13:04.616864+0000","last_undegraded":"2024-09-16T00:13:04.616864+0000","last_fullsized":"2024-09-16T00:13:04.616864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:59.381944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605240+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:13:04.605240+0000","last_peered":"2024-09-16T00:13:04.605240+0000","last_clean":"2024-09-16T00:13:04.605240+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:13:04.605240+0000","last_undegraded":"2024-09-16T00:13:04.605240+0000","last_fullsized":"2024-09-16T00:13:04.605240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663870+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:13:04.663870+0000","last_peered":"2024-09-16T00:13:04.663870+0000","last_clean":"2024-09-16T00:13:04.663870+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:13:04.663870+0000","last_undegraded":"2024-09-16T00:13:04.663870+0000","last_fullsized":"2024-09-16T00:13:04.663870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605535+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:13:04.605535+0000","last_peered":"2024-09-16T00:13:04.605535+0000","last_clean":"2024-09-16T00:13:04.605535+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:13:04.605535+0000","last_undegraded":"2024-09-16T00:13:04.605535+0000","last_fullsized":"2024-09-16T00:13:04.605535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606766+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:13:04.606766+0000","last_peered":"2024-09-16T00:13:04.606766+0000","last_clean":"2024-09-16T00:13:04.606766+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:13:04.606766+0000","last_undegraded":"2024-09-16T00:13:04.606766+0000","last_fullsized":"2024-09-16T00:13:04.606766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663880+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:13:04.663880+0000","last_peered":"2024-09-16T00:13:04.663880+0000","last_clean":"2024-09-16T00:13:04.663880+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:13:04.663880+0000","last_undegraded":"2024-09-16T00:13:04.663880+0000","last_fullsized":"2024-09-16T00:13:04.663880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611528+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:13:04.611528+0000","last_peered":"2024-09-16T00:13:04.611528+0000","last_clean":"2024-09-16T00:13:04.611528+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:13:04.611528+0000","last_undegraded":"2024-09-16T00:13:04.611528+0000","last_fullsized":"2024-09-16T00:13:04.611528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609303+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:13:04.609303+0000","last_peered":"2024-09-16T00:13:04.609303+0000","last_clean":"2024-09-16T00:13:04.609303+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:13:04.609303+0000","last_undegraded":"2024-09-16T00:13:04.609303+0000","last_fullsized":"2024-09-16T00:13:04.609303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663938+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:13:04.663938+0000","last_peered":"2024-09-16T00:13:04.663938+0000","last_clean":"2024-09-16T00:13:04.663938+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:13:04.663938+0000","last_undegraded":"2024-09-16T00:13:04.663938+0000","last_fullsized":"2024-09-16T00:13:04.663938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605284+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:13:04.605284+0000","last_peered":"2024-09-16T00:13:04.605284+0000","last_clean":"2024-09-16T00:13:04.605284+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:13:04.605284+0000","last_undegraded":"2024-09-16T00:13:04.605284+0000","last_fullsized":"2024-09-16T00:13:04.605284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663375+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:13:04.663375+0000","last_peered":"2024-09-16T00:13:04.663375+0000","last_clean":"2024-09-16T00:13:04.663375+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:13:04.663375+0000","last_undegraded":"2024-09-16T00:13:04.663375+0000","last_fullsized":"2024-09-16T00:13:04.663375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663627+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:13:04.663627+0000","last_peered":"2024-09-16T00:13:04.663627+0000","last_clean":"2024-09-16T00:13:04.663627+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:13:04.663627+0000","last_undegraded":"2024-09-16T00:13:04.663627+0000","last_fullsized":"2024-09-16T00:13:04.663627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606741+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:13:04.606741+0000","last_peered":"2024-09-16T00:13:04.606741+0000","last_clean":"2024-09-16T00:13:04.606741+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:13:04.606741+0000","last_undegraded":"2024-09-16T00:13:04.606741+0000","last_fullsized":"2024-09-16T00:13:04.606741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611683+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:13:04.611683+0000","last_peered":"2024-09-16T00:13:04.611683+0000","last_clean":"2024-09-16T00:13:04.611683+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:13:04.611683+0000","last_undegraded":"2024-09-16T00:13:04.611683+0000","last_fullsized":"2024-09-16T00:13:04.611683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649341+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:13:04.649341+0000","last_peered":"2024-09-16T00:13:04.649341+0000","last_clean":"2024-09-16T00:13:04.649341+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:13:04.649341+0000","last_undegraded":"2024-09-16T00:13:04.649341+0000","last_fullsized":"2024-09-16T00:13:04.649341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605811+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:13:04.605811+0000","last_peered":"2024-09-16T00:13:04.605811+0000","last_clean":"2024-09-16T00:13:04.605811+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:13:04.605811+0000","last_undegraded":"2024-09-16T00:13:04.605811+0000","last_fullsized":"2024-09-16T00:13:04.605811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663898+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:13:04.663898+0000","last_peered":"2024-09-16T00:13:04.663898+0000","last_clean":"2024-09-16T00:13:04.663898+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:13:04.663898+0000","last_undegraded":"2024-09-16T00:13:04.663898+0000","last_fullsized":"2024-09-16T00:13:04.663898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649554+0000","last_change":"2024-09-16T00:12:04.028342+0000","last_active":"2024-09-16T00:13:04.649554+0000","last_peered":"2024-09-16T00:13:04.649554+0000","last_clean":"2024-09-16T00:13:04.649554+0000","last_became_active":"2024-09-16T00:12:04.027889+0000","last_became_peered":"2024-09-16T00:12:04.027889+0000","last_unstale":"2024-09-16T00:13:04.649554+0000","last_undegraded":"2024-09-16T00:13:04.649554+0000","last_fullsized":"2024-09-16T00:13:04.649554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:47.840428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664117+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:13:04.664117+0000","last_peered":"2024-09-16T00:13:04.664117+0000","last_clean":"2024-09-16T00:13:04.664117+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:13:04.664117+0000","last_undegraded":"2024-09-16T00:13:04.664117+0000","last_fullsized":"2024-09-16T00:13:04.664117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663932+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:13:04.663932+0000","last_peered":"2024-09-16T00:13:04.663932+0000","last_clean":"2024-09-16T00:13:04.663932+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:13:04.663932+0000","last_undegraded":"2024-09-16T00:13:04.663932+0000","last_fullsized":"2024-09-16T00:13:04.663932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611689+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:13:04.611689+0000","last_peered":"2024-09-16T00:13:04.611689+0000","last_clean":"2024-09-16T00:13:04.611689+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:13:04.611689+0000","last_undegraded":"2024-09-16T00:13:04.611689+0000","last_fullsized":"2024-09-16T00:13:04.611689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664072+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:13:04.664072+0000","last_peered":"2024-09-16T00:13:04.664072+0000","last_clean":"2024-09-16T00:13:04.664072+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:13:04.664072+0000","last_undegraded":"2024-09-16T00:13:04.664072+0000","last_fullsized":"2024-09-16T00:13:04.664072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664019+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:13:04.664019+0000","last_peered":"2024-09-16T00:13:04.664019+0000","last_clean":"2024-09-16T00:13:04.664019+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:13:04.664019+0000","last_undegraded":"2024-09-16T00:13:04.664019+0000","last_fullsized":"2024-09-16T00:13:04.664019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80437,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1176,"num_read_kb":677,"num_write":282,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":241311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579949,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":515263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808882,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1744896,"data_stored":1174325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201340,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430272,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1112771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593798,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822731,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36972,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708180,"statfs":{"total":95995035648,"available":95957176320,"internally_reserved":0,"allocated":1667072,"data_stored":1107033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":515164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215190,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":509871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":584108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":583439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":512153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":74667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":74516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:13:39.202 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:13:39.202 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:13:39.463 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:39.739 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:39 smithi078 ceph-mon[31157]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-16T00:13:39.740 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:39 smithi078 ceph-mon[31157]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:39 smithi193 ceph-mon[35484]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-16T00:13:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:39 smithi193 ceph-mon[35484]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:39 smithi148 ceph-mon[35599]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-16T00:13:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:39 smithi148 ceph-mon[35599]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:39.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.976+0000 7f8652668640 1 -- 172.21.15.78:0/2619092228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c100a80 msgr2=0x7f864c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.976+0000 7f8652668640 1 --2- 172.21.15.78:0/2619092228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c100a80 0x7f864c100ee0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8640009930 tx=0x7f864002a420 comp rx=0 tx=0).stop 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.976+0000 7f8652668640 1 -- 172.21.15.78:0/2619092228 shutdown_connections 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.976+0000 7f8652668640 1 --2- 172.21.15.78:0/2619092228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864c101420 0x7f864c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.976+0000 7f8652668640 1 --2- 172.21.15.78:0/2619092228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c100a80 0x7f864c100ee0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.976+0000 7f8652668640 1 --2- 172.21.15.78:0/2619092228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f864c106a80 0x7f864c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.976+0000 7f8652668640 1 -- 172.21.15.78:0/2619092228 >> 172.21.15.78:0/2619092228 conn(0x7f864c0fc7a0 msgr2=0x7f864c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 -- 172.21.15.78:0/2619092228 shutdown_connections 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 -- 172.21.15.78:0/2619092228 wait complete. 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 Processor -- start 2024-09-16T00:13:39.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 -- start start 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864c100a80 0x7f864c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c101420 0x7f864c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f864c106a80 0x7f864c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f864c110ea0 con 0x7f864c101420 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f864c110d20 con 0x7f864c100a80 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.977+0000 7f8652668640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f864c111020 con 0x7f864c106a80 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c101420 0x7f864c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c101420 0x7f864c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54008/0 (socket says 172.21.15.78:54008) 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 -- 172.21.15.78:0/1614059126 learned_addr learned my addr 172.21.15.78:0/1614059126 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864bfff640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864c100a80 0x7f864c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f8650c05640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f864c106a80 0x7f864c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 -- 172.21.15.78:0/1614059126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f864c106a80 msgr2=0x7f864c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:39.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f864c106a80 0x7f864c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 -- 172.21.15.78:0/1614059126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864c100a80 msgr2=0x7f864c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864c100a80 0x7f864c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 -- 172.21.15.78:0/1614059126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864c19d440 con 0x7f864c101420 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f8650c05640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f864c106a80 0x7f864c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864bfff640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864c100a80 0x7f864c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f864b7fe640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c101420 0x7f864c1989d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8640009a50 tx=0x7f8640004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f86497fa640 1 -- 172.21.15.78:0/1614059126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864003f030 con 0x7f864c101420 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f86497fa640 1 -- 172.21.15.78:0/1614059126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8640003ce0 con 0x7f864c101420 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864c19d6d0 con 0x7f864c101420 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f86497fa640 1 -- 172.21.15.78:0/1614059126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640035070 con 0x7f864c101420 2024-09-16T00:13:39.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.978+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864c1a5010 con 0x7f864c101420 2024-09-16T00:13:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.980+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864c1021c0 con 0x7f864c101420 2024-09-16T00:13:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.981+0000 7f86497fa640 1 -- 172.21.15.78:0/1614059126 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f8640042030 con 0x7f864c101420 2024-09-16T00:13:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.981+0000 7f86497fa640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86340a32d0 0x7f86340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.982+0000 7f86497fa640 1 -- 172.21.15.78:0/1614059126 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f86400f0100 con 0x7f864c101420 2024-09-16T00:13:39.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.982+0000 7f864bfff640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86340a32d0 0x7f86340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:39.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.982+0000 7f864bfff640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86340a32d0 0x7f86340a5790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f863c009930 tx=0x7f863c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:39.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:39.986+0000 7f86497fa640 1 -- 172.21.15.78:0/1614059126 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86400bc960 con 0x7f864c101420 2024-09-16T00:13:40.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.150+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 --> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f864c062630 con 0x7f86340a32d0 2024-09-16T00:13:40.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.158+0000 7f86497fa640 1 -- 172.21.15.78:0/1614059126 <== mgr.14150 v2:172.21.15.78:6800/119552845 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340528 (secure 0 0 0) 0x7f864c062630 con 0x7f86340a32d0 2024-09-16T00:13:40.162 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:40.163 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86340a32d0 msgr2=0x7f86340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86340a32d0 0x7f86340a5790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f863c009930 tx=0x7f863c008040 comp rx=0 tx=0).stop 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c101420 msgr2=0x7f864c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c101420 0x7f864c1989d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8640009a50 tx=0x7f8640004270 comp rx=0 tx=0).stop 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 shutdown_connections 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86340a32d0 0x7f86340a5790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f864c106a80 0x7f864c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f864c101420 0x7f864c1989d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 --2- 172.21.15.78:0/1614059126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f864c100a80 0x7f864c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:40.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 >> 172.21.15.78:0/1614059126 conn(0x7f864c0fc7a0 msgr2=0x7f864c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:40.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 shutdown_connections 2024-09-16T00:13:40.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:40.161+0000 7f8652668640 1 -- 172.21.15.78:0/1614059126 wait complete. 2024-09-16T00:13:40.929 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-16T00:13:38.433120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":671039,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1302,"num_read_kb":786,"num_write":465,"num_write_kb":3111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":879,"ondisk_log_size":879,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359460,"kb_used_data":14360,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582364,"statfs":{"total":1151940427776,"available":1151572340736,"internally_reserved":0,"allocated":14704640,"data_stored":8158475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9459,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":71,"num_write":16,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003066"},"pg_stats":[{"pgid":"2.1f","version":"90'14","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:34.460100+0000","last_change":"2024-09-16T00:11:02.617158+0000","last_active":"2024-09-16T00:13:34.460100+0000","last_peered":"2024-09-16T00:13:34.460100+0000","last_clean":"2024-09-16T00:13:34.460100+0000","last_became_active":"2024-09-16T00:11:02.616819+0000","last_became_peered":"2024-09-16T00:11:02.616819+0000","last_unstale":"2024-09-16T00:13:34.460100+0000","last_undegraded":"2024-09-16T00:13:34.460100+0000","last_fullsized":"2024-09-16T00:13:34.460100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:16.929870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.645270+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.644093+0000","last_became_peered":"2024-09-16T00:11:04.644093+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:13.993170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610803+0000","last_change":"2024-09-16T00:12:04.132674+0000","last_active":"2024-09-16T00:13:04.610803+0000","last_peered":"2024-09-16T00:13:04.610803+0000","last_clean":"2024-09-16T00:13:04.610803+0000","last_became_active":"2024-09-16T00:12:04.132108+0000","last_became_peered":"2024-09-16T00:12:04.132108+0000","last_unstale":"2024-09-16T00:13:04.610803+0000","last_undegraded":"2024-09-16T00:13:04.610803+0000","last_fullsized":"2024-09-16T00:13:04.610803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:10.569289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664962+0000","last_change":"2024-09-16T00:11:02.809191+0000","last_active":"2024-09-16T00:13:04.664962+0000","last_peered":"2024-09-16T00:13:04.664962+0000","last_clean":"2024-09-16T00:13:04.664962+0000","last_became_active":"2024-09-16T00:11:02.808894+0000","last_became_peered":"2024-09-16T00:11:02.808894+0000","last_unstale":"2024-09-16T00:13:04.664962+0000","last_undegraded":"2024-09-16T00:13:04.664962+0000","last_fullsized":"2024-09-16T00:13:04.664962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:13.545139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663575+0000","last_change":"2024-09-16T00:11:04.639045+0000","last_active":"2024-09-16T00:13:04.663575+0000","last_peered":"2024-09-16T00:13:04.663575+0000","last_clean":"2024-09-16T00:13:04.663575+0000","last_became_active":"2024-09-16T00:11:04.638196+0000","last_became_peered":"2024-09-16T00:11:04.638196+0000","last_unstale":"2024-09-16T00:13:04.663575+0000","last_undegraded":"2024-09-16T00:13:04.663575+0000","last_fullsized":"2024-09-16T00:13:04.663575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:24.873248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611137+0000","last_change":"2024-09-16T00:11:11.679542+0000","last_active":"2024-09-16T00:13:04.611137+0000","last_peered":"2024-09-16T00:13:04.611137+0000","last_clean":"2024-09-16T00:13:04.611137+0000","last_became_active":"2024-09-16T00:11:11.679197+0000","last_became_peered":"2024-09-16T00:11:11.679197+0000","last_unstale":"2024-09-16T00:13:04.611137+0000","last_undegraded":"2024-09-16T00:13:04.611137+0000","last_fullsized":"2024-09-16T00:13:04.611137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:05:28.164430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664592+0000","last_change":"2024-09-16T00:11:02.617773+0000","last_active":"2024-09-16T00:13:04.664592+0000","last_peered":"2024-09-16T00:13:04.664592+0000","last_clean":"2024-09-16T00:13:04.664592+0000","last_became_active":"2024-09-16T00:11:02.617435+0000","last_became_peered":"2024-09-16T00:11:02.617435+0000","last_unstale":"2024-09-16T00:13:04.664592+0000","last_undegraded":"2024-09-16T00:13:04.664592+0000","last_fullsized":"2024-09-16T00:13:04.664592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:49:37.492598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605487+0000","last_change":"2024-09-16T00:11:04.638532+0000","last_active":"2024-09-16T00:13:04.605487+0000","last_peered":"2024-09-16T00:13:04.605487+0000","last_clean":"2024-09-16T00:13:04.605487+0000","last_became_active":"2024-09-16T00:11:04.638225+0000","last_became_peered":"2024-09-16T00:11:04.638225+0000","last_unstale":"2024-09-16T00:13:04.605487+0000","last_undegraded":"2024-09-16T00:13:04.605487+0000","last_fullsized":"2024-09-16T00:13:04.605487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:07:10.892046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611375+0000","last_change":"2024-09-16T00:11:11.683183+0000","last_active":"2024-09-16T00:13:04.611375+0000","last_peered":"2024-09-16T00:13:04.611375+0000","last_clean":"2024-09-16T00:13:04.611375+0000","last_became_active":"2024-09-16T00:11:11.682816+0000","last_became_peered":"2024-09-16T00:11:11.682816+0000","last_unstale":"2024-09-16T00:13:04.611375+0000","last_undegraded":"2024-09-16T00:13:04.611375+0000","last_fullsized":"2024-09-16T00:13:04.611375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:56.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664399+0000","last_change":"2024-09-16T00:11:02.616654+0000","last_active":"2024-09-16T00:13:04.664399+0000","last_peered":"2024-09-16T00:13:04.664399+0000","last_clean":"2024-09-16T00:13:04.664399+0000","last_became_active":"2024-09-16T00:11:02.615981+0000","last_became_peered":"2024-09-16T00:11:02.615981+0000","last_unstale":"2024-09-16T00:13:04.664399+0000","last_undegraded":"2024-09-16T00:13:04.664399+0000","last_fullsized":"2024-09-16T00:13:04.664399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:32.244391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605665+0000","last_change":"2024-09-16T00:11:04.643353+0000","last_active":"2024-09-16T00:13:04.605665+0000","last_peered":"2024-09-16T00:13:04.605665+0000","last_clean":"2024-09-16T00:13:04.605665+0000","last_became_active":"2024-09-16T00:11:04.643047+0000","last_became_peered":"2024-09-16T00:11:04.643047+0000","last_unstale":"2024-09-16T00:13:04.605665+0000","last_undegraded":"2024-09-16T00:13:04.605665+0000","last_fullsized":"2024-09-16T00:13:04.605665+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:59.563796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611103+0000","last_change":"2024-09-16T00:11:11.678736+0000","last_active":"2024-09-16T00:13:04.611103+0000","last_peered":"2024-09-16T00:13:04.611103+0000","last_clean":"2024-09-16T00:13:04.611103+0000","last_became_active":"2024-09-16T00:11:11.678225+0000","last_became_peered":"2024-09-16T00:11:11.678225+0000","last_unstale":"2024-09-16T00:13:04.611103+0000","last_undegraded":"2024-09-16T00:13:04.611103+0000","last_fullsized":"2024-09-16T00:13:04.611103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:02.720857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649281+0000","last_change":"2024-09-16T00:11:04.637487+0000","last_active":"2024-09-16T00:13:04.649281+0000","last_peered":"2024-09-16T00:13:04.649281+0000","last_clean":"2024-09-16T00:13:04.649281+0000","last_became_active":"2024-09-16T00:11:04.637184+0000","last_became_peered":"2024-09-16T00:11:04.637184+0000","last_unstale":"2024-09-16T00:13:04.649281+0000","last_undegraded":"2024-09-16T00:13:04.649281+0000","last_fullsized":"2024-09-16T00:13:04.649281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:28.880255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664252+0000","last_change":"2024-09-16T00:12:04.033464+0000","last_active":"2024-09-16T00:13:04.664252+0000","last_peered":"2024-09-16T00:13:04.664252+0000","last_clean":"2024-09-16T00:13:04.664252+0000","last_became_active":"2024-09-16T00:12:04.028916+0000","last_became_peered":"2024-09-16T00:12:04.028916+0000","last_unstale":"2024-09-16T00:13:04.664252+0000","last_undegraded":"2024-09-16T00:13:04.664252+0000","last_fullsized":"2024-09-16T00:13:04.664252+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:47.359979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611584+0000","last_change":"2024-09-16T00:11:11.687427+0000","last_active":"2024-09-16T00:13:04.611584+0000","last_peered":"2024-09-16T00:13:04.611584+0000","last_clean":"2024-09-16T00:13:04.611584+0000","last_became_active":"2024-09-16T00:11:11.686979+0000","last_became_peered":"2024-09-16T00:11:11.686979+0000","last_unstale":"2024-09-16T00:13:04.611584+0000","last_undegraded":"2024-09-16T00:13:04.611584+0000","last_fullsized":"2024-09-16T00:13:04.611584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:39.562454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664652+0000","last_change":"2024-09-16T00:11:04.639301+0000","last_active":"2024-09-16T00:13:04.664652+0000","last_peered":"2024-09-16T00:13:04.664652+0000","last_clean":"2024-09-16T00:13:04.664652+0000","last_became_active":"2024-09-16T00:11:04.638812+0000","last_became_peered":"2024-09-16T00:11:04.638812+0000","last_unstale":"2024-09-16T00:13:04.664652+0000","last_undegraded":"2024-09-16T00:13:04.664652+0000","last_fullsized":"2024-09-16T00:13:04.664652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:17.286074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:15.431074+0000","last_change":"2024-09-16T00:11:02.619233+0000","last_active":"2024-09-16T00:13:15.431074+0000","last_peered":"2024-09-16T00:13:15.431074+0000","last_clean":"2024-09-16T00:13:15.431074+0000","last_became_active":"2024-09-16T00:11:02.618724+0000","last_became_peered":"2024-09-16T00:11:02.618724+0000","last_unstale":"2024-09-16T00:13:15.431074+0000","last_undegraded":"2024-09-16T00:13:15.431074+0000","last_fullsized":"2024-09-16T00:13:15.431074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:43.096003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611391+0000","last_change":"2024-09-16T00:11:11.681641+0000","last_active":"2024-09-16T00:13:04.611391+0000","last_peered":"2024-09-16T00:13:04.611391+0000","last_clean":"2024-09-16T00:13:04.611391+0000","last_became_active":"2024-09-16T00:11:11.680713+0000","last_became_peered":"2024-09-16T00:11:11.680713+0000","last_unstale":"2024-09-16T00:13:04.611391+0000","last_undegraded":"2024-09-16T00:13:04.611391+0000","last_fullsized":"2024-09-16T00:13:04.611391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:01.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663782+0000","last_change":"2024-09-16T00:11:04.641042+0000","last_active":"2024-09-16T00:13:04.663782+0000","last_peered":"2024-09-16T00:13:04.663782+0000","last_clean":"2024-09-16T00:13:04.663782+0000","last_became_active":"2024-09-16T00:11:04.640610+0000","last_became_peered":"2024-09-16T00:11:04.640610+0000","last_unstale":"2024-09-16T00:13:04.663782+0000","last_undegraded":"2024-09-16T00:13:04.663782+0000","last_fullsized":"2024-09-16T00:13:04.663782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:29:14.510990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664498+0000","last_change":"2024-09-16T00:11:02.615794+0000","last_active":"2024-09-16T00:13:04.664498+0000","last_peered":"2024-09-16T00:13:04.664498+0000","last_clean":"2024-09-16T00:13:04.664498+0000","last_became_active":"2024-09-16T00:11:02.614742+0000","last_became_peered":"2024-09-16T00:11:02.614742+0000","last_unstale":"2024-09-16T00:13:04.664498+0000","last_undegraded":"2024-09-16T00:13:04.664498+0000","last_fullsized":"2024-09-16T00:13:04.664498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:42.767506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609374+0000","last_change":"2024-09-16T00:11:11.686604+0000","last_active":"2024-09-16T00:13:04.609374+0000","last_peered":"2024-09-16T00:13:04.609374+0000","last_clean":"2024-09-16T00:13:04.609374+0000","last_became_active":"2024-09-16T00:11:11.686219+0000","last_became_peered":"2024-09-16T00:11:11.686219+0000","last_unstale":"2024-09-16T00:13:04.609374+0000","last_undegraded":"2024-09-16T00:13:04.609374+0000","last_fullsized":"2024-09-16T00:13:04.609374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:48.636298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.642630+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.642260+0000","last_became_peered":"2024-09-16T00:11:04.642260+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:06.580702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605426+0000","last_change":"2024-09-16T00:12:04.034002+0000","last_active":"2024-09-16T00:13:04.605426+0000","last_peered":"2024-09-16T00:13:04.605426+0000","last_clean":"2024-09-16T00:13:04.605426+0000","last_became_active":"2024-09-16T00:12:04.030959+0000","last_became_peered":"2024-09-16T00:12:04.030959+0000","last_unstale":"2024-09-16T00:13:04.605426+0000","last_undegraded":"2024-09-16T00:13:04.605426+0000","last_fullsized":"2024-09-16T00:13:04.605426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:47.730692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611324+0000","last_change":"2024-09-16T00:12:04.029906+0000","last_active":"2024-09-16T00:13:04.611324+0000","last_peered":"2024-09-16T00:13:04.611324+0000","last_clean":"2024-09-16T00:13:04.611324+0000","last_became_active":"2024-09-16T00:12:04.029259+0000","last_became_peered":"2024-09-16T00:12:04.029259+0000","last_unstale":"2024-09-16T00:13:04.611324+0000","last_undegraded":"2024-09-16T00:13:04.611324+0000","last_fullsized":"2024-09-16T00:13:04.611324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:23.399762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,1],"acting":[4,8,10,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663413+0000","last_change":"2024-09-16T00:11:04.649272+0000","last_active":"2024-09-16T00:13:04.663413+0000","last_peered":"2024-09-16T00:13:04.663413+0000","last_clean":"2024-09-16T00:13:04.663413+0000","last_became_active":"2024-09-16T00:11:04.648967+0000","last_became_peered":"2024-09-16T00:11:04.648967+0000","last_unstale":"2024-09-16T00:13:04.663413+0000","last_undegraded":"2024-09-16T00:13:04.663413+0000","last_fullsized":"2024-09-16T00:13:04.663413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:38:59.152507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611856+0000","last_change":"2024-09-16T00:11:03.000121+0000","last_active":"2024-09-16T00:13:04.611856+0000","last_peered":"2024-09-16T00:13:04.611856+0000","last_clean":"2024-09-16T00:13:04.611856+0000","last_became_active":"2024-09-16T00:11:02.999812+0000","last_became_peered":"2024-09-16T00:11:02.999812+0000","last_unstale":"2024-09-16T00:13:04.611856+0000","last_undegraded":"2024-09-16T00:13:04.611856+0000","last_fullsized":"2024-09-16T00:13:04.611856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:10.919517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610909+0000","last_change":"2024-09-16T00:11:11.686331+0000","last_active":"2024-09-16T00:13:04.610909+0000","last_peered":"2024-09-16T00:13:04.610909+0000","last_clean":"2024-09-16T00:13:04.610909+0000","last_became_active":"2024-09-16T00:11:11.685912+0000","last_became_peered":"2024-09-16T00:11:11.685912+0000","last_unstale":"2024-09-16T00:13:04.610909+0000","last_undegraded":"2024-09-16T00:13:04.610909+0000","last_fullsized":"2024-09-16T00:13:04.610909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:39.993302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.665046+0000","last_change":"2024-09-16T00:11:04.644514+0000","last_active":"2024-09-16T00:13:04.665046+0000","last_peered":"2024-09-16T00:13:04.665046+0000","last_clean":"2024-09-16T00:13:04.665046+0000","last_became_active":"2024-09-16T00:11:04.644225+0000","last_became_peered":"2024-09-16T00:11:04.644225+0000","last_unstale":"2024-09-16T00:13:04.665046+0000","last_undegraded":"2024-09-16T00:13:04.665046+0000","last_fullsized":"2024-09-16T00:13:04.665046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:12.344487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'21","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:25.652660+0000","last_change":"2024-09-16T00:11:02.614313+0000","last_active":"2024-09-16T00:13:25.652660+0000","last_peered":"2024-09-16T00:13:25.652660+0000","last_clean":"2024-09-16T00:13:25.652660+0000","last_became_active":"2024-09-16T00:11:02.613918+0000","last_became_peered":"2024-09-16T00:11:02.613918+0000","last_unstale":"2024-09-16T00:13:25.652660+0000","last_undegraded":"2024-09-16T00:13:25.652660+0000","last_fullsized":"2024-09-16T00:13:25.652660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:01.727854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664101+0000","last_change":"2024-09-16T00:11:11.681254+0000","last_active":"2024-09-16T00:13:04.664101+0000","last_peered":"2024-09-16T00:13:04.664101+0000","last_clean":"2024-09-16T00:13:04.664101+0000","last_became_active":"2024-09-16T00:11:11.680871+0000","last_became_peered":"2024-09-16T00:11:11.680871+0000","last_unstale":"2024-09-16T00:13:04.664101+0000","last_undegraded":"2024-09-16T00:13:04.664101+0000","last_fullsized":"2024-09-16T00:13:04.664101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:05:16.798306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606866+0000","last_change":"2024-09-16T00:11:04.640934+0000","last_active":"2024-09-16T00:13:04.606866+0000","last_peered":"2024-09-16T00:13:04.606866+0000","last_clean":"2024-09-16T00:13:04.606866+0000","last_became_active":"2024-09-16T00:11:04.640617+0000","last_became_peered":"2024-09-16T00:11:04.640617+0000","last_unstale":"2024-09-16T00:13:04.606866+0000","last_undegraded":"2024-09-16T00:13:04.606866+0000","last_fullsized":"2024-09-16T00:13:04.606866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:39:27.532702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664634+0000","last_change":"2024-09-16T00:11:02.810123+0000","last_active":"2024-09-16T00:13:04.664634+0000","last_peered":"2024-09-16T00:13:04.664634+0000","last_clean":"2024-09-16T00:13:04.664634+0000","last_became_active":"2024-09-16T00:11:02.809443+0000","last_became_peered":"2024-09-16T00:11:02.809443+0000","last_unstale":"2024-09-16T00:13:04.664634+0000","last_undegraded":"2024-09-16T00:13:04.664634+0000","last_fullsized":"2024-09-16T00:13:04.664634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:14.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664409+0000","last_change":"2024-09-16T00:11:11.682083+0000","last_active":"2024-09-16T00:13:04.664409+0000","last_peered":"2024-09-16T00:13:04.664409+0000","last_clean":"2024-09-16T00:13:04.664409+0000","last_became_active":"2024-09-16T00:11:11.681552+0000","last_became_peered":"2024-09-16T00:11:11.681552+0000","last_unstale":"2024-09-16T00:13:04.664409+0000","last_undegraded":"2024-09-16T00:13:04.664409+0000","last_fullsized":"2024-09-16T00:13:04.664409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:10.019646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664046+0000","last_change":"2024-09-16T00:11:04.637781+0000","last_active":"2024-09-16T00:13:04.664046+0000","last_peered":"2024-09-16T00:13:04.664046+0000","last_clean":"2024-09-16T00:13:04.664046+0000","last_became_active":"2024-09-16T00:11:04.637467+0000","last_became_peered":"2024-09-16T00:11:04.637467+0000","last_unstale":"2024-09-16T00:13:04.664046+0000","last_undegraded":"2024-09-16T00:13:04.664046+0000","last_fullsized":"2024-09-16T00:13:04.664046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:54.954702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609377+0000","last_change":"2024-09-16T00:11:02.810306+0000","last_active":"2024-09-16T00:13:04.609377+0000","last_peered":"2024-09-16T00:13:04.609377+0000","last_clean":"2024-09-16T00:13:04.609377+0000","last_became_active":"2024-09-16T00:11:02.809970+0000","last_became_peered":"2024-09-16T00:11:02.809970+0000","last_unstale":"2024-09-16T00:13:04.609377+0000","last_undegraded":"2024-09-16T00:13:04.609377+0000","last_fullsized":"2024-09-16T00:13:04.609377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:24:06.208694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664159+0000","last_change":"2024-09-16T00:11:11.683554+0000","last_active":"2024-09-16T00:13:04.664159+0000","last_peered":"2024-09-16T00:13:04.664159+0000","last_clean":"2024-09-16T00:13:04.664159+0000","last_became_active":"2024-09-16T00:11:11.683156+0000","last_became_peered":"2024-09-16T00:11:11.683156+0000","last_unstale":"2024-09-16T00:13:04.664159+0000","last_undegraded":"2024-09-16T00:13:04.664159+0000","last_fullsized":"2024-09-16T00:13:04.664159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:52.688833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664057+0000","last_change":"2024-09-16T00:11:04.650139+0000","last_active":"2024-09-16T00:13:04.664057+0000","last_peered":"2024-09-16T00:13:04.664057+0000","last_clean":"2024-09-16T00:13:04.664057+0000","last_became_active":"2024-09-16T00:11:04.649772+0000","last_became_peered":"2024-09-16T00:11:04.649772+0000","last_unstale":"2024-09-16T00:13:04.664057+0000","last_undegraded":"2024-09-16T00:13:04.664057+0000","last_fullsized":"2024-09-16T00:13:04.664057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:24.872674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:31.694904+0000","last_change":"2024-09-16T00:11:02.618087+0000","last_active":"2024-09-16T00:13:31.694904+0000","last_peered":"2024-09-16T00:13:31.694904+0000","last_clean":"2024-09-16T00:13:31.694904+0000","last_became_active":"2024-09-16T00:11:02.617696+0000","last_became_peered":"2024-09-16T00:11:02.617696+0000","last_unstale":"2024-09-16T00:13:31.694904+0000","last_undegraded":"2024-09-16T00:13:31.694904+0000","last_fullsized":"2024-09-16T00:13:31.694904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:29:52.180331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606730+0000","last_change":"2024-09-16T00:11:11.680044+0000","last_active":"2024-09-16T00:13:04.606730+0000","last_peered":"2024-09-16T00:13:04.606730+0000","last_clean":"2024-09-16T00:13:04.606730+0000","last_became_active":"2024-09-16T00:11:11.679638+0000","last_became_peered":"2024-09-16T00:11:11.679638+0000","last_unstale":"2024-09-16T00:13:04.606730+0000","last_undegraded":"2024-09-16T00:13:04.606730+0000","last_fullsized":"2024-09-16T00:13:04.606730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:54.241320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605309+0000","last_change":"2024-09-16T00:11:04.650104+0000","last_active":"2024-09-16T00:13:04.605309+0000","last_peered":"2024-09-16T00:13:04.605309+0000","last_clean":"2024-09-16T00:13:04.605309+0000","last_became_active":"2024-09-16T00:11:04.649801+0000","last_became_peered":"2024-09-16T00:11:04.649801+0000","last_unstale":"2024-09-16T00:13:04.605309+0000","last_undegraded":"2024-09-16T00:13:04.605309+0000","last_fullsized":"2024-09-16T00:13:04.605309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:28.508489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:36.746691+0000","last_change":"2024-09-16T00:11:02.809205+0000","last_active":"2024-09-16T00:13:36.746691+0000","last_peered":"2024-09-16T00:13:36.746691+0000","last_clean":"2024-09-16T00:13:36.746691+0000","last_became_active":"2024-09-16T00:11:02.808866+0000","last_became_peered":"2024-09-16T00:11:02.808866+0000","last_unstale":"2024-09-16T00:13:36.746691+0000","last_undegraded":"2024-09-16T00:13:36.746691+0000","last_fullsized":"2024-09-16T00:13:36.746691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:05.041502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606979+0000","last_change":"2024-09-16T00:11:11.679168+0000","last_active":"2024-09-16T00:13:04.606979+0000","last_peered":"2024-09-16T00:13:04.606979+0000","last_clean":"2024-09-16T00:13:04.606979+0000","last_became_active":"2024-09-16T00:11:11.678843+0000","last_became_peered":"2024-09-16T00:11:11.678843+0000","last_unstale":"2024-09-16T00:13:04.606979+0000","last_undegraded":"2024-09-16T00:13:04.606979+0000","last_fullsized":"2024-09-16T00:13:04.606979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:44.617547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606488+0000","last_change":"2024-09-16T00:12:04.027859+0000","last_active":"2024-09-16T00:13:04.606488+0000","last_peered":"2024-09-16T00:13:04.606488+0000","last_clean":"2024-09-16T00:13:04.606488+0000","last_became_active":"2024-09-16T00:12:04.027506+0000","last_became_peered":"2024-09-16T00:12:04.027506+0000","last_unstale":"2024-09-16T00:13:04.606488+0000","last_undegraded":"2024-09-16T00:13:04.606488+0000","last_fullsized":"2024-09-16T00:13:04.606488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:34.450319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649981+0000","last_change":"2024-09-16T00:11:02.620778+0000","last_active":"2024-09-16T00:13:04.649981+0000","last_peered":"2024-09-16T00:13:04.649981+0000","last_clean":"2024-09-16T00:13:04.649981+0000","last_became_active":"2024-09-16T00:11:02.620467+0000","last_became_peered":"2024-09-16T00:11:02.620467+0000","last_unstale":"2024-09-16T00:13:04.649981+0000","last_undegraded":"2024-09-16T00:13:04.649981+0000","last_fullsized":"2024-09-16T00:13:04.649981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:49.771281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605678+0000","last_change":"2024-09-16T00:12:04.029471+0000","last_active":"2024-09-16T00:13:04.605678+0000","last_peered":"2024-09-16T00:13:04.605678+0000","last_clean":"2024-09-16T00:13:04.605678+0000","last_became_active":"2024-09-16T00:12:04.029054+0000","last_became_peered":"2024-09-16T00:12:04.029054+0000","last_unstale":"2024-09-16T00:13:04.605678+0000","last_undegraded":"2024-09-16T00:13:04.605678+0000","last_fullsized":"2024-09-16T00:13:04.605678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:52.054218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664553+0000","last_change":"2024-09-16T00:11:04.644143+0000","last_active":"2024-09-16T00:13:04.664553+0000","last_peered":"2024-09-16T00:13:04.664553+0000","last_clean":"2024-09-16T00:13:04.664553+0000","last_became_active":"2024-09-16T00:11:04.643598+0000","last_became_peered":"2024-09-16T00:11:04.643598+0000","last_unstale":"2024-09-16T00:13:04.664553+0000","last_undegraded":"2024-09-16T00:13:04.664553+0000","last_fullsized":"2024-09-16T00:13:04.664553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:02:00.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611832+0000","last_change":"2024-09-16T00:11:02.614246+0000","last_active":"2024-09-16T00:13:04.611832+0000","last_peered":"2024-09-16T00:13:04.611832+0000","last_clean":"2024-09-16T00:13:04.611832+0000","last_became_active":"2024-09-16T00:11:02.613784+0000","last_became_peered":"2024-09-16T00:11:02.613784+0000","last_unstale":"2024-09-16T00:13:04.611832+0000","last_undegraded":"2024-09-16T00:13:04.611832+0000","last_fullsized":"2024-09-16T00:13:04.611832+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:56.397435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649917+0000","last_change":"2024-09-16T00:09:47.914919+0000","last_active":"2024-09-16T00:13:04.649917+0000","last_peered":"2024-09-16T00:13:04.649917+0000","last_clean":"2024-09-16T00:13:04.649917+0000","last_became_active":"2024-09-16T00:09:47.914389+0000","last_became_peered":"2024-09-16T00:09:47.914389+0000","last_unstale":"2024-09-16T00:13:04.649917+0000","last_undegraded":"2024-09-16T00:13:04.649917+0000","last_fullsized":"2024-09-16T00:13:04.649917+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:06:21.032381+0000","last_clean_scrub_stamp":"2024-09-16T00:06:21.032381+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:26.558186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650098+0000","last_change":"2024-09-16T00:11:11.682930+0000","last_active":"2024-09-16T00:13:04.650098+0000","last_peered":"2024-09-16T00:13:04.650098+0000","last_clean":"2024-09-16T00:13:04.650098+0000","last_became_active":"2024-09-16T00:11:11.682504+0000","last_became_peered":"2024-09-16T00:11:11.682504+0000","last_unstale":"2024-09-16T00:13:04.650098+0000","last_undegraded":"2024-09-16T00:13:04.650098+0000","last_fullsized":"2024-09-16T00:13:04.650098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.693538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649329+0000","last_change":"2024-09-16T00:11:04.640008+0000","last_active":"2024-09-16T00:13:04.649329+0000","last_peered":"2024-09-16T00:13:04.649329+0000","last_clean":"2024-09-16T00:13:04.649329+0000","last_became_active":"2024-09-16T00:11:04.639653+0000","last_became_peered":"2024-09-16T00:11:04.639653+0000","last_unstale":"2024-09-16T00:13:04.649329+0000","last_undegraded":"2024-09-16T00:13:04.649329+0000","last_fullsized":"2024-09-16T00:13:04.649329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:42.375048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:33.709720+0000","last_change":"2024-09-16T00:11:03.000011+0000","last_active":"2024-09-16T00:13:33.709720+0000","last_peered":"2024-09-16T00:13:33.709720+0000","last_clean":"2024-09-16T00:13:33.709720+0000","last_became_active":"2024-09-16T00:11:02.999538+0000","last_became_peered":"2024-09-16T00:11:02.999538+0000","last_unstale":"2024-09-16T00:13:33.709720+0000","last_undegraded":"2024-09-16T00:13:33.709720+0000","last_fullsized":"2024-09-16T00:13:33.709720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:55.226557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605291+0000","last_change":"2024-09-16T00:11:11.678584+0000","last_active":"2024-09-16T00:13:04.605291+0000","last_peered":"2024-09-16T00:13:04.605291+0000","last_clean":"2024-09-16T00:13:04.605291+0000","last_became_active":"2024-09-16T00:11:11.678212+0000","last_became_peered":"2024-09-16T00:11:11.678212+0000","last_unstale":"2024-09-16T00:13:04.605291+0000","last_undegraded":"2024-09-16T00:13:04.605291+0000","last_fullsized":"2024-09-16T00:13:04.605291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:28:06.993200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649345+0000","last_change":"2024-09-16T00:11:04.641594+0000","last_active":"2024-09-16T00:13:04.649345+0000","last_peered":"2024-09-16T00:13:04.649345+0000","last_clean":"2024-09-16T00:13:04.649345+0000","last_became_active":"2024-09-16T00:11:04.641267+0000","last_became_peered":"2024-09-16T00:11:04.641267+0000","last_unstale":"2024-09-16T00:13:04.649345+0000","last_undegraded":"2024-09-16T00:13:04.649345+0000","last_fullsized":"2024-09-16T00:13:04.649345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:31.792850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605828+0000","last_change":"2024-09-16T00:11:02.618910+0000","last_active":"2024-09-16T00:13:04.605828+0000","last_peered":"2024-09-16T00:13:04.605828+0000","last_clean":"2024-09-16T00:13:04.605828+0000","last_became_active":"2024-09-16T00:11:02.617907+0000","last_became_peered":"2024-09-16T00:11:02.617907+0000","last_unstale":"2024-09-16T00:13:04.605828+0000","last_undegraded":"2024-09-16T00:13:04.605828+0000","last_fullsized":"2024-09-16T00:13:04.605828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:18:19.906970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663968+0000","last_change":"2024-09-16T00:11:11.679051+0000","last_active":"2024-09-16T00:13:04.663968+0000","last_peered":"2024-09-16T00:13:04.663968+0000","last_clean":"2024-09-16T00:13:04.663968+0000","last_became_active":"2024-09-16T00:11:11.678717+0000","last_became_peered":"2024-09-16T00:11:11.678717+0000","last_unstale":"2024-09-16T00:13:04.663968+0000","last_undegraded":"2024-09-16T00:13:04.663968+0000","last_fullsized":"2024-09-16T00:13:04.663968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:08.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650076+0000","last_change":"2024-09-16T00:11:04.643010+0000","last_active":"2024-09-16T00:13:04.650076+0000","last_peered":"2024-09-16T00:13:04.650076+0000","last_clean":"2024-09-16T00:13:04.650076+0000","last_became_active":"2024-09-16T00:11:04.642610+0000","last_became_peered":"2024-09-16T00:11:04.642610+0000","last_unstale":"2024-09-16T00:13:04.650076+0000","last_undegraded":"2024-09-16T00:13:04.650076+0000","last_fullsized":"2024-09-16T00:13:04.650076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:49.870634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.607017+0000","last_change":"2024-09-16T00:11:02.619692+0000","last_active":"2024-09-16T00:13:04.607017+0000","last_peered":"2024-09-16T00:13:04.607017+0000","last_clean":"2024-09-16T00:13:04.607017+0000","last_became_active":"2024-09-16T00:11:02.619410+0000","last_became_peered":"2024-09-16T00:11:02.619410+0000","last_unstale":"2024-09-16T00:13:04.607017+0000","last_undegraded":"2024-09-16T00:13:04.607017+0000","last_fullsized":"2024-09-16T00:13:04.607017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:44.751121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605023+0000","last_change":"2024-09-16T00:11:11.687430+0000","last_active":"2024-09-16T00:13:04.605023+0000","last_peered":"2024-09-16T00:13:04.605023+0000","last_clean":"2024-09-16T00:13:04.605023+0000","last_became_active":"2024-09-16T00:11:11.687050+0000","last_became_peered":"2024-09-16T00:11:11.687050+0000","last_unstale":"2024-09-16T00:13:04.605023+0000","last_undegraded":"2024-09-16T00:13:04.605023+0000","last_fullsized":"2024-09-16T00:13:04.605023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:35.745658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663799+0000","last_change":"2024-09-16T00:11:04.651449+0000","last_active":"2024-09-16T00:13:04.663799+0000","last_peered":"2024-09-16T00:13:04.663799+0000","last_clean":"2024-09-16T00:13:04.663799+0000","last_became_active":"2024-09-16T00:11:04.651128+0000","last_became_peered":"2024-09-16T00:11:04.651128+0000","last_unstale":"2024-09-16T00:13:04.663799+0000","last_undegraded":"2024-09-16T00:13:04.663799+0000","last_fullsized":"2024-09-16T00:13:04.663799+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:34:33.767253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:19.532889+0000","last_change":"2024-09-16T00:11:02.617489+0000","last_active":"2024-09-16T00:13:19.532889+0000","last_peered":"2024-09-16T00:13:19.532889+0000","last_clean":"2024-09-16T00:13:19.532889+0000","last_became_active":"2024-09-16T00:11:02.617123+0000","last_became_peered":"2024-09-16T00:11:02.617123+0000","last_unstale":"2024-09-16T00:13:19.532889+0000","last_undegraded":"2024-09-16T00:13:19.532889+0000","last_fullsized":"2024-09-16T00:13:19.532889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:01.621660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605103+0000","last_change":"2024-09-16T00:11:11.678860+0000","last_active":"2024-09-16T00:13:04.605103+0000","last_peered":"2024-09-16T00:13:04.605103+0000","last_clean":"2024-09-16T00:13:04.605103+0000","last_became_active":"2024-09-16T00:11:11.678434+0000","last_became_peered":"2024-09-16T00:11:11.678434+0000","last_unstale":"2024-09-16T00:13:04.605103+0000","last_undegraded":"2024-09-16T00:13:04.605103+0000","last_fullsized":"2024-09-16T00:13:04.605103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:40.768022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605218+0000","last_change":"2024-09-16T00:11:04.649338+0000","last_active":"2024-09-16T00:13:04.605218+0000","last_peered":"2024-09-16T00:13:04.605218+0000","last_clean":"2024-09-16T00:13:04.605218+0000","last_became_active":"2024-09-16T00:11:04.649029+0000","last_became_peered":"2024-09-16T00:11:04.649029+0000","last_unstale":"2024-09-16T00:13:04.605218+0000","last_undegraded":"2024-09-16T00:13:04.605218+0000","last_fullsized":"2024-09-16T00:13:04.605218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:33.896640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664161+0000","last_change":"2024-09-16T00:11:02.809797+0000","last_active":"2024-09-16T00:13:04.664161+0000","last_peered":"2024-09-16T00:13:04.664161+0000","last_clean":"2024-09-16T00:13:04.664161+0000","last_became_active":"2024-09-16T00:11:02.809151+0000","last_became_peered":"2024-09-16T00:11:02.809151+0000","last_unstale":"2024-09-16T00:13:04.664161+0000","last_undegraded":"2024-09-16T00:13:04.664161+0000","last_fullsized":"2024-09-16T00:13:04.664161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:46.512791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.607004+0000","last_change":"2024-09-16T00:11:11.677610+0000","last_active":"2024-09-16T00:13:04.607004+0000","last_peered":"2024-09-16T00:13:04.607004+0000","last_clean":"2024-09-16T00:13:04.607004+0000","last_became_active":"2024-09-16T00:11:11.677275+0000","last_became_peered":"2024-09-16T00:11:11.677275+0000","last_unstale":"2024-09-16T00:13:04.607004+0000","last_undegraded":"2024-09-16T00:13:04.607004+0000","last_fullsized":"2024-09-16T00:13:04.607004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:48.291701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649323+0000","last_change":"2024-09-16T00:11:04.645611+0000","last_active":"2024-09-16T00:13:04.649323+0000","last_peered":"2024-09-16T00:13:04.649323+0000","last_clean":"2024-09-16T00:13:04.649323+0000","last_became_active":"2024-09-16T00:11:04.645276+0000","last_became_peered":"2024-09-16T00:11:04.645276+0000","last_unstale":"2024-09-16T00:13:04.649323+0000","last_undegraded":"2024-09-16T00:13:04.649323+0000","last_fullsized":"2024-09-16T00:13:04.649323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:22.624488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:36.747287+0000","last_change":"2024-09-16T00:11:02.617673+0000","last_active":"2024-09-16T00:13:36.747287+0000","last_peered":"2024-09-16T00:13:36.747287+0000","last_clean":"2024-09-16T00:13:36.747287+0000","last_became_active":"2024-09-16T00:11:02.617261+0000","last_became_peered":"2024-09-16T00:11:02.617261+0000","last_unstale":"2024-09-16T00:13:36.747287+0000","last_undegraded":"2024-09-16T00:13:36.747287+0000","last_fullsized":"2024-09-16T00:13:36.747287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:38.537456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609415+0000","last_change":"2024-09-16T00:11:11.680173+0000","last_active":"2024-09-16T00:13:04.609415+0000","last_peered":"2024-09-16T00:13:04.609415+0000","last_clean":"2024-09-16T00:13:04.609415+0000","last_became_active":"2024-09-16T00:11:11.679453+0000","last_became_peered":"2024-09-16T00:11:11.679453+0000","last_unstale":"2024-09-16T00:13:04.609415+0000","last_undegraded":"2024-09-16T00:13:04.609415+0000","last_fullsized":"2024-09-16T00:13:04.609415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:33:07.321347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649661+0000","last_change":"2024-09-16T00:11:04.649242+0000","last_active":"2024-09-16T00:13:04.649661+0000","last_peered":"2024-09-16T00:13:04.649661+0000","last_clean":"2024-09-16T00:13:04.649661+0000","last_became_active":"2024-09-16T00:11:04.648895+0000","last_became_peered":"2024-09-16T00:11:04.648895+0000","last_unstale":"2024-09-16T00:13:04.649661+0000","last_undegraded":"2024-09-16T00:13:04.649661+0000","last_fullsized":"2024-09-16T00:13:04.649661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:03:05.023653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605440+0000","last_change":"2024-09-16T00:11:02.616254+0000","last_active":"2024-09-16T00:13:04.605440+0000","last_peered":"2024-09-16T00:13:04.605440+0000","last_clean":"2024-09-16T00:13:04.605440+0000","last_became_active":"2024-09-16T00:11:02.615847+0000","last_became_peered":"2024-09-16T00:11:02.615847+0000","last_unstale":"2024-09-16T00:13:04.605440+0000","last_undegraded":"2024-09-16T00:13:04.605440+0000","last_fullsized":"2024-09-16T00:13:04.605440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:28.437446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664603+0000","last_change":"2024-09-16T00:11:11.679678+0000","last_active":"2024-09-16T00:13:04.664603+0000","last_peered":"2024-09-16T00:13:04.664603+0000","last_clean":"2024-09-16T00:13:04.664603+0000","last_became_active":"2024-09-16T00:11:11.679331+0000","last_became_peered":"2024-09-16T00:11:11.679331+0000","last_unstale":"2024-09-16T00:13:04.664603+0000","last_undegraded":"2024-09-16T00:13:04.664603+0000","last_fullsized":"2024-09-16T00:13:04.664603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:53:40.102112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606631+0000","last_change":"2024-09-16T00:11:04.637472+0000","last_active":"2024-09-16T00:13:04.606631+0000","last_peered":"2024-09-16T00:13:04.606631+0000","last_clean":"2024-09-16T00:13:04.606631+0000","last_became_active":"2024-09-16T00:11:04.637210+0000","last_became_peered":"2024-09-16T00:11:04.637210+0000","last_unstale":"2024-09-16T00:13:04.606631+0000","last_undegraded":"2024-09-16T00:13:04.606631+0000","last_fullsized":"2024-09-16T00:13:04.606631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:06.689449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611195+0000","last_change":"2024-09-16T00:11:02.616574+0000","last_active":"2024-09-16T00:13:04.611195+0000","last_peered":"2024-09-16T00:13:04.611195+0000","last_clean":"2024-09-16T00:13:04.611195+0000","last_became_active":"2024-09-16T00:11:02.616246+0000","last_became_peered":"2024-09-16T00:11:02.616246+0000","last_unstale":"2024-09-16T00:13:04.611195+0000","last_undegraded":"2024-09-16T00:13:04.611195+0000","last_fullsized":"2024-09-16T00:13:04.611195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:15.000147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605618+0000","last_change":"2024-09-16T00:11:11.678927+0000","last_active":"2024-09-16T00:13:04.605618+0000","last_peered":"2024-09-16T00:13:04.605618+0000","last_clean":"2024-09-16T00:13:04.605618+0000","last_became_active":"2024-09-16T00:11:11.678540+0000","last_became_peered":"2024-09-16T00:11:11.678540+0000","last_unstale":"2024-09-16T00:13:04.605618+0000","last_undegraded":"2024-09-16T00:13:04.605618+0000","last_fullsized":"2024-09-16T00:13:04.605618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:55.650795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664419+0000","last_change":"2024-09-16T00:11:04.646038+0000","last_active":"2024-09-16T00:13:04.664419+0000","last_peered":"2024-09-16T00:13:04.664419+0000","last_clean":"2024-09-16T00:13:04.664419+0000","last_became_active":"2024-09-16T00:11:04.645033+0000","last_became_peered":"2024-09-16T00:11:04.645033+0000","last_unstale":"2024-09-16T00:13:04.664419+0000","last_undegraded":"2024-09-16T00:13:04.664419+0000","last_fullsized":"2024-09-16T00:13:04.664419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:58.319233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'43","reported_seq":321,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:34.711137+0000","last_change":"2024-09-16T00:11:02.621244+0000","last_active":"2024-09-16T00:13:34.711137+0000","last_peered":"2024-09-16T00:13:34.711137+0000","last_clean":"2024-09-16T00:13:34.711137+0000","last_became_active":"2024-09-16T00:11:02.620786+0000","last_became_peered":"2024-09-16T00:11:02.620786+0000","last_unstale":"2024-09-16T00:13:34.711137+0000","last_undegraded":"2024-09-16T00:13:34.711137+0000","last_fullsized":"2024-09-16T00:13:34.711137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:43.049606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71853,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":190,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609191+0000","last_change":"2024-09-16T00:11:11.681381+0000","last_active":"2024-09-16T00:13:04.609191+0000","last_peered":"2024-09-16T00:13:04.609191+0000","last_clean":"2024-09-16T00:13:04.609191+0000","last_became_active":"2024-09-16T00:11:11.680919+0000","last_became_peered":"2024-09-16T00:11:11.680919+0000","last_unstale":"2024-09-16T00:13:04.609191+0000","last_undegraded":"2024-09-16T00:13:04.609191+0000","last_fullsized":"2024-09-16T00:13:04.609191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:11.688564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605601+0000","last_change":"2024-09-16T00:11:04.647123+0000","last_active":"2024-09-16T00:13:04.605601+0000","last_peered":"2024-09-16T00:13:04.605601+0000","last_clean":"2024-09-16T00:13:04.605601+0000","last_became_active":"2024-09-16T00:11:04.646562+0000","last_became_peered":"2024-09-16T00:11:04.646562+0000","last_unstale":"2024-09-16T00:13:04.605601+0000","last_undegraded":"2024-09-16T00:13:04.605601+0000","last_fullsized":"2024-09-16T00:13:04.605601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:36.297904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605792+0000","last_change":"2024-09-16T00:11:02.809564+0000","last_active":"2024-09-16T00:13:04.605792+0000","last_peered":"2024-09-16T00:13:04.605792+0000","last_clean":"2024-09-16T00:13:04.605792+0000","last_became_active":"2024-09-16T00:11:02.809093+0000","last_became_peered":"2024-09-16T00:11:02.809093+0000","last_unstale":"2024-09-16T00:13:04.605792+0000","last_undegraded":"2024-09-16T00:13:04.605792+0000","last_fullsized":"2024-09-16T00:13:04.605792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:11.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611640+0000","last_change":"2024-09-16T00:11:11.686326+0000","last_active":"2024-09-16T00:13:04.611640+0000","last_peered":"2024-09-16T00:13:04.611640+0000","last_clean":"2024-09-16T00:13:04.611640+0000","last_became_active":"2024-09-16T00:11:11.685875+0000","last_became_peered":"2024-09-16T00:11:11.685875+0000","last_unstale":"2024-09-16T00:13:04.611640+0000","last_undegraded":"2024-09-16T00:13:04.611640+0000","last_fullsized":"2024-09-16T00:13:04.611640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:57.635063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663456+0000","last_change":"2024-09-16T00:11:04.644839+0000","last_active":"2024-09-16T00:13:04.663456+0000","last_peered":"2024-09-16T00:13:04.663456+0000","last_clean":"2024-09-16T00:13:04.663456+0000","last_became_active":"2024-09-16T00:11:04.642277+0000","last_became_peered":"2024-09-16T00:11:04.642277+0000","last_unstale":"2024-09-16T00:13:04.663456+0000","last_undegraded":"2024-09-16T00:13:04.663456+0000","last_fullsized":"2024-09-16T00:13:04.663456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:18.374328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.650055+0000","last_change":"2024-09-16T00:11:02.619839+0000","last_active":"2024-09-16T00:13:04.650055+0000","last_peered":"2024-09-16T00:13:04.650055+0000","last_clean":"2024-09-16T00:13:04.650055+0000","last_became_active":"2024-09-16T00:11:02.619479+0000","last_became_peered":"2024-09-16T00:11:02.619479+0000","last_unstale":"2024-09-16T00:13:04.650055+0000","last_undegraded":"2024-09-16T00:13:04.650055+0000","last_fullsized":"2024-09-16T00:13:04.650055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:43:11.420981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.608952+0000","last_change":"2024-09-16T00:11:11.688281+0000","last_active":"2024-09-16T00:13:04.608952+0000","last_peered":"2024-09-16T00:13:04.608952+0000","last_clean":"2024-09-16T00:13:04.608952+0000","last_became_active":"2024-09-16T00:11:11.687907+0000","last_became_peered":"2024-09-16T00:11:11.687907+0000","last_unstale":"2024-09-16T00:13:04.608952+0000","last_undegraded":"2024-09-16T00:13:04.608952+0000","last_fullsized":"2024-09-16T00:13:04.608952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:22.196139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605753+0000","last_change":"2024-09-16T00:11:04.645958+0000","last_active":"2024-09-16T00:13:04.605753+0000","last_peered":"2024-09-16T00:13:04.605753+0000","last_clean":"2024-09-16T00:13:04.605753+0000","last_became_active":"2024-09-16T00:11:04.645599+0000","last_became_peered":"2024-09-16T00:11:04.645599+0000","last_unstale":"2024-09-16T00:13:04.605753+0000","last_undegraded":"2024-09-16T00:13:04.605753+0000","last_fullsized":"2024-09-16T00:13:04.605753+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:27:10.606395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":269,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:34.710559+0000","last_change":"2024-09-16T00:11:02.621421+0000","last_active":"2024-09-16T00:13:34.710559+0000","last_peered":"2024-09-16T00:13:34.710559+0000","last_clean":"2024-09-16T00:13:34.710559+0000","last_became_active":"2024-09-16T00:11:02.620943+0000","last_became_peered":"2024-09-16T00:11:02.620943+0000","last_unstale":"2024-09-16T00:13:34.710559+0000","last_undegraded":"2024-09-16T00:13:34.710559+0000","last_fullsized":"2024-09-16T00:13:34.710559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:10.777143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664396+0000","last_change":"2024-09-16T00:11:11.679501+0000","last_active":"2024-09-16T00:13:04.664396+0000","last_peered":"2024-09-16T00:13:04.664396+0000","last_clean":"2024-09-16T00:13:04.664396+0000","last_became_active":"2024-09-16T00:11:11.677977+0000","last_became_peered":"2024-09-16T00:11:11.677977+0000","last_unstale":"2024-09-16T00:13:04.664396+0000","last_undegraded":"2024-09-16T00:13:04.664396+0000","last_fullsized":"2024-09-16T00:13:04.664396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:42.799599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663978+0000","last_change":"2024-09-16T00:11:11.679036+0000","last_active":"2024-09-16T00:13:04.663978+0000","last_peered":"2024-09-16T00:13:04.663978+0000","last_clean":"2024-09-16T00:13:04.663978+0000","last_became_active":"2024-09-16T00:11:11.678667+0000","last_became_peered":"2024-09-16T00:11:11.678667+0000","last_unstale":"2024-09-16T00:13:04.663978+0000","last_undegraded":"2024-09-16T00:13:04.663978+0000","last_fullsized":"2024-09-16T00:13:04.663978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:07.771996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609055+0000","last_change":"2024-09-16T00:11:04.643534+0000","last_active":"2024-09-16T00:13:04.609055+0000","last_peered":"2024-09-16T00:13:04.609055+0000","last_clean":"2024-09-16T00:13:04.609055+0000","last_became_active":"2024-09-16T00:11:04.643215+0000","last_became_peered":"2024-09-16T00:11:04.643215+0000","last_unstale":"2024-09-16T00:13:04.609055+0000","last_undegraded":"2024-09-16T00:13:04.609055+0000","last_fullsized":"2024-09-16T00:13:04.609055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:52.383896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605507+0000","last_change":"2024-09-16T00:11:02.618306+0000","last_active":"2024-09-16T00:13:04.605507+0000","last_peered":"2024-09-16T00:13:04.605507+0000","last_clean":"2024-09-16T00:13:04.605507+0000","last_became_active":"2024-09-16T00:11:02.617697+0000","last_became_peered":"2024-09-16T00:11:02.617697+0000","last_unstale":"2024-09-16T00:13:04.605507+0000","last_undegraded":"2024-09-16T00:13:04.605507+0000","last_fullsized":"2024-09-16T00:13:04.605507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:13.620345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664904+0000","last_change":"2024-09-16T00:11:11.682926+0000","last_active":"2024-09-16T00:13:04.664904+0000","last_peered":"2024-09-16T00:13:04.664904+0000","last_clean":"2024-09-16T00:13:04.664904+0000","last_became_active":"2024-09-16T00:11:11.682201+0000","last_became_peered":"2024-09-16T00:11:11.682201+0000","last_unstale":"2024-09-16T00:13:04.664904+0000","last_undegraded":"2024-09-16T00:13:04.664904+0000","last_fullsized":"2024-09-16T00:13:04.664904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:29.210012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663729+0000","last_change":"2024-09-16T00:11:04.640123+0000","last_active":"2024-09-16T00:13:04.663729+0000","last_peered":"2024-09-16T00:13:04.663729+0000","last_clean":"2024-09-16T00:13:04.663729+0000","last_became_active":"2024-09-16T00:11:04.639853+0000","last_became_peered":"2024-09-16T00:11:04.639853+0000","last_unstale":"2024-09-16T00:13:04.663729+0000","last_undegraded":"2024-09-16T00:13:04.663729+0000","last_fullsized":"2024-09-16T00:13:04.663729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:23.378924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606517+0000","last_change":"2024-09-16T00:11:02.613139+0000","last_active":"2024-09-16T00:13:04.606517+0000","last_peered":"2024-09-16T00:13:04.606517+0000","last_clean":"2024-09-16T00:13:04.606517+0000","last_became_active":"2024-09-16T00:11:02.612664+0000","last_became_peered":"2024-09-16T00:11:02.612664+0000","last_unstale":"2024-09-16T00:13:04.606517+0000","last_undegraded":"2024-09-16T00:13:04.606517+0000","last_fullsized":"2024-09-16T00:13:04.606517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:31:12.414173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664822+0000","last_change":"2024-09-16T00:11:02.619313+0000","last_active":"2024-09-16T00:13:04.664822+0000","last_peered":"2024-09-16T00:13:04.664822+0000","last_clean":"2024-09-16T00:13:04.664822+0000","last_became_active":"2024-09-16T00:11:02.618941+0000","last_became_peered":"2024-09-16T00:11:02.618941+0000","last_unstale":"2024-09-16T00:13:04.664822+0000","last_undegraded":"2024-09-16T00:13:04.664822+0000","last_fullsized":"2024-09-16T00:13:04.664822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:17.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605506+0000","last_change":"2024-09-16T00:11:04.649352+0000","last_active":"2024-09-16T00:13:04.605506+0000","last_peered":"2024-09-16T00:13:04.605506+0000","last_clean":"2024-09-16T00:13:04.605506+0000","last_became_active":"2024-09-16T00:11:04.649060+0000","last_became_peered":"2024-09-16T00:11:04.649060+0000","last_unstale":"2024-09-16T00:13:04.605506+0000","last_undegraded":"2024-09-16T00:13:04.605506+0000","last_fullsized":"2024-09-16T00:13:04.605506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:56:34.304450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610913+0000","last_change":"2024-09-16T00:11:11.689377+0000","last_active":"2024-09-16T00:13:04.610913+0000","last_peered":"2024-09-16T00:13:04.610913+0000","last_clean":"2024-09-16T00:13:04.610913+0000","last_became_active":"2024-09-16T00:11:11.688986+0000","last_became_peered":"2024-09-16T00:11:11.688986+0000","last_unstale":"2024-09-16T00:13:04.610913+0000","last_undegraded":"2024-09-16T00:13:04.610913+0000","last_fullsized":"2024-09-16T00:13:04.610913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:31.691709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:35.712969+0000","last_change":"2024-09-16T00:11:02.620113+0000","last_active":"2024-09-16T00:13:35.712969+0000","last_peered":"2024-09-16T00:13:35.712969+0000","last_clean":"2024-09-16T00:13:35.712969+0000","last_became_active":"2024-09-16T00:11:02.619696+0000","last_became_peered":"2024-09-16T00:11:02.619696+0000","last_unstale":"2024-09-16T00:13:35.712969+0000","last_undegraded":"2024-09-16T00:13:35.712969+0000","last_fullsized":"2024-09-16T00:13:35.712969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:01.600365+0000","last_clean_scrub_stamp":"2024-09-16T00:11:01.600365+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:59.281112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663780+0000","last_change":"2024-09-16T00:11:04.636372+0000","last_active":"2024-09-16T00:13:04.663780+0000","last_peered":"2024-09-16T00:13:04.663780+0000","last_clean":"2024-09-16T00:13:04.663780+0000","last_became_active":"2024-09-16T00:11:04.635666+0000","last_became_peered":"2024-09-16T00:11:04.635666+0000","last_unstale":"2024-09-16T00:13:04.663780+0000","last_undegraded":"2024-09-16T00:13:04.663780+0000","last_fullsized":"2024-09-16T00:13:04.663780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:56.242142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611136+0000","last_change":"2024-09-16T00:11:11.689864+0000","last_active":"2024-09-16T00:13:04.611136+0000","last_peered":"2024-09-16T00:13:04.611136+0000","last_clean":"2024-09-16T00:13:04.611136+0000","last_became_active":"2024-09-16T00:11:11.689449+0000","last_became_peered":"2024-09-16T00:11:11.689449+0000","last_unstale":"2024-09-16T00:13:04.611136+0000","last_undegraded":"2024-09-16T00:13:04.611136+0000","last_fullsized":"2024-09-16T00:13:04.611136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:10.657468+0000","last_clean_scrub_stamp":"2024-09-16T00:11:10.657468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:05.704252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663783+0000","last_change":"2024-09-16T00:11:04.642803+0000","last_active":"2024-09-16T00:13:04.663783+0000","last_peered":"2024-09-16T00:13:04.663783+0000","last_clean":"2024-09-16T00:13:04.663783+0000","last_became_active":"2024-09-16T00:11:04.642469+0000","last_became_peered":"2024-09-16T00:11:04.642469+0000","last_unstale":"2024-09-16T00:13:04.663783+0000","last_undegraded":"2024-09-16T00:13:04.663783+0000","last_fullsized":"2024-09-16T00:13:04.663783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:24.722449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611426+0000","last_change":"2024-09-16T00:11:04.645390+0000","last_active":"2024-09-16T00:13:04.611426+0000","last_peered":"2024-09-16T00:13:04.611426+0000","last_clean":"2024-09-16T00:13:04.611426+0000","last_became_active":"2024-09-16T00:11:04.645057+0000","last_became_peered":"2024-09-16T00:11:04.645057+0000","last_unstale":"2024-09-16T00:13:04.611426+0000","last_undegraded":"2024-09-16T00:13:04.611426+0000","last_fullsized":"2024-09-16T00:13:04.611426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:48.755317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.610934+0000","last_change":"2024-09-16T00:11:04.641139+0000","last_active":"2024-09-16T00:13:04.610934+0000","last_peered":"2024-09-16T00:13:04.610934+0000","last_clean":"2024-09-16T00:13:04.610934+0000","last_became_active":"2024-09-16T00:11:04.640849+0000","last_became_peered":"2024-09-16T00:11:04.640849+0000","last_unstale":"2024-09-16T00:13:04.610934+0000","last_undegraded":"2024-09-16T00:13:04.610934+0000","last_fullsized":"2024-09-16T00:13:04.610934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:50:26.878692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663979+0000","last_change":"2024-09-16T00:12:04.029280+0000","last_active":"2024-09-16T00:13:04.663979+0000","last_peered":"2024-09-16T00:13:04.663979+0000","last_clean":"2024-09-16T00:13:04.663979+0000","last_became_active":"2024-09-16T00:12:04.028973+0000","last_became_peered":"2024-09-16T00:12:04.028973+0000","last_unstale":"2024-09-16T00:13:04.663979+0000","last_undegraded":"2024-09-16T00:13:04.663979+0000","last_fullsized":"2024-09-16T00:13:04.663979+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:40:09.806915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606846+0000","last_change":"2024-09-16T00:12:04.028319+0000","last_active":"2024-09-16T00:13:04.606846+0000","last_peered":"2024-09-16T00:13:04.606846+0000","last_clean":"2024-09-16T00:13:04.606846+0000","last_became_active":"2024-09-16T00:12:04.027985+0000","last_became_peered":"2024-09-16T00:12:04.027985+0000","last_unstale":"2024-09-16T00:13:04.606846+0000","last_undegraded":"2024-09-16T00:13:04.606846+0000","last_fullsized":"2024-09-16T00:13:04.606846+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:12.119428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663790+0000","last_change":"2024-09-16T00:11:04.647988+0000","last_active":"2024-09-16T00:13:04.663790+0000","last_peered":"2024-09-16T00:13:04.663790+0000","last_clean":"2024-09-16T00:13:04.663790+0000","last_became_active":"2024-09-16T00:11:04.647610+0000","last_became_peered":"2024-09-16T00:11:04.647610+0000","last_unstale":"2024-09-16T00:13:04.663790+0000","last_undegraded":"2024-09-16T00:13:04.663790+0000","last_fullsized":"2024-09-16T00:13:04.663790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:47.009158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611369+0000","last_change":"2024-09-16T00:11:04.648257+0000","last_active":"2024-09-16T00:13:04.611369+0000","last_peered":"2024-09-16T00:13:04.611369+0000","last_clean":"2024-09-16T00:13:04.611369+0000","last_became_active":"2024-09-16T00:11:04.647942+0000","last_became_peered":"2024-09-16T00:11:04.647942+0000","last_unstale":"2024-09-16T00:13:04.611369+0000","last_undegraded":"2024-09-16T00:13:04.611369+0000","last_fullsized":"2024-09-16T00:13:04.611369+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:43.061943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.616864+0000","last_change":"2024-09-16T00:13:04.612598+0000","last_active":"2024-09-16T00:13:04.616864+0000","last_peered":"2024-09-16T00:13:04.616864+0000","last_clean":"2024-09-16T00:13:04.616864+0000","last_became_active":"2024-09-16T00:13:04.612310+0000","last_became_peered":"2024-09-16T00:13:04.612310+0000","last_unstale":"2024-09-16T00:13:04.616864+0000","last_undegraded":"2024-09-16T00:13:04.616864+0000","last_fullsized":"2024-09-16T00:13:04.616864+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:59.381944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605240+0000","last_change":"2024-09-16T00:11:04.642643+0000","last_active":"2024-09-16T00:13:04.605240+0000","last_peered":"2024-09-16T00:13:04.605240+0000","last_clean":"2024-09-16T00:13:04.605240+0000","last_became_active":"2024-09-16T00:11:04.642336+0000","last_became_peered":"2024-09-16T00:11:04.642336+0000","last_unstale":"2024-09-16T00:13:04.605240+0000","last_undegraded":"2024-09-16T00:13:04.605240+0000","last_fullsized":"2024-09-16T00:13:04.605240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:29.396072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663870+0000","last_change":"2024-09-16T00:11:04.640636+0000","last_active":"2024-09-16T00:13:04.663870+0000","last_peered":"2024-09-16T00:13:04.663870+0000","last_clean":"2024-09-16T00:13:04.663870+0000","last_became_active":"2024-09-16T00:11:04.640325+0000","last_became_peered":"2024-09-16T00:11:04.640325+0000","last_unstale":"2024-09-16T00:13:04.663870+0000","last_undegraded":"2024-09-16T00:13:04.663870+0000","last_fullsized":"2024-09-16T00:13:04.663870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:37.673533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605535+0000","last_change":"2024-09-16T00:11:04.639903+0000","last_active":"2024-09-16T00:13:04.605535+0000","last_peered":"2024-09-16T00:13:04.605535+0000","last_clean":"2024-09-16T00:13:04.605535+0000","last_became_active":"2024-09-16T00:11:04.639535+0000","last_became_peered":"2024-09-16T00:11:04.639535+0000","last_unstale":"2024-09-16T00:13:04.605535+0000","last_undegraded":"2024-09-16T00:13:04.605535+0000","last_fullsized":"2024-09-16T00:13:04.605535+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:14.545787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606766+0000","last_change":"2024-09-16T00:11:04.635278+0000","last_active":"2024-09-16T00:13:04.606766+0000","last_peered":"2024-09-16T00:13:04.606766+0000","last_clean":"2024-09-16T00:13:04.606766+0000","last_became_active":"2024-09-16T00:11:04.634986+0000","last_became_peered":"2024-09-16T00:11:04.634986+0000","last_unstale":"2024-09-16T00:13:04.606766+0000","last_undegraded":"2024-09-16T00:13:04.606766+0000","last_fullsized":"2024-09-16T00:13:04.606766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:02:48.181788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663880+0000","last_change":"2024-09-16T00:11:04.650515+0000","last_active":"2024-09-16T00:13:04.663880+0000","last_peered":"2024-09-16T00:13:04.663880+0000","last_clean":"2024-09-16T00:13:04.663880+0000","last_became_active":"2024-09-16T00:11:04.650167+0000","last_became_peered":"2024-09-16T00:11:04.650167+0000","last_unstale":"2024-09-16T00:13:04.663880+0000","last_undegraded":"2024-09-16T00:13:04.663880+0000","last_fullsized":"2024-09-16T00:13:04.663880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:16:20.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611528+0000","last_change":"2024-09-16T00:11:04.639113+0000","last_active":"2024-09-16T00:13:04.611528+0000","last_peered":"2024-09-16T00:13:04.611528+0000","last_clean":"2024-09-16T00:13:04.611528+0000","last_became_active":"2024-09-16T00:11:04.638769+0000","last_became_peered":"2024-09-16T00:11:04.638769+0000","last_unstale":"2024-09-16T00:13:04.611528+0000","last_undegraded":"2024-09-16T00:13:04.611528+0000","last_fullsized":"2024-09-16T00:13:04.611528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:57.042061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.609303+0000","last_change":"2024-09-16T00:11:04.644900+0000","last_active":"2024-09-16T00:13:04.609303+0000","last_peered":"2024-09-16T00:13:04.609303+0000","last_clean":"2024-09-16T00:13:04.609303+0000","last_became_active":"2024-09-16T00:11:04.644569+0000","last_became_peered":"2024-09-16T00:11:04.644569+0000","last_unstale":"2024-09-16T00:13:04.609303+0000","last_undegraded":"2024-09-16T00:13:04.609303+0000","last_fullsized":"2024-09-16T00:13:04.609303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:54.377784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663938+0000","last_change":"2024-09-16T00:11:04.649345+0000","last_active":"2024-09-16T00:13:04.663938+0000","last_peered":"2024-09-16T00:13:04.663938+0000","last_clean":"2024-09-16T00:13:04.663938+0000","last_became_active":"2024-09-16T00:11:04.649033+0000","last_became_peered":"2024-09-16T00:11:04.649033+0000","last_unstale":"2024-09-16T00:13:04.663938+0000","last_undegraded":"2024-09-16T00:13:04.663938+0000","last_fullsized":"2024-09-16T00:13:04.663938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:24.697878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605284+0000","last_change":"2024-09-16T00:11:04.637470+0000","last_active":"2024-09-16T00:13:04.605284+0000","last_peered":"2024-09-16T00:13:04.605284+0000","last_clean":"2024-09-16T00:13:04.605284+0000","last_became_active":"2024-09-16T00:11:04.637119+0000","last_became_peered":"2024-09-16T00:11:04.637119+0000","last_unstale":"2024-09-16T00:13:04.605284+0000","last_undegraded":"2024-09-16T00:13:04.605284+0000","last_fullsized":"2024-09-16T00:13:04.605284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:19:38.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663375+0000","last_change":"2024-09-16T00:11:04.643086+0000","last_active":"2024-09-16T00:13:04.663375+0000","last_peered":"2024-09-16T00:13:04.663375+0000","last_clean":"2024-09-16T00:13:04.663375+0000","last_became_active":"2024-09-16T00:11:04.642317+0000","last_became_peered":"2024-09-16T00:11:04.642317+0000","last_unstale":"2024-09-16T00:13:04.663375+0000","last_undegraded":"2024-09-16T00:13:04.663375+0000","last_fullsized":"2024-09-16T00:13:04.663375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:25.197095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663627+0000","last_change":"2024-09-16T00:11:04.647025+0000","last_active":"2024-09-16T00:13:04.663627+0000","last_peered":"2024-09-16T00:13:04.663627+0000","last_clean":"2024-09-16T00:13:04.663627+0000","last_became_active":"2024-09-16T00:11:04.646720+0000","last_became_peered":"2024-09-16T00:11:04.646720+0000","last_unstale":"2024-09-16T00:13:04.663627+0000","last_undegraded":"2024-09-16T00:13:04.663627+0000","last_fullsized":"2024-09-16T00:13:04.663627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:53.697004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.606741+0000","last_change":"2024-09-16T00:11:04.646729+0000","last_active":"2024-09-16T00:13:04.606741+0000","last_peered":"2024-09-16T00:13:04.606741+0000","last_clean":"2024-09-16T00:13:04.606741+0000","last_became_active":"2024-09-16T00:11:04.646407+0000","last_became_peered":"2024-09-16T00:11:04.646407+0000","last_unstale":"2024-09-16T00:13:04.606741+0000","last_undegraded":"2024-09-16T00:13:04.606741+0000","last_fullsized":"2024-09-16T00:13:04.606741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:25.349749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611683+0000","last_change":"2024-09-16T00:11:04.651151+0000","last_active":"2024-09-16T00:13:04.611683+0000","last_peered":"2024-09-16T00:13:04.611683+0000","last_clean":"2024-09-16T00:13:04.611683+0000","last_became_active":"2024-09-16T00:11:04.650433+0000","last_became_peered":"2024-09-16T00:11:04.650433+0000","last_unstale":"2024-09-16T00:13:04.611683+0000","last_undegraded":"2024-09-16T00:13:04.611683+0000","last_fullsized":"2024-09-16T00:13:04.611683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:59:14.456405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649341+0000","last_change":"2024-09-16T00:11:04.645054+0000","last_active":"2024-09-16T00:13:04.649341+0000","last_peered":"2024-09-16T00:13:04.649341+0000","last_clean":"2024-09-16T00:13:04.649341+0000","last_became_active":"2024-09-16T00:11:04.644583+0000","last_became_peered":"2024-09-16T00:11:04.644583+0000","last_unstale":"2024-09-16T00:13:04.649341+0000","last_undegraded":"2024-09-16T00:13:04.649341+0000","last_fullsized":"2024-09-16T00:13:04.649341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:39.104670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664053+0000","last_change":"2024-09-16T00:11:04.650914+0000","last_active":"2024-09-16T00:13:04.664053+0000","last_peered":"2024-09-16T00:13:04.664053+0000","last_clean":"2024-09-16T00:13:04.664053+0000","last_became_active":"2024-09-16T00:11:04.650585+0000","last_became_peered":"2024-09-16T00:11:04.650585+0000","last_unstale":"2024-09-16T00:13:04.664053+0000","last_undegraded":"2024-09-16T00:13:04.664053+0000","last_fullsized":"2024-09-16T00:13:04.664053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:47.275811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.605811+0000","last_change":"2024-09-16T00:11:04.641746+0000","last_active":"2024-09-16T00:13:04.605811+0000","last_peered":"2024-09-16T00:13:04.605811+0000","last_clean":"2024-09-16T00:13:04.605811+0000","last_became_active":"2024-09-16T00:11:04.641417+0000","last_became_peered":"2024-09-16T00:11:04.641417+0000","last_unstale":"2024-09-16T00:13:04.605811+0000","last_undegraded":"2024-09-16T00:13:04.605811+0000","last_fullsized":"2024-09-16T00:13:04.605811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:53.584786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663898+0000","last_change":"2024-09-16T00:11:04.644288+0000","last_active":"2024-09-16T00:13:04.663898+0000","last_peered":"2024-09-16T00:13:04.663898+0000","last_clean":"2024-09-16T00:13:04.663898+0000","last_became_active":"2024-09-16T00:11:04.643940+0000","last_became_peered":"2024-09-16T00:11:04.643940+0000","last_unstale":"2024-09-16T00:13:04.663898+0000","last_undegraded":"2024-09-16T00:13:04.663898+0000","last_fullsized":"2024-09-16T00:13:04.663898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:35.462746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.649554+0000","last_change":"2024-09-16T00:12:04.028342+0000","last_active":"2024-09-16T00:13:04.649554+0000","last_peered":"2024-09-16T00:13:04.649554+0000","last_clean":"2024-09-16T00:13:04.649554+0000","last_became_active":"2024-09-16T00:12:04.027889+0000","last_became_peered":"2024-09-16T00:12:04.027889+0000","last_unstale":"2024-09-16T00:13:04.649554+0000","last_undegraded":"2024-09-16T00:13:04.649554+0000","last_fullsized":"2024-09-16T00:13:04.649554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:47.840428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664117+0000","last_change":"2024-09-16T00:11:04.649368+0000","last_active":"2024-09-16T00:13:04.664117+0000","last_peered":"2024-09-16T00:13:04.664117+0000","last_clean":"2024-09-16T00:13:04.664117+0000","last_became_active":"2024-09-16T00:11:04.648983+0000","last_became_peered":"2024-09-16T00:11:04.648983+0000","last_unstale":"2024-09-16T00:13:04.664117+0000","last_undegraded":"2024-09-16T00:13:04.664117+0000","last_fullsized":"2024-09-16T00:13:04.664117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:20.922627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663932+0000","last_change":"2024-09-16T00:11:04.640159+0000","last_active":"2024-09-16T00:13:04.663932+0000","last_peered":"2024-09-16T00:13:04.663932+0000","last_clean":"2024-09-16T00:13:04.663932+0000","last_became_active":"2024-09-16T00:11:04.636869+0000","last_became_peered":"2024-09-16T00:11:04.636869+0000","last_unstale":"2024-09-16T00:13:04.663932+0000","last_undegraded":"2024-09-16T00:13:04.663932+0000","last_fullsized":"2024-09-16T00:13:04.663932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:08.332938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.611689+0000","last_change":"2024-09-16T00:11:04.652005+0000","last_active":"2024-09-16T00:13:04.611689+0000","last_peered":"2024-09-16T00:13:04.611689+0000","last_clean":"2024-09-16T00:13:04.611689+0000","last_became_active":"2024-09-16T00:11:04.651554+0000","last_became_peered":"2024-09-16T00:11:04.651554+0000","last_unstale":"2024-09-16T00:13:04.611689+0000","last_undegraded":"2024-09-16T00:13:04.611689+0000","last_fullsized":"2024-09-16T00:13:04.611689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:30:13.320812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.663700+0000","last_change":"2024-09-16T00:11:04.639681+0000","last_active":"2024-09-16T00:13:04.663700+0000","last_peered":"2024-09-16T00:13:04.663700+0000","last_clean":"2024-09-16T00:13:04.663700+0000","last_became_active":"2024-09-16T00:11:04.639376+0000","last_became_peered":"2024-09-16T00:11:04.639376+0000","last_unstale":"2024-09-16T00:13:04.663700+0000","last_undegraded":"2024-09-16T00:13:04.663700+0000","last_fullsized":"2024-09-16T00:13:04.663700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:00.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664072+0000","last_change":"2024-09-16T00:11:04.646361+0000","last_active":"2024-09-16T00:13:04.664072+0000","last_peered":"2024-09-16T00:13:04.664072+0000","last_clean":"2024-09-16T00:13:04.664072+0000","last_became_active":"2024-09-16T00:11:04.646074+0000","last_became_peered":"2024-09-16T00:11:04.646074+0000","last_unstale":"2024-09-16T00:13:04.664072+0000","last_undegraded":"2024-09-16T00:13:04.664072+0000","last_fullsized":"2024-09-16T00:13:04.664072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:54:05.979702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:13:04.664019+0000","last_change":"2024-09-16T00:11:04.649275+0000","last_active":"2024-09-16T00:13:04.664019+0000","last_peered":"2024-09-16T00:13:04.664019+0000","last_clean":"2024-09-16T00:13:04.664019+0000","last_became_active":"2024-09-16T00:11:04.648950+0000","last_became_peered":"2024-09-16T00:11:04.648950+0000","last_unstale":"2024-09-16T00:13:04.664019+0000","last_undegraded":"2024-09-16T00:13:04.664019+0000","last_fullsized":"2024-09-16T00:13:04.664019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:11:03.612500+0000","last_clean_scrub_stamp":"2024-09-16T00:11:03.612500+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:51:14.229486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80437,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1176,"num_read_kb":677,"num_write":282,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":241311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579949,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":515263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808882,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1744896,"data_stored":1174325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":518755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201340,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430272,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1658880,"data_stored":1112771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593798,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822731,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36972,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708180,"statfs":{"total":95995035648,"available":95957176320,"internally_reserved":0,"allocated":1667072,"data_stored":1107033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":515164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215190,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":509871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":584108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":583439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1064960,"data_stored":512153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":74667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":74516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:13:40.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:13:41.189 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:41.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:41 smithi078 ceph-mon[31157]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:41.589 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:41 smithi078 ceph-mon[31157]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-16T00:13:41.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 -- 172.21.15.78:0/1538414440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 msgr2=0x7ffa841006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:41.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/1538414440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 0x7ffa841006d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009930 tx=0x7ffa7402a420 comp rx=0 tx=0).stop 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 -- 172.21.15.78:0/1538414440 shutdown_connections 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/1538414440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa84106f50 0x7ffa8410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/1538414440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa84100ca0 0x7ffa84106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/1538414440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 0x7ffa841006d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 -- 172.21.15.78:0/1538414440 >> 172.21.15.78:0/1538414440 conn(0x7ffa84077210 msgr2=0x7ffa840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 -- 172.21.15.78:0/1538414440 shutdown_connections 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.668+0000 7ffa8c3df640 1 -- 172.21.15.78:0/1538414440 wait complete. 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8c3df640 1 Processor -- start 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8c3df640 1 -- start start 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8c3df640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 0x7ffa841984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8c3df640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa84100ca0 0x7ffa841989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8c3df640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa84106f50 0x7ffa8419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:41.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8c3df640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84111fa0 con 0x7ffa841002f0 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8c3df640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84111e20 con 0x7ffa84106f50 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8c3df640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa84112120 con 0x7ffa84100ca0 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 0x7ffa841984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 0x7ffa841984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54022/0 (socket says 172.21.15.78:54022) 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 -- 172.21.15.78:0/3876731367 learned_addr learned my addr 172.21.15.78:0/3876731367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8997a640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa84100ca0 0x7ffa841989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 -- 172.21.15.78:0/3876731367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa84100ca0 msgr2=0x7ffa841989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa84100ca0 0x7ffa841989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 -- 172.21.15.78:0/3876731367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa84106f50 msgr2=0x7ffa8419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a97c640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa84106f50 0x7ffa8419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa84106f50 0x7ffa8419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 -- 172.21.15.78:0/3876731367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8419d450 con 0x7ffa841002f0 2024-09-16T00:13:41.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8997a640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa84100ca0 0x7ffa841989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a97c640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa84106f50 0x7ffa8419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.669+0000 7ffa8a17b640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 0x7ffa841984a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009a50 tx=0x7ffa74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.670+0000 7ffa7b7fe640 1 -- 172.21.15.78:0/3876731367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7403f030 con 0x7ffa841002f0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.670+0000 7ffa7b7fe640 1 -- 172.21.15.78:0/3876731367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74003ce0 con 0x7ffa841002f0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.670+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8419d6e0 con 0x7ffa841002f0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.670+0000 7ffa7b7fe640 1 -- 172.21.15.78:0/3876731367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74035070 con 0x7ffa841002f0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.670+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa841a5020 con 0x7ffa841002f0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.671+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa841033b0 con 0x7ffa841002f0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.672+0000 7ffa7b7fe640 1 -- 172.21.15.78:0/3876731367 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7ffa74042030 con 0x7ffa841002f0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.672+0000 7ffa7b7fe640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa700a32d0 0x7ffa700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.672+0000 7ffa8997a640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa700a32d0 0x7ffa700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.672+0000 7ffa7b7fe640 1 -- 172.21.15.78:0/3876731367 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ffa740f0470 con 0x7ffa841002f0 2024-09-16T00:13:41.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.673+0000 7ffa8997a640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa700a32d0 0x7ffa700a5790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffa84104830 tx=0x7ffa80005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:41.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.676+0000 7ffa7b7fe640 1 -- 172.21.15.78:0/3876731367 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa740bccd0 con 0x7ffa841002f0 2024-09-16T00:13:41.756 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:41 smithi148 ceph-mon[35599]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:41.757 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:41 smithi148 ceph-mon[35599]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-16T00:13:41.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.876+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa8410a120 con 0x7ffa841002f0 2024-09-16T00:13:41.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.877+0000 7ffa7b7fe640 1 -- 172.21.15.78:0/3876731367 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7ffa740bc5d0 con 0x7ffa841002f0 2024-09-16T00:13:41.878 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:41.878 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-16T00:13:25:778566+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:25.728293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:13:41.878 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-16T00:13:41.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa700a32d0 msgr2=0x7ffa700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:41.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa700a32d0 0x7ffa700a5790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffa84104830 tx=0x7ffa80005d90 comp rx=0 tx=0).stop 2024-09-16T00:13:41.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 msgr2=0x7ffa841984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:41.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 0x7ffa841984a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009a50 tx=0x7ffa74004270 comp rx=0 tx=0).stop 2024-09-16T00:13:41.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 shutdown_connections 2024-09-16T00:13:41.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa700a32d0 0x7ffa700a5790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa84106f50 0x7ffa8419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa84100ca0 0x7ffa841989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 --2- 172.21.15.78:0/3876731367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa841002f0 0x7ffa841984a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:41.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 >> 172.21.15.78:0/3876731367 conn(0x7ffa84077210 msgr2=0x7ffa840fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:41.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 shutdown_connections 2024-09-16T00:13:41.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:41.879+0000 7ffa8c3df640 1 -- 172.21.15.78:0/3876731367 wait complete. 2024-09-16T00:13:41.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:41 smithi193 ceph-mon[35484]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:13:41.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:41 smithi193 ceph-mon[35484]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-16T00:13:42.583 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:13:42.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:42 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3876731367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:42.830 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:42.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:42 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3876731367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:42.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:42 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3876731367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:43.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 -- 172.21.15.78:0/2690458067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c102790 msgr2=0x7fce0c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 --2- 172.21.15.78:0/2690458067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c102790 0x7fce0c1067e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009900 tx=0x7fcdf402a410 comp rx=0 tx=0).stop 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 -- 172.21.15.78:0/2690458067 shutdown_connections 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 --2- 172.21.15.78:0/2690458067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce0c106f10 0x7fce0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 --2- 172.21.15.78:0/2690458067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c102790 0x7fce0c1067e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 --2- 172.21.15.78:0/2690458067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce0c101de0 0x7fce0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 -- 172.21.15.78:0/2690458067 >> 172.21.15.78:0/2690458067 conn(0x7fce0c0fd8c0 msgr2=0x7fce0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 -- 172.21.15.78:0/2690458067 shutdown_connections 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.337+0000 7fce1304e640 1 -- 172.21.15.78:0/2690458067 wait complete. 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce1304e640 1 Processor -- start 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce1304e640 1 -- start start 2024-09-16T00:13:43.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce1304e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c101de0 0x7fce0c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce1304e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce0c102790 0x7fce0c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce1304e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce0c106f10 0x7fce0c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce1304e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c111f90 con 0x7fce0c101de0 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce1304e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c111e10 con 0x7fce0c106f10 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce1304e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c112110 con 0x7fce0c102790 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce10dea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c101de0 0x7fce0c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce10dea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c101de0 0x7fce0c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54032/0 (socket says 172.21.15.78:54032) 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce10dea640 1 -- 172.21.15.78:0/4012434203 learned_addr learned my addr 172.21.15.78:0/4012434203 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce115eb640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce0c106f10 0x7fce0c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce10dea640 1 -- 172.21.15.78:0/4012434203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce0c102790 msgr2=0x7fce0c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce10dea640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce0c102790 0x7fce0c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce10dea640 1 -- 172.21.15.78:0/4012434203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce0c106f10 msgr2=0x7fce0c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce10dea640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce0c106f10 0x7fce0c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.338+0000 7fce10dea640 1 -- 172.21.15.78:0/4012434203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c1a6930 con 0x7fce0c101de0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.339+0000 7fce10dea640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c101de0 0x7fce0c0744f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00d950 tx=0x7fcdfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.339+0000 7fce09ffb640 1 -- 172.21.15.78:0/4012434203 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc017070 con 0x7fce0c101de0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.339+0000 7fce09ffb640 1 -- 172.21.15.78:0/4012434203 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdfc009c30 con 0x7fce0c101de0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.339+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c1a6c20 con 0x7fce0c101de0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.339+0000 7fce09ffb640 1 -- 172.21.15.78:0/4012434203 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00f6b0 con 0x7fce0c101de0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.339+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c1a71b0 con 0x7fce0c101de0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.340+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c1021c0 con 0x7fce0c101de0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.341+0000 7fce09ffb640 1 -- 172.21.15.78:0/4012434203 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fcdfc0109d0 con 0x7fce0c101de0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.342+0000 7fce09ffb640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcdf80a3350 0x7fcdf80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.342+0000 7fce0bfff640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcdf80a3350 0x7fcdf80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:43.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.342+0000 7fce09ffb640 1 -- 172.21.15.78:0/4012434203 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcdfc0cfa40 con 0x7fce0c101de0 2024-09-16T00:13:43.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.342+0000 7fce0bfff640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcdf80a3350 0x7fcdf80a5810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcdf402a930 tx=0x7fcdf4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:43.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.346+0000 7fce09ffb640 1 -- 172.21.15.78:0/4012434203 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcdfc009de0 con 0x7fce0c101de0 2024-09-16T00:13:43.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:43 smithi078 ceph-mon[31157]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T00:13:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.548+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce0c075a20 con 0x7fce0c101de0 2024-09-16T00:13:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.549+0000 7fce09ffb640 1 -- 172.21.15.78:0/4012434203 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7fcdfc09c2a0 con 0x7fce0c101de0 2024-09-16T00:13:43.550 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:43.550 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-16T00:13:25:778566+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:25.728293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:13:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-16T00:13:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcdf80a3350 msgr2=0x7fcdf80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcdf80a3350 0x7fcdf80a5810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcdf402a930 tx=0x7fcdf4000bc0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c101de0 msgr2=0x7fce0c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c101de0 0x7fce0c0744f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00d950 tx=0x7fcdfc00de10 comp rx=0 tx=0).stop 2024-09-16T00:13:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 shutdown_connections 2024-09-16T00:13:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcdf80a3350 0x7fcdf80a5810 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce0c106f10 0x7fce0c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce0c102790 0x7fce0c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 --2- 172.21.15.78:0/4012434203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce0c101de0 0x7fce0c0744f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 >> 172.21.15.78:0/4012434203 conn(0x7fce0c0fd8c0 msgr2=0x7fce0c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 shutdown_connections 2024-09-16T00:13:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:43.551+0000 7fce1304e640 1 -- 172.21.15.78:0/4012434203 wait complete. 2024-09-16T00:13:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:43 smithi193 ceph-mon[35484]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T00:13:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:43 smithi148 ceph-mon[35599]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-16T00:13:44.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:13:44.474 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:44.753 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:44 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4012434203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:44 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4012434203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:44 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4012434203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:45.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 -- 172.21.15.78:0/468538913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 msgr2=0x7f7a5810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:45.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/468538913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 0x7f7a5810bbb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009900 tx=0x7f7a4c02a480 comp rx=0 tx=0).stop 2024-09-16T00:13:45.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 -- 172.21.15.78:0/468538913 shutdown_connections 2024-09-16T00:13:45.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/468538913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a5810c180 0x7f7a5810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/468538913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 0x7f7a5810bbb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/468538913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a58102da0 0x7f7a58103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 -- 172.21.15.78:0/468538913 >> 172.21.15.78:0/468538913 conn(0x7f7a580fc7a0 msgr2=0x7f7a580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 -- 172.21.15.78:0/468538913 shutdown_connections 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.994+0000 7f7a5de4f640 1 -- 172.21.15.78:0/468538913 wait complete. 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a5de4f640 1 Processor -- start 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a5de4f640 1 -- start start 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a5de4f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a58102da0 0x7f7a581983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a5de4f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 0x7f7a58198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a5de4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a5810c180 0x7f7a5819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a5de4f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58111f60 con 0x7f7a581036c0 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a5de4f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a58111de0 con 0x7f7a5810c180 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a5de4f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a581120e0 con 0x7f7a58102da0 2024-09-16T00:13:45.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 0x7f7a58198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 0x7f7a58198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54062/0 (socket says 172.21.15.78:54062) 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a56ffd640 1 -- 172.21.15.78:0/2086147924 learned_addr learned my addr 172.21.15.78:0/2086147924 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.995+0000 7f7a577fe640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a58102da0 0x7f7a581983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a57fff640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a5810c180 0x7f7a5819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a56ffd640 1 -- 172.21.15.78:0/2086147924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a58102da0 msgr2=0x7f7a581983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a56ffd640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a58102da0 0x7f7a581983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a56ffd640 1 -- 172.21.15.78:0/2086147924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a5810c180 msgr2=0x7f7a5819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a56ffd640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a5810c180 0x7f7a5819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a56ffd640 1 -- 172.21.15.78:0/2086147924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5819d390 con 0x7f7a581036c0 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a56ffd640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 0x7f7a58198920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c006010 tx=0x7f7a4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a54ff9640 1 -- 172.21.15.78:0/2086147924 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c03f030 con 0x7f7a581036c0 2024-09-16T00:13:45.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a54ff9640 1 -- 172.21.15.78:0/2086147924 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a4c003ce0 con 0x7f7a581036c0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5819d620 con 0x7f7a581036c0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a54ff9640 1 -- 172.21.15.78:0/2086147924 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c035070 con 0x7f7a581036c0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.996+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a581a4f60 con 0x7f7a581036c0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.997+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5806b010 con 0x7f7a581036c0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.998+0000 7f7a54ff9640 1 -- 172.21.15.78:0/2086147924 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f7a4c042030 con 0x7f7a581036c0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.999+0000 7f7a54ff9640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a400a32d0 0x7f7a400a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.999+0000 7f7a577fe640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a400a32d0 0x7f7a400a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.999+0000 7f7a54ff9640 1 -- 172.21.15.78:0/2086147924 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7a4c0f04c0 con 0x7f7a581036c0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:44.999+0000 7f7a577fe640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a400a32d0 0x7f7a400a5790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7a48005fd0 tx=0x7f7a4800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.003+0000 7f7a54ff9640 1 -- 172.21.15.78:0/2086147924 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a4c0bcde0 con 0x7f7a581036c0 2024-09-16T00:13:45.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.171+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a581071c0 con 0x7f7a581036c0 2024-09-16T00:13:45.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.173+0000 7f7a54ff9640 1 -- 172.21.15.78:0/2086147924 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24757 (secure 0 0 0) 0x7f7a4c0bc6e0 con 0x7f7a581036c0 2024-09-16T00:13:45.174 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:45.174 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:13:04.600749+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a400a32d0 msgr2=0x7f7a400a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a400a32d0 0x7f7a400a5790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7a48005fd0 tx=0x7f7a4800b390 comp rx=0 tx=0).stop 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 msgr2=0x7f7a58198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 0x7f7a58198920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c006010 tx=0x7f7a4c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 shutdown_connections 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a400a32d0 0x7f7a400a5790 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a5810c180 0x7f7a5819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a581036c0 0x7f7a58198920 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 --2- 172.21.15.78:0/2086147924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a58102da0 0x7f7a581983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:45.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 >> 172.21.15.78:0/2086147924 conn(0x7f7a580fc7a0 msgr2=0x7f7a580fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:45.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 shutdown_connections 2024-09-16T00:13:45.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:45.175+0000 7f7a5de4f640 1 -- 172.21.15.78:0/2086147924 wait complete. 2024-09-16T00:13:45.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:45 smithi078 ceph-mon[31157]: pgmap v371: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:13:45.632 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:45 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2086147924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:45.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-16T00:13:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:45 smithi193 ceph-mon[35484]: pgmap v371: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:13:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:45 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2086147924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:45 smithi148 ceph-mon[35599]: pgmap v371: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:13:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:45 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2086147924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:13:46.112 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:47.354 INFO:teuthology.orchestra.run.smithi078.stdout:{"run_dir":"/var/run/ceph"} 2024-09-16T00:13:47.354 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-16T00:13:47.355 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3393 2024-09-16T00:13:47.355 DEBUG:tasks.quiescer:Yielding 2024-09-16T00:13:47.355 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T00:13:47.367 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T00:13:47.368 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi078.front.sepia.ceph.com 2024-09-16T00:13:47.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-16T00:13:47.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:13:47.407 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:47.419 INFO:teuthology.orchestra.run.smithi078.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-16T00:13:47.419 INFO:teuthology.orchestra.run.smithi078.stdout:0@eb73a744-73be-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/21dc2d71-c2a3-4518-91bc-47ab979ecd32 1.1T 352M 1.1T 1% /tmp/cephfs 2024-09-16T00:13:47.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-16T00:13:47.513 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-16T00:13:47.552 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-16T00:13:47.615 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:47 smithi078 ceph-mon[31157]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T00:13:47.629 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:47 smithi193 ceph-mon[35484]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T00:13:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:47 smithi148 ceph-mon[35599]: pgmap v372: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T00:13:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 -- 172.21.15.78:0/643503161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 msgr2=0x7f6ee41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/643503161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 0x7f6ee41029b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009900 tx=0x7f6ecc02a480 comp rx=0 tx=0).stop 2024-09-16T00:13:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 -- 172.21.15.78:0/643503161 shutdown_connections 2024-09-16T00:13:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/643503161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ee4102ef0 0x7f6ee410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/643503161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 0x7f6ee41029b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/643503161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ee4100e20 0x7f6ee4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 -- 172.21.15.78:0/643503161 >> 172.21.15.78:0/643503161 conn(0x7f6ee40fc7d0 msgr2=0x7f6ee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 -- 172.21.15.78:0/643503161 shutdown_connections 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 -- 172.21.15.78:0/643503161 wait complete. 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.152+0000 7f6ee98d1640 1 Processor -- start 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee98d1640 1 -- start start 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee98d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ee4100e20 0x7f6ee41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee98d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 0x7f6ee4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee98d1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ee4102ef0 0x7f6ee41a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee98d1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee4110f80 con 0x7f6ee4102570 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee98d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee4110e00 con 0x7f6ee4100e20 2024-09-16T00:13:48.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee98d1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee4111100 con 0x7f6ee4102ef0 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 0x7f6ee4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 0x7f6ee4199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33156/0 (socket says 172.21.15.78:33156) 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 -- 172.21.15.78:0/1599604994 learned_addr learned my addr 172.21.15.78:0/1599604994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee2ffd640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ee4100e20 0x7f6ee41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee37fe640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ee4102ef0 0x7f6ee41a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 -- 172.21.15.78:0/1599604994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ee4102ef0 msgr2=0x7f6ee41a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ee4102ef0 0x7f6ee41a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 -- 172.21.15.78:0/1599604994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ee4100e20 msgr2=0x7f6ee41992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ee4100e20 0x7f6ee41992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 -- 172.21.15.78:0/1599604994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee41a4890 con 0x7f6ee4102570 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.153+0000 7f6ee27fc640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 0x7f6ee4199810 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc006010 tx=0x7f6ecc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:48.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.154+0000 7f6ee88cf640 1 -- 172.21.15.78:0/1599604994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc03f070 con 0x7f6ee4102570 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.154+0000 7f6ee88cf640 1 -- 172.21.15.78:0/1599604994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc031e30 con 0x7f6ee4102570 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.154+0000 7f6ee88cf640 1 -- 172.21.15.78:0/1599604994 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc035030 con 0x7f6ee4102570 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.154+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee41a4a60 con 0x7f6ee4102570 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.154+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee41a4eb0 con 0x7f6ee4102570 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.155+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee4101200 con 0x7f6ee4102570 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.156+0000 7f6ee88cf640 1 -- 172.21.15.78:0/1599604994 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f6ecc042070 con 0x7f6ee4102570 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.157+0000 7f6ee88cf640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ed00a32d0 0x7f6ed00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.157+0000 7f6ee2ffd640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ed00a32d0 0x7f6ed00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.157+0000 7f6ee88cf640 1 -- 172.21.15.78:0/1599604994 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6ecc0f0e80 con 0x7f6ee4102570 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.157+0000 7f6ee2ffd640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ed00a32d0 0x7f6ed00a5790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400abe0 tx=0x7f6ed400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.164+0000 7f6ee88cf640 1 -- 172.21.15.78:0/1599604994 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ecc0bd6e0 con 0x7f6ee4102570 2024-09-16T00:13:48.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.373+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ee41a50b0 con 0x7f6ee4102570 2024-09-16T00:13:48.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.374+0000 7f6ee88cf640 1 -- 172.21.15.78:0/1599604994 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7f6ecc0bcfe0 con 0x7f6ee4102570 2024-09-16T00:13:48.375 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:48.376 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-16T00:13:25:778566+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:25.728293+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:13:48.376 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-16T00:13:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ed00a32d0 msgr2=0x7f6ed00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ed00a32d0 0x7f6ed00a5790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400abe0 tx=0x7f6ed400a2d0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 msgr2=0x7f6ee4199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 0x7f6ee4199810 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc006010 tx=0x7f6ecc003c80 comp rx=0 tx=0).stop 2024-09-16T00:13:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 shutdown_connections 2024-09-16T00:13:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ed00a32d0 0x7f6ed00a5790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ee4102ef0 0x7f6ee41a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee4102570 0x7f6ee4199810 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 --2- 172.21.15.78:0/1599604994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ee4100e20 0x7f6ee41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.377+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 >> 172.21.15.78:0/1599604994 conn(0x7f6ee40fc7d0 msgr2=0x7f6ee4106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.378+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 shutdown_connections 2024-09-16T00:13:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:48.378+0000 7f6ee98d1640 1 -- 172.21.15.78:0/1599604994 wait complete. 2024-09-16T00:13:48.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:48 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1599604994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:48.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:48 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1599604994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:48.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:48 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1599604994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:13:49.676 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:49 smithi078 ceph-mon[31157]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:49.677 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:13:49.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-16T00:13:49.755 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-16T00:13:49.757 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-16T00:13:49.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:11:05.234467+0000', 'modified': '2024-09-16T00:13:25.728293+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24445, 'mds_2': 24440, 'mds_3': 24427, 'mds_4': 24421}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6839/1764484583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6838', 'nonce': 1764484583}, {'type': 'v1', 'addr': '172.21.15.78:6839', 'nonce': 1764484583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14583': {'gid': 14583, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.78:6841/1822173762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6840', 'nonce': 1822173762}, {'type': 'v1', 'addr': '172.21.15.78:6841', 'nonce': 1822173762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/1629662299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 1629662299}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 1629662299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6833/3203333506', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 3203333506}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 3203333506}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6835/903408403', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 903408403}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 903408403}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.193:6835/2292206006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 2292206006}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 2292206006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.193:6837/1139945107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1139945107}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1139945107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.148:6837/3780846668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 3780846668}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 3780846668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.193:6839/3313451522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3313451522}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3313451522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.148:6839/831570873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 831570873}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 831570873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24470, 24445, 24440, 24427, 24421]} 2024-09-16T00:13:49.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-16T00:13:49.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.f status 2024-09-16T00:13:49.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-16T00:13:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/: 2024-09-16T00:13:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:13:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 .. 2024-09-16T00:13:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 data 2024-09-16T00:13:49.912 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:49.912 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data: 2024-09-16T00:13:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:49.913 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 .. 2024-09-16T00:13:49.914 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-16T00:13:49.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:49 smithi193 ceph-mon[35484]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:13:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:49 smithi148 ceph-mon[35599]: pgmap v373: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-16T00:13:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:13:49.998 INFO:teuthology.orchestra.run.smithi078.stdout:[Service] 2024-09-16T00:13:49.999 INFO:teuthology.orchestra.run.smithi078.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-16T00:13:49.999 INFO:teuthology.orchestra.run.smithi078.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-16T00:13:50.001 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-16T00:13:50.089 INFO:teuthology.orchestra.run.smithi078.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-16T00:13:50.135 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:50.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 -- 172.21.15.78:0/413631025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374106f00 msgr2=0x7fc37410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 --2- 172.21.15.78:0/413631025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374106f00 0x7fc37410ea80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009900 tx=0x7fc36c02a410 comp rx=0 tx=0).stop 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 -- 172.21.15.78:0/413631025 shutdown_connections 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 --2- 172.21.15.78:0/413631025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374106f00 0x7fc37410ea80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 --2- 172.21.15.78:0/413631025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc374102780 0x7fc3741067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 --2- 172.21.15.78:0/413631025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc374101dd0 0x7fc3741021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 -- 172.21.15.78:0/413631025 >> 172.21.15.78:0/413631025 conn(0x7fc3740fd8b0 msgr2=0x7fc3740ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 -- 172.21.15.78:0/413631025 shutdown_connections 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.633+0000 7fc378b6a640 1 -- 172.21.15.78:0/413631025 wait complete. 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc378b6a640 1 Processor -- start 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc378b6a640 1 -- start start 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc378b6a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374101dd0 0x7fc3740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc378b6a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc374102780 0x7fc374074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc378b6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc374106f00 0x7fc374074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc378b6a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc374111f60 con 0x7fc374101dd0 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc378b6a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc374111de0 con 0x7fc374102780 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc378b6a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3741120e0 con 0x7fc374106f00 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc37259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374101dd0 0x7fc3740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc37259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374101dd0 0x7fc3740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33182/0 (socket says 172.21.15.78:33182) 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc371d9b640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc374102780 0x7fc374074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc37259c640 1 -- 172.21.15.78:0/3014586439 learned_addr learned my addr 172.21.15.78:0/3014586439 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc372d9d640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc374106f00 0x7fc374074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc37259c640 1 -- 172.21.15.78:0/3014586439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc374106f00 msgr2=0x7fc374074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc37259c640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc374106f00 0x7fc374074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc37259c640 1 -- 172.21.15.78:0/3014586439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc374102780 msgr2=0x7fc374074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc37259c640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc374102780 0x7fc374074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc37259c640 1 -- 172.21.15.78:0/3014586439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3741a68a0 con 0x7fc374101dd0 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.634+0000 7fc371d9b640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc374102780 0x7fc374074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.635+0000 7fc37259c640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374101dd0 0x7fc3740744f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc36400ebc0 tx=0x7fc36400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.635+0000 7fc36b7fe640 1 -- 172.21.15.78:0/3014586439 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364010630 con 0x7fc374101dd0 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.635+0000 7fc36b7fe640 1 -- 172.21.15.78:0/3014586439 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc364010cb0 con 0x7fc374101dd0 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.635+0000 7fc378b6a640 1 -- 172.21.15.78:0/3014586439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3741a6b90 con 0x7fc374101dd0 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.635+0000 7fc36b7fe640 1 -- 172.21.15.78:0/3014586439 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364018bd0 con 0x7fc374101dd0 2024-09-16T00:13:50.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.635+0000 7fc378b6a640 1 -- 172.21.15.78:0/3014586439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3741a7120 con 0x7fc374101dd0 2024-09-16T00:13:50.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.637+0000 7fc36b7fe640 1 -- 172.21.15.78:0/3014586439 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fc3640107d0 con 0x7fc374101dd0 2024-09-16T00:13:50.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.637+0000 7fc37af43740 1 Processor -- start 2024-09-16T00:13:50.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc37af43740 1 -- start start 2024-09-16T00:13:50.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc37af43740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f8ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc37af43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5fa1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc37af43740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5fa23c0 0x55a2c5fa47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc37af43740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5ef5090 con 0x55a2c5fa23c0 2024-09-16T00:13:50.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc37af43740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5e9ca70 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc37af43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5ef1050 con 0x55a2c5f8f210 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc372d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f8ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc371d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5fa23c0 0x55a2c5fa47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc371d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5fa23c0 0x55a2c5fa47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33188/0 (socket says 172.21.15.78:33188) 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc372d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f8ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42530/0 (socket says 172.21.15.78:42530) 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc371d9b640 1 -- 172.21.15.78:0/1208472746 learned_addr learned my addr 172.21.15.78:0/1208472746 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc37259c640 1 --2- 172.21.15.78:0/1208472746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5fa1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc36b7fe640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3600a3350 0x7fc3600a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc371d9b640 1 -- 172.21.15.78:0/1208472746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 msgr2=0x55a2c5fa1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc371d9b640 1 --2- 172.21.15.78:0/1208472746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5fa1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc371d9b640 1 -- 172.21.15.78:0/1208472746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 msgr2=0x55a2c5f8ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc371d9b640 1 --2- 172.21.15.78:0/1208472746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f8ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc371d9b640 1 -- 172.21.15.78:0/1208472746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c5ea9130 con 0x55a2c5fa23c0 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc372d9d640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3600a3350 0x7fc3600a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.638+0000 7fc36b7fe640 1 -- 172.21.15.78:0/3014586439 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc3640d0300 con 0x7fc374101dd0 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.639+0000 7fc371d9b640 1 --2- 172.21.15.78:0/1208472746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5fa23c0 0x55a2c5fa47b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0103d0 tx=0x7fc35c010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18c89931d09b6fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.639+0000 7fc368ff9640 1 -- 172.21.15.78:0/1208472746 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c011700 con 0x55a2c5fa23c0 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.639+0000 7fc368ff9640 1 -- 172.21.15.78:0/1208472746 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c011d80 con 0x55a2c5fa23c0 2024-09-16T00:13:50.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.639+0000 7fc368ff9640 1 -- 172.21.15.78:0/1208472746 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c01a380 con 0x55a2c5fa23c0 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 -- 172.21.15.78:0/1208472746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5fa23c0 msgr2=0x55a2c5fa47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc372d9d640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3600a3350 0x7fc3600a5810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0096c0 tx=0x7fc36c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 --2- 172.21.15.78:0/1208472746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5fa23c0 0x55a2c5fa47b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0103d0 tx=0x7fc35c010890 comp rx=0 tx=0).stop 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 -- 172.21.15.78:0/1208472746 shutdown_connections 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 --2- 172.21.15.78:0/1208472746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5fa23c0 0x55a2c5fa47b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 --2- 172.21.15.78:0/1208472746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5fa1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 --2- 172.21.15.78:0/1208472746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f8ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 -- 172.21.15.78:0/1208472746 >> 172.21.15.78:0/1208472746 conn(0x55a2c5f8af60 msgr2=0x55a2c5f8b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 -- 172.21.15.78:0/1208472746 shutdown_connections 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 -- 172.21.15.78:0/1208472746 wait complete. 2024-09-16T00:13:50.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 -- start start 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.640+0000 7fc37af43740 1 Processor -- start 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fa1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5f9f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f9f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5fa4f30 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5eaff20 con 0x55a2c5fa23c0 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5fa50b0 con 0x55a2c5f8f210 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 10 client.0 _ll_register_callbacks cb 0x55a2c5e79030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fa1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fa1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33204/0 (socket says 172.21.15.78:33204) 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 -- 172.21.15.78:0/2299504043 learned_addr learned my addr 172.21.15.78:0/2299504043 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc372d9d640 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f9f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc371d9b640 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5f9f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 -- 172.21.15.78:0/2299504043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 msgr2=0x55a2c5f9f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5f9f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 -- 172.21.15.78:0/2299504043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 msgr2=0x55a2c5f9f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f9f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 -- 172.21.15.78:0/2299504043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c5c662b0 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37259c640 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fa1ce0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc36409c2b0 tx=0x7fc364098970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16068da7b89d7fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc359ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:50.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc359ffb640 1 -- 172.21.15.78:0/2299504043 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364010e60 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc359ffb640 1 -- 172.21.15.78:0/2299504043 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc36409ba00 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 1 -- 172.21.15.78:0/2299504043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35c011d80 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc359ffb640 1 -- 172.21.15.78:0/2299504043 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc36409c380 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 10 client.15447 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.641+0000 7fc37af43740 1 -- 172.21.15.78:0/2299504043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2c5fb09d0 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc359ffb640 1 -- 172.21.15.78:0/2299504043 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc35c011d80 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc359ffb640 1 -- 172.21.15.78:0/2299504043 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc36409b470 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc37af43740 10 client.15447 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc37af43740 10 client.15447 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc37af43740 20 client.15447 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc37af43740 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a2c5fb2f90 0x55a2c5fb4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc37af43740 4 client.15447 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc37af43740 1 -- 172.21.15.78:0/2299504043 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2c5f02600 con 0x55a2c5fb2f90 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.642+0000 7fc371d9b640 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a2c5fb2f90 0x55a2c5fb4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc371d9b640 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a2c5fb2f90 0x55a2c5fb4400 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5303a6f55d7f3f37 server_cookie=2ced2ad9d97ca266 in_seq=0 out_seq=0 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc359ffb640 10 client.15447 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:13:50.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc359ffb640 1 -- 172.21.15.78:0/2299504043 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2c5f02600 con 0x55a2c5fb2f90 2024-09-16T00:13:50.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc359ffb640 10 client.15447 handle_command_reply: tid=0 2024-09-16T00:13:50.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 client.15447 shutdown 2024-09-16T00:13:50.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 -- 172.21.15.78:0/2299504043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 msgr2=0x55a2c5fa1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fa1ce0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc36409c2b0 tx=0x7fc364098970 comp rx=0 tx=0).stop 2024-09-16T00:13:50.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 -- 172.21.15.78:0/2299504043 shutdown_connections 2024-09-16T00:13:50.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f9f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a2c5fb2f90 0x55a2c5fb4400 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5f9f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 --2- 172.21.15.78:0/2299504043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fa1ce0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.643+0000 7fc37af43740 1 -- 172.21.15.78:0/2299504043 >> 172.21.15.78:0/2299504043 conn(0x55a2c5f8af60 msgr2=0x55a2c5f90bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:50.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.644+0000 7fc359ffb640 0 client.15447 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:13:50.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.644+0000 7fc37af43740 1 -- 172.21.15.78:0/2299504043 shutdown_connections 2024-09-16T00:13:50.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.644+0000 7fc37af43740 1 -- 172.21.15.78:0/2299504043 wait complete. 2024-09-16T00:13:50.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.644+0000 7fc37af43740 20 client.15447 trim_cache size 0 max 16384 2024-09-16T00:13:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.662+0000 7fc37af43740 1 Processor -- start 2024-09-16T00:13:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.662+0000 7fc37af43740 1 -- start start 2024-09-16T00:13:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37af43740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fb28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37af43740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8f210 0x55a2c5fb3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37af43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f85ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37af43740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5fb2740 con 0x55a2c5f8f210 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37af43740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5fb2e10 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37af43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5fb34e0 con 0x55a2c5fa23c0 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc371d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8f210 0x55a2c5fb3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fb28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc371d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8f210 0x55a2c5fb3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33208/0 (socket says 172.21.15.78:33208) 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc371d9b640 1 -- 172.21.15.78:0/593038413 learned_addr learned my addr 172.21.15.78:0/593038413 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37259c640 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fb28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42550/0 (socket says 172.21.15.78:42550) 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc372d9d640 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f85ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37259c640 1 -- 172.21.15.78:0/593038413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5fa23c0 msgr2=0x55a2c5f85ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37259c640 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f85ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37259c640 1 -- 172.21.15.78:0/593038413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8f210 msgr2=0x55a2c5fb3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37259c640 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8f210 0x55a2c5fb3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37259c640 1 -- 172.21.15.78:0/593038413 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2c5f9fb70 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc371d9b640 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8f210 0x55a2c5fb3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.663+0000 7fc37259c640 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fb28c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc36409c540 tx=0x7fc36400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d339d7a82816b7b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.664+0000 7fc35bfff640 1 -- 172.21.15.78:0/593038413 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364010e60 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.664+0000 7fc35bfff640 1 -- 172.21.15.78:0/593038413 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3640a5170 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.664+0000 7fc35bfff640 1 -- 172.21.15.78:0/593038413 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36409bf10 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.664+0000 7fc37af43740 1 -- 172.21.15.78:0/593038413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 msgr2=0x55a2c5fb28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.664+0000 7fc37af43740 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fb28c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc36409c540 tx=0x7fc36400cf90 comp rx=0 tx=0).stop 2024-09-16T00:13:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 -- 172.21.15.78:0/593038413 shutdown_connections 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8f210 0x55a2c5fb3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 --2- 172.21.15.78:0/593038413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5fb28c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 -- 172.21.15.78:0/593038413 >> 172.21.15.78:0/593038413 conn(0x55a2c5f8af60 msgr2=0x55a2c5f8b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 -- 172.21.15.78:0/593038413 shutdown_connections 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 -- 172.21.15.78:0/593038413 wait complete. 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 -- start start 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 Processor -- start 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f9b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5f9b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f9be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5fb4280 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5fb3bb0 con 0x55a2c5fa23c0 2024-09-16T00:13:50.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a2c5fb4950 con 0x55a2c5f8f210 2024-09-16T00:13:50.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37af43740 10 client.0 _ll_register_callbacks cb 0x7fc3640099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:50.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f9b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f9b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33212/0 (socket says 172.21.15.78:33212) 2024-09-16T00:13:50.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc37259c640 1 -- 172.21.15.78:0/801636017 learned_addr learned my addr 172.21.15.78:0/801636017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.665+0000 7fc372d9d640 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f9be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc371d9b640 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5f9b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37259c640 1 -- 172.21.15.78:0/801636017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 msgr2=0x55a2c5f9b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37259c640 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5f9b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37259c640 1 -- 172.21.15.78:0/801636017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 msgr2=0x55a2c5f9be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37259c640 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f9be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37259c640 1 -- 172.21.15.78:0/801636017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2c5fbb8f0 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37259c640 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f9b400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc36400cba0 tx=0x7fc3640d3ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f27c8137ce283ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc359ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:50.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc359ffb640 1 -- 172.21.15.78:0/801636017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364010e60 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc359ffb640 1 -- 172.21.15.78:0/801636017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc3640a1a40 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37af43740 1 -- 172.21.15.78:0/801636017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3640a5170 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc359ffb640 1 -- 172.21.15.78:0/801636017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc36409c0f0 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37af43740 10 client.15453 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc37af43740 1 -- 172.21.15.78:0/801636017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2c5fbcbb0 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.666+0000 7fc359ffb640 1 -- 172.21.15.78:0/801636017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3640a5170 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc359ffb640 1 -- 172.21.15.78:0/801636017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc3640b0a20 con 0x55a2c5f8e8f0 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc37af43740 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc37af43740 10 client.15453 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T00:13:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc37af43740 20 client.15453 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:50.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc37af43740 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a2c6021290 0x55a2c601ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:50.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc37af43740 4 client.15453 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:50.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc37af43740 1 -- 172.21.15.78:0/801636017 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "status"}) -- 0x55a2c5efcb10 con 0x55a2c6021290 2024-09-16T00:13:50.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc371d9b640 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a2c6021290 0x55a2c601ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:50.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc371d9b640 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a2c6021290 0x55a2c601ffa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4de92473d30d3c0c server_cookie=5253cf832f6b5fe9 in_seq=0 out_seq=0 2024-09-16T00:13:50.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.667+0000 7fc359ffb640 10 client.15453 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:13:50.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc359ffb640 1 -- 172.21.15.78:0/801636017 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55a2c5efcb10 con 0x55a2c6021290 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc359ffb640 10 client.15453 handle_command_reply: tid=0 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 client.15453 shutdown 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 -- 172.21.15.78:0/801636017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 msgr2=0x55a2c5f9b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f9b400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc36400cba0 tx=0x7fc3640d3ee0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 -- 172.21.15.78:0/801636017 shutdown_connections 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2c5fa23c0 0x55a2c5f9be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a2c6021290 0x55a2c601ffa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2c5f8f210 0x55a2c5f9b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 --2- 172.21.15.78:0/801636017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a2c5f8e8f0 0x55a2c5f9b400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 -- 172.21.15.78:0/801636017 >> 172.21.15.78:0/801636017 conn(0x55a2c5f8af60 msgr2=0x55a2c5f85ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:50.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc359ffb640 0 client.15453 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:13:50.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 -- 172.21.15.78:0/801636017 shutdown_connections 2024-09-16T00:13:50.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 1 -- 172.21.15.78:0/801636017 wait complete. 2024-09-16T00:13:50.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc37af43740 20 client.15453 trim_cache size 0 max 16384 2024-09-16T00:13:50.681 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 4, 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24421, 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 128.13296029, 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 29, 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-09-16T00:13:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 194.81710455999999 2024-09-16T00:13:50.683 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:50.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc378b6a640 1 -- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3600a3350 msgr2=0x7fc3600a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc378b6a640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3600a3350 0x7fc3600a5810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0096c0 tx=0x7fc36c009260 comp rx=0 tx=0).stop 2024-09-16T00:13:50.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc378b6a640 1 -- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374101dd0 msgr2=0x7fc3740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.668+0000 7fc378b6a640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374101dd0 0x7fc3740744f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc36400ebc0 tx=0x7fc36400c6a0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.669+0000 7fc378b6a640 1 -- 172.21.15.78:0/3014586439 shutdown_connections 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.669+0000 7fc378b6a640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3600a3350 0x7fc3600a5810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.669+0000 7fc378b6a640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc374106f00 0x7fc374074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.669+0000 7fc378b6a640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc374102780 0x7fc374074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.669+0000 7fc378b6a640 1 --2- 172.21.15.78:0/3014586439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc374101dd0 0x7fc3740744f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.669+0000 7fc378b6a640 1 -- 172.21.15.78:0/3014586439 >> 172.21.15.78:0/3014586439 conn(0x7fc3740fd8b0 msgr2=0x7fc3740fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.669+0000 7fc378b6a640 1 -- 172.21.15.78:0/3014586439 shutdown_connections 2024-09-16T00:13:50.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:50.669+0000 7fc378b6a640 1 -- 172.21.15.78:0/3014586439 wait complete. 2024-09-16T00:13:51.636 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:51 smithi078 ceph-mon[31157]: pgmap v374: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:51.670 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.c status 2024-09-16T00:13:51.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:51 smithi193 ceph-mon[35484]: pgmap v374: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:51.931 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:51 smithi148 ceph-mon[35599]: pgmap v374: 129 pgs: 129 active+clean; 655 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 -- 172.21.15.78:0/2395203522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 msgr2=0x7fb8b410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/2395203522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 0x7fb8b410ec30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009cd0 tx=0x7fb8ac02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 -- 172.21.15.78:0/2395203522 shutdown_connections 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/2395203522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 0x7fb8b410ec30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/2395203522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4103740 0x7fb8b4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/2395203522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b4102d90 0x7fb8b4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 -- 172.21.15.78:0/2395203522 >> 172.21.15.78:0/2395203522 conn(0x7fb8b40fc7d0 msgr2=0x7fb8b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 -- 172.21.15.78:0/2395203522 shutdown_connections 2024-09-16T00:13:52.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 -- 172.21.15.78:0/2395203522 wait complete. 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.411+0000 7fb8bb13f640 1 Processor -- start 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8bb13f640 1 -- start start 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8bb13f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102d90 0x7fb8b41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8bb13f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b4103740 0x7fb8b4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8bb13f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 0x7fb8b419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8bb13f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41120d0 con 0x7fb8b4108290 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8bb13f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4111f50 con 0x7fb8b4103740 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8bb13f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4112250 con 0x7fb8b4102d90 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 0x7fb8b419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 0x7fb8b419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33232/0 (socket says 172.21.15.78:33232) 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 -- 172.21.15.78:0/1278608060 learned_addr learned my addr 172.21.15.78:0/1278608060 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b4103740 0x7fb8b4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102d90 0x7fb8b41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 -- 172.21.15.78:0/1278608060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102d90 msgr2=0x7fb8b41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102d90 0x7fb8b41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 -- 172.21.15.78:0/1278608060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b4103740 msgr2=0x7fb8b4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b4103740 0x7fb8b4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 -- 172.21.15.78:0/1278608060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b419d560 con 0x7fb8b4108290 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b96dc640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 0x7fb8b419ce80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005cc0 tx=0x7fb8ac003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b1ffb640 1 -- 172.21.15.78:0/1278608060 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac004040 con 0x7fb8b4108290 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8b1ffb640 1 -- 172.21.15.78:0/1278608060 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac0046c0 con 0x7fb8b4108290 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.412+0000 7fb8bb13f640 1 -- 172.21.15.78:0/1278608060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b419d7f0 con 0x7fb8b4108290 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.413+0000 7fb8b1ffb640 1 -- 172.21.15.78:0/1278608060 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac030e70 con 0x7fb8b4108290 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.413+0000 7fb8bb13f640 1 -- 172.21.15.78:0/1278608060 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41a5130 con 0x7fb8b4108290 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.415+0000 7fb8b1ffb640 1 -- 172.21.15.78:0/1278608060 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fb8ac0305a0 con 0x7fb8b4108290 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.415+0000 7fb8bd518740 1 Processor -- start 2024-09-16T00:13:52.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.415+0000 7fb8bd518740 1 -- start start 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 0x5615dc01adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc025670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc027fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8bd518740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615dbf81090 con 0x5615dc01b2f0 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8bd518740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5615dbf28a70 con 0x5615dc025bb0 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8bd518740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5615dbf7d050 con 0x5615dc01a9d0 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc025670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc025670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33244/0 (socket says 172.21.15.78:33244) 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 -- 172.21.15.78:0/1674466532 learned_addr learned my addr 172.21.15.78:0/1674466532 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b1ffb640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb8a40a32d0 0x7fb8a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc027fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 0x5615dc01adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b96dc640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb8a40a32d0 0x7fb8a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 -- 172.21.15.78:0/1674466532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 msgr2=0x5615dc01adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 0x5615dc01adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 -- 172.21.15.78:0/1674466532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 msgr2=0x5615dc027fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc027fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 -- 172.21.15.78:0/1674466532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615dbf35130 con 0x5615dc01b2f0 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b1ffb640 1 -- 172.21.15.78:0/1278608060 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb8ac02e030 con 0x7fb8b4108290 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc025670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800bcb0 tx=0x7fb8a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7440182de6e846c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb89effd640 1 -- 172.21.15.78:0/1674466532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8015070 con 0x5615dc01b2f0 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.416+0000 7fb8b96dc640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb8a40a32d0 0x7fb8a40a5790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0b9a90 tx=0x7fb8ac0bcf10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb89effd640 1 -- 172.21.15.78:0/1674466532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a801a940 con 0x5615dc01b2f0 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb89effd640 1 -- 172.21.15.78:0/1674466532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8011a30 con 0x5615dc01b2f0 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 -- 172.21.15.78:0/1674466532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 msgr2=0x5615dc025670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc025670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800bcb0 tx=0x7fb8a800f730 comp rx=0 tx=0).stop 2024-09-16T00:13:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 -- 172.21.15.78:0/1674466532 shutdown_connections 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc027fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc025670 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1674466532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 0x5615dc01adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 -- 172.21.15.78:0/1674466532 >> 172.21.15.78:0/1674466532 conn(0x5615dc016f60 msgr2=0x5615dc017340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 -- 172.21.15.78:0/1674466532 shutdown_connections 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.417+0000 7fb8bd518740 1 -- 172.21.15.78:0/1674466532 wait complete. 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 1 -- start start 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 1 Processor -- start 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 0x5615dc025430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc022780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc022cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615dc028780 con 0x5615dc01b2f0 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5615dbf3bf20 con 0x5615dc025bb0 2024-09-16T00:13:52.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5615dc028900 con 0x5615dc01a9d0 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8bd518740 10 client.0 _ll_register_callbacks cb 0x5615dbf05030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8b8edb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc022780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8b8edb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc022780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33260/0 (socket says 172.21.15.78:33260) 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.418+0000 7fb8b8edb640 1 -- 172.21.15.78:0/4043771815 learned_addr learned my addr 172.21.15.78:0/4043771815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc022cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8b8edb640 1 -- 172.21.15.78:0/4043771815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 msgr2=0x5615dc025430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 0x5615dc025430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8b8edb640 1 -- 172.21.15.78:0/4043771815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 msgr2=0x5615dc022cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc022cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8b8edb640 1 -- 172.21.15.78:0/4043771815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615dbcf22b0 con 0x5615dc01b2f0 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc022cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc022780 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80115b0 tx=0x7fb8a800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4371dd8cc5f315 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:52.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb893fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb893fff640 1 -- 172.21.15.78:0/4043771815 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8028060 con 0x5615dc01b2f0 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb893fff640 1 -- 172.21.15.78:0/4043771815 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb8a801a730 con 0x5615dc01b2f0 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8bd518740 1 -- 172.21.15.78:0/4043771815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a801a940 con 0x5615dc01b2f0 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb893fff640 1 -- 172.21.15.78:0/4043771815 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8a802f050 con 0x5615dc01b2f0 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8bd518740 10 client.15465 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb8bd518740 1 -- 172.21.15.78:0/4043771815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615dc034090 con 0x5615dc01b2f0 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.419+0000 7fb893fff640 1 -- 172.21.15.78:0/4043771815 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a801a940 con 0x5615dc01b2f0 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb893fff640 1 -- 172.21.15.78:0/4043771815 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb8a8011840 con 0x5615dc01b2f0 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb8bd518740 10 client.15465 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb8bd518740 10 client.15465 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb8bd518740 20 client.15465 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb8bd518740 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5615dc0363c0 0x5615dc037830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb8bd518740 4 client.15465 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb8bd518740 1 -- 172.21.15.78:0/4043771815 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615dbf8e600 con 0x5615dc0363c0 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5615dc0363c0 0x5615dc037830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5615dc0363c0 0x5615dc037830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b77bf2eff727d2c6 server_cookie=aad1df4945efa282 in_seq=0 out_seq=0 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.420+0000 7fb893fff640 10 client.15465 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.421+0000 7fb893fff640 1 -- 172.21.15.78:0/4043771815 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615dbf8e600 con 0x5615dc0363c0 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.421+0000 7fb893fff640 10 client.15465 handle_command_reply: tid=0 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.421+0000 7fb8bd518740 1 client.15465 shutdown 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.421+0000 7fb8bd518740 1 -- 172.21.15.78:0/4043771815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 msgr2=0x5615dc022780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.421+0000 7fb8bd518740 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc022780 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80115b0 tx=0x7fb8a800bfd0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 1 -- 172.21.15.78:0/4043771815 shutdown_connections 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5615dc0363c0 0x5615dc037830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc025bb0 0x5615dc022cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc022780 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb893fff640 0 client.15465 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 1 --2- 172.21.15.78:0/4043771815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01a9d0 0x5615dc025430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 1 -- 172.21.15.78:0/4043771815 >> 172.21.15.78:0/4043771815 conn(0x5615dc016f60 msgr2=0x5615dc01cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 1 -- 172.21.15.78:0/4043771815 shutdown_connections 2024-09-16T00:13:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 1 -- 172.21.15.78:0/4043771815 wait complete. 2024-09-16T00:13:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.422+0000 7fb8bd518740 20 client.15465 trim_cache size 0 max 16384 2024-09-16T00:13:52.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.439+0000 7fb8bd518740 1 Processor -- start 2024-09-16T00:13:52.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.439+0000 7fb8bd518740 1 -- start start 2024-09-16T00:13:52.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.439+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 0x5615dc03ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.439+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01b2f0 0x5615dc01f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.439+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc025bb0 0x5615dc01ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.439+0000 7fb8bd518740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615dc035b70 con 0x5615dc025bb0 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.439+0000 7fb8bd518740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5615dc036240 con 0x5615dc01a9d0 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.439+0000 7fb8bd518740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5615dc036910 con 0x5615dc01b2f0 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc025bb0 0x5615dc01ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc025bb0 0x5615dc01ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33268/0 (socket says 172.21.15.78:33268) 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 -- 172.21.15.78:0/2290472010 learned_addr learned my addr 172.21.15.78:0/2290472010 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 0x5615dc03ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01b2f0 0x5615dc01f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 -- 172.21.15.78:0/2290472010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01b2f0 msgr2=0x5615dc01f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:52.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01b2f0 0x5615dc01f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 -- 172.21.15.78:0/2290472010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 msgr2=0x5615dc03ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:52.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 0x5615dc03ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 -- 172.21.15.78:0/2290472010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615dc01a800 con 0x5615dc025bb0 2024-09-16T00:13:52.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01b2f0 0x5615dc01f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:52.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc025bb0 0x5615dc01ff20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5615dc0182e0 tx=0x7fb8a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52ef7e7c871d60b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:52.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb89e7fc640 1 -- 172.21.15.78:0/2290472010 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0016860 con 0x5615dc025bb0 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb89e7fc640 1 -- 172.21.15.78:0/2290472010 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a00153c0 con 0x5615dc025bb0 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.440+0000 7fb89e7fc640 1 -- 172.21.15.78:0/2290472010 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a001e420 con 0x5615dc025bb0 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 -- 172.21.15.78:0/2290472010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc025bb0 msgr2=0x5615dc01ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc025bb0 0x5615dc01ff20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5615dc0182e0 tx=0x7fb8a0008040 comp rx=0 tx=0).stop 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 -- 172.21.15.78:0/2290472010 shutdown_connections 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc025bb0 0x5615dc01ff20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc01b2f0 0x5615dc01f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 --2- 172.21.15.78:0/2290472010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 0x5615dc03ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 -- 172.21.15.78:0/2290472010 >> 172.21.15.78:0/2290472010 conn(0x5615dc016f60 msgr2=0x5615dc021050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 -- 172.21.15.78:0/2290472010 shutdown_connections 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 -- 172.21.15.78:0/2290472010 wait complete. 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 -- start start 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.441+0000 7fb8bd518740 1 Processor -- start 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 0x5615dc09c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc09c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8bd518740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc025bb0 0x5615dc03f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8bd518740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5615dc0376b0 con 0x5615dc01b2f0 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8bd518740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5615dc036fe0 con 0x5615dc01a9d0 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8bd518740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5615dc037d80 con 0x5615dc025bb0 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8bd518740 10 client.0 _ll_register_callbacks cb 0x7fb8a00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc09c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc09c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33276/0 (socket says 172.21.15.78:33276) 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 -- 172.21.15.78:0/1729836976 learned_addr learned my addr 172.21.15.78:0/1729836976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 0x5615dc09c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc025bb0 0x5615dc03f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 -- 172.21.15.78:0/1729836976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc025bb0 msgr2=0x5615dc03f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc025bb0 0x5615dc03f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 -- 172.21.15.78:0/1729836976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 msgr2=0x5615dc09c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 0x5615dc09c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 -- 172.21.15.78:0/1729836976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615dc03fe60 con 0x5615dc01b2f0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb8b8edb640 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc09c7b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8011c10 tx=0x7fb8a800b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d34771d8a46021 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.442+0000 7fb893fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb893fff640 1 -- 172.21.15.78:0/1729836976 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8038060 con 0x5615dc01b2f0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb893fff640 1 -- 172.21.15.78:0/1729836976 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb8a8004680 con 0x5615dc01b2f0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 1 -- 172.21.15.78:0/1729836976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a00153c0 con 0x5615dc01b2f0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb893fff640 1 -- 172.21.15.78:0/1729836976 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8a803f050 con 0x5615dc01b2f0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 1 -- 172.21.15.78:0/1729836976 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615dc0a4b90 con 0x5615dc01b2f0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb893fff640 1 -- 172.21.15.78:0/1729836976 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a00153c0 con 0x5615dc01b2f0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb893fff640 1 -- 172.21.15.78:0/1729836976 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb8a801a930 con 0x5615dc01b2f0 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:52.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 10 client.15471 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T00:13:52.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 20 client.15471 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:52.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5615dc0a9690 0x5615dc0a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:52.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 4 client.15471 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:52.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8bd518740 1 -- 172.21.15.78:0/1729836976 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "status"}) -- 0x5615dbf88b10 con 0x5615dc0a9690 2024-09-16T00:13:52.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.443+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5615dc0a9690 0x5615dc0a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:52.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb893fff640 10 client.15471 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:13:52.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8b3fff640 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5615dc0a9690 0x5615dc0a83a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=418059bf1d07f7ff server_cookie=cf0b8a724131bfd1 in_seq=0 out_seq=0 2024-09-16T00:13:52.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb893fff640 1 -- 172.21.15.78:0/1729836976 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5615dbf88b10 con 0x5615dc0a9690 2024-09-16T00:13:52.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb893fff640 10 client.15471 handle_command_reply: tid=0 2024-09-16T00:13:52.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 client.15471 shutdown 2024-09-16T00:13:52.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 -- 172.21.15.78:0/1729836976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 msgr2=0x5615dc09c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc09c7b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8011c10 tx=0x7fb8a800b9f0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 -- 172.21.15.78:0/1729836976 shutdown_connections 2024-09-16T00:13:52.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615dc025bb0 0x5615dc03f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5615dc01b2f0 0x5615dc09c7b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5615dc0a9690 0x5615dc0a83a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 --2- 172.21.15.78:0/1729836976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5615dc01a9d0 0x5615dc09c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb893fff640 0 client.15471 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 -- 172.21.15.78:0/1729836976 >> 172.21.15.78:0/1729836976 conn(0x5615dc016f60 msgr2=0x5615dc021470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 -- 172.21.15.78:0/1729836976 shutdown_connections 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.444+0000 7fb8bd518740 1 -- 172.21.15.78:0/1729836976 wait complete. 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bd518740 20 client.15471 trim_cache size 0 max 16384 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 3, 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24427, 2024-09-16T00:13:52.453 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 130.924105035, 2024-09-16T00:13:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 29, 2024-09-16T00:13:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-09-16T00:13:52.454 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 193.979153943 2024-09-16T00:13:52.454 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:52.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 -- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb8a40a32d0 msgr2=0x7fb8a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb8a40a32d0 0x7fb8a40a5790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0b9a90 tx=0x7fb8ac0bcf10 comp rx=0 tx=0).stop 2024-09-16T00:13:52.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 -- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 msgr2=0x7fb8b419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:52.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 0x7fb8b419ce80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005cc0 tx=0x7fb8ac003970 comp rx=0 tx=0).stop 2024-09-16T00:13:52.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 -- 172.21.15.78:0/1278608060 shutdown_connections 2024-09-16T00:13:52.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb8a40a32d0 0x7fb8a40a5790 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8b4108290 0x7fb8b419ce80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb8b4103740 0x7fb8b4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 --2- 172.21.15.78:0/1278608060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b4102d90 0x7fb8b41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:52.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 -- 172.21.15.78:0/1278608060 >> 172.21.15.78:0/1278608060 conn(0x7fb8b40fc7d0 msgr2=0x7fb8b4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:52.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 -- 172.21.15.78:0/1278608060 shutdown_connections 2024-09-16T00:13:52.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:52.445+0000 7fb8bb13f640 1 -- 172.21.15.78:0/1278608060 wait complete. 2024-09-16T00:13:53.239 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:13:53.495 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:53.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:53 smithi078 ceph-mon[31157]: pgmap v375: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-16T00:13:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:53 smithi193 ceph-mon[35484]: pgmap v375: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-16T00:13:53.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:53 smithi148 ceph-mon[35599]: pgmap v375: 129 pgs: 129 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 45 KiB/s wr, 13 op/s 2024-09-16T00:13:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 -- 172.21.15.78:0/2926612697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102570 msgr2=0x7f21a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 --2- 172.21.15.78:0/2926612697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102570 0x7f21a01029b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2190009900 tx=0x7f219002a480 comp rx=0 tx=0).stop 2024-09-16T00:13:54.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 -- 172.21.15.78:0/2926612697 shutdown_connections 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 --2- 172.21.15.78:0/2926612697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a0102ef0 0x7f21a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 --2- 172.21.15.78:0/2926612697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102570 0x7f21a01029b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 --2- 172.21.15.78:0/2926612697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a0100e20 0x7f21a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 -- 172.21.15.78:0/2926612697 >> 172.21.15.78:0/2926612697 conn(0x7f21a00fc7d0 msgr2=0x7f21a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 -- 172.21.15.78:0/2926612697 shutdown_connections 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.015+0000 7f21a75d9640 1 -- 172.21.15.78:0/2926612697 wait complete. 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a75d9640 1 Processor -- start 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a75d9640 1 -- start start 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a75d9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a0100e20 0x7f21a0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a75d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a0102570 0x7f21a0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a75d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102ef0 0x7f21a019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a75d9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0110f50 con 0x7f21a0102ef0 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a75d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0110dd0 con 0x7f21a0102570 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a75d9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01110d0 con 0x7f21a0100e20 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a5b76640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102ef0 0x7f21a019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a5b76640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102ef0 0x7f21a019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33308/0 (socket says 172.21.15.78:33308) 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a5b76640 1 -- 172.21.15.78:0/3366229804 learned_addr learned my addr 172.21.15.78:0/3366229804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.016+0000 7f21a4b74640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a0102570 0x7f21a0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a5b76640 1 -- 172.21.15.78:0/3366229804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a0100e20 msgr2=0x7f21a0198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a5375640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a0100e20 0x7f21a0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a5b76640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a0100e20 0x7f21a0198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a5b76640 1 -- 172.21.15.78:0/3366229804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a0102570 msgr2=0x7f21a0198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a5b76640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a0102570 0x7f21a0198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a5b76640 1 -- 172.21.15.78:0/3366229804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a019d4d0 con 0x7f21a0102ef0 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a5b76640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102ef0 0x7f21a019cdf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f219800cce0 tx=0x7f219800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21967fc640 1 -- 172.21.15.78:0/3366229804 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219800f450 con 0x7f21a0102ef0 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21967fc640 1 -- 172.21.15.78:0/3366229804 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f219800fad0 con 0x7f21a0102ef0 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a75d9640 1 -- 172.21.15.78:0/3366229804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a019d7c0 con 0x7f21a0102ef0 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21967fc640 1 -- 172.21.15.78:0/3366229804 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198018c10 con 0x7f21a0102ef0 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.017+0000 7f21a75d9640 1 -- 172.21.15.78:0/3366229804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a01a50a0 con 0x7f21a0102ef0 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.019+0000 7f21967fc640 1 -- 172.21.15.78:0/3366229804 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f219800f5f0 con 0x7f21a0102ef0 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a99b2740 1 Processor -- start 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a99b2740 1 -- start start 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854319d0 0x564f85431db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854322f0 0x564f8543c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 0x564f8543efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a99b2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f85398090 con 0x564f854319d0 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a99b2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f8533fa70 con 0x564f8543cbb0 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a99b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564f85394050 con 0x564f854322f0 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.020+0000 7f21a4b74640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854319d0 0x564f85431db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854319d0 0x564f85431db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33320/0 (socket says 172.21.15.78:33320) 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 -- 172.21.15.78:0/2722133054 learned_addr learned my addr 172.21.15.78:0/2722133054 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21967fc640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f218c0a32d0 0x7f218c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 0x564f8543efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a5375640 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854322f0 0x564f8543c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a5b76640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f218c0a32d0 0x7f218c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 -- 172.21.15.78:0/2722133054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854322f0 msgr2=0x564f8543c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854322f0 0x564f8543c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 -- 172.21.15.78:0/2722133054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 msgr2=0x564f8543efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:54.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 0x564f8543efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 -- 172.21.15.78:0/2722133054 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8534c130 con 0x564f854319d0 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21967fc640 1 -- 172.21.15.78:0/3366229804 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f21980d0490 con 0x7f21a0102ef0 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f21a4b74640 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854319d0 0x564f85431db0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f21900096c0 tx=0x7f2190000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a744d60493b056 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f218b7fe640 1 -- 172.21.15.78:0/2722133054 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219004c070 con 0x564f854319d0 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f218b7fe640 1 -- 172.21.15.78:0/2722133054 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2190003cd0 con 0x564f854319d0 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.021+0000 7f218b7fe640 1 -- 172.21.15.78:0/2722133054 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190035030 con 0x564f854319d0 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 -- 172.21.15.78:0/2722133054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854319d0 msgr2=0x564f85431db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854319d0 0x564f85431db0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f21900096c0 tx=0x7f2190000bc0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a5b76640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f218c0a32d0 0x7f218c0a5790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2198097030 tx=0x7f2198098190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 -- 172.21.15.78:0/2722133054 shutdown_connections 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 0x564f8543efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854322f0 0x564f8543c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2722133054 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854319d0 0x564f85431db0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 -- 172.21.15.78:0/2722133054 >> 172.21.15.78:0/2722133054 conn(0x564f8542df60 msgr2=0x564f8542e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 -- 172.21.15.78:0/2722133054 shutdown_connections 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 -- 172.21.15.78:0/2722133054 wait complete. 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.022+0000 7f21a99b2740 1 -- start start 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a99b2740 1 Processor -- start 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f8543c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85439860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 0x564f85439dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a99b2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f8543f780 con 0x564f854322f0 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a99b2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f85352f20 con 0x564f854319d0 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a99b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564f8543f900 con 0x564f8543cbb0 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a99b2740 10 client.0 _ll_register_callbacks cb 0x564f8531c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a5375640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85439860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a4b74640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f8543c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a4b74640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f8543c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42642/0 (socket says 172.21.15.78:42642) 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a5375640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85439860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33336/0 (socket says 172.21.15.78:33336) 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a4b74640 1 -- 172.21.15.78:0/2273334389 learned_addr learned my addr 172.21.15.78:0/2273334389 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.023+0000 7f21a4b74640 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 0x564f85439dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a4b74640 1 -- 172.21.15.78:0/2273334389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 msgr2=0x564f85439dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a4b74640 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 0x564f85439dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a4b74640 1 -- 172.21.15.78:0/2273334389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 msgr2=0x564f85439860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a4b74640 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85439860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a4b74640 1 -- 172.21.15.78:0/2273334389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f851092b0 con 0x564f854319d0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a5375640 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85439860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a4b74640 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f8543c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2190006010 tx=0x7f2190031960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7c862d3aa81d6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f2188ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f2188ff9640 1 -- 172.21.15.78:0/2273334389 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219004c040 con 0x564f854319d0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a99b2740 1 -- 172.21.15.78:0/2273334389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2190003cd0 con 0x564f854319d0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f2188ff9640 1 -- 172.21.15.78:0/2273334389 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f21900035f0 con 0x564f854319d0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f2188ff9640 1 -- 172.21.15.78:0/2273334389 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2190057050 con 0x564f854319d0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a99b2740 10 client.24986 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f21a99b2740 1 -- 172.21.15.78:0/2273334389 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f8544b150 con 0x564f854319d0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.024+0000 7f2188ff9640 1 -- 172.21.15.78:0/2273334389 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2190003cd0 con 0x564f854319d0 2024-09-16T00:13:54.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f2188ff9640 1 -- 172.21.15.78:0/2273334389 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f21900046c0 con 0x564f854319d0 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f21a99b2740 10 client.24986 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f21a99b2740 10 client.24986 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f21a99b2740 20 client.24986 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x564f8544c4a0 0x564f8544e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f21a99b2740 4 client.24986 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f21a99b2740 1 -- 172.21.15.78:0/2273334389 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f853a5600 con 0x564f8544c4a0 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f21a5375640 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x564f8544c4a0 0x564f8544e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f21a5375640 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x564f8544c4a0 0x564f8544e640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=528549e80f5a704e server_cookie=830e9c92dc5a8bd2 in_seq=0 out_seq=0 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.025+0000 7f2188ff9640 10 client.24986 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f2188ff9640 1 -- 172.21.15.78:0/2273334389 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f853a5600 con 0x564f8544c4a0 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f2188ff9640 10 client.24986 handle_command_reply: tid=0 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 client.24986 shutdown 2024-09-16T00:13:54.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 -- 172.21.15.78:0/2273334389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 msgr2=0x564f8543c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f8543c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2190006010 tx=0x7f2190031960 comp rx=0 tx=0).stop 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 -- 172.21.15.78:0/2273334389 shutdown_connections 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 0x564f85439dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85439860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x564f8544c4a0 0x564f8544e640 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 --2- 172.21.15.78:0/2273334389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f8543c510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f2188ff9640 0 client.24986 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.026+0000 7f21a99b2740 1 -- 172.21.15.78:0/2273334389 >> 172.21.15.78:0/2273334389 conn(0x564f8542df60 msgr2=0x564f85433c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.027+0000 7f21a99b2740 1 -- 172.21.15.78:0/2273334389 shutdown_connections 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.027+0000 7f21a99b2740 1 -- 172.21.15.78:0/2273334389 wait complete. 2024-09-16T00:13:54.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.027+0000 7f21a99b2740 20 client.24986 trim_cache size 0 max 16384 2024-09-16T00:13:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a99b2740 1 Processor -- start 2024-09-16T00:13:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a99b2740 1 -- start start 2024-09-16T00:13:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854319d0 0x564f8544cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f8544d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 0x564f85439830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a99b2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f8544c980 con 0x564f854322f0 2024-09-16T00:13:54.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a99b2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f8544d050 con 0x564f8543cbb0 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a99b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564f8544d720 con 0x564f854319d0 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a5375640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f8544d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a5375640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f8544d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33350/0 (socket says 172.21.15.78:33350) 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a5375640 1 -- 172.21.15.78:0/4131247592 learned_addr learned my addr 172.21.15.78:0/4131247592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a4b74640 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 0x564f85439830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a4b74640 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854319d0 0x564f8544cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a5375640 1 -- 172.21.15.78:0/4131247592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854319d0 msgr2=0x564f8544cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a5375640 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854319d0 0x564f8544cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a5375640 1 -- 172.21.15.78:0/4131247592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 msgr2=0x564f85439830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a5375640 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 0x564f85439830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a5375640 1 -- 172.21.15.78:0/4131247592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f8543efa0 con 0x564f854322f0 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.046+0000 7f21a4b74640 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854319d0 0x564f8544cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:13:54.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.047+0000 7f21a5375640 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f8544d8a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x564f854390f0 tx=0x7f219c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea0963f8751a35a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.047+0000 7f218affd640 1 -- 172.21.15.78:0/4131247592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c0140d0 con 0x564f854322f0 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.047+0000 7f218affd640 1 -- 172.21.15.78:0/4131247592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f219c014270 con 0x564f854322f0 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.047+0000 7f218affd640 1 -- 172.21.15.78:0/4131247592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219c012450 con 0x564f854322f0 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 -- 172.21.15.78:0/4131247592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 msgr2=0x564f8544d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f8544d8a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x564f854390f0 tx=0x7f219c014840 comp rx=0 tx=0).stop 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 -- 172.21.15.78:0/4131247592 shutdown_connections 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f8543cbb0 0x564f85439830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f8544d8a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 --2- 172.21.15.78:0/4131247592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f854319d0 0x564f8544cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 -- 172.21.15.78:0/4131247592 >> 172.21.15.78:0/4131247592 conn(0x564f8542df60 msgr2=0x564f85439120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 -- 172.21.15.78:0/4131247592 shutdown_connections 2024-09-16T00:13:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 -- 172.21.15.78:0/4131247592 wait complete. 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 -- start start 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.048+0000 7f21a99b2740 1 Processor -- start 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f854519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85451ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a99b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 0x564f854b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a99b2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f8544e4c0 con 0x564f854322f0 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a99b2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f8544ddf0 con 0x564f854319d0 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a99b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564f8544eb90 con 0x564f8543cbb0 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a99b2740 10 client.0 _ll_register_callbacks cb 0x7f219004a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f854519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f854519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42660/0 (socket says 172.21.15.78:42660) 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 -- 172.21.15.78:0/1617809628 learned_addr learned my addr 172.21.15.78:0/1617809628 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:54.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a5375640 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85451ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 0x564f854b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 -- 172.21.15.78:0/1617809628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 msgr2=0x564f854b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 0x564f854b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 -- 172.21.15.78:0/1617809628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 msgr2=0x564f85451ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85451ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.049+0000 7f21a4b74640 1 -- 172.21.15.78:0/1617809628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f854b5e10 con 0x564f854319d0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a4b74640 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f854519a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2190009900 tx=0x7f2190046040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b8d22e62fdec59 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f2188ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f2188ff9640 1 -- 172.21.15.78:0/1617809628 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219004c070 con 0x564f854319d0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 1 -- 172.21.15.78:0/1617809628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f219c014270 con 0x564f854319d0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f2188ff9640 1 -- 172.21.15.78:0/1617809628 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f21900035f0 con 0x564f854319d0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f2188ff9640 1 -- 172.21.15.78:0/1617809628 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f219006b050 con 0x564f854319d0 2024-09-16T00:13:54.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 10 client.24992 fetch_fsmap learned FSMap version 29 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 1 -- 172.21.15.78:0/1617809628 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f854b6ed0 con 0x564f854319d0 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f2188ff9640 1 -- 172.21.15.78:0/1617809628 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f219c014270 con 0x564f854319d0 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f2188ff9640 1 -- 172.21.15.78:0/1617809628 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f21900052b0 con 0x564f854319d0 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 10 client.24992 fetch_fsmap finished waiting for FSMap version 29 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 10 client.24992 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 20 client.24992 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x564f854bba30 0x564f854ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 4 client.24992 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.050+0000 7f21a99b2740 1 -- 172.21.15.78:0/1617809628 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "status"}) -- 0x564f8539fb10 con 0x564f854bba30 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.051+0000 7f21a5375640 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x564f854bba30 0x564f854ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.051+0000 7f2188ff9640 10 client.24992 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.051+0000 7f21a5375640 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x564f854bba30 0x564f854ba740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=efc934cc4631b0b6 server_cookie=3ad88873506b4b8c in_seq=0 out_seq=0 2024-09-16T00:13:54.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.051+0000 7f2188ff9640 1 -- 172.21.15.78:0/1617809628 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564f8539fb10 con 0x564f854bba30 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.051+0000 7f2188ff9640 10 client.24992 handle_command_reply: tid=0 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.051+0000 7f21a99b2740 1 client.24992 shutdown 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.051+0000 7f21a99b2740 1 -- 172.21.15.78:0/1617809628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 msgr2=0x564f854519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.051+0000 7f21a99b2740 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f854519a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2190009900 tx=0x7f2190046040 comp rx=0 tx=0).stop 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 1 -- 172.21.15.78:0/1617809628 shutdown_connections 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f8543cbb0 0x564f854b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f854322f0 0x564f85451ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x564f854bba30 0x564f854ba740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 1 --2- 172.21.15.78:0/1617809628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f854319d0 0x564f854519a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f2188ff9640 0 client.24992 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 1 -- 172.21.15.78:0/1617809628 >> 172.21.15.78:0/1617809628 conn(0x564f8542df60 msgr2=0x564f8542c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 1 -- 172.21.15.78:0/1617809628 shutdown_connections 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 1 -- 172.21.15.78:0/1617809628 wait complete. 2024-09-16T00:13:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a99b2740 20 client.24992 trim_cache size 0 max 16384 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 2, 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24440, 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 133.54360450499999, 2024-09-16T00:13:54.064 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 29, 2024-09-16T00:13:54.065 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:54.065 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-09-16T00:13:54.065 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 192.98076310299999 2024-09-16T00:13:54.065 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 -- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f218c0a32d0 msgr2=0x7f218c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f218c0a32d0 0x7f218c0a5790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2198097030 tx=0x7f2198098190 comp rx=0 tx=0).stop 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 -- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102ef0 msgr2=0x7f21a019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102ef0 0x7f21a019cdf0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f219800cce0 tx=0x7f219800a620 comp rx=0 tx=0).stop 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 -- 172.21.15.78:0/3366229804 shutdown_connections 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f218c0a32d0 0x7f218c0a5790 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21a0102ef0 0x7f21a019cdf0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f21a0102570 0x7f21a0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.052+0000 7f21a75d9640 1 --2- 172.21.15.78:0/3366229804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21a0100e20 0x7f21a0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:54.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.053+0000 7f21a75d9640 1 -- 172.21.15.78:0/3366229804 >> 172.21.15.78:0/3366229804 conn(0x7f21a00fc7d0 msgr2=0x7f21a006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:54.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.053+0000 7f21a75d9640 1 -- 172.21.15.78:0/3366229804 shutdown_connections 2024-09-16T00:13:54.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:54.053+0000 7f21a75d9640 1 -- 172.21.15.78:0/3366229804 wait complete. 2024-09-16T00:13:54.458 INFO:teuthology.orchestra.run.smithi078.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-16T00:13:54.461 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-16T00:13:54.506 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/: 2024-09-16T00:13:54.506 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.506 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 .. 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 data 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data: 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:total 35 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 . 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 .. 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 base 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 16 00:13 global 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 log 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_commit_ts 2024-09-16T00:13:54.507 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_dynshmem 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 16 00:13 pg_hba.conf 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 16 00:13 pg_ident.conf 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 pg_logical 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 pg_multixact 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_notify 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_replslot 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_serial 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_snapshots 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_stat 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_stat_tmp 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 pg_subtrans 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_tblspc 2024-09-16T00:13:54.508 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_twophase 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 PG_VERSION 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 pg_wal 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 pg_xact 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 16 00:13 postgresql.auto.conf 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 16 00:13 postgresql.conf 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base: 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 . 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 1 2024-09-16T00:13:54.509 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 13493 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 13494 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout:total 7777 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 . 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 .. 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 112 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 113 2024-09-16T00:13:54.510 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 1247 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1247_fsm 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1247_vm 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 16 00:13 1249 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1249_fsm 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1249_vm 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 16 00:13 1255 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1255_fsm 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1255_vm 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 16 00:13 1259 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1259_fsm 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1259_vm 2024-09-16T00:13:54.511 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 16 00:13 13344 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13344_fsm 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13344_vm 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13346 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13348 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13349_fsm 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349_vm 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13351 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13353 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13354_fsm 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354_vm 2024-09-16T00:13:54.512 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13356 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13358 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13359_fsm 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359_vm 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13361 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13363 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1417 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1418 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 174 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 175 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2187 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2224 2024-09-16T00:13:54.513 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2328 2024-09-16T00:13:54.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2336 2024-09-16T00:13:54.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2337 2024-09-16T00:13:54.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2579 2024-09-16T00:13:54.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2600 2024-09-16T00:13:54.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2600_fsm 2024-09-16T00:13:54.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2600_vm 2024-09-16T00:13:54.514 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601 2024-09-16T00:13:54.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2601_fsm 2024-09-16T00:13:54.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601_vm 2024-09-16T00:13:54.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2602 2024-09-16T00:13:54.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2602_fsm 2024-09-16T00:13:54.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2602_vm 2024-09-16T00:13:54.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2603 2024-09-16T00:13:54.515 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2603_fsm 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2603_vm 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2604 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2605 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2605_fsm 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2605_vm 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2606_fsm 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606_vm 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2607 2024-09-16T00:13:54.516 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2607_fsm 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2607_vm 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 16 00:13 2608 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2608_fsm 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2608_vm 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 16 00:13 2609 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2609_fsm 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2609_vm 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2610 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2610_fsm 2024-09-16T00:13:54.517 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2610_vm 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2611 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2612_fsm 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612_vm 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2613 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2615_fsm 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615_vm 2024-09-16T00:13:54.518 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2616 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2616_fsm 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2616_vm 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 16 00:13 2617 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2617_fsm 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2617_vm 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 16 00:13 2618 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2618_fsm 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2618_vm 2024-09-16T00:13:54.519 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 16 00:13 2619 2024-09-16T00:13:54.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2619_fsm 2024-09-16T00:13:54.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2619_vm 2024-09-16T00:13:54.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2620 2024-09-16T00:13:54.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2650 2024-09-16T00:13:54.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2651 2024-09-16T00:13:54.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2652 2024-09-16T00:13:54.520 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2653 2024-09-16T00:13:54.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2654 2024-09-16T00:13:54.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2655 2024-09-16T00:13:54.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2656 2024-09-16T00:13:54.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2657 2024-09-16T00:13:54.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 16 00:13 2658 2024-09-16T00:13:54.521 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 2659 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2660 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2661 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2662 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2663 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2664 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2665 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2666 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2667 2024-09-16T00:13:54.522 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2668 2024-09-16T00:13:54.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2669 2024-09-16T00:13:54.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2670 2024-09-16T00:13:54.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 16 00:13 2673 2024-09-16T00:13:54.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 16 00:13 2674 2024-09-16T00:13:54.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 16 00:13 2675 2024-09-16T00:13:54.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2678 2024-09-16T00:13:54.523 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2679 2024-09-16T00:13:54.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2680 2024-09-16T00:13:54.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2681 2024-09-16T00:13:54.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2682 2024-09-16T00:13:54.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2683 2024-09-16T00:13:54.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2684 2024-09-16T00:13:54.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2685 2024-09-16T00:13:54.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2686 2024-09-16T00:13:54.524 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2687 2024-09-16T00:13:54.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2688 2024-09-16T00:13:54.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2689 2024-09-16T00:13:54.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 16 00:13 2690 2024-09-16T00:13:54.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 16 00:13 2691 2024-09-16T00:13:54.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2692 2024-09-16T00:13:54.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2693 2024-09-16T00:13:54.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2696 2024-09-16T00:13:54.525 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2699 2024-09-16T00:13:54.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2701 2024-09-16T00:13:54.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2702 2024-09-16T00:13:54.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2703 2024-09-16T00:13:54.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2704 2024-09-16T00:13:54.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2753 2024-09-16T00:13:54.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2753_fsm 2024-09-16T00:13:54.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2753_vm 2024-09-16T00:13:54.526 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2754 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2755 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2756 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2757 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2830 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2831 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2832 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2833 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2834 2024-09-16T00:13:54.527 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2835 2024-09-16T00:13:54.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2836 2024-09-16T00:13:54.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2837 2024-09-16T00:13:54.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 16 00:13 2838 2024-09-16T00:13:54.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2838_fsm 2024-09-16T00:13:54.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2838_vm 2024-09-16T00:13:54.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2839 2024-09-16T00:13:54.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840 2024-09-16T00:13:54.528 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840_fsm 2024-09-16T00:13:54.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2840_vm 2024-09-16T00:13:54.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2841 2024-09-16T00:13:54.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2995 2024-09-16T00:13:54.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2996 2024-09-16T00:13:54.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079 2024-09-16T00:13:54.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3079_fsm 2024-09-16T00:13:54.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079_vm 2024-09-16T00:13:54.529 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3080 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3081 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3085 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3118 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3119 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 16 00:13 3164 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3256 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3257 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3258 2024-09-16T00:13:54.530 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3350 2024-09-16T00:13:54.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3351 2024-09-16T00:13:54.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3379 2024-09-16T00:13:54.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3380 2024-09-16T00:13:54.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3381 2024-09-16T00:13:54.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3394 2024-09-16T00:13:54.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3394_fsm 2024-09-16T00:13:54.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3394_vm 2024-09-16T00:13:54.531 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3395 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3429 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3430 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3431 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3433 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3439 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3440 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3455 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 16 00:13 3456 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3456_fsm 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3456_vm 2024-09-16T00:13:54.532 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3466 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3467 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3468 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3501 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3502 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3503 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3534 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3541_fsm 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541_vm 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3542 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3574 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3575 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3576 2024-09-16T00:13:54.533 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3596 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3597 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3598 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3599 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3600_fsm 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600_vm 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3601_fsm 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601_vm 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3602_fsm 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602_vm 2024-09-16T00:13:54.534 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603_fsm 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3603_vm 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3604 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3605 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3606 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3607 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3608 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 3609 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3712 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764 2024-09-16T00:13:54.535 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3764_fsm 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764_vm 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3766 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3767 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3997 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4143 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4144 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4145 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4146 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4147 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4148 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4149 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4150 2024-09-16T00:13:54.536 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4151 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4152 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4153 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4154 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4155 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4156 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4157 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4158 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4159 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4160 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4161 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4162 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4163 2024-09-16T00:13:54.537 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4164 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4165 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4166 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4167 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4168 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4169 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4170 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4171 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4172 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4173 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4174 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 5002 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 548 2024-09-16T00:13:54.538 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 549 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6102 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6104 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6106 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6110 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6111 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6112 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6113 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6117 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 826 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 827 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 828 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 16 00:13 pg_filenode.map 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 PG_VERSION 2024-09-16T00:13:54.539 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:total 7777 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 . 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 .. 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 112 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 113 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 1247 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1247_fsm 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1247_vm 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 16 00:13 1249 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1249_fsm 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1249_vm 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 16 00:13 1255 2024-09-16T00:13:54.540 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1255_fsm 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1255_vm 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 16 00:13 1259 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1259_fsm 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1259_vm 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 16 00:13 13344 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13344_fsm 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13344_vm 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13346 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13348 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13349_fsm 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349_vm 2024-09-16T00:13:54.541 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13351 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13353 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13354_fsm 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354_vm 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13356 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13358 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13359_fsm 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359_vm 2024-09-16T00:13:54.542 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13361 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13363 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1417 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1418 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 174 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 175 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2187 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2224 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2328 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2336 2024-09-16T00:13:54.543 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2337 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2579 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2600 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2600_fsm 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2600_vm 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2601_fsm 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601_vm 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2602 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2602_fsm 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2602_vm 2024-09-16T00:13:54.544 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2603 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2603_fsm 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2603_vm 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2604 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2605 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2605_fsm 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2605_vm 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2606_fsm 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606_vm 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2607 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2607_fsm 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2607_vm 2024-09-16T00:13:54.545 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 16 00:13 2608 2024-09-16T00:13:54.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2608_fsm 2024-09-16T00:13:54.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2608_vm 2024-09-16T00:13:54.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 16 00:13 2609 2024-09-16T00:13:54.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2609_fsm 2024-09-16T00:13:54.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2609_vm 2024-09-16T00:13:54.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2610 2024-09-16T00:13:54.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2610_fsm 2024-09-16T00:13:54.546 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2610_vm 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2611 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2612_fsm 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612_vm 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2613 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2615_fsm 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615_vm 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2616 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2616_fsm 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2616_vm 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 16 00:13 2617 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2617_fsm 2024-09-16T00:13:54.547 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2617_vm 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 16 00:13 2618 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2618_fsm 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2618_vm 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 16 00:13 2619 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2619_fsm 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2619_vm 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2620 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2650 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2651 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2652 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2653 2024-09-16T00:13:54.548 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2654 2024-09-16T00:13:54.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2655 2024-09-16T00:13:54.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2656 2024-09-16T00:13:54.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2657 2024-09-16T00:13:54.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 16 00:13 2658 2024-09-16T00:13:54.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 2659 2024-09-16T00:13:54.549 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2660 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2661 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2662 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2663 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2664 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2665 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2666 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2667 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2668 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2669 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2670 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 16 00:13 2673 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 16 00:13 2674 2024-09-16T00:13:54.550 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 16 00:13 2675 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2678 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2679 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2680 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2681 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2682 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2683 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2684 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2685 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2686 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2687 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2688 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2689 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 16 00:13 2690 2024-09-16T00:13:54.551 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 16 00:13 2691 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2692 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2693 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2696 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2699 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2701 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2702 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2703 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2704 2024-09-16T00:13:54.552 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2753 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2753_fsm 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2753_vm 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2754 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2755 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2756 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2757 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2830 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2831 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2832 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2833 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2834 2024-09-16T00:13:54.553 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2835 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2836 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2837 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 16 00:13 2838 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2838_fsm 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2838_vm 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2839 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840_fsm 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2840_vm 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2841 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2995 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2996 2024-09-16T00:13:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3079_fsm 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079_vm 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3080 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3081 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3085 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3118 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3119 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 16 00:13 3164 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3256 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3257 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3258 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3350 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3351 2024-09-16T00:13:54.555 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3379 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3380 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3381 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3394 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3394_fsm 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3394_vm 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3395 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3429 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3430 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3431 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3433 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3439 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3440 2024-09-16T00:13:54.556 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3455 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 16 00:13 3456 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3456_fsm 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3456_vm 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3466 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3467 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3468 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3501 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3502 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3503 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3534 2024-09-16T00:13:54.557 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3541_fsm 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541_vm 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3542 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3574 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3575 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3576 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3596 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3597 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3598 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3599 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600 2024-09-16T00:13:54.558 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3600_fsm 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600_vm 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3601_fsm 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601_vm 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3602_fsm 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602_vm 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603 2024-09-16T00:13:54.559 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603_fsm 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3603_vm 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3604 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3605 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3606 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3607 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3608 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 3609 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3712 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3764_fsm 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764_vm 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3766 2024-09-16T00:13:54.560 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3767 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3997 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4143 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4144 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4145 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4146 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4147 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4148 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4149 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4150 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4151 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4152 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4153 2024-09-16T00:13:54.561 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4154 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4155 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4156 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4157 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4158 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4159 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4160 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4161 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4162 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4163 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4164 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4165 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4166 2024-09-16T00:13:54.562 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4167 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4168 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4169 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4170 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4171 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4172 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4173 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4174 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 5002 2024-09-16T00:13:54.563 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 548 2024-09-16T00:13:54.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 549 2024-09-16T00:13:54.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6102 2024-09-16T00:13:54.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6104 2024-09-16T00:13:54.566 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6106 2024-09-16T00:13:54.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6110 2024-09-16T00:13:54.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6111 2024-09-16T00:13:54.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6112 2024-09-16T00:13:54.567 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6113 2024-09-16T00:13:54.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6117 2024-09-16T00:13:54.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 826 2024-09-16T00:13:54.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 827 2024-09-16T00:13:54.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 828 2024-09-16T00:13:54.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 16 00:13 pg_filenode.map 2024-09-16T00:13:54.568 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 PG_VERSION 2024-09-16T00:13:54.569 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.569 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-16T00:13:54.569 INFO:teuthology.orchestra.run.smithi078.stdout:total 7777 2024-09-16T00:13:54.569 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 . 2024-09-16T00:13:54.569 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 .. 2024-09-16T00:13:54.569 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 112 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 113 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 1247 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1247_fsm 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1247_vm 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 16 00:13 1249 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1249_fsm 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1249_vm 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 16 00:13 1255 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1255_fsm 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1255_vm 2024-09-16T00:13:54.570 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 16 00:13 1259 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1259_fsm 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1259_vm 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 16 00:13 13344 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13344_fsm 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13344_vm 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13346 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13348 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13349_fsm 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349_vm 2024-09-16T00:13:54.571 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13351 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13353 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13354_fsm 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354_vm 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13356 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13358 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13359_fsm 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359_vm 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13361 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13363 2024-09-16T00:13:54.572 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1417 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1418 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 174 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 175 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2187 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2224 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2328 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2336 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2337 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2579 2024-09-16T00:13:54.573 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2600 2024-09-16T00:13:54.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2600_fsm 2024-09-16T00:13:54.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2600_vm 2024-09-16T00:13:54.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601 2024-09-16T00:13:54.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2601_fsm 2024-09-16T00:13:54.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601_vm 2024-09-16T00:13:54.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2602 2024-09-16T00:13:54.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2602_fsm 2024-09-16T00:13:54.574 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2602_vm 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2603 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2603_fsm 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2603_vm 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2604 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2605 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2605_fsm 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2605_vm 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2606_fsm 2024-09-16T00:13:54.575 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606_vm 2024-09-16T00:13:54.576 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2607 2024-09-16T00:13:54.576 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2607_fsm 2024-09-16T00:13:54.576 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2607_vm 2024-09-16T00:13:54.576 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 16 00:13 2608 2024-09-16T00:13:54.576 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2608_fsm 2024-09-16T00:13:54.576 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2608_vm 2024-09-16T00:13:54.576 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 16 00:13 2609 2024-09-16T00:13:54.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2609_fsm 2024-09-16T00:13:54.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2609_vm 2024-09-16T00:13:54.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2610 2024-09-16T00:13:54.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2610_fsm 2024-09-16T00:13:54.577 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2610_vm 2024-09-16T00:13:54.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2611 2024-09-16T00:13:54.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612 2024-09-16T00:13:54.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2612_fsm 2024-09-16T00:13:54.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612_vm 2024-09-16T00:13:54.578 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2613 2024-09-16T00:13:54.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615 2024-09-16T00:13:54.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2615_fsm 2024-09-16T00:13:54.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615_vm 2024-09-16T00:13:54.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2616 2024-09-16T00:13:54.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2616_fsm 2024-09-16T00:13:54.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2616_vm 2024-09-16T00:13:54.579 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 16 00:13 2617 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2617_fsm 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2617_vm 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 16 00:13 2618 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2618_fsm 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2618_vm 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 16 00:13 2619 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2619_fsm 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2619_vm 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2620 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2650 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2651 2024-09-16T00:13:54.580 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2652 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2653 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2654 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2655 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2656 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2657 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 16 00:13 2658 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 2659 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2660 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2661 2024-09-16T00:13:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2662 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2663 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2664 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2665 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2666 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2667 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2668 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2669 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2670 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 16 00:13 2673 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 16 00:13 2674 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 16 00:13 2675 2024-09-16T00:13:54.582 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2678 2024-09-16T00:13:54.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2679 2024-09-16T00:13:54.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2680 2024-09-16T00:13:54.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2681 2024-09-16T00:13:54.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2682 2024-09-16T00:13:54.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2683 2024-09-16T00:13:54.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2684 2024-09-16T00:13:54.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2685 2024-09-16T00:13:54.583 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2686 2024-09-16T00:13:54.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2687 2024-09-16T00:13:54.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2688 2024-09-16T00:13:54.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2689 2024-09-16T00:13:54.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 16 00:13 2690 2024-09-16T00:13:54.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 16 00:13 2691 2024-09-16T00:13:54.584 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2692 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2693 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2696 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2699 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2701 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2702 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2703 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2704 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2753 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2753_fsm 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2753_vm 2024-09-16T00:13:54.585 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2754 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2755 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2756 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2757 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2830 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2831 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2832 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2833 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2834 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2835 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2836 2024-09-16T00:13:54.586 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2837 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 16 00:13 2838 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2838_fsm 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2838_vm 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2839 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840_fsm 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2840_vm 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2841 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2995 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2996 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3079_fsm 2024-09-16T00:13:54.587 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079_vm 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3080 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3081 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3085 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3118 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3119 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 16 00:13 3164 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3256 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3257 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3258 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3350 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3351 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3379 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3380 2024-09-16T00:13:54.588 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3381 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3394 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3394_fsm 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3394_vm 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3395 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3429 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3430 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3431 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3433 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3439 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3440 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3455 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 16 00:13 3456 2024-09-16T00:13:54.589 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3456_fsm 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3456_vm 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3466 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3467 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3468 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3501 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3502 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3503 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3534 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3541_fsm 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541_vm 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3542 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3574 2024-09-16T00:13:54.590 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3575 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3576 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3596 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3597 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3598 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3599 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3600_fsm 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600_vm 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601 2024-09-16T00:13:54.591 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3601_fsm 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601_vm 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3602_fsm 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602_vm 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603_fsm 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3603_vm 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3604 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3605 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3606 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3607 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3608 2024-09-16T00:13:54.592 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 3609 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3712 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3764_fsm 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764_vm 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3766 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3767 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3997 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4143 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4144 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4145 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4146 2024-09-16T00:13:54.593 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4147 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4148 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4149 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4150 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4151 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4152 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4153 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4154 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4155 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4156 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4157 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4158 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4159 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4160 2024-09-16T00:13:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4161 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4162 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4163 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4164 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4165 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4166 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4167 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4168 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4169 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4170 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4171 2024-09-16T00:13:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4172 2024-09-16T00:13:54.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4173 2024-09-16T00:13:54.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4174 2024-09-16T00:13:54.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 5002 2024-09-16T00:13:54.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 548 2024-09-16T00:13:54.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 549 2024-09-16T00:13:54.596 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6102 2024-09-16T00:13:54.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6104 2024-09-16T00:13:54.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6106 2024-09-16T00:13:54.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6110 2024-09-16T00:13:54.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6111 2024-09-16T00:13:54.597 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6112 2024-09-16T00:13:54.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6113 2024-09-16T00:13:54.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6117 2024-09-16T00:13:54.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 826 2024-09-16T00:13:54.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 827 2024-09-16T00:13:54.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 828 2024-09-16T00:13:54.598 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 16 00:13 pg_filenode.map 2024-09-16T00:13:54.599 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 PG_VERSION 2024-09-16T00:13:54.599 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.599 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/global: 2024-09-16T00:13:54.599 INFO:teuthology.orchestra.run.smithi078.stdout:total 537 2024-09-16T00:13:54.599 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 16 00:13 . 2024-09-16T00:13:54.599 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1213 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1213_fsm 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1213_vm 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1214 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1214_fsm 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1214_vm 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 1232 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 1233 2024-09-16T00:13:54.600 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1260 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1260_fsm 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1260_vm 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1261 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1261_fsm 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1261_vm 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1262 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1262_fsm 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1262_vm 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2396 2024-09-16T00:13:54.601 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2396_fsm 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2396_vm 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2397 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2671 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2672 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2676 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2677 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2694 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2695 2024-09-16T00:13:54.602 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2697 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2698 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2846 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2847 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2964 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2965 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2966 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2967 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3592 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3593 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4060 2024-09-16T00:13:54.603 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4061 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4175 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4176 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4177 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4178 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4181 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4182 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4183 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4184 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4185 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4186 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6000 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6001 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6002 2024-09-16T00:13:54.604 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6100 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6114 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6115 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 pg_control 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 16 00:13 pg_filenode.map 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/log: 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.605 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:total 1 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 . 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 mappings 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 16 00:13 replorigin_checkpoint 2024-09-16T00:13:54.606 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 snapshots 2024-09-16T00:13:54.607 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.607 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-16T00:13:54.607 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.607 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.607 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 .. 2024-09-16T00:13:54.607 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.608 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-16T00:13:54.608 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.608 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.608 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 .. 2024-09-16T00:13:54.608 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.608 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-16T00:13:54.609 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.609 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 . 2024-09-16T00:13:54.609 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.609 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 members 2024-09-16T00:13:54.609 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 offsets 2024-09-16T00:13:54.609 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.610 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-16T00:13:54.610 INFO:teuthology.orchestra.run.smithi078.stdout:total 8 2024-09-16T00:13:54.610 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:13:54.610 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 .. 2024-09-16T00:13:54.610 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 0000 2024-09-16T00:13:54.611 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.611 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-16T00:13:54.611 INFO:teuthology.orchestra.run.smithi078.stdout:total 8 2024-09-16T00:13:54.611 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:13:54.611 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 .. 2024-09-16T00:13:54.611 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 0000 2024-09-16T00:13:54.612 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.612 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-16T00:13:54.612 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.612 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.612 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.612 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.612 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-16T00:13:54.612 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-16T00:13:54.613 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-16T00:13:54.614 INFO:teuthology.orchestra.run.smithi078.stdout:total 8 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 0000 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.615 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:total 16384 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 . 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:13 000000010000000000000001 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 archive_status 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 .. 2024-09-16T00:13:54.616 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:13:54.617 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-16T00:13:54.617 INFO:teuthology.orchestra.run.smithi078.stdout:total 8 2024-09-16T00:13:54.617 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:13:54.617 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 16 00:13 .. 2024-09-16T00:13:54.617 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 0000 2024-09-16T00:13:54.617 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-16T00:13:54.805 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-16T00:13:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:dropping old tables... 2024-09-16T00:13:54.854 INFO:teuthology.orchestra.run.smithi078.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-16T00:13:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-16T00:13:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-16T00:13:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-16T00:13:54.855 INFO:teuthology.orchestra.run.smithi078.stderr:creating tables... 2024-09-16T00:13:54.869 INFO:teuthology.orchestra.run.smithi078.stderr:generating data (client-side)... 2024-09-16T00:13:54.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-16T00:13:54.947 INFO:teuthology.orchestra.run.smithi078.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.86 s) 2024-09-16T00:13:55.120 INFO:teuthology.orchestra.run.smithi078.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.23 s, remaining 3.49 s) 2024-09-16T00:13:55.216 INFO:teuthology.orchestra.run.smithi078.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.33 s, remaining 3.18 s) 2024-09-16T00:13:55.223 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:55.351 INFO:teuthology.orchestra.run.smithi078.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.46 s, remaining 3.25 s) 2024-09-16T00:13:55.511 INFO:teuthology.orchestra.run.smithi078.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.62 s, remaining 3.37 s) 2024-09-16T00:13:55.661 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:55 smithi078 ceph-mon[31157]: pgmap v376: 129 pgs: 129 active+clean; 18 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 MiB/s wr, 25 op/s 2024-09-16T00:13:55.661 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:55 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:55.670 INFO:teuthology.orchestra.run.smithi078.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.78 s, remaining 3.39 s) 2024-09-16T00:13:55.754 INFO:teuthology.orchestra.run.smithi078.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.87 s, remaining 3.10 s) 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 -- 172.21.15.78:0/4236944678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4101dd0 msgr2=0x7f4ae41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 --2- 172.21.15.78:0/4236944678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4101dd0 0x7f4ae41021b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009900 tx=0x7f4ad402a410 comp rx=0 tx=0).stop 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 -- 172.21.15.78:0/4236944678 shutdown_connections 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 --2- 172.21.15.78:0/4236944678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ae4106f00 0x7f4ae410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 --2- 172.21.15.78:0/4236944678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4102780 0x7f4ae41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 --2- 172.21.15.78:0/4236944678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4101dd0 0x7f4ae41021b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 -- 172.21.15.78:0/4236944678 >> 172.21.15.78:0/4236944678 conn(0x7f4ae40fd8b0 msgr2=0x7f4ae40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 -- 172.21.15.78:0/4236944678 shutdown_connections 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.747+0000 7f4ae937b640 1 -- 172.21.15.78:0/4236944678 wait complete. 2024-09-16T00:13:55.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae937b640 1 Processor -- start 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae937b640 1 -- start start 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae937b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ae4101dd0 0x7f4ae41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae937b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4102780 0x7f4ae4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae937b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4106f00 0x7f4ae419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae937b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111fb0 con 0x7f4ae4102780 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae937b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111e30 con 0x7f4ae4101dd0 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae937b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4112130 con 0x7f4ae4106f00 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4102780 0x7f4ae4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4102780 0x7f4ae4198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33378/0 (socket says 172.21.15.78:33378) 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4adbfff640 1 -- 172.21.15.78:0/2643687165 learned_addr learned my addr 172.21.15.78:0/2643687165 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ae4101dd0 0x7f4ae41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae37fe640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4106f00 0x7f4ae419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4adbfff640 1 -- 172.21.15.78:0/2643687165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4106f00 msgr2=0x7f4ae419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4adbfff640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4106f00 0x7f4ae419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4adbfff640 1 -- 172.21.15.78:0/2643687165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ae4101dd0 msgr2=0x7f4ae41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4adbfff640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ae4101dd0 0x7f4ae41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4adbfff640 1 -- 172.21.15.78:0/2643687165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae419d480 con 0x7f4ae4102780 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.748+0000 7f4ae37fe640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4106f00 0x7f4ae419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.749+0000 7f4adbfff640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4102780 0x7f4ae4198a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00e9f0 tx=0x7f4acc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.749+0000 7f4ae0ff9640 1 -- 172.21.15.78:0/2643687165 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc00cd80 con 0x7f4ae4102780 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.749+0000 7f4ae937b640 1 -- 172.21.15.78:0/2643687165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae419d770 con 0x7f4ae4102780 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.749+0000 7f4ae0ff9640 1 -- 172.21.15.78:0/2643687165 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4acc005500 con 0x7f4ae4102780 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.749+0000 7f4ae0ff9640 1 -- 172.21.15.78:0/2643687165 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc010690 con 0x7f4ae4102780 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.749+0000 7f4ae937b640 1 -- 172.21.15.78:0/2643687165 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae41a5050 con 0x7f4ae4102780 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4aeb754740 1 Processor -- start 2024-09-16T00:13:55.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4aeb754740 1 -- start start 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 0x563569c08cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c17a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c1a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4aeb754740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563569b6f090 con 0x563569c091f0 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4aeb754740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563569b16a70 con 0x563569c088d0 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4aeb754740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563569b6b050 con 0x563569c17fc0 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4ae0ff9640 1 -- 172.21.15.78:0/2643687165 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f4acc0046c0 con 0x7f4ae4102780 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c17a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c17a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33386/0 (socket says 172.21.15.78:33386) 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4ae37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 0x563569c08cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.752+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/2083933629 learned_addr learned my addr 172.21.15.78:0/2083933629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4adbfff640 1 --2- 172.21.15.78:0/2083933629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c1a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/2083933629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 msgr2=0x563569c1a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/2083933629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c1a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/2083933629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 msgr2=0x563569c08cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/2083933629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 0x563569c08cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/2083933629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563569b23130 con 0x563569c091f0 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/2083933629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c17a80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009900 tx=0x7f4ad4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d1623f73cf4a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae0ff9640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4ad00a32d0 0x7f4ad00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae37fe640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4ad00a32d0 0x7f4ad00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ad97fa640 1 -- 172.21.15.78:0/2083933629 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4035040 con 0x563569c091f0 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ae0ff9640 1 -- 172.21.15.78:0/2643687165 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4acc0d0290 con 0x7f4ae4102780 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ad97fa640 1 -- 172.21.15.78:0/2083933629 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad4005520 con 0x563569c091f0 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.753+0000 7f4ad97fa640 1 -- 172.21.15.78:0/2083933629 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4003850 con 0x563569c091f0 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 -- 172.21.15.78:0/2083933629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 msgr2=0x563569c17a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4ae37fe640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4ad00a32d0 0x7f4ad00a5790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4adc005fd0 tx=0x7f4adc009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:55.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 --2- 172.21.15.78:0/2083933629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c17a80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009900 tx=0x7f4ad4004040 comp rx=0 tx=0).stop 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 -- 172.21.15.78:0/2083933629 shutdown_connections 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 --2- 172.21.15.78:0/2083933629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c1a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 --2- 172.21.15.78:0/2083933629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c17a80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 --2- 172.21.15.78:0/2083933629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 0x563569c08cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 -- 172.21.15.78:0/2083933629 >> 172.21.15.78:0/2083933629 conn(0x563569c04f60 msgr2=0x563569c05340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 -- 172.21.15.78:0/2083933629 shutdown_connections 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.754+0000 7f4aeb754740 1 -- 172.21.15.78:0/2083933629 wait complete. 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 1 -- start start 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 1 Processor -- start 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c17910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c091f0 0x563569c14c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c17fc0 0x563569c151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563569c1ab90 con 0x563569c088d0 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563569b29f20 con 0x563569c17fc0 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563569c1ad10 con 0x563569c091f0 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4aeb754740 10 client.0 _ll_register_callbacks cb 0x563569af3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c17910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c17910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33398/0 (socket says 172.21.15.78:33398) 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/3998509961 learned_addr learned my addr 172.21.15.78:0/3998509961 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4ae37fe640 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c17fc0 0x563569c151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4adbfff640 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c091f0 0x563569c14c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.755+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/3998509961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c091f0 msgr2=0x563569c14c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c091f0 0x563569c14c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/3998509961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c17fc0 msgr2=0x563569c151c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c17fc0 0x563569c151c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/3998509961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635698e02b0 con 0x563569c088d0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c17910 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4005bc0 tx=0x7f4ad402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba1a5b33488322a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aca7fc640 1 -- 172.21.15.78:0/3998509961 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4035050 con 0x563569c088d0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aca7fc640 1 -- 172.21.15.78:0/3998509961 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4ad4004e60 con 0x563569c088d0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aeb754740 1 -- 172.21.15.78:0/3998509961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad4005520 con 0x563569c088d0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aca7fc640 1 -- 172.21.15.78:0/3998509961 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4ad4009520 con 0x563569c088d0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aeb754740 10 client.15501 fetch_fsmap learned FSMap version 30 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aeb754740 1 -- 172.21.15.78:0/3998509961 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563569c0b610 con 0x563569c088d0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aca7fc640 1 -- 172.21.15.78:0/3998509961 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad4005520 con 0x563569c088d0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aca7fc640 1 -- 172.21.15.78:0/3998509961 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4ad40315b0 con 0x563569c088d0 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.756+0000 7f4aeb754740 10 client.15501 fetch_fsmap finished waiting for FSMap version 30 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.757+0000 7f4aeb754740 10 client.15501 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.757+0000 7f4aeb754740 20 client.15501 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.757+0000 7f4aeb754740 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x563569c0d820 0x563569c0ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.757+0000 7f4aeb754740 4 client.15501 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.757+0000 7f4aeb754740 1 -- 172.21.15.78:0/3998509961 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563569b7c600 con 0x563569c0d820 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.757+0000 7f4adbfff640 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x563569c0d820 0x563569c0ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.757+0000 7f4aca7fc640 10 client.15501 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.757+0000 7f4adbfff640 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x563569c0d820 0x563569c0ca80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d2cd079dda51e08 server_cookie=be9303cad172ca5a in_seq=0 out_seq=0 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aca7fc640 1 -- 172.21.15.78:0/3998509961 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563569b7c600 con 0x563569c0d820 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aca7fc640 10 client.15501 handle_command_reply: tid=0 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 client.15501 shutdown 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 -- 172.21.15.78:0/3998509961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 msgr2=0x563569c17910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c17910 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4005bc0 tx=0x7f4ad402a410 comp rx=0 tx=0).stop 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 -- 172.21.15.78:0/3998509961 shutdown_connections 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x563569c0d820 0x563569c0ca80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c17fc0 0x563569c151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c091f0 0x563569c14c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 --2- 172.21.15.78:0/3998509961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c17910 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 -- 172.21.15.78:0/3998509961 >> 172.21.15.78:0/3998509961 conn(0x563569c04f60 msgr2=0x563569c0ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aca7fc640 0 client.15501 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 -- 172.21.15.78:0/3998509961 shutdown_connections 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 1 -- 172.21.15.78:0/3998509961 wait complete. 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.758+0000 7f4aeb754740 20 client.15501 trim_cache size 0 max 16384 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4aeb754740 1 Processor -- start 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4aeb754740 1 -- start start 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 0x563569c0def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c0e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c0d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4aeb754740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563569c0cfd0 con 0x563569c091f0 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4aeb754740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563569c0d6a0 con 0x563569c088d0 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4aeb754740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563569c0dd70 con 0x563569c17fc0 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.776+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c0e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c0e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33410/0 (socket says 172.21.15.78:33410) 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4adbfff640 1 -- 172.21.15.78:0/2918502131 learned_addr learned my addr 172.21.15.78:0/2918502131 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4ae37fe640 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c0d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 0x563569c0def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4adbfff640 1 -- 172.21.15.78:0/2918502131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 msgr2=0x563569c0d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4adbfff640 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c0d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4adbfff640 1 -- 172.21.15.78:0/2918502131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 msgr2=0x563569c0def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4adbfff640 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 0x563569c0def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4adbfff640 1 -- 172.21.15.78:0/2918502131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563569c09020 con 0x563569c091f0 2024-09-16T00:13:55.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4adbfff640 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c0e5c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00c6a0 tx=0x7f4acc0d2780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e8f95fb8f80214 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:55.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4ad8ff9640 1 -- 172.21.15.78:0/2918502131 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc098690 con 0x563569c091f0 2024-09-16T00:13:55.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4ad8ff9640 1 -- 172.21.15.78:0/2918502131 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4acc09caf0 con 0x563569c091f0 2024-09-16T00:13:55.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.777+0000 7f4ad8ff9640 1 -- 172.21.15.78:0/2918502131 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4acc09bd50 con 0x563569c091f0 2024-09-16T00:13:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 -- 172.21.15.78:0/2918502131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 msgr2=0x563569c0e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c0e5c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00c6a0 tx=0x7f4acc0d2780 comp rx=0 tx=0).stop 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 -- 172.21.15.78:0/2918502131 shutdown_connections 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c0d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c091f0 0x563569c0e5c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 --2- 172.21.15.78:0/2918502131 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c088d0 0x563569c0def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 -- 172.21.15.78:0/2918502131 >> 172.21.15.78:0/2918502131 conn(0x563569c04f60 msgr2=0x563569c05340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 -- 172.21.15.78:0/2918502131 shutdown_connections 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.778+0000 7f4aeb754740 1 -- 172.21.15.78:0/2918502131 wait complete. 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 1 -- start start 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 1 Processor -- start 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c091f0 0x563569c26d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c27240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563569c0eb10 con 0x563569c088d0 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563569c0e440 con 0x563569c091f0 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563569c0f1e0 con 0x563569c17fc0 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4aeb754740 10 client.0 _ll_register_callbacks cb 0x7f4adc009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.779+0000 7f4ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c267c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33424/0 (socket says 172.21.15.78:33424) 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/1523612865 learned_addr learned my addr 172.21.15.78:0/1523612865 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4adbfff640 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c091f0 0x563569c26d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4ae37fe640 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c27240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/1523612865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 msgr2=0x563569c27240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c27240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/1523612865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c091f0 msgr2=0x563569c26d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c091f0 0x563569c26d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4ae2ffd640 1 -- 172.21.15.78:0/1523612865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563569c8cac0 con 0x563569c088d0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4adbfff640 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c091f0 0x563569c26d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4ae2ffd640 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c267c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40096c0 tx=0x7f4ad40317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70ea015062861be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4aca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4aca7fc640 1 -- 172.21.15.78:0/1523612865 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4035040 con 0x563569c088d0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4aca7fc640 1 -- 172.21.15.78:0/1523612865 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4ad404f8d0 con 0x563569c088d0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4aeb754740 1 -- 172.21.15.78:0/1523612865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4acc09caf0 con 0x563569c088d0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4aca7fc640 1 -- 172.21.15.78:0/1523612865 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4ad406c050 con 0x563569c088d0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4aeb754740 10 client.15507 fetch_fsmap learned FSMap version 30 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.780+0000 7f4aeb754740 1 -- 172.21.15.78:0/1523612865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563569c8dd80 con 0x563569c088d0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4aca7fc640 1 -- 172.21.15.78:0/1523612865 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4acc09caf0 con 0x563569c088d0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4aca7fc640 1 -- 172.21.15.78:0/1523612865 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f4ad40046c0 con 0x563569c088d0 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4aeb754740 10 client.15507 fetch_fsmap finished waiting for FSMap version 30 2024-09-16T00:13:55.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4aeb754740 10 client.15507 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4aeb754740 20 client.15507 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4aeb754740 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x563569c92970 0x563569c91650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4aeb754740 4 client.15507 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4aeb754740 1 -- 172.21.15.78:0/1523612865 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "status"}) -- 0x563569b76b10 con 0x563569c92970 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.781+0000 7f4adbfff640 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x563569c92970 0x563569c91650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4adbfff640 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x563569c92970 0x563569c91650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a2c4ca73c0a489b server_cookie=fa2608d3c586dc02 in_seq=0 out_seq=0 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aca7fc640 10 client.15507 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aca7fc640 1 -- 172.21.15.78:0/1523612865 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563569b76b10 con 0x563569c92970 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aca7fc640 10 client.15507 handle_command_reply: tid=0 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 client.15507 shutdown 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 -- 172.21.15.78:0/1523612865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 msgr2=0x563569c267c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c267c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4ad40096c0 tx=0x7f4ad40317c0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 -- 172.21.15.78:0/1523612865 shutdown_connections 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x563569c92970 0x563569c91650 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563569c17fc0 0x563569c27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563569c091f0 0x563569c26d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 --2- 172.21.15.78:0/1523612865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563569c088d0 0x563569c267c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 -- 172.21.15.78:0/1523612865 >> 172.21.15.78:0/1523612865 conn(0x563569c04f60 msgr2=0x563569c102d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aca7fc640 0 client.15507 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 -- 172.21.15.78:0/1523612865 shutdown_connections 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 1 -- 172.21.15.78:0/1523612865 wait complete. 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.782+0000 7f4aeb754740 20 client.15507 trim_cache size 0 max 16384 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:55.798 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 1, 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24445, 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 136.28349395199999, 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 30, 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 192.10981169199999 2024-09-16T00:13:55.799 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 -- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4ad00a32d0 msgr2=0x7f4ad00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4ad00a32d0 0x7f4ad00a5790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4adc005fd0 tx=0x7f4adc009780 comp rx=0 tx=0).stop 2024-09-16T00:13:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 -- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4102780 msgr2=0x7f4ae4198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4102780 0x7f4ae4198a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4acc00e9f0 tx=0x7f4acc00edf0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 -- 172.21.15.78:0/2643687165 shutdown_connections 2024-09-16T00:13:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4ad00a32d0 0x7f4ad00a5790 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ae4106f00 0x7f4ae419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ae4102780 0x7f4ae4198a10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 --2- 172.21.15.78:0/2643687165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4ae4101dd0 0x7f4ae41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:55.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 -- 172.21.15.78:0/2643687165 >> 172.21.15.78:0/2643687165 conn(0x7f4ae40fd8b0 msgr2=0x7f4ae40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:55.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 -- 172.21.15.78:0/2643687165 shutdown_connections 2024-09-16T00:13:55.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:55.783+0000 7f4ae937b640 1 -- 172.21.15.78:0/2643687165 wait complete. 2024-09-16T00:13:55.913 INFO:teuthology.orchestra.run.smithi078.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.03 s, remaining 3.08 s) 2024-09-16T00:13:55.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:55 smithi193 ceph-mon[35484]: pgmap v376: 129 pgs: 129 active+clean; 18 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 MiB/s wr, 25 op/s 2024-09-16T00:13:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:55 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:55 smithi148 ceph-mon[35599]: pgmap v376: 129 pgs: 129 active+clean; 18 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 MiB/s wr, 25 op/s 2024-09-16T00:13:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:55 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.10 s, remaining 2.81 s) 2024-09-16T00:13:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.27 s, remaining 2.79 s) 2024-09-16T00:13:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.42 s, remaining 2.70 s) 2024-09-16T00:13:56.407 INFO:teuthology.orchestra.run.smithi078.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.52 s, remaining 2.53 s) 2024-09-16T00:13:56.550 INFO:teuthology.orchestra.run.smithi078.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.66 s, remaining 2.43 s) 2024-09-16T00:13:56.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:13:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.78 s, remaining 2.29 s) 2024-09-16T00:13:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.93 s, remaining 2.19 s) 2024-09-16T00:13:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:13:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.08 s, remaining 2.08 s) 2024-09-16T00:13:57.050 INFO:teuthology.orchestra.run.smithi078.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.16 s, remaining 1.91 s) 2024-09-16T00:13:57.228 INFO:teuthology.orchestra.run.smithi078.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.34 s, remaining 1.82 s) 2024-09-16T00:13:57.368 INFO:teuthology.orchestra.run.smithi078.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.48 s, remaining 1.70 s) 2024-09-16T00:13:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/1237542394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 msgr2=0x7f5a44101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/1237542394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 0x7f5a44101200 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009900 tx=0x7f5a3802a480 comp rx=0 tx=0).stop 2024-09-16T00:13:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/1237542394 shutdown_connections 2024-09-16T00:13:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/1237542394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a44102ef0 0x7f5a4410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/1237542394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a44102570 0x7f5a441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/1237542394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 0x7f5a44101200 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/1237542394 >> 172.21.15.78:0/1237542394 conn(0x7f5a440fc7d0 msgr2=0x7f5a440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/1237542394 shutdown_connections 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.449+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/1237542394 wait complete. 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.450+0000 7f5a4b5bd640 1 Processor -- start 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.450+0000 7f5a4b5bd640 1 -- start start 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.450+0000 7f5a4b5bd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 0x7f5a441974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.450+0000 7f5a4b5bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a44102570 0x7f5a441979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.450+0000 7f5a4b5bd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a44102ef0 0x7f5a4419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.450+0000 7f5a4b5bd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44110f80 con 0x7f5a44100e20 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.450+0000 7f5a4b5bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44110e00 con 0x7f5a44102570 2024-09-16T00:13:57.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.450+0000 7f5a4b5bd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44111100 con 0x7f5a44102ef0 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 0x7f5a441974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 0x7f5a441974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33762/0 (socket says 172.21.15.78:33762) 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a48b58640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a44102570 0x7f5a441979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 -- 172.21.15.78:0/78864644 learned_addr learned my addr 172.21.15.78:0/78864644 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a44102ef0 0x7f5a4419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 -- 172.21.15.78:0/78864644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a44102ef0 msgr2=0x7f5a4419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a44102ef0 0x7f5a4419bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 -- 172.21.15.78:0/78864644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a44102570 msgr2=0x7f5a441979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a44102570 0x7f5a441979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 -- 172.21.15.78:0/78864644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4419c450 con 0x7f5a44100e20 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a48b58640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a44102570 0x7f5a441979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a49359640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 0x7f5a441974a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5a38006010 tx=0x7f5a38003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:57.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a367fc640 1 -- 172.21.15.78:0/78864644 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3803f070 con 0x7f5a44100e20 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a367fc640 1 -- 172.21.15.78:0/78864644 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a38031c30 con 0x7f5a44100e20 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a367fc640 1 -- 172.21.15.78:0/78864644 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38035030 con 0x7f5a44100e20 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/78864644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4419c6e0 con 0x7f5a44100e20 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.451+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/78864644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a44069f70 con 0x7f5a44100e20 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a4d996740 1 Processor -- start 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a367fc640 1 -- 172.21.15.78:0/78864644 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f5a38042070 con 0x7f5a44100e20 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a4d996740 1 -- start start 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736707c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736716a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 0x55d7367193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a4d996740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d73666e090 con 0x55d736707890 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a4d996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d736615a70 con 0x55d736716fc0 2024-09-16T00:13:57.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a4d996740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d73666a050 con 0x55d7367081b0 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.454+0000 7f5a49b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736707c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736707c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33764/0 (socket says 172.21.15.78:33764) 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2358991441 learned_addr learned my addr 172.21.15.78:0/2358991441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49359640 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736716a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a48b58640 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 0x55d7367193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2358991441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 msgr2=0x55d736716a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736716a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2358991441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 msgr2=0x55d7367193b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 0x55d7367193b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2358991441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d736622130 con 0x55d736707890 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a367fc640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5a300a3350 0x7f5a300a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736707c70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009f70 tx=0x7f5a3c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a11d1288769b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:57.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a367fc640 1 -- 172.21.15.78:0/78864644 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5a380f0590 con 0x7f5a44100e20 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a49359640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5a300a3350 0x7f5a300a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.455+0000 7f5a2f7fe640 1 -- 172.21.15.78:0/2358991441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c016070 con 0x55d736707890 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a2f7fe640 1 -- 172.21.15.78:0/2358991441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c004d20 con 0x55d736707890 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a2f7fe640 1 -- 172.21.15.78:0/2358991441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c012950 con 0x55d736707890 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a4d996740 1 -- 172.21.15.78:0/2358991441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 msgr2=0x55d736707c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736707c70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009f70 tx=0x7f5a3c010800 comp rx=0 tx=0).stop 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a49359640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5a300a3350 0x7f5a300a5810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005ab0 tx=0x7f5a3802a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a4d996740 1 -- 172.21.15.78:0/2358991441 shutdown_connections 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 0x55d7367193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736716a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2358991441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736707c70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.456+0000 7f5a4d996740 1 -- 172.21.15.78:0/2358991441 >> 172.21.15.78:0/2358991441 conn(0x55d736703f60 msgr2=0x55d736704340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.457+0000 7f5a4d996740 1 -- 172.21.15.78:0/2358991441 shutdown_connections 2024-09-16T00:13:57.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.457+0000 7f5a4d996740 1 -- 172.21.15.78:0/2358991441 wait complete. 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.457+0000 7f5a4d996740 1 -- start start 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.457+0000 7f5a4d996740 1 Processor -- start 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736716930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 0x55d736713c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736716fc0 0x55d7367141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a4d996740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d736719b90 con 0x55d736707890 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a4d996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d736628f20 con 0x55d7367081b0 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a4d996740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d736719d10 con 0x55d736716fc0 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a4d996740 10 client.0 _ll_register_callbacks cb 0x55d7365f2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736716930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736716930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33770/0 (socket says 172.21.15.78:33770) 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2740355565 learned_addr learned my addr 172.21.15.78:0/2740355565 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a48b58640 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 0x55d736713c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736716fc0 0x55d7367141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2740355565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736716fc0 msgr2=0x55d7367141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736716fc0 0x55d7367141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2740355565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 msgr2=0x55d736713c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 0x55d736713c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2740355565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7363df2b0 con 0x55d736707890 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736716930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0049d0 tx=0x7f5a3c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ad8f9cf2ac4921 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.458+0000 7f5a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a2cff9640 1 -- 172.21.15.78:0/2740355565 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c016070 con 0x55d736707890 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a2cff9640 1 -- 172.21.15.78:0/2740355565 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5a3c0051f0 con 0x55d736707890 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 1 -- 172.21.15.78:0/2740355565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3c004d20 con 0x55d736707890 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a2cff9640 1 -- 172.21.15.78:0/2740355565 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c032050 con 0x55d736707890 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 10 client.15519 fetch_fsmap learned FSMap version 31 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 1 -- 172.21.15.78:0/2740355565 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d7367255a0 con 0x55d736707890 2024-09-16T00:13:57.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a2cff9640 1 -- 172.21.15.78:0/2740355565 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c004d20 con 0x55d736707890 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a2cff9640 1 -- 172.21.15.78:0/2740355565 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5a3c00f040 con 0x55d736707890 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 10 client.15519 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 10 client.15519 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 20 client.15519 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d736726950 0x55d736728470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 4 client.15519 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.459+0000 7f5a4d996740 1 -- 172.21.15.78:0/2740355565 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d73667b600 con 0x55d736726950 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.460+0000 7f5a48b58640 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d736726950 0x55d736728470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.460+0000 7f5a48b58640 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d736726950 0x55d736728470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a647a90cbc4601c server_cookie=e5e00d281983037e in_seq=0 out_seq=0 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.460+0000 7f5a2cff9640 10 client.15519 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a2cff9640 1 -- 172.21.15.78:0/2740355565 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d73667b600 con 0x55d736726950 2024-09-16T00:13:57.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a2cff9640 10 client.15519 handle_command_reply: tid=0 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 client.15519 shutdown 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 -- 172.21.15.78:0/2740355565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 msgr2=0x55d736716930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736716930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0049d0 tx=0x7f5a3c012450 comp rx=0 tx=0).stop 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 -- 172.21.15.78:0/2740355565 shutdown_connections 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d736726950 0x55d736728470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736716fc0 0x55d7367141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 0x55d736713c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2740355565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d736716930 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a2cff9640 0 client.15519 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 -- 172.21.15.78:0/2740355565 >> 172.21.15.78:0/2740355565 conn(0x55d736703f60 msgr2=0x55d736709cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 -- 172.21.15.78:0/2740355565 shutdown_connections 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 1 -- 172.21.15.78:0/2740355565 wait complete. 2024-09-16T00:13:57.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.461+0000 7f5a4d996740 20 client.15519 trim_cache size 0 max 16384 2024-09-16T00:13:57.481 INFO:teuthology.orchestra.run.smithi078.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.59 s, remaining 1.55 s) 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.479+0000 7f5a4d996740 1 Processor -- start 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.479+0000 7f5a4d996740 1 -- start start 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.479+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736707890 0x55d736727da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.479+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 0x55d736728470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.479+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736716fc0 0x55d736727000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a4d996740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d736726e80 con 0x55d736716fc0 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a4d996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d736727550 con 0x55d7367081b0 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a4d996740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d736727c20 con 0x55d736707890 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736716fc0 0x55d736727000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736716fc0 0x55d736727000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33784/0 (socket says 172.21.15.78:33784) 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2763161823 learned_addr learned my addr 172.21.15.78:0/2763161823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a48b58640 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 0x55d736728470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736707890 0x55d736727da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2763161823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 msgr2=0x55d736728470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 0x55d736728470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2763161823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736716fc0 msgr2=0x55d736727000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736716fc0 0x55d736727000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 -- 172.21.15.78:0/2763161823 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d736714550 con 0x55d736707890 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736707890 0x55d736727da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c01a630 tx=0x7f5a3c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c19b64da7d0c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.480+0000 7f5a2effd640 1 -- 172.21.15.78:0/2763161823 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c016040 con 0x55d736707890 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.481+0000 7f5a2effd640 1 -- 172.21.15.78:0/2763161823 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c010d00 con 0x55d736707890 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.481+0000 7f5a2effd640 1 -- 172.21.15.78:0/2763161823 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c01ac80 con 0x55d736707890 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 -- 172.21.15.78:0/2763161823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736707890 msgr2=0x55d736727da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736707890 0x55d736727da0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c01a630 tx=0x7f5a3c000f40 comp rx=0 tx=0).stop 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 -- 172.21.15.78:0/2763161823 shutdown_connections 2024-09-16T00:13:57.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736716fc0 0x55d736727000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d7367081b0 0x55d736728470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 --2- 172.21.15.78:0/2763161823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d736707890 0x55d736727da0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 -- 172.21.15.78:0/2763161823 >> 172.21.15.78:0/2763161823 conn(0x55d736703f60 msgr2=0x55d7366fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:57.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 -- 172.21.15.78:0/2763161823 shutdown_connections 2024-09-16T00:13:57.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 -- 172.21.15.78:0/2763161823 wait complete. 2024-09-16T00:13:57.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 -- start start 2024-09-16T00:13:57.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.482+0000 7f5a4d996740 1 Processor -- start 2024-09-16T00:13:57.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d7367311d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736731710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a4d996740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 0x55d73672a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a4d996740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d7367289c0 con 0x55d736707890 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a4d996740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d7367282f0 con 0x55d736716fc0 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a4d996740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7367290f0 con 0x55d7367081b0 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a4d996740 10 client.0 _ll_register_callbacks cb 0x7f5a3c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d7367311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d7367311d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33796/0 (socket says 172.21.15.78:33796) 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 -- 172.21.15.78:0/3173264672 learned_addr learned my addr 172.21.15.78:0/3173264672 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a48b58640 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736731710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 -- 172.21.15.78:0/3173264672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 msgr2=0x55d736731710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:57.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736731710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 -- 172.21.15.78:0/3173264672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 msgr2=0x55d73672a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 0x55d73672a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 -- 172.21.15.78:0/3173264672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d73672ae90 con 0x55d736707890 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a48b58640 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736731710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a49b5a640 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d7367311d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004ec0 tx=0x7f5a3c01a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f114902b396f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.483+0000 7f5a2cff9640 1 -- 172.21.15.78:0/3173264672 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c03b060 con 0x55d736707890 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a2cff9640 1 -- 172.21.15.78:0/3173264672 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5a3c0053f0 con 0x55d736707890 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 1 -- 172.21.15.78:0/3173264672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3c010d00 con 0x55d736707890 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a2cff9640 1 -- 172.21.15.78:0/3173264672 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c042050 con 0x55d736707890 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 10 client.15525 fetch_fsmap learned FSMap version 31 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 1 -- 172.21.15.78:0/3173264672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d73672f970 con 0x55d736707890 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a2cff9640 1 -- 172.21.15.78:0/3173264672 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c010d00 con 0x55d736707890 2024-09-16T00:13:57.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a2cff9640 1 -- 172.21.15.78:0/3173264672 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5a3c02b430 con 0x55d736707890 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 10 client.15525 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 10 client.15525 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 20 client.15525 populate_metadata read hostname 'smithi078' 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d736796280 0x55d736794830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 4 client.15525 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.484+0000 7f5a4d996740 1 -- 172.21.15.78:0/3173264672 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "status"}) -- 0x55d736675b10 con 0x55d736796280 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a48b58640 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d736796280 0x55d736794830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a48b58640 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d736796280 0x55d736794830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=531531615c668681 server_cookie=8cfe242dcdfc5b77 in_seq=0 out_seq=0 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a2cff9640 10 client.15525 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a2cff9640 1 -- 172.21.15.78:0/3173264672 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d736675b10 con 0x55d736796280 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a2cff9640 10 client.15525 handle_command_reply: tid=0 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 client.15525 shutdown 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 -- 172.21.15.78:0/3173264672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 msgr2=0x55d7367311d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d7367311d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004ec0 tx=0x7f5a3c01a7f0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 -- 172.21.15.78:0/3173264672 shutdown_connections 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d736796280 0x55d736794830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d736716fc0 0x55d73672a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a2cff9640 0 client.15525 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7367081b0 0x55d736731710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 --2- 172.21.15.78:0/3173264672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d736707890 0x55d7367311d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 -- 172.21.15.78:0/3173264672 >> 172.21.15.78:0/3173264672 conn(0x55d736703f60 msgr2=0x55d7367027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 -- 172.21.15.78:0/3173264672 shutdown_connections 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.485+0000 7f5a4d996740 1 -- 172.21.15.78:0/3173264672 wait complete. 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4d996740 20 client.15525 trim_cache size 0 max 16384 2024-09-16T00:13:57.492 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "eb73a744-73be-11ef-bceb-c7b262605968", 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 24470, 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 171.858809538, 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 31, 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 90, 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 90, 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 183.65204121299999 2024-09-16T00:13:57.493 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:13:57.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5a300a3350 msgr2=0x7f5a300a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5a300a3350 0x7f5a300a5810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005ab0 tx=0x7f5a3802a480 comp rx=0 tx=0).stop 2024-09-16T00:13:57.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 msgr2=0x7f5a441974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:13:57.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 0x7f5a441974a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5a38006010 tx=0x7f5a38003c10 comp rx=0 tx=0).stop 2024-09-16T00:13:57.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/78864644 shutdown_connections 2024-09-16T00:13:57.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5a300a3350 0x7f5a300a5810 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a44102ef0 0x7f5a4419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5a44102570 0x7f5a441979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 --2- 172.21.15.78:0/78864644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5a44100e20 0x7f5a441974a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:13:57.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/78864644 >> 172.21.15.78:0/78864644 conn(0x7f5a440fc7d0 msgr2=0x7f5a440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:13:57.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/78864644 shutdown_connections 2024-09-16T00:13:57.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:13:57.486+0000 7f5a4b5bd640 1 -- 172.21.15.78:0/78864644 wait complete. 2024-09-16T00:13:57.607 INFO:teuthology.orchestra.run.smithi078.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.72 s, remaining 1.42 s) 2024-09-16T00:13:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.87 s, remaining 1.31 s) 2024-09-16T00:13:57.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:57 smithi078 ceph-mon[31157]: pgmap v377: 129 pgs: 129 active+clean; 43 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 4.4 MiB/s wr, 88 op/s 2024-09-16T00:13:57.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:57 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:57.853 INFO:teuthology.orchestra.run.smithi078.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.97 s, remaining 1.16 s) 2024-09-16T00:13:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:57 smithi193 ceph-mon[35484]: pgmap v377: 129 pgs: 129 active+clean; 43 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 4.4 MiB/s wr, 88 op/s 2024-09-16T00:13:57.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:57 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:57 smithi148 ceph-mon[35599]: pgmap v377: 129 pgs: 129 active+clean; 43 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 4.4 MiB/s wr, 88 op/s 2024-09-16T00:13:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:57 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:13:57.971 INFO:teuthology.orchestra.run.smithi078.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.08 s, remaining 1.03 s) 2024-09-16T00:13:58.146 INFO:teuthology.orchestra.run.smithi078.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.26 s, remaining 0.91 s) 2024-09-16T00:13:58.237 INFO:teuthology.orchestra.run.smithi078.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.35 s, remaining 0.77 s) 2024-09-16T00:13:58.242 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-16T00:13:58.374 INFO:teuthology.orchestra.run.smithi078.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.49 s, remaining 0.65 s) 2024-09-16T00:13:58.522 INFO:teuthology.orchestra.run.smithi078.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.63 s, remaining 0.52 s) 2024-09-16T00:13:58.602 INFO:teuthology.orchestra.run.smithi078.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.72 s, remaining 0.38 s) 2024-09-16T00:13:58.747 INFO:teuthology.orchestra.run.smithi078.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.86 s, remaining 0.26 s) 2024-09-16T00:13:58.844 INFO:teuthology.orchestra.run.smithi078.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.96 s, remaining 0.13 s) 2024-09-16T00:13:59.013 INFO:teuthology.orchestra.run.smithi078.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.13 s, remaining 0.00 s) 2024-09-16T00:13:59.026 INFO:teuthology.orchestra.run.smithi078.stderr:vacuuming... 2024-09-16T00:13:59.642 INFO:teuthology.orchestra.run.smithi078.stderr:creating primary keys... 2024-09-16T00:13:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:13:59 smithi193 ceph-mon[35484]: pgmap v378: 129 pgs: 129 active+clean; 161 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 23 MiB/s wr, 137 op/s 2024-09-16T00:13:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:13:59 smithi148 ceph-mon[35599]: pgmap v378: 129 pgs: 129 active+clean; 161 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 23 MiB/s wr, 137 op/s 2024-09-16T00:14:00.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:13:59 smithi078 ceph-mon[31157]: pgmap v378: 129 pgs: 129 active+clean; 161 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 23 MiB/s wr, 137 op/s 2024-09-16T00:14:00.763 INFO:teuthology.orchestra.run.smithi078.stderr:done in 5.91 s (drop tables 0.00 s, create tables 0.02 s, client-side generate 4.16 s, vacuum 0.62 s, primary keys 1.12 s). 2024-09-16T00:14:00.767 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-16T00:14:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:starting vacuum...end. 2024-09-16T00:14:01.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:01 smithi193 ceph-mon[35484]: pgmap v379: 129 pgs: 129 active+clean; 245 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 37 MiB/s wr, 145 op/s 2024-09-16T00:14:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:01 smithi148 ceph-mon[35599]: pgmap v379: 129 pgs: 129 active+clean; 245 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 37 MiB/s wr, 145 op/s 2024-09-16T00:14:02.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:01 smithi078 ceph-mon[31157]: pgmap v379: 129 pgs: 129 active+clean; 245 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 37 MiB/s wr, 145 op/s 2024-09-16T00:14:03.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:03 smithi193 ceph-mon[35484]: pgmap v380: 129 pgs: 129 active+clean; 414 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 63 MiB/s wr, 159 op/s 2024-09-16T00:14:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:03 smithi148 ceph-mon[35599]: pgmap v380: 129 pgs: 129 active+clean; 414 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 63 MiB/s wr, 159 op/s 2024-09-16T00:14:04.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:03 smithi078 ceph-mon[31157]: pgmap v380: 129 pgs: 129 active+clean; 414 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 63 MiB/s wr, 159 op/s 2024-09-16T00:14:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 5.0 s, 4055.5 tps, lat 24.250 ms stddev 28.811 2024-09-16T00:14:05.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:05 smithi193 ceph-mon[35484]: pgmap v381: 129 pgs: 129 active+clean; 485 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 72 MiB/s wr, 212 op/s 2024-09-16T00:14:05.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:05 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:14:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:05 smithi148 ceph-mon[35599]: pgmap v381: 129 pgs: 129 active+clean; 485 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 72 MiB/s wr, 212 op/s 2024-09-16T00:14:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:05 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:14:06.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:05 smithi078 ceph-mon[31157]: pgmap v381: 129 pgs: 129 active+clean; 485 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 72 MiB/s wr, 212 op/s 2024-09-16T00:14:06.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:05 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:14:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:06 smithi193 ceph-mon[35484]: pgmap v382: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 76 MiB/s wr, 294 op/s 2024-09-16T00:14:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:14:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:14:07.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:06 smithi148 ceph-mon[35599]: pgmap v382: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 76 MiB/s wr, 294 op/s 2024-09-16T00:14:07.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:14:07.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:14:07.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:06 smithi078 ceph-mon[31157]: pgmap v382: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 76 MiB/s wr, 294 op/s 2024-09-16T00:14:07.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:14:07.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:14:07.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:14:07.724 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:14:08.129 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:08.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.749+0000 7feb94bc8640 1 -- 172.21.15.78:0/1624892381 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c030950 con 0x7feb901068d0 2024-09-16T00:14:08.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.749+0000 7feb96bcc640 1 -- 172.21.15.78:0/1624892381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 msgr2=0x7feb90106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.749+0000 7feb96bcc640 1 --2- 172.21.15.78:0/1624892381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 0x7feb90106cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009860 tx=0x7feb8c02a340 comp rx=0 tx=0).stop 2024-09-16T00:14:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.749+0000 7feb96bcc640 1 -- 172.21.15.78:0/1624892381 shutdown_connections 2024-09-16T00:14:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.749+0000 7feb96bcc640 1 --2- 172.21.15.78:0/1624892381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb90101270 0x7feb90069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.749+0000 7feb96bcc640 1 --2- 172.21.15.78:0/1624892381 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb901008d0 0x7feb90100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.749+0000 7feb96bcc640 1 --2- 172.21.15.78:0/1624892381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 0x7feb90106cb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.750+0000 7feb96bcc640 1 -- 172.21.15.78:0/1624892381 >> 172.21.15.78:0/1624892381 conn(0x7feb900fc610 msgr2=0x7feb900fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.750+0000 7feb96bcc640 1 -- 172.21.15.78:0/1624892381 shutdown_connections 2024-09-16T00:14:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.750+0000 7feb96bcc640 1 -- 172.21.15.78:0/1624892381 wait complete. 2024-09-16T00:14:08.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.750+0000 7feb96bcc640 1 Processor -- start 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb96bcc640 1 -- start start 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb96bcc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb901008d0 0x7feb90116150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb96bcc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb90101270 0x7feb90116690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb96bcc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 0x7feb9011be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb96bcc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010fc20 con 0x7feb90101270 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb96bcc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010faa0 con 0x7feb901068d0 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb96bcc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb9010fda0 con 0x7feb901008d0 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb963cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 0x7feb9011be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb963cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 0x7feb9011be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52374/0 (socket says 172.21.15.78:52374) 2024-09-16T00:14:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.751+0000 7feb963cb640 1 -- 172.21.15.78:0/3315989115 learned_addr learned my addr 172.21.15.78:0/3315989115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.752+0000 7feb963cb640 1 -- 172.21.15.78:0/3315989115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb901008d0 msgr2=0x7feb90116150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.752+0000 7feb963cb640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb901008d0 0x7feb90116150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.752+0000 7feb963cb640 1 -- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb90101270 msgr2=0x7feb90116690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.752+0000 7feb963cb640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb90101270 0x7feb90116690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.752+0000 7feb963cb640 1 -- 172.21.15.78:0/3315989115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb90111420 con 0x7feb901068d0 2024-09-16T00:14:08.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.752+0000 7feb963cb640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 0x7feb9011be70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feb8800efd0 tx=0x7feb8800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.753+0000 7feb82ffd640 1 -- 172.21.15.78:0/3315989115 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88003b70 con 0x7feb901068d0 2024-09-16T00:14:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.753+0000 7feb82ffd640 1 -- 172.21.15.78:0/3315989115 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb88003d10 con 0x7feb901068d0 2024-09-16T00:14:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.753+0000 7feb82ffd640 1 -- 172.21.15.78:0/3315989115 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88010990 con 0x7feb901068d0 2024-09-16T00:14:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.754+0000 7feb96bcc640 1 -- 172.21.15.78:0/3315989115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb90111710 con 0x7feb901068d0 2024-09-16T00:14:08.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.754+0000 7feb96bcc640 1 -- 172.21.15.78:0/3315989115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb90111bd0 con 0x7feb901068d0 2024-09-16T00:14:08.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.758+0000 7feb9a207740 1 Processor -- start 2024-09-16T00:14:08.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.758+0000 7feb82ffd640 1 -- 172.21.15.78:0/3315989115 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7feb88004620 con 0x7feb901068d0 2024-09-16T00:14:08.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb9a207740 1 -- start start 2024-09-16T00:14:08.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb9a207740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255ae9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb9a207740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9f90 0x557255af4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb9a207740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb9a207740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557255a4f2e0 con 0x557255af4850 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb9a207740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5572559f6ce0 con 0x557255ae9670 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb9a207740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557255a4b2a0 con 0x557255ae9f90 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb953c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255ae9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb953c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255ae9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52388/0 (socket says 172.21.15.78:52388) 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.759+0000 7feb953c9640 1 -- 172.21.15.78:0/554377655 learned_addr learned my addr 172.21.15.78:0/554377655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb953c9640 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb953c9640 1 -- 172.21.15.78:0/554377655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9f90 msgr2=0x557255af4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:08.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb95bca640 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9f90 0x557255af4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb953c9640 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9f90 0x557255af4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb953c9640 1 -- 172.21.15.78:0/554377655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 msgr2=0x557255af6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb953c9640 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb953c9640 1 -- 172.21.15.78:0/554377655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572559db440 con 0x557255ae9670 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb82ffd640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb7c0a3420 0x7feb7c0a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb95bca640 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9f90 0x557255af4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.760+0000 7feb82ffd640 1 -- 172.21.15.78:0/3315989115 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feb880d05e0 con 0x7feb901068d0 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.761+0000 7feb953c9640 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255ae9a50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb840094e0 tx=0x7feb840096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661a4cb3f8dbf902 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.761+0000 7feb95bca640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb7c0a3420 0x7feb7c0a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.761+0000 7feb7bfff640 1 -- 172.21.15.78:0/554377655 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84004050 con 0x557255ae9670 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.761+0000 7feb7bfff640 1 -- 172.21.15.78:0/554377655 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb84016030 con 0x557255ae9670 2024-09-16T00:14:08.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.762+0000 7feb7bfff640 1 -- 172.21.15.78:0/554377655 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb840116a0 con 0x557255ae9670 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.762+0000 7feb9a207740 1 -- 172.21.15.78:0/554377655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 msgr2=0x557255ae9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.762+0000 7feb9a207740 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255ae9a50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb840094e0 tx=0x7feb840096d0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.762+0000 7feb95bca640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb7c0a3420 0x7feb7c0a58e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feb8c005b60 tx=0x7feb8c005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.762+0000 7feb9a207740 1 -- 172.21.15.78:0/554377655 shutdown_connections 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.762+0000 7feb9a207740 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.763+0000 7feb9a207740 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9f90 0x557255af4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.763+0000 7feb9a207740 1 --2- 172.21.15.78:0/554377655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255ae9a50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.763+0000 7feb9a207740 1 -- 172.21.15.78:0/554377655 >> 172.21.15.78:0/554377655 conn(0x557255ae5c00 msgr2=0x557255ae5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:08.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.763+0000 7feb9a207740 1 -- 172.21.15.78:0/554377655 shutdown_connections 2024-09-16T00:14:08.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.763+0000 7feb9a207740 1 -- 172.21.15.78:0/554377655 wait complete. 2024-09-16T00:14:08.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.764+0000 7feb9a207740 1 -- start start 2024-09-16T00:14:08.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.764+0000 7feb9a207740 1 Processor -- start 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb9a207740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255af40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb9a207740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9f90 0x557255af13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb9a207740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb9a207740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557255af7470 con 0x557255af4850 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb9a207740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557255a0a050 con 0x557255ae9f90 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb9a207740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557255af75f0 con 0x557255ae9670 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb9a207740 10 client.0 _ll_register_callbacks cb 0x557255a4a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb963cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb963cb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50084/0 (socket says 172.21.15.78:50084) 2024-09-16T00:14:08.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb963cb640 1 -- 172.21.15.78:0/2293402494 learned_addr learned my addr 172.21.15.78:0/2293402494 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.765+0000 7feb95bca640 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255af40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.766+0000 7feb95bca640 1 -- 172.21.15.78:0/2293402494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9f90 msgr2=0x557255af13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.766+0000 7feb95bca640 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9f90 0x557255af13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.766+0000 7feb95bca640 1 -- 172.21.15.78:0/2293402494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 msgr2=0x557255af18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.766+0000 7feb95bca640 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.766+0000 7feb95bca640 1 -- 172.21.15.78:0/2293402494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572557c03c0 con 0x557255ae9670 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.766+0000 7feb95bca640 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255af40f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feb8c02a930 tx=0x7feb8c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c0f0059c9bfc34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.767+0000 7feb797fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.767+0000 7feb797fa640 1 -- 172.21.15.78:0/2293402494 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c005260 con 0x557255ae9670 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.767+0000 7feb797fa640 1 -- 172.21.15.78:0/2293402494 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feb8c005400 con 0x557255ae9670 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.767+0000 7feb9a207740 1 -- 172.21.15.78:0/2293402494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb84016030 con 0x557255ae9670 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.767+0000 7feb797fa640 1 -- 172.21.15.78:0/2293402494 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feb8c055050 con 0x557255ae9670 2024-09-16T00:14:08.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.767+0000 7feb9a207740 10 client.24958 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:08.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.767+0000 7feb9a207740 1 -- 172.21.15.78:0/2293402494 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557255b02d50 con 0x557255ae9670 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.768+0000 7feb797fa640 1 -- 172.21.15.78:0/2293402494 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb84016030 con 0x557255ae9670 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.768+0000 7feb797fa640 1 -- 172.21.15.78:0/2293402494 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7feb8c003990 con 0x557255ae9670 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.768+0000 7feb9a207740 10 client.24958 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.768+0000 7feb9a207740 10 client.24958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.768+0000 7feb9a207740 20 client.24958 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.768+0000 7feb9a207740 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557255b046b0 0x557255b06b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.768+0000 7feb9a207740 4 client.24958 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.768+0000 7feb9a207740 1 -- 172.21.15.78:0/2293402494 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557255a5c850 con 0x557255b046b0 2024-09-16T00:14:08.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.769+0000 7feb953c9640 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557255b046b0 0x557255b06b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.770+0000 7feb953c9640 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557255b046b0 0x557255b06b50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7161af53c66c67 server_cookie=a3850a4558b8fbdd in_seq=0 out_seq=0 2024-09-16T00:14:08.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.770+0000 7feb797fa640 10 client.24958 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:08.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.775+0000 7feb797fa640 1 -- 172.21.15.78:0/2293402494 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557255a5c850 con 0x557255b046b0 2024-09-16T00:14:08.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.775+0000 7feb797fa640 10 client.24958 handle_command_reply: tid=0 2024-09-16T00:14:08.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.776+0000 7feb9a207740 1 client.24958 shutdown 2024-09-16T00:14:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 -- 172.21.15.78:0/2293402494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 msgr2=0x557255af40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255af40f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feb8c02a930 tx=0x7feb8c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 -- 172.21.15.78:0/2293402494 shutdown_connections 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557255b046b0 0x557255b06b50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255af4850 0x557255af18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9f90 0x557255af13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 --2- 172.21.15.78:0/2293402494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255af40f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 -- 172.21.15.78:0/2293402494 >> 172.21.15.78:0/2293402494 conn(0x557255ae5c00 msgr2=0x557255ae4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb797fa640 0 client.24958 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 -- 172.21.15.78:0/2293402494 shutdown_connections 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.778+0000 7feb9a207740 1 -- 172.21.15.78:0/2293402494 wait complete. 2024-09-16T00:14:08.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.779+0000 7feb9a207740 20 client.24958 trim_cache size 0 max 16384 2024-09-16T00:14:08.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb9a207740 1 Processor -- start 2024-09-16T00:14:08.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb9a207740 1 -- start start 2024-09-16T00:14:08.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb9a207740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255b00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb9a207740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 0x557255b066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb9a207740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255af4850 0x557255aec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb9a207740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb901c8330 con 0x557255ae9f90 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb9a207740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb901c6090 con 0x557255ae9670 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb9a207740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb901c7c60 con 0x557255af4850 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb95bca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255b00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb95bca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255b00720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52398/0 (socket says 172.21.15.78:52398) 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb95bca640 1 -- 172.21.15.78:0/145347976 learned_addr learned my addr 172.21.15.78:0/145347976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb963cb640 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255af4850 0x557255aec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.817+0000 7feb953c9640 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 0x557255b066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb963cb640 1 -- 172.21.15.78:0/145347976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 msgr2=0x557255b00720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:08.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb963cb640 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255b00720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb963cb640 1 -- 172.21.15.78:0/145347976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 msgr2=0x557255b066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:08.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb963cb640 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 0x557255b066c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb963cb640 1 -- 172.21.15.78:0/145347976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557255af0350 con 0x557255af4850 2024-09-16T00:14:08.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb963cb640 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255af4850 0x557255aec9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb88009270 tx=0x7feb8809c340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2e3e814d5e1d69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:08.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb7b7fe640 1 -- 172.21.15.78:0/145347976 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880b4070 con 0x557255af4850 2024-09-16T00:14:08.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb7b7fe640 1 -- 172.21.15.78:0/145347976 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb880a53b0 con 0x557255af4850 2024-09-16T00:14:08.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.818+0000 7feb7b7fe640 1 -- 172.21.15.78:0/145347976 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004b40 con 0x557255af4850 2024-09-16T00:14:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 -- 172.21.15.78:0/145347976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255af4850 msgr2=0x557255aec9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255af4850 0x557255aec9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb88009270 tx=0x7feb8809c340 comp rx=0 tx=0).stop 2024-09-16T00:14:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 -- 172.21.15.78:0/145347976 shutdown_connections 2024-09-16T00:14:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255af4850 0x557255aec9c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 0x557255b066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 --2- 172.21.15.78:0/145347976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255ae9670 0x557255b00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 -- 172.21.15.78:0/145347976 >> 172.21.15.78:0/145347976 conn(0x557255ae5c00 msgr2=0x557255aec2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 -- 172.21.15.78:0/145347976 shutdown_connections 2024-09-16T00:14:08.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 -- 172.21.15.78:0/145347976 wait complete. 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.819+0000 7feb9a207740 1 -- start start 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.820+0000 7feb9a207740 1 Processor -- start 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.820+0000 7feb9a207740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255afc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.820+0000 7feb9a207740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 0x557255afcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.820+0000 7feb9a207740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255af4850 0x557255aed9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.820+0000 7feb9a207740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb901c6e90 con 0x557255ae9f90 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.820+0000 7feb9a207740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb901c6760 con 0x557255af4850 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb963cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255af4850 0x557255aed9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb953c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 0x557255afcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb963cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255af4850 0x557255aed9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52408/0 (socket says 172.21.15.78:52408) 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb963cb640 1 -- 172.21.15.78:0/3703690565 learned_addr learned my addr 172.21.15.78:0/3703690565 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb95bca640 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255afc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb901c7560 con 0x557255ae9670 2024-09-16T00:14:08.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb9a207740 10 client.0 _ll_register_callbacks cb 0x7feb8c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb95bca640 1 -- 172.21.15.78:0/3703690565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255af4850 msgr2=0x557255aed9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb95bca640 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255af4850 0x557255aed9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb95bca640 1 -- 172.21.15.78:0/3703690565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 msgr2=0x557255afcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb95bca640 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 0x557255afcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb95bca640 1 -- 172.21.15.78:0/3703690565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557255aedf10 con 0x557255ae9670 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb963cb640 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255af4850 0x557255aed9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb95bca640 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255afc890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb8c004010 tx=0x7feb8c005490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31bd97c8e57a716c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.821+0000 7feb797fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb797fa640 1 -- 172.21.15.78:0/3703690565 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c005600 con 0x557255ae9670 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb880a53b0 con 0x557255ae9670 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb797fa640 1 -- 172.21.15.78:0/3703690565 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feb8c0035f0 con 0x557255ae9670 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb797fa640 1 -- 172.21.15.78:0/3703690565 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feb8c06b050 con 0x557255ae9670 2024-09-16T00:14:08.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb9a207740 10 client.24964 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557255b6eb80 con 0x557255ae9670 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb797fa640 1 -- 172.21.15.78:0/3703690565 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb880a53b0 con 0x557255ae9670 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb797fa640 1 -- 172.21.15.78:0/3703690565 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7feb8c0046c0 con 0x557255ae9670 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb9a207740 10 client.24964 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb9a207740 10 client.24964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb9a207740 20 client.24964 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.822+0000 7feb9a207740 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557255b6f2d0 0x557255b716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.823+0000 7feb9a207740 4 client.24964 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:14:08.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.823+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557255a56d60 con 0x557255b6f2d0 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.823+0000 7feb953c9640 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557255b6f2d0 0x557255b716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.823+0000 7feb797fa640 10 client.24964 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.823+0000 7feb953c9640 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557255b6f2d0 0x557255b716f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb24e7b9fb42fd83 server_cookie=746c309a04269a2e in_seq=0 out_seq=0 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.823+0000 7feb797fa640 1 -- 172.21.15.78:0/3703690565 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557255a56d60 con 0x557255b6f2d0 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.823+0000 7feb797fa640 10 client.24964 handle_command_reply: tid=0 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.823+0000 7feb9a207740 1 client.24964 shutdown 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 msgr2=0x557255afc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255afc890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb8c004010 tx=0x7feb8c005490 comp rx=0 tx=0).stop 2024-09-16T00:14:08.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 shutdown_connections 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557255b6f2d0 0x557255b716f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557255af4850 0x557255aed9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb797fa640 0 client.24964 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557255ae9f90 0x557255afcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 --2- 172.21.15.78:0/3703690565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557255ae9670 0x557255afc890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 >> 172.21.15.78:0/3703690565 conn(0x557255ae5c00 msgr2=0x557255ae4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 shutdown_connections 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 1 -- 172.21.15.78:0/3703690565 wait complete. 2024-09-16T00:14:08.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.824+0000 7feb9a207740 20 client.24964 trim_cache size 0 max 16384 2024-09-16T00:14:08.834 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:14:08.834 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:14:08.834 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:14:08.834 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 -- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb7c0a3420 msgr2=0x7feb7c0a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb7c0a3420 0x7feb7c0a58e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feb8c005b60 tx=0x7feb8c005a90 comp rx=0 tx=0).stop 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 -- 172.21.15.78:0/3315989115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 msgr2=0x7feb9011be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 0x7feb9011be70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feb8800efd0 tx=0x7feb8800c6a0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 -- 172.21.15.78:0/3315989115 shutdown_connections 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb7c0a3420 0x7feb7c0a58e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb901068d0 0x7feb9011be70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb90101270 0x7feb90116690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 --2- 172.21.15.78:0/3315989115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb901008d0 0x7feb90116150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 -- 172.21.15.78:0/3315989115 >> 172.21.15.78:0/3315989115 conn(0x7feb900fc610 msgr2=0x7feb90107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 -- 172.21.15.78:0/3315989115 shutdown_connections 2024-09-16T00:14:08.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:08.825+0000 7feb96bcc640 1 -- 172.21.15.78:0/3315989115 wait complete. 2024-09-16T00:14:09.646 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:09 smithi078 ceph-mon[31157]: pgmap v383: 129 pgs: 129 active+clean; 884 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 75 MiB/s wr, 372 op/s 2024-09-16T00:14:09.694 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:14:09.695 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:14:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:09 smithi193 ceph-mon[35484]: pgmap v383: 129 pgs: 129 active+clean; 884 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 75 MiB/s wr, 372 op/s 2024-09-16T00:14:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:09 smithi148 ceph-mon[35599]: pgmap v383: 129 pgs: 129 active+clean; 884 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 75 MiB/s wr, 372 op/s 2024-09-16T00:14:10.124 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:10.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 -- 172.21.15.78:0/2467789039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc980108c30 msgr2=0x7fc98010cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 --2- 172.21.15.78:0/2467789039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc980108c30 0x7fc98010cc70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc970009900 tx=0x7fc97002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 -- 172.21.15.78:0/2467789039 shutdown_connections 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 --2- 172.21.15.78:0/2467789039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc98010d3a0 0x7fc980114f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 --2- 172.21.15.78:0/2467789039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc980108c30 0x7fc98010cc70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 --2- 172.21.15.78:0/2467789039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc980108280 0x7fc980108660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 -- 172.21.15.78:0/2467789039 >> 172.21.15.78:0/2467789039 conn(0x7fc980077a00 msgr2=0x7fc980079e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 -- 172.21.15.78:0/2467789039 shutdown_connections 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.799+0000 7fc98825c640 1 -- 172.21.15.78:0/2467789039 wait complete. 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.800+0000 7fc98825c640 1 Processor -- start 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.800+0000 7fc98825c640 1 -- start start 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc98825c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc980108280 0x7fc980073970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:10.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc98825c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc980108c30 0x7fc980073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:10.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc98825c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc98010d3a0 0x7fc98006cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:10.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc98825c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc9801183f0 con 0x7fc980108c30 2024-09-16T00:14:10.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc98825c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc980118270 con 0x7fc98010d3a0 2024-09-16T00:14:10.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc98825c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc980118570 con 0x7fc980108280 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc9867f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc98010d3a0 0x7fc98006cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc9867f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc98010d3a0 0x7fc98006cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52428/0 (socket says 172.21.15.78:52428) 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc9867f9640 1 -- 172.21.15.78:0/3031722452 learned_addr learned my addr 172.21.15.78:0/3031722452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.801+0000 7fc9857f7640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc980108c30 0x7fc980073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc9867f9640 1 -- 172.21.15.78:0/3031722452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc980108280 msgr2=0x7fc980073970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc9867f9640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc980108280 0x7fc980073970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc9867f9640 1 -- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc980108c30 msgr2=0x7fc980073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc9867f9640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc980108c30 0x7fc980073eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc9867f9640 1 -- 172.21.15.78:0/3031722452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98006d470 con 0x7fc98010d3a0 2024-09-16T00:14:10.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc9867f9640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc98010d3a0 0x7fc98006cbb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc97800ea60 tx=0x7fc97800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:10.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc976ffd640 1 -- 172.21.15.78:0/3031722452 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97800ce40 con 0x7fc98010d3a0 2024-09-16T00:14:10.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc976ffd640 1 -- 172.21.15.78:0/3031722452 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc978005020 con 0x7fc98010d3a0 2024-09-16T00:14:10.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.802+0000 7fc976ffd640 1 -- 172.21.15.78:0/3031722452 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978010690 con 0x7fc98010d3a0 2024-09-16T00:14:10.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.803+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc98006d760 con 0x7fc98010d3a0 2024-09-16T00:14:10.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.803+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9800716d0 con 0x7fc98010d3a0 2024-09-16T00:14:10.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.805+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc980109750 con 0x7fc98010d3a0 2024-09-16T00:14:10.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.806+0000 7fc976ffd640 1 -- 172.21.15.78:0/3031722452 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fc9780051d0 con 0x7fc98010d3a0 2024-09-16T00:14:10.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.807+0000 7fc976ffd640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc96c0a3420 0x7fc96c0a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:10.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.807+0000 7fc976ffd640 1 -- 172.21.15.78:0/3031722452 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc9780d0350 con 0x7fc98010d3a0 2024-09-16T00:14:10.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.807+0000 7fc985ff8640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc96c0a3420 0x7fc96c0a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:10.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.808+0000 7fc985ff8640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc96c0a3420 0x7fc96c0a58e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00b990 tx=0x7fc97c00b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:10.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:10.812+0000 7fc976ffd640 1 -- 172.21.15.78:0/3031722452 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc97809cbb0 con 0x7fc98010d3a0 2024-09-16T00:14:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 10.0 s, 3015.0 tps, lat 33.168 ms stddev 37.476 2024-09-16T00:14:11.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.179+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc980072160 con 0x7fc98010d3a0 2024-09-16T00:14:11.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.180+0000 7fc976ffd640 1 -- 172.21.15.78:0/3031722452 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7fc97809c4b0 con 0x7fc98010d3a0 2024-09-16T00:14:11.181 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:14:11.181 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:14:11.181 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc96c0a3420 msgr2=0x7fc96c0a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc96c0a3420 0x7fc96c0a58e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00b990 tx=0x7fc97c00b370 comp rx=0 tx=0).stop 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc98010d3a0 msgr2=0x7fc98006cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc98010d3a0 0x7fc98006cbb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc97800ea60 tx=0x7fc97800ef20 comp rx=0 tx=0).stop 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 shutdown_connections 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc96c0a3420 0x7fc96c0a58e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc98010d3a0 0x7fc98006cbb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc980108c30 0x7fc980073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 --2- 172.21.15.78:0/3031722452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc980108280 0x7fc980073970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.183+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 >> 172.21.15.78:0/3031722452 conn(0x7fc980077a00 msgr2=0x7fc980078d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.184+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 shutdown_connections 2024-09-16T00:14:11.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:11.184+0000 7fc98825c640 1 -- 172.21.15.78:0/3031722452 wait complete. 2024-09-16T00:14:11.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:11 smithi078 ceph-mon[31157]: pgmap v384: 129 pgs: 129 active+clean; 896 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 56 MiB/s wr, 359 op/s 2024-09-16T00:14:11.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:11 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3031722452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:11.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:11 smithi193 ceph-mon[35484]: pgmap v384: 129 pgs: 129 active+clean; 896 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 56 MiB/s wr, 359 op/s 2024-09-16T00:14:11.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:11 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3031722452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:11 smithi148 ceph-mon[35599]: pgmap v384: 129 pgs: 129 active+clean; 896 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 56 MiB/s wr, 359 op/s 2024-09-16T00:14:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:11 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3031722452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:13.243 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.2 2024-09-16T00:14:13.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:14:13.583 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:13 smithi078 ceph-mon[31157]: pgmap v385: 129 pgs: 129 active+clean; 921 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 45 MiB/s wr, 524 op/s 2024-09-16T00:14:13.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:13 smithi193 ceph-mon[35484]: pgmap v385: 129 pgs: 129 active+clean; 921 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 45 MiB/s wr, 524 op/s 2024-09-16T00:14:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:13 smithi148 ceph-mon[35599]: pgmap v385: 129 pgs: 129 active+clean; 921 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 45 MiB/s wr, 524 op/s 2024-09-16T00:14:15.777 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:15.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:14:15.780 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:15 smithi078 ceph-mon[31157]: pgmap v386: 129 pgs: 129 active+clean; 929 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 578 op/s 2024-09-16T00:14:15.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 15.0 s, 3387.8 tps, lat 29.471 ms stddev 40.130 2024-09-16T00:14:15.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:15 smithi193 ceph-mon[35484]: pgmap v386: 129 pgs: 129 active+clean; 929 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 578 op/s 2024-09-16T00:14:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:15 smithi148 ceph-mon[35599]: pgmap v386: 129 pgs: 129 active+clean; 929 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 578 op/s 2024-09-16T00:14:17.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.456+0000 7f127a5ea640 1 -- 172.21.15.78:0/100951879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1274110190 msgr2=0x7f127406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:17.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.456+0000 7f127a5ea640 1 --2- 172.21.15.78:0/100951879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1274110190 0x7f127406e4f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1260009900 tx=0x7f126002a410 comp rx=0 tx=0).stop 2024-09-16T00:14:17.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.457+0000 7f127a5ea640 1 -- 172.21.15.78:0/100951879 shutdown_connections 2024-09-16T00:14:17.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.457+0000 7f127a5ea640 1 --2- 172.21.15.78:0/100951879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f127411b140 0x7f127411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.457+0000 7f127a5ea640 1 --2- 172.21.15.78:0/100951879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f127406ea30 0x7f127406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.457+0000 7f127a5ea640 1 --2- 172.21.15.78:0/100951879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1274110190 0x7f127406e4f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.457+0000 7f127a5ea640 1 -- 172.21.15.78:0/100951879 >> 172.21.15.78:0/100951879 conn(0x7f1274073180 msgr2=0x7f1274073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:17.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.457+0000 7f127a5ea640 1 -- 172.21.15.78:0/100951879 shutdown_connections 2024-09-16T00:14:17.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.457+0000 7f127a5ea640 1 -- 172.21.15.78:0/100951879 wait complete. 2024-09-16T00:14:17.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.458+0000 7f127a5ea640 1 Processor -- start 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.458+0000 7f127a5ea640 1 -- start start 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.458+0000 7f127a5ea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f127406ea30 0x7f127411a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.458+0000 7f127a5ea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1274110190 0x7f12741137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.458+0000 7f127a5ea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f127411b140 0x7f1274113d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.458+0000 7f127a5ea640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1274120c60 con 0x7f1274110190 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.459+0000 7f127a5ea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1274120ae0 con 0x7f127406ea30 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.459+0000 7f127a5ea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1274120de0 con 0x7f127411b140 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.459+0000 7f1278b87640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f127411b140 0x7f1274113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.459+0000 7f12737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1274110190 0x7f12741137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.459+0000 7f12737fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1274110190 0x7f12741137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47652/0 (socket says 172.21.15.78:47652) 2024-09-16T00:14:17.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.459+0000 7f12737fe640 1 -- 172.21.15.78:0/329918655 learned_addr learned my addr 172.21.15.78:0/329918655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:17.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.459+0000 7f1273fff640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f127406ea30 0x7f127411a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:17.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.460+0000 7f1273fff640 1 -- 172.21.15.78:0/329918655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f127411b140 msgr2=0x7f1274113d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.460+0000 7f1273fff640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f127411b140 0x7f1274113d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.460+0000 7f1273fff640 1 -- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1274110190 msgr2=0x7f12741137e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.460+0000 7f1273fff640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1274110190 0x7f12741137e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.460+0000 7f1273fff640 1 -- 172.21.15.78:0/329918655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1274114310 con 0x7f127406ea30 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.460+0000 7f1278b87640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f127411b140 0x7f1274113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.461+0000 7f1273fff640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f127406ea30 0x7f127411a6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f126002a930 tx=0x7f1260003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.461+0000 7f12717fa640 1 -- 172.21.15.78:0/329918655 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126003f070 con 0x7f127406ea30 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.461+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12741145a0 con 0x7f127406ea30 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.461+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1274118510 con 0x7f127406ea30 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.461+0000 7f12717fa640 1 -- 172.21.15.78:0/329918655 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1260031c90 con 0x7f127406ea30 2024-09-16T00:14:17.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.462+0000 7f12717fa640 1 -- 172.21.15.78:0/329918655 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260035030 con 0x7f127406ea30 2024-09-16T00:14:17.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.464+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1258005290 con 0x7f127406ea30 2024-09-16T00:14:17.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.466+0000 7f12717fa640 1 -- 172.21.15.78:0/329918655 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f1260042070 con 0x7f127406ea30 2024-09-16T00:14:17.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.467+0000 7f12717fa640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12640a34a0 0x7f12640a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:17.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.467+0000 7f12737fe640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12640a34a0 0x7f12640a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:17.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.468+0000 7f12737fe640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12640a34a0 0x7f12640a5960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f126c000be0 tx=0x7f126c005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:17.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.468+0000 7f12717fa640 1 -- 172.21.15.78:0/329918655 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f12600f0360 con 0x7f127406ea30 2024-09-16T00:14:17.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.471+0000 7f12717fa640 1 -- 172.21.15.78:0/329918655 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1260049030 con 0x7f127406ea30 2024-09-16T00:14:17.526 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:17.659 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:17 smithi078 ceph-mon[31157]: pgmap v387: 129 pgs: 129 active+clean; 949 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-09-16T00:14:17.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.862+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1258005850 con 0x7f127406ea30 2024-09-16T00:14:17.869 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:14:17.869 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.863+0000 7f12717fa640 1 -- 172.21.15.78:0/329918655 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7f1260043e90 con 0x7f127406ea30 2024-09-16T00:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.866+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12640a34a0 msgr2=0x7f12640a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.866+0000 7f127a5ea640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12640a34a0 0x7f12640a5960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f126c000be0 tx=0x7f126c005040 comp rx=0 tx=0).stop 2024-09-16T00:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.866+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f127406ea30 msgr2=0x7f127411a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f127406ea30 0x7f127411a6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f126002a930 tx=0x7f1260003b90 comp rx=0 tx=0).stop 2024-09-16T00:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 shutdown_connections 2024-09-16T00:14:17.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12640a34a0 0x7f12640a5960 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f127411b140 0x7f1274113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1274110190 0x7f12741137e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 --2- 172.21.15.78:0/329918655 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f127406ea30 0x7f127411a6e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 >> 172.21.15.78:0/329918655 conn(0x7f1274073180 msgr2=0x7f127411c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 shutdown_connections 2024-09-16T00:14:17.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:17.867+0000 7f127a5ea640 1 -- 172.21.15.78:0/329918655 wait complete. 2024-09-16T00:14:17.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:17 smithi193 ceph-mon[35484]: pgmap v387: 129 pgs: 129 active+clean; 949 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-09-16T00:14:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:17 smithi148 ceph-mon[35599]: pgmap v387: 129 pgs: 129 active+clean; 949 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 636 op/s 2024-09-16T00:14:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.254+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1363723101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e6010bfd0 msgr2=0x7f5e6010e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.254+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1363723101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e6010bfd0 0x7f5e6010e460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009be0 tx=0x7f5e5802e8f0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.254+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1363723101 shutdown_connections 2024-09-16T00:14:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.254+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1363723101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e6010bfd0 0x7f5e6010e460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.254+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1363723101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e60103550 0x7f5e6010ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.254+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1363723101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e60102c30 0x7f5e60103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.254+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1363723101 >> 172.21.15.78:0/1363723101 conn(0x7f5e600fc610 msgr2=0x7f5e600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.255+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1363723101 shutdown_connections 2024-09-16T00:14:18.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.255+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1363723101 wait complete. 2024-09-16T00:14:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.256+0000 7f5e5ffff640 1 Processor -- start 2024-09-16T00:14:18.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5ffff640 1 -- start start 2024-09-16T00:14:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e60102c30 0x7f5e60195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e60103550 0x7f5e60196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6010bfd0 0x7f5e6019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5ffff640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010fc20 con 0x7f5e60103550 2024-09-16T00:14:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5ffff640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010faa0 con 0x7f5e6010bfd0 2024-09-16T00:14:18.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5ffff640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6010fda0 con 0x7f5e60102c30 2024-09-16T00:14:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e60102c30 0x7f5e60195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e60102c30 0x7f5e60195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:54990/0 (socket says 172.21.15.78:54990) 2024-09-16T00:14:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5effd640 1 -- 172.21.15.78:0/1690615632 learned_addr learned my addr 172.21.15.78:0/1690615632 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6010bfd0 0x7f5e6019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.257+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e60103550 0x7f5e60196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.258+0000 7f5e5effd640 1 -- 172.21.15.78:0/1690615632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6010bfd0 msgr2=0x7f5e6019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.258+0000 7f5e5effd640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6010bfd0 0x7f5e6019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.258+0000 7f5e5effd640 1 -- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e60103550 msgr2=0x7f5e60196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.258+0000 7f5e5effd640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e60103550 0x7f5e60196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.258+0000 7f5e5effd640 1 -- 172.21.15.78:0/1690615632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6019adf0 con 0x7f5e60102c30 2024-09-16T00:14:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.259+0000 7f5e5effd640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e60102c30 0x7f5e60195e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000e9f0 tx=0x7f5e5000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.259+0000 7f5e47fff640 1 -- 172.21.15.78:0/1690615632 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000cd80 con 0x7f5e60102c30 2024-09-16T00:14:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.259+0000 7f5e47fff640 1 -- 172.21.15.78:0/1690615632 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e50010910 con 0x7f5e60102c30 2024-09-16T00:14:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.259+0000 7f5e47fff640 1 -- 172.21.15.78:0/1690615632 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50018980 con 0x7f5e60102c30 2024-09-16T00:14:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.259+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1690615632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6019b0e0 con 0x7f5e60102c30 2024-09-16T00:14:18.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.259+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1690615632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6019b510 con 0x7f5e60102c30 2024-09-16T00:14:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.263+0000 7f5e67667740 1 Processor -- start 2024-09-16T00:14:18.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.264+0000 7f5e47fff640 1 -- 172.21.15.78:0/1690615632 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f5e50010ac0 con 0x7f5e60102c30 2024-09-16T00:14:18.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.264+0000 7f5e67667740 1 -- start start 2024-09-16T00:14:18.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.264+0000 7f5e67667740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece871270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.264+0000 7f5e67667740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 0x559ece871bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.264+0000 7f5e67667740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece87e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e67667740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ece7d72f0 con 0x559ece872d20 2024-09-16T00:14:18.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e67667740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559ece77ecf0 con 0x559ece87c550 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e67667740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559ece7d32b0 con 0x559ece8717b0 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece871270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 0x559ece871bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece871270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47674/0 (socket says 172.21.15.78:47674) 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/3848107479 learned_addr learned my addr 172.21.15.78:0/3848107479 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece87e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/3848107479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 msgr2=0x559ece871bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 0x559ece871bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/3848107479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 msgr2=0x559ece87e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece87e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/3848107479 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ece7633a0 con 0x559ece872d20 2024-09-16T00:14:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece87e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.265+0000 7f5e47fff640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5e480a33d0 0x7f5e480a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.266+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece871270 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5e58038000 tx=0x7f5e5800a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986bf2e64c829205 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:18.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.266+0000 7f5e47fff640 1 -- 172.21.15.78:0/1690615632 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5e500d00e0 con 0x7f5e60102c30 2024-09-16T00:14:18.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.266+0000 7f5e5effd640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5e480a33d0 0x7f5e480a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.266+0000 7f5e457fa640 1 -- 172.21.15.78:0/3848107479 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580347e0 con 0x559ece872d20 2024-09-16T00:14:18.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.267+0000 7f5e5effd640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5e480a33d0 0x7f5e480a5890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000c6a0 tx=0x7f5e5000cf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:18.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.267+0000 7f5e457fa640 1 -- 172.21.15.78:0/3848107479 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58003970 con 0x559ece872d20 2024-09-16T00:14:18.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.267+0000 7f5e457fa640 1 -- 172.21.15.78:0/3848107479 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580044c0 con 0x559ece872d20 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 -- 172.21.15.78:0/3848107479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 msgr2=0x559ece871270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece871270 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5e58038000 tx=0x7f5e5800a0f0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 -- 172.21.15.78:0/3848107479 shutdown_connections 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece87e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 0x559ece871bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 --2- 172.21.15.78:0/3848107479 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece871270 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 -- 172.21.15.78:0/3848107479 >> 172.21.15.78:0/3848107479 conn(0x559ece83a310 msgr2=0x559ece86de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 -- 172.21.15.78:0/3848107479 shutdown_connections 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.268+0000 7f5e67667740 1 -- 172.21.15.78:0/3848107479 wait complete. 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 1 -- start start 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 1 Processor -- start 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece8717b0 0x559ece87bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece872d20 0x559ece8791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece8797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e5f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece8717b0 0x559ece87bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e5f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece8717b0 0x559ece87bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47690/0 (socket says 172.21.15.78:47690) 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559ece87f240 con 0x559ece8717b0 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559ece792060 con 0x559ece87c550 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/1000876597 learned_addr learned my addr 172.21.15.78:0/1000876597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559ece87f3c0 con 0x559ece872d20 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.269+0000 7f5e67667740 10 client.0 _ll_register_callbacks cb 0x559ece4e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.270+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece8797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.270+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece872d20 0x559ece8791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.270+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/1000876597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece872d20 msgr2=0x559ece8791c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.270+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece872d20 0x559ece8791c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.270+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/1000876597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 msgr2=0x559ece8797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.270+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece8797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.270+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/1000876597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ece5483c0 con 0x559ece8717b0 2024-09-16T00:14:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.271+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece8717b0 0x559ece87bf10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009be0 tx=0x7f5e58004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f50b397193c3b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:18.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.271+0000 7f5e42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.271+0000 7f5e42ff5640 1 -- 172.21.15.78:0/1000876597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58004240 con 0x559ece8717b0 2024-09-16T00:14:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.271+0000 7f5e42ff5640 1 -- 172.21.15.78:0/1000876597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5e580043e0 con 0x559ece8717b0 2024-09-16T00:14:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.272+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e58003970 con 0x559ece8717b0 2024-09-16T00:14:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.272+0000 7f5e42ff5640 1 -- 172.21.15.78:0/1000876597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e580045e0 con 0x559ece8717b0 2024-09-16T00:14:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.272+0000 7f5e67667740 10 client.15561 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.272+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559ece88ac10 con 0x559ece8717b0 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.273+0000 7f5e42ff5640 1 -- 172.21.15.78:0/1000876597 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58003970 con 0x559ece8717b0 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.273+0000 7f5e42ff5640 1 -- 172.21.15.78:0/1000876597 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5e58035480 con 0x559ece8717b0 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.273+0000 7f5e67667740 10 client.15561 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.273+0000 7f5e67667740 10 client.15561 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.273+0000 7f5e67667740 20 client.15561 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.273+0000 7f5e67667740 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559ece88bf60 0x559ece88e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.273+0000 7f5e67667740 4 client.15561 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.273+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ece7e4860 con 0x559ece88bf60 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.274+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559ece88bf60 0x559ece88e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.275+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559ece88bf60 0x559ece88e350 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8caa6ecfdffad3f5 server_cookie=4ec0425da58a5ee7 in_seq=0 out_seq=0 2024-09-16T00:14:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.275+0000 7f5e42ff5640 10 client.15561 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.275+0000 7f5e42ff5640 1 -- 172.21.15.78:0/1000876597 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ece7e4860 con 0x559ece88bf60 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.275+0000 7f5e42ff5640 10 client.15561 handle_command_reply: tid=0 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.275+0000 7f5e67667740 1 client.15561 shutdown 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece8717b0 msgr2=0x559ece87bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece8717b0 0x559ece87bf10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009be0 tx=0x7f5e58004160 comp rx=0 tx=0).stop 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 shutdown_connections 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece8797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559ece88bf60 0x559ece88e350 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece872d20 0x559ece8791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 --2- 172.21.15.78:0/1000876597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece8717b0 0x559ece87bf10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 >> 172.21.15.78:0/1000876597 conn(0x559ece83a310 msgr2=0x559ece873b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e42ff5640 0 client.15561 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:14:18.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 shutdown_connections 2024-09-16T00:14:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.276+0000 7f5e67667740 1 -- 172.21.15.78:0/1000876597 wait complete. 2024-09-16T00:14:18.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.277+0000 7f5e67667740 20 client.15561 trim_cache size 0 max 16384 2024-09-16T00:14:18.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.299+0000 7f5e67667740 1 Processor -- start 2024-09-16T00:14:18.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e67667740 1 -- start start 2024-09-16T00:14:18.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e67667740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece8717b0 0x559ece878a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e67667740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece87b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e67667740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece87c550 0x559ece874300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e67667740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601ac840 con 0x559ece872d20 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e67667740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601ae410 con 0x559ece8717b0 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e67667740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601acf10 con 0x559ece87c550 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e5f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece8717b0 0x559ece878a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e5f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece87c550 0x559ece874300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e5f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece8717b0 0x559ece878a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46580/0 (socket says 172.21.15.78:46580) 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.300+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/1617496567 learned_addr learned my addr 172.21.15.78:0/1617496567 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/1617496567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece87b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/1617496567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece87c550 msgr2=0x559ece874300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/1617496567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece87c550 0x559ece874300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/1617496567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 msgr2=0x559ece87b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/1617496567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece87b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e5f7fe640 1 -- 172.21.15.78:0/1617496567 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ece87c280 con 0x559ece8717b0 2024-09-16T00:14:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/1617496567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece8717b0 0x559ece878a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e580098b0 tx=0x7f5e5802e8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc9d71d056c40a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e44ff9640 1 -- 172.21.15.78:0/1617496567 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58034e30 con 0x559ece8717b0 2024-09-16T00:14:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e44ff9640 1 -- 172.21.15.78:0/1617496567 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5800e070 con 0x559ece8717b0 2024-09-16T00:14:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.301+0000 7f5e44ff9640 1 -- 172.21.15.78:0/1617496567 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800f4d0 con 0x559ece8717b0 2024-09-16T00:14:18.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 -- 172.21.15.78:0/1617496567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece8717b0 msgr2=0x559ece878a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 --2- 172.21.15.78:0/1617496567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece8717b0 0x559ece878a80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e580098b0 tx=0x7f5e5802e8f0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 -- 172.21.15.78:0/1617496567 shutdown_connections 2024-09-16T00:14:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 --2- 172.21.15.78:0/1617496567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece87c550 0x559ece874300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 --2- 172.21.15.78:0/1617496567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece87b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 --2- 172.21.15.78:0/1617496567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece8717b0 0x559ece878a80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 -- 172.21.15.78:0/1617496567 >> 172.21.15.78:0/1617496567 conn(0x559ece83a310 msgr2=0x559ece87aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 -- 172.21.15.78:0/1617496567 shutdown_connections 2024-09-16T00:14:18.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 -- 172.21.15.78:0/1617496567 wait complete. 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.303+0000 7f5e67667740 1 -- start start 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e67667740 1 Processor -- start 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e67667740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 0x559ece88f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e67667740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece88f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e67667740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece88faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e67667740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601add10 con 0x559ece872d20 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e67667740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601ad640 con 0x559ece87c550 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e67667740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601aa820 con 0x559ece8717b0 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e67667740 10 client.0 _ll_register_callbacks cb 0x7f5e5800a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e5e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece88f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e5e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece88f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47698/0 (socket says 172.21.15.78:47698) 2024-09-16T00:14:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.304+0000 7f5e5e7fc640 1 -- 172.21.15.78:0/694314356 learned_addr learned my addr 172.21.15.78:0/694314356 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.305+0000 7f5e5e7fc640 1 -- 172.21.15.78:0/694314356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 msgr2=0x559ece88f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.305+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 0x559ece88f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.305+0000 7f5e5e7fc640 1 -- 172.21.15.78:0/694314356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 msgr2=0x559ece88faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.305+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece88faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.305+0000 7f5e5e7fc640 1 -- 172.21.15.78:0/694314356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ece8f58a0 con 0x559ece872d20 2024-09-16T00:14:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.305+0000 7f5e5e7fc640 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece88f560 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x559ece878a50 tx=0x7f5e5400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaf1bb079cbc0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.306+0000 7f5e42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.306+0000 7f5e42ff5640 1 -- 172.21.15.78:0/694314356 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54023470 con 0x559ece872d20 2024-09-16T00:14:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.306+0000 7f5e67667740 1 -- 172.21.15.78:0/694314356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e5800e070 con 0x559ece872d20 2024-09-16T00:14:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.307+0000 7f5e42ff5640 1 -- 172.21.15.78:0/694314356 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5e54023610 con 0x559ece872d20 2024-09-16T00:14:18.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.307+0000 7f5e42ff5640 1 -- 172.21.15.78:0/694314356 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5e5402c050 con 0x559ece872d20 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.307+0000 7f5e67667740 10 client.15567 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.307+0000 7f5e67667740 1 -- 172.21.15.78:0/694314356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559ece8f6c50 con 0x559ece872d20 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.307+0000 7f5e42ff5640 1 -- 172.21.15.78:0/694314356 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e5800e070 con 0x559ece872d20 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.308+0000 7f5e42ff5640 1 -- 172.21.15.78:0/694314356 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5e540118b0 con 0x559ece872d20 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.308+0000 7f5e67667740 10 client.15567 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.308+0000 7f5e67667740 10 client.15567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.308+0000 7f5e67667740 20 client.15567 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.308+0000 7f5e67667740 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559ece8f7340 0x559ece8f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.308+0000 7f5e67667740 4 client.15567 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.308+0000 7f5e67667740 1 -- 172.21.15.78:0/694314356 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ece7ded70 con 0x559ece8f7340 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.308+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559ece8f7340 0x559ece8f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:18.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.309+0000 7f5e42ff5640 10 client.15567 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.309+0000 7f5e5f7fe640 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559ece8f7340 0x559ece8f9760 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49ba41f19f96f688 server_cookie=eb561feb193e3e76 in_seq=0 out_seq=0 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.309+0000 7f5e42ff5640 1 -- 172.21.15.78:0/694314356 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ece7ded70 con 0x559ece8f7340 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.309+0000 7f5e42ff5640 10 client.15567 handle_command_reply: tid=0 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.309+0000 7f5e67667740 1 client.15567 shutdown 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.309+0000 7f5e67667740 1 -- 172.21.15.78:0/694314356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 msgr2=0x559ece88f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece88f560 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x559ece878a50 tx=0x7f5e5400bde0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 -- 172.21.15.78:0/694314356 shutdown_connections 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559ece87c550 0x559ece88faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559ece872d20 0x559ece88f560 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559ece8f7340 0x559ece8f9760 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 --2- 172.21.15.78:0/694314356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559ece8717b0 0x559ece88f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e42ff5640 0 client.15567 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:14:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 -- 172.21.15.78:0/694314356 >> 172.21.15.78:0/694314356 conn(0x559ece83a310 msgr2=0x559ece86cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 -- 172.21.15.78:0/694314356 shutdown_connections 2024-09-16T00:14:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 1 -- 172.21.15.78:0/694314356 wait complete. 2024-09-16T00:14:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.310+0000 7f5e67667740 20 client.15567 trim_cache size 0 max 16384 2024-09-16T00:14:18.314 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:14:18.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5e480a33d0 msgr2=0x7f5e480a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5e480a33d0 0x7f5e480a5890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000c6a0 tx=0x7f5e5000cf20 comp rx=0 tx=0).stop 2024-09-16T00:14:18.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1690615632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e60102c30 msgr2=0x7f5e60195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:18.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e60102c30 0x7f5e60195e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000e9f0 tx=0x7f5e5000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1690615632 shutdown_connections 2024-09-16T00:14:18.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5e480a33d0 0x7f5e480a5890 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e6010bfd0 0x7f5e6019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e60103550 0x7f5e60196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 --2- 172.21.15.78:0/1690615632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e60102c30 0x7f5e60195e40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:18.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.311+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1690615632 >> 172.21.15.78:0/1690615632 conn(0x7f5e600fc610 msgr2=0x7f5e6010d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:18.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.312+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1690615632 shutdown_connections 2024-09-16T00:14:18.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:18.312+0000 7f5e5ffff640 1 -- 172.21.15.78:0/1690615632 wait complete. 2024-09-16T00:14:18.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:18 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/329918655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:18.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:18 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/329918655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:19.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:18 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/329918655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:19.146 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:14:19.609 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0' 2024-09-16T00:14:19.610 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0 2024-09-16T00:14:19.707 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:19.847 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:19 smithi078 ceph-mon[31157]: pgmap v388: 129 pgs: 129 active+clean; 974 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 854 op/s 2024-09-16T00:14:19.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:19 smithi193 ceph-mon[35484]: pgmap v388: 129 pgs: 129 active+clean; 974 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 854 op/s 2024-09-16T00:14:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:19 smithi148 ceph-mon[35599]: pgmap v388: 129 pgs: 129 active+clean; 974 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 MiB/s wr, 854 op/s 2024-09-16T00:14:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 20.0 s, 3295.2 tps, lat 30.370 ms stddev 34.311 2024-09-16T00:14:20.940 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:14:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.295+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2850403742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc24100a80 msgr2=0x7fdc24100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.295+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2850403742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc24100a80 0x7fdc24100ee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009900 tx=0x7fdc1002a3f0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.297+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2850403742 shutdown_connections 2024-09-16T00:14:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.297+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2850403742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc24101420 0x7fdc2410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.297+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2850403742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc24100a80 0x7fdc24100ee0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.297+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2850403742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc24106a80 0x7fdc24106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.297+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2850403742 >> 172.21.15.78:0/2850403742 conn(0x7fdc240fc7a0 msgr2=0x7fdc240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.298+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2850403742 shutdown_connections 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.299+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2850403742 wait complete. 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.299+0000 7fdc2ad1f640 1 Processor -- start 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc2ad1f640 1 -- start start 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc2ad1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc24100a80 0x7fdc24115f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc2ad1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc24101420 0x7fdc24116440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc2ad1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc24106a80 0x7fdc24110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc2ad1f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410ebc0 con 0x7fdc24106a80 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc2ad1f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410ea40 con 0x7fdc24100a80 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc2ad1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2410ed40 con 0x7fdc24101420 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc24101420 0x7fdc24116440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc24101420 0x7fdc24116440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:55050/0 (socket says 172.21.15.78:55050) 2024-09-16T00:14:20.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.300+0000 7fdc23fff640 1 -- 172.21.15.78:0/2295907128 learned_addr learned my addr 172.21.15.78:0/2295907128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.301+0000 7fdc292bc640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc24106a80 0x7fdc24110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.301+0000 7fdc28abb640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc24100a80 0x7fdc24115f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.302+0000 7fdc292bc640 1 -- 172.21.15.78:0/2295907128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc24101420 msgr2=0x7fdc24116440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.302+0000 7fdc292bc640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc24101420 0x7fdc24116440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.302+0000 7fdc292bc640 1 -- 172.21.15.78:0/2295907128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc24100a80 msgr2=0x7fdc24115f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.302+0000 7fdc292bc640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc24100a80 0x7fdc24115f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.302+0000 7fdc292bc640 1 -- 172.21.15.78:0/2295907128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc24110a50 con 0x7fdc24106a80 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.303+0000 7fdc292bc640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc24106a80 0x7fdc24110190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00be20 tx=0x7fdc1c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.304+0000 7fdc21ffb640 1 -- 172.21.15.78:0/2295907128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c019070 con 0x7fdc24106a80 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.304+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2295907128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc24110d40 con 0x7fdc24106a80 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.304+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2295907128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc241b3ba0 con 0x7fdc24106a80 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.305+0000 7fdc21ffb640 1 -- 172.21.15.78:0/2295907128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc1c014920 con 0x7fdc24106a80 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.305+0000 7fdc21ffb640 1 -- 172.21.15.78:0/2295907128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0128e0 con 0x7fdc24106a80 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.317+0000 7fdc2d0f8740 1 Processor -- start 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.319+0000 7fdc2d0f8740 1 -- start start 2024-09-16T00:14:20.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.319+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.319+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d6ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.319+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d7b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.320+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a299cd42f0 con 0x55a299d6e5d0 2024-09-16T00:14:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.320+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a299c7bcf0 con 0x55a299d6fb40 2024-09-16T00:14:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.320+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a299cd02b0 con 0x55a299d79370 2024-09-16T00:14:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.320+0000 7fdc21ffb640 1 -- 172.21.15.78:0/2295907128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fdc1c01c070 con 0x7fdc24106a80 2024-09-16T00:14:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.320+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.320+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d6e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46620/0 (socket says 172.21.15.78:46620) 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.320+0000 7fdc23fff640 1 -- 172.21.15.78:0/4039711022 learned_addr learned my addr 172.21.15.78:0/4039711022 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.321+0000 7fdc28abb640 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d6ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.324+0000 7fdc23fff640 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d7b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.325+0000 7fdc21ffb640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc140a34f0 0x7fdc140a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.326+0000 7fdc292bc640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc140a34f0 0x7fdc140a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.326+0000 7fdc21ffb640 1 -- 172.21.15.78:0/2295907128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdc1c0cf310 con 0x7fdc24106a80 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.327+0000 7fdc23fff640 1 -- 172.21.15.78:0/4039711022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 msgr2=0x55a299d7b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.327+0000 7fdc23fff640 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d7b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.327+0000 7fdc23fff640 1 -- 172.21.15.78:0/4039711022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 msgr2=0x55a299d6ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.327+0000 7fdc23fff640 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d6ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.327+0000 7fdc23fff640 1 -- 172.21.15.78:0/4039711022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a299c603a0 con 0x55a299d6fb40 2024-09-16T00:14:20.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.328+0000 7fdc23fff640 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d6e090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55a299d6f070 tx=0x7fdc10000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fcc80a756f44fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.328+0000 7fdc0effd640 1 -- 172.21.15.78:0/4039711022 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10044030 con 0x55a299d6fb40 2024-09-16T00:14:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.328+0000 7fdc292bc640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc140a34f0 0x7fdc140a59b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c09b560 tx=0x7fdc1c00e280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.328+0000 7fdc0effd640 1 -- 172.21.15.78:0/4039711022 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc10032b70 con 0x55a299d6fb40 2024-09-16T00:14:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.328+0000 7fdc0effd640 1 -- 172.21.15.78:0/4039711022 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10035070 con 0x55a299d6fb40 2024-09-16T00:14:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.330+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/4039711022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 msgr2=0x55a299d6e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.330+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d6e090 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55a299d6f070 tx=0x7fdc10000bc0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.330+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/4039711022 shutdown_connections 2024-09-16T00:14:20.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.330+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d7b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.330+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d6ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.330+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/4039711022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d6e090 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.330+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/4039711022 >> 172.21.15.78:0/4039711022 conn(0x55a299d37310 msgr2=0x55a299d6ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.331+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/4039711022 shutdown_connections 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.331+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/4039711022 wait complete. 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.331+0000 7fdc2d0f8740 1 -- start start 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.332+0000 7fdc2d0f8740 1 Processor -- start 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.332+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d78da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.332+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d76050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.332+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.332+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a299d8d020 con 0x55a299d6e5d0 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.332+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a299c8f060 con 0x55a299d6fb40 2024-09-16T00:14:20.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.332+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a299d8d1a0 con 0x55a299d79370 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.332+0000 7fdc2d0f8740 10 client.0 _ll_register_callbacks cb 0x55a2999e28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d78da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d78da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47722/0 (socket says 172.21.15.78:47722) 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 -- 172.21.15.78:0/7219441 learned_addr learned my addr 172.21.15.78:0/7219441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc28abb640 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d76050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 -- 172.21.15.78:0/7219441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 msgr2=0x55a299d76050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d76050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 -- 172.21.15.78:0/7219441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 msgr2=0x55a299d78da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d78da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.333+0000 7fdc23fff640 1 -- 172.21.15.78:0/7219441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a299a453c0 con 0x55a299d79370 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.334+0000 7fdc23fff640 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d767e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc100054c0 tx=0x7fdc10032550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=715ef77c5e1a7f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:20.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.334+0000 7fdc03fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.335+0000 7fdc03fff640 1 -- 172.21.15.78:0/7219441 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10035040 con 0x55a299d79370 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.335+0000 7fdc03fff640 1 -- 172.21.15.78:0/7219441 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdc10004460 con 0x55a299d79370 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.335+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/7219441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc10032b70 con 0x55a299d79370 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.337+0000 7fdc03fff640 1 -- 172.21.15.78:0/7219441 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc10056050 con 0x55a299d79370 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.337+0000 7fdc2d0f8740 10 client.24994 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.337+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/7219441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a299d8d020 con 0x55a299d79370 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.341+0000 7fdc03fff640 1 -- 172.21.15.78:0/7219441 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc10032b70 con 0x55a299d79370 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.342+0000 7fdc03fff640 1 -- 172.21.15.78:0/7219441 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fdc10059020 con 0x55a299d79370 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.342+0000 7fdc2d0f8740 10 client.24994 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.342+0000 7fdc2d0f8740 10 client.24994 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.342+0000 7fdc2d0f8740 20 client.24994 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:20.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.342+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a299d99d10 0x55a299d9c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.342+0000 7fdc2d0f8740 4 client.24994 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.342+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/7219441 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a299ce1860 con 0x55a299d99d10 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.343+0000 7fdc28abb640 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a299d99d10 0x55a299d9c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.346+0000 7fdc03fff640 10 client.24994 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.346+0000 7fdc28abb640 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a299d99d10 0x55a299d9c100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8337fd1394b2f7a server_cookie=92f4d0024137ca82 in_seq=0 out_seq=0 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.347+0000 7fdc03fff640 1 -- 172.21.15.78:0/7219441 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a299ce1860 con 0x55a299d99d10 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.347+0000 7fdc03fff640 10 client.24994 handle_command_reply: tid=0 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.347+0000 7fdc2d0f8740 1 client.24994 shutdown 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 msgr2=0x55a299d767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d767e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdc100054c0 tx=0x7fdc10032550 comp rx=0 tx=0).stop 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/7219441 shutdown_connections 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d79370 0x55a299d767e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a299d99d10 0x55a299d9c100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d76050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/7219441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d6e5d0 0x55a299d78da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/7219441 >> 172.21.15.78:0/7219441 conn(0x55a299d37310 msgr2=0x55a299d70930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.349+0000 7fdc03fff640 0 client.24994 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.350+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/7219441 shutdown_connections 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.350+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/7219441 wait complete. 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.350+0000 7fdc2d0f8740 20 client.24994 trim_cache size 0 max 16384 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc2d0f8740 1 Processor -- start 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc2d0f8740 1 -- start start 2024-09-16T00:14:20.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d74f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d77fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d79370 0x55a299d6a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0800fb90 con 0x55a299d79370 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08011760 con 0x55a299d6fb40 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08010260 con 0x55a299d6e5d0 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc28abb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d77fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc28abb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d77fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46640/0 (socket says 172.21.15.78:46640) 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.370+0000 7fdc28abb640 1 -- 172.21.15.78:0/997288234 learned_addr learned my addr 172.21.15.78:0/997288234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc28abb640 1 -- 172.21.15.78:0/997288234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 msgr2=0x55a299d74f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc28abb640 1 --2- 172.21.15.78:0/997288234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d74f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc28abb640 1 -- 172.21.15.78:0/997288234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d79370 msgr2=0x55a299d6a510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:14:20.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc28abb640 1 --2- 172.21.15.78:0/997288234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d79370 0x55a299d6a510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc28abb640 1 -- 172.21.15.78:0/997288234 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a299d7bad0 con 0x55a299d6fb40 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc28abb640 1 --2- 172.21.15.78:0/997288234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d77fc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a299d758e0 tx=0x7fdc18005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc8f70b1284c9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc0e7fc640 1 -- 172.21.15.78:0/997288234 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18004250 con 0x55a299d6fb40 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc0e7fc640 1 -- 172.21.15.78:0/997288234 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc180043f0 con 0x55a299d6fb40 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.371+0000 7fdc0e7fc640 1 -- 172.21.15.78:0/997288234 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18027930 con 0x55a299d6fb40 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/997288234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 msgr2=0x55a299d77fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/997288234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d77fc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55a299d758e0 tx=0x7fdc18005f90 comp rx=0 tx=0).stop 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/997288234 shutdown_connections 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/997288234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d79370 0x55a299d6a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/997288234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d77fc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/997288234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d74f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/997288234 >> 172.21.15.78:0/997288234 conn(0x55a299d37310 msgr2=0x55a299d778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/997288234 shutdown_connections 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.372+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/997288234 wait complete. 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.373+0000 7fdc2d0f8740 1 -- start start 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.373+0000 7fdc2d0f8740 1 Processor -- start 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d8e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc2d0f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d79370 0x55a299d8ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc2d0f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08011060 con 0x55a299d79370 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc28abb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc23fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d8e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:55084/0 (socket says 172.21.15.78:55084) 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc23fff640 1 -- 172.21.15.78:0/1850433080 learned_addr learned my addr 172.21.15.78:0/1850433080 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:20.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.374+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc08010990 con 0x55a299d6fb40 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.375+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0800db70 con 0x55a299d6e5d0 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.375+0000 7fdc2d0f8740 10 client.0 _ll_register_callbacks cb 0x7fdc18009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.375+0000 7fdc23fff640 1 -- 172.21.15.78:0/1850433080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 msgr2=0x55a299d8e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.375+0000 7fdc23fff640 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d8e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.375+0000 7fdc23fff640 1 -- 172.21.15.78:0/1850433080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d79370 msgr2=0x55a299d8ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.375+0000 7fdc23fff640 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d79370 0x55a299d8ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.375+0000 7fdc23fff640 1 -- 172.21.15.78:0/1850433080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a299e037f0 con 0x55a299d6e5d0 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.376+0000 7fdc23fff640 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d8e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc10001e50 tx=0x7fdc100092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356c734470561966 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.376+0000 7fdc03fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.376+0000 7fdc03fff640 1 -- 172.21.15.78:0/1850433080 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10044070 con 0x55a299d6e5d0 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.376+0000 7fdc03fff640 1 -- 172.21.15.78:0/1850433080 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdc100035f0 con 0x55a299d6e5d0 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.376+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc180043f0 con 0x55a299d6e5d0 2024-09-16T00:14:20.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.377+0000 7fdc03fff640 1 -- 172.21.15.78:0/1850433080 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc10035070 con 0x55a299d6e5d0 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.377+0000 7fdc2d0f8740 10 client.24997 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.377+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a299e04ab0 con 0x55a299d6e5d0 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc03fff640 1 -- 172.21.15.78:0/1850433080 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc180043f0 con 0x55a299d6e5d0 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc03fff640 1 -- 172.21.15.78:0/1850433080 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fdc10059040 con 0x55a299d6e5d0 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc2d0f8740 10 client.24997 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc2d0f8740 10 client.24997 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc2d0f8740 20 client.24997 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a299e05120 0x55a299e07540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc2d0f8740 4 client.24997 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a299cdbd70 con 0x55a299e05120 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.378+0000 7fdc28abb640 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a299e05120 0x55a299e07540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.379+0000 7fdc03fff640 10 client.24997 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:14:20.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.379+0000 7fdc28abb640 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a299e05120 0x55a299e07540 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3bcb0e3730f9432d server_cookie=f09cd98750ab009d in_seq=0 out_seq=0 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.379+0000 7fdc03fff640 1 -- 172.21.15.78:0/1850433080 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a299cdbd70 con 0x55a299e05120 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.379+0000 7fdc03fff640 10 client.24997 handle_command_reply: tid=0 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.381+0000 7fdc2d0f8740 1 client.24997 shutdown 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.381+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 msgr2=0x55a299d8e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.381+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d8e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdc10001e50 tx=0x7fdc100092d0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 shutdown_connections 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a299d79370 0x55a299d8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55a299e05120 0x55a299e07540 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a299d6fb40 0x55a299d8e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 1 --2- 172.21.15.78:0/1850433080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a299d6e5d0 0x55a299d8e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 >> 172.21.15.78:0/1850433080 conn(0x55a299d37310 msgr2=0x55a299d780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc03fff640 0 client.24997 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:14:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 shutdown_connections 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 1 -- 172.21.15.78:0/1850433080 wait complete. 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.382+0000 7fdc2d0f8740 20 client.24997 trim_cache size 0 max 16384 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc140a34f0 msgr2=0x7fdc140a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc140a34f0 0x7fdc140a59b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c09b560 tx=0x7fdc1c00e280 comp rx=0 tx=0).stop 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc24106a80 msgr2=0x7fdc24110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc24106a80 0x7fdc24110190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00be20 tx=0x7fdc1c00bf10 comp rx=0 tx=0).stop 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2295907128 shutdown_connections 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc140a34f0 0x7fdc140a59b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc24106a80 0x7fdc24110190 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc24101420 0x7fdc24116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 --2- 172.21.15.78:0/2295907128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc24100a80 0x7fdc24115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2295907128 >> 172.21.15.78:0/2295907128 conn(0x7fdc240fc7a0 msgr2=0x7fdc240fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2295907128 shutdown_connections 2024-09-16T00:14:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:20.384+0000 7fdc2ad1f640 1 -- 172.21.15.78:0/2295907128 wait complete. 2024-09-16T00:14:21.705 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:21 smithi078 ceph-mon[31157]: pgmap v389: 129 pgs: 129 active+clean; 978 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 777 op/s 2024-09-16T00:14:21.715 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:21.760 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:14:21.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:21 smithi193 ceph-mon[35484]: pgmap v389: 129 pgs: 129 active+clean; 978 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 777 op/s 2024-09-16T00:14:21.957 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:21 smithi148 ceph-mon[35599]: pgmap v389: 129 pgs: 129 active+clean; 978 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 777 op/s 2024-09-16T00:14:23.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:23 smithi193 ceph-mon[35484]: pgmap v390: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 901 op/s 2024-09-16T00:14:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:23 smithi148 ceph-mon[35599]: pgmap v390: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 901 op/s 2024-09-16T00:14:24.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:23 smithi078 ceph-mon[31157]: pgmap v390: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 901 op/s 2024-09-16T00:14:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/719831560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb811b140 msgr2=0x7f3cb811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/719831560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb811b140 0x7f3cb811d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009860 tx=0x7f3cb002a380 comp rx=0 tx=0).stop 2024-09-16T00:14:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/719831560 shutdown_connections 2024-09-16T00:14:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/719831560 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb811b140 0x7f3cb811d530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/719831560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cb806ea30 0x7f3cb806ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/719831560 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb8110190 0x7f3cb806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/719831560 >> 172.21.15.78:0/719831560 conn(0x7f3cb8073180 msgr2=0x7f3cb8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:24.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/719831560 shutdown_connections 2024-09-16T00:14:24.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.376+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/719831560 wait complete. 2024-09-16T00:14:24.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.377+0000 7f3cbd7b8640 1 Processor -- start 2024-09-16T00:14:24.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.377+0000 7f3cbd7b8640 1 -- start start 2024-09-16T00:14:24.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.378+0000 7f3cbd7b8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cb806ea30 0x7f3cb81a7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.378+0000 7f3cbd7b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb8110190 0x7f3cb81a7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.378+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cb806ea30 0x7f3cb81a7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.378+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cb806ea30 0x7f3cb81a7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:55098/0 (socket says 172.21.15.78:55098) 2024-09-16T00:14:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.378+0000 7f3cb7fff640 1 -- 172.21.15.78:0/2189371387 learned_addr learned my addr 172.21.15.78:0/2189371387 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.378+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb811b140 0x7f3cb81aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.379+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb8110190 0x7f3cb81a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.379+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8120bd0 con 0x7f3cb811b140 2024-09-16T00:14:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.379+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8120a50 con 0x7f3cb8110190 2024-09-16T00:14:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.379+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb8120d50 con 0x7f3cb806ea30 2024-09-16T00:14:24.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.380+0000 7f3cb77fe640 1 -- 172.21.15.78:0/2189371387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cb806ea30 msgr2=0x7f3cb81a7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:24.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.380+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cb806ea30 0x7f3cb81a7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.380+0000 7f3cb77fe640 1 -- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb811b140 msgr2=0x7f3cb81aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:24.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.380+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb811b140 0x7f3cb81aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.380+0000 7f3cb77fe640 1 -- 172.21.15.78:0/2189371387 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb81ac100 con 0x7f3cb8110190 2024-09-16T00:14:24.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.380+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb8110190 0x7f3cb81a7690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00d6d0 tx=0x7f3cac00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:24.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.380+0000 7f3cb57fa640 1 -- 172.21.15.78:0/2189371387 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac017070 con 0x7f3cb8110190 2024-09-16T00:14:24.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.381+0000 7f3cb57fa640 1 -- 172.21.15.78:0/2189371387 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cac010460 con 0x7f3cb8110190 2024-09-16T00:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.381+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb81ac3f0 con 0x7f3cb8110190 2024-09-16T00:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.381+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb81b3cd0 con 0x7f3cb8110190 2024-09-16T00:14:24.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.381+0000 7f3cb57fa640 1 -- 172.21.15.78:0/2189371387 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00f620 con 0x7f3cb8110190 2024-09-16T00:14:24.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.384+0000 7f3cb57fa640 1 -- 172.21.15.78:0/2189371387 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f3cac01a070 con 0x7f3cb8110190 2024-09-16T00:14:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.385+0000 7f3cc0df3740 1 Processor -- start 2024-09-16T00:14:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.385+0000 7f3cc0df3740 1 -- start start 2024-09-16T00:14:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.385+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b66ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.385+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.385+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 0x5589f2b73ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.385+0000 7f3cc0df3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5589f2acc2a0 con 0x5589f2b71ae0 2024-09-16T00:14:24.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.385+0000 7f3cc0df3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5589f2a73b10 con 0x5589f2b67220 2024-09-16T00:14:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.385+0000 7f3cc0df3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5589f2ac8260 con 0x5589f2b66900 2024-09-16T00:14:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.386+0000 7f3cbcfb7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b66ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.386+0000 7f3cbcfb7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b66ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:55102/0 (socket says 172.21.15.78:55102) 2024-09-16T00:14:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.386+0000 7f3cbcfb7640 1 -- 172.21.15.78:0/3122187358 learned_addr learned my addr 172.21.15.78:0/3122187358 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.386+0000 7f3cbcfb7640 1 -- 172.21.15.78:0/3122187358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 msgr2=0x5589f2b715a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:14:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.386+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/3122187358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b715a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.386+0000 7f3cbcfb7640 1 -- 172.21.15.78:0/3122187358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 msgr2=0x5589f2b73ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:24.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.387+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/3122187358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 0x5589f2b73ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.387+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/3122187358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 0x5589f2b73ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.387+0000 7f3cbcfb7640 1 -- 172.21.15.78:0/3122187358 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f283db90 con 0x5589f2b66900 2024-09-16T00:14:24.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.388+0000 7f3cb57fa640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ca00a3420 0x7f3ca00a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.388+0000 7f3cb57fa640 1 -- 172.21.15.78:0/2189371387 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3cac0cf490 con 0x7f3cb8110190 2024-09-16T00:14:24.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.389+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/3122187358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b66ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0035040 tx=0x7f3cb0009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2daa16773cfe33 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:24.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.389+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ca00a3420 0x7f3ca00a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.391+0000 7f3c9e7fc640 1 -- 172.21.15.78:0/3122187358 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0037040 con 0x5589f2b66900 2024-09-16T00:14:24.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.391+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ca00a3420 0x7f3ca00a58e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5589f2b63cf0 tx=0x7f3ca800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:24.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.391+0000 7f3c9e7fc640 1 -- 172.21.15.78:0/3122187358 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb00305b0 con 0x5589f2b66900 2024-09-16T00:14:24.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.392+0000 7f3c9e7fc640 1 -- 172.21.15.78:0/3122187358 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00053b0 con 0x5589f2b66900 2024-09-16T00:14:24.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.392+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3122187358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 msgr2=0x5589f2b66ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:24.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.392+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3122187358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b66ce0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0035040 tx=0x7f3cb0009370 comp rx=0 tx=0).stop 2024-09-16T00:14:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.392+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3122187358 shutdown_connections 2024-09-16T00:14:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.392+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3122187358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 0x5589f2b73ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.392+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3122187358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.392+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3122187358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b66ce0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.392+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3122187358 >> 172.21.15.78:0/3122187358 conn(0x5589f2b62e90 msgr2=0x5589f2b63270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.393+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3122187358 shutdown_connections 2024-09-16T00:14:24.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.393+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3122187358 wait complete. 2024-09-16T00:14:24.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.393+0000 7f3cc0df3740 1 -- start start 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.393+0000 7f3cc0df3740 1 Processor -- start 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b71ae0 0x5589f2b6f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cc0df3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5589f2b74700 con 0x5589f2b66900 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cc0df3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5589f2a546e0 con 0x5589f2b67220 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cc0df3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5589f2b74880 con 0x5589f2b71ae0 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cc0df3740 10 client.0 _ll_register_callbacks cb 0x5589f2a78a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47776/0 (socket says 172.21.15.78:47776) 2024-09-16T00:14:24.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cb7fff640 1 -- 172.21.15.78:0/131463841 learned_addr learned my addr 172.21.15.78:0/131463841 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b71ae0 0x5589f2b6f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.394+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cbcfb7640 1 -- 172.21.15.78:0/131463841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 msgr2=0x5589f2b6eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b6eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cbcfb7640 1 -- 172.21.15.78:0/131463841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 msgr2=0x5589f2b6e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cbcfb7640 1 -- 172.21.15.78:0/131463841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f2b7dca0 con 0x5589f2b71ae0 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b71ae0 0x5589f2b6f410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009860 tx=0x7f3cb0030e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a29378b67572fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3c977fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3c977fe640 1 -- 172.21.15.78:0/131463841 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0048020 con 0x5589f2b71ae0 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cc0df3740 1 -- 172.21.15.78:0/131463841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cb00305b0 con 0x5589f2b71ae0 2024-09-16T00:14:24.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3c977fe640 1 -- 172.21.15.78:0/131463841 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3cb00046c0 con 0x5589f2b71ae0 2024-09-16T00:14:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3c977fe640 1 -- 172.21.15.78:0/131463841 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3cb0037040 con 0x5589f2b71ae0 2024-09-16T00:14:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cc0df3740 10 client.25009 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.395+0000 7f3cc0df3740 1 -- 172.21.15.78:0/131463841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5589f2b74700 con 0x5589f2b71ae0 2024-09-16T00:14:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.396+0000 7f3c977fe640 1 -- 172.21.15.78:0/131463841 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb00305b0 con 0x5589f2b71ae0 2024-09-16T00:14:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.396+0000 7f3c977fe640 1 -- 172.21.15.78:0/131463841 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f3cb003a070 con 0x5589f2b71ae0 2024-09-16T00:14:24.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.396+0000 7f3cc0df3740 10 client.25009 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.396+0000 7f3cc0df3740 10 client.25009 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:14:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.396+0000 7f3cc0df3740 20 client.25009 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.396+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5589f2b81b10 0x5589f2b83f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.396+0000 7f3cc0df3740 4 client.25009 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:24.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.396+0000 7f3cc0df3740 1 -- 172.21.15.78:0/131463841 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589f2ad99d0 con 0x5589f2b81b10 2024-09-16T00:14:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.397+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5589f2b81b10 0x5589f2b83f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.399+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5589f2b81b10 0x5589f2b83f70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835548dc9af2377f server_cookie=68360effc94ec77e in_seq=0 out_seq=0 2024-09-16T00:14:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.399+0000 7f3c977fe640 10 client.25009 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.399+0000 7f3c977fe640 1 -- 172.21.15.78:0/131463841 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589f2ad99d0 con 0x5589f2b81b10 2024-09-16T00:14:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.399+0000 7f3c977fe640 10 client.25009 handle_command_reply: tid=0 2024-09-16T00:14:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.399+0000 7f3cc0df3740 1 client.25009 shutdown 2024-09-16T00:14:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 -- 172.21.15.78:0/131463841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b71ae0 msgr2=0x5589f2b6f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:24.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b71ae0 0x5589f2b6f410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009860 tx=0x7f3cb0030e10 comp rx=0 tx=0).stop 2024-09-16T00:14:24.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 -- 172.21.15.78:0/131463841 shutdown_connections 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5589f2b81b10 0x5589f2b83f70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b71ae0 0x5589f2b6f410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b6eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/131463841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 -- 172.21.15.78:0/131463841 >> 172.21.15.78:0/131463841 conn(0x5589f2b62e90 msgr2=0x5589f2b68b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3c977fe640 0 client.25009 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 -- 172.21.15.78:0/131463841 shutdown_connections 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 1 -- 172.21.15.78:0/131463841 wait complete. 2024-09-16T00:14:24.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.400+0000 7f3cc0df3740 20 client.25009 trim_cache size 0 max 16384 2024-09-16T00:14:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cc0df3740 1 Processor -- start 2024-09-16T00:14:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cc0df3740 1 -- start start 2024-09-16T00:14:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b67220 0x5589f2b89840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b71ae0 0x5589f2b89d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cc0df3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca400faa0 con 0x5589f2b66900 2024-09-16T00:14:24.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cc0df3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4011670 con 0x5589f2b71ae0 2024-09-16T00:14:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b67220 0x5589f2b89840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cb77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b67220 0x5589f2b89840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:55132/0 (socket says 172.21.15.78:55132) 2024-09-16T00:14:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cb77fe640 1 -- 172.21.15.78:0/338819259 learned_addr learned my addr 172.21.15.78:0/338819259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.431+0000 7f3cc0df3740 1 -- 172.21.15.78:0/338819259 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4010170 con 0x5589f2b67220 2024-09-16T00:14:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.432+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.432+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b71ae0 0x5589f2b89d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.432+0000 7f3cb77fe640 1 -- 172.21.15.78:0/338819259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b71ae0 msgr2=0x5589f2b89d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.432+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b71ae0 0x5589f2b89d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.432+0000 7f3cb77fe640 1 -- 172.21.15.78:0/338819259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 msgr2=0x5589f2b6d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.432+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.432+0000 7f3cb77fe640 1 -- 172.21.15.78:0/338819259 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f2b67750 con 0x5589f2b67220 2024-09-16T00:14:24.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.432+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b67220 0x5589f2b89840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3cac09b290 tx=0x7f3cac09bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1e9821a6b16f570 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.433+0000 7f3c9dffb640 1 -- 172.21.15.78:0/338819259 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac021040 con 0x5589f2b67220 2024-09-16T00:14:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.433+0000 7f3c9dffb640 1 -- 172.21.15.78:0/338819259 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cac09f940 con 0x5589f2b67220 2024-09-16T00:14:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.433+0000 7f3c9dffb640 1 -- 172.21.15.78:0/338819259 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cac09a9e0 con 0x5589f2b67220 2024-09-16T00:14:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.434+0000 7f3cc0df3740 1 -- 172.21.15.78:0/338819259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b67220 msgr2=0x5589f2b89840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.434+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b67220 0x5589f2b89840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3cac09b290 tx=0x7f3cac09bc90 comp rx=0 tx=0).stop 2024-09-16T00:14:24.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.434+0000 7f3cc0df3740 1 -- 172.21.15.78:0/338819259 shutdown_connections 2024-09-16T00:14:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.434+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b71ae0 0x5589f2b89d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.434+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b67220 0x5589f2b89840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.434+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/338819259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b66900 0x5589f2b6d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.434+0000 7f3cc0df3740 1 -- 172.21.15.78:0/338819259 >> 172.21.15.78:0/338819259 conn(0x5589f2b62e90 msgr2=0x5589f2b70a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.435+0000 7f3cc0df3740 1 -- 172.21.15.78:0/338819259 shutdown_connections 2024-09-16T00:14:24.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.435+0000 7f3cc0df3740 1 -- 172.21.15.78:0/338819259 wait complete. 2024-09-16T00:14:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.435+0000 7f3cc0df3740 1 -- start start 2024-09-16T00:14:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.436+0000 7f3cc0df3740 1 Processor -- start 2024-09-16T00:14:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.436+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b7a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.436+0000 7f3cc0df3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b6b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.436+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b7a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.436+0000 7f3cb7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b7a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:55142/0 (socket says 172.21.15.78:55142) 2024-09-16T00:14:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cb7fff640 1 -- 172.21.15.78:0/3354647592 learned_addr learned my addr 172.21.15.78:0/3354647592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 0x5589f2b6ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4010f70 con 0x5589f2b71ae0 2024-09-16T00:14:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca40108a0 con 0x5589f2b67220 2024-09-16T00:14:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca400da80 con 0x5589f2b66900 2024-09-16T00:14:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cbcfb7640 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 0x5589f2b6ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cc0df3740 10 client.0 _ll_register_callbacks cb 0x7f3cb0030cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cb7fff640 1 -- 172.21.15.78:0/3354647592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 msgr2=0x5589f2b6b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b6b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b6b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cb7fff640 1 -- 172.21.15.78:0/3354647592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 msgr2=0x5589f2b6ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 0x5589f2b6ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.437+0000 7f3cb7fff640 1 -- 172.21.15.78:0/3354647592 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f2b6c170 con 0x5589f2b66900 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.438+0000 7f3cb7fff640 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b7a180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8015c90 tx=0x7f3ca8004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb15f666f1962fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.438+0000 7f3c977fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.438+0000 7f3c977fe640 1 -- 172.21.15.78:0/3354647592 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca802f020 con 0x5589f2b66900 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.438+0000 7f3c977fe640 1 -- 172.21.15.78:0/3354647592 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3ca802b660 con 0x5589f2b66900 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.438+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cac09f940 con 0x5589f2b66900 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3c977fe640 1 -- 172.21.15.78:0/3354647592 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3ca802b860 con 0x5589f2b66900 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3cc0df3740 10 client.25015 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5589f2bec710 con 0x5589f2b66900 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3c977fe640 1 -- 172.21.15.78:0/3354647592 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cac09f940 con 0x5589f2b66900 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3c977fe640 1 -- 172.21.15.78:0/3354647592 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f3ca803c020 con 0x5589f2b66900 2024-09-16T00:14:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3cc0df3740 10 client.25015 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3cc0df3740 10 client.25015 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:14:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3cc0df3740 20 client.25015 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.439+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5589f2becf60 0x5589f2bef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.440+0000 7f3cc0df3740 4 client.25015 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}] 2024-09-16T00:14:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.440+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}) -- 0x5589f2ad3d20 con 0x5589f2becf60 2024-09-16T00:14:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.440+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5589f2becf60 0x5589f2bef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.440+0000 7f3c977fe640 10 client.25015 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:24.441+0000 7f3cb77fe640 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5589f2becf60 0x5589f2bef3a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e07113bebdd61db server_cookie=711425f37f404f1f in_seq=0 out_seq=0 2024-09-16T00:14:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:25.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.452+0000 7f3c977fe640 1 -- 172.21.15.78:0/3354647592 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5589f2ad3d20 con 0x5589f2becf60 2024-09-16T00:14:25.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.452+0000 7f3c977fe640 10 client.25015 handle_command_reply: tid=0 2024-09-16T00:14:25.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.452+0000 7f3cc0df3740 1 client.25015 shutdown 2024-09-16T00:14:25.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.452+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 msgr2=0x5589f2b7a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:25.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.452+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b7a180 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8015c90 tx=0x7f3ca8004b50 comp rx=0 tx=0).stop 2024-09-16T00:14:25.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 shutdown_connections 2024-09-16T00:14:25.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5589f2becf60 0x5589f2bef3a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589f2b71ae0 0x5589f2b6ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3c977fe640 0 client.25015 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5589f2b67220 0x5589f2b6b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 1 --2- 172.21.15.78:0/3354647592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2b66900 0x5589f2b7a180 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 >> 172.21.15.78:0/3354647592 conn(0x5589f2b62e90 msgr2=0x5589f2b617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 shutdown_connections 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 1 -- 172.21.15.78:0/3354647592 wait complete. 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.453+0000 7f3cc0df3740 20 client.25015 trim_cache size 0 max 16384 2024-09-16T00:14:25.456 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 31, 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 2, 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "56b5046b": { 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 2, 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:14:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 150.2, 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:25.459 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:14:25.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ca00a3420 msgr2=0x7f3ca00a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:25.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ca00a3420 0x7f3ca00a58e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5589f2b63cf0 tx=0x7f3ca800b040 comp rx=0 tx=0).stop 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb8110190 msgr2=0x7f3cb81a7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb8110190 0x7f3cb81a7690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00d6d0 tx=0x7f3cac00db90 comp rx=0 tx=0).stop 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 shutdown_connections 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ca00a3420 0x7f3ca00a58e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cb811b140 0x7f3cb81aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3cb8110190 0x7f3cb81a7690 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 --2- 172.21.15.78:0/2189371387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cb806ea30 0x7f3cb81a7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 >> 172.21.15.78:0/2189371387 conn(0x7f3cb8073180 msgr2=0x7f3cb8119370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 shutdown_connections 2024-09-16T00:14:25.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.454+0000 7f3cbd7b8640 1 -- 172.21.15.78:0/2189371387 wait complete. 2024-09-16T00:14:25.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:25 smithi078 ceph-mon[31157]: pgmap v391: 129 pgs: 129 active+clean; 969 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 812 op/s 2024-09-16T00:14:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 25.0 s, 2446.3 tps, lat 29.508 ms stddev 40.415 2024-09-16T00:14:25.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:25 smithi193 ceph-mon[35484]: pgmap v391: 129 pgs: 129 active+clean; 969 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 812 op/s 2024-09-16T00:14:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:25 smithi148 ceph-mon[35599]: pgmap v391: 129 pgs: 129 active+clean; 969 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 812 op/s 2024-09-16T00:14:26.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 -- 172.21.15.78:0/2432442529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 msgr2=0x7f977c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 --2- 172.21.15.78:0/2432442529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 0x7f977c1029b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9764009900 tx=0x7f976402a480 comp rx=0 tx=0).stop 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 -- 172.21.15.78:0/2432442529 shutdown_connections 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 --2- 172.21.15.78:0/2432442529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f977c102ef0 0x7f977c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 --2- 172.21.15.78:0/2432442529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 0x7f977c1029b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 --2- 172.21.15.78:0/2432442529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f977c100e20 0x7f977c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 -- 172.21.15.78:0/2432442529 >> 172.21.15.78:0/2432442529 conn(0x7f977c0fc7d0 msgr2=0x7f977c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 -- 172.21.15.78:0/2432442529 shutdown_connections 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.922+0000 7f9781a85640 1 -- 172.21.15.78:0/2432442529 wait complete. 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.923+0000 7f9781a85640 1 Processor -- start 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.923+0000 7f9781a85640 1 -- start start 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.923+0000 7f9781a85640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f977c100e20 0x7f977c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.923+0000 7f9781a85640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 0x7f977c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.923+0000 7f9781a85640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f977c102ef0 0x7f977c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.923+0000 7f9781a85640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f977c110f80 con 0x7f977c102570 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.923+0000 7f9781a85640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f977c110e00 con 0x7f977c102ef0 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.923+0000 7f9781a85640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f977c111100 con 0x7f977c100e20 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 0x7f977c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 0x7f977c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47818/0 (socket says 172.21.15.78:47818) 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977affd640 1 -- 172.21.15.78:0/3771542427 learned_addr learned my addr 172.21.15.78:0/3771542427 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f977c102ef0 0x7f977c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977b7fe640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f977c100e20 0x7f977c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977bfff640 1 -- 172.21.15.78:0/3771542427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f977c100e20 msgr2=0x7f977c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977bfff640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f977c100e20 0x7f977c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977bfff640 1 -- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 msgr2=0x7f977c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977bfff640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 0x7f977c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977bfff640 1 -- 172.21.15.78:0/3771542427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c1a69c0 con 0x7f977c102ef0 2024-09-16T00:14:26.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977affd640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 0x7f977c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f977bfff640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f977c102ef0 0x7f977c0fd700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c970 tx=0x7f976c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f9778ff9640 1 -- 172.21.15.78:0/3771542427 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c007bb0 con 0x7f977c102ef0 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f9781a85640 1 -- 172.21.15.78:0/3771542427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1a6cb0 con 0x7f977c102ef0 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f9778ff9640 1 -- 172.21.15.78:0/3771542427 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c007d50 con 0x7f977c102ef0 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f9778ff9640 1 -- 172.21.15.78:0/3771542427 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c018770 con 0x7f977c102ef0 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.924+0000 7f9781a85640 1 -- 172.21.15.78:0/3771542427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1a71a0 con 0x7f977c102ef0 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.927+0000 7f9778ff9640 1 -- 172.21.15.78:0/3771542427 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f976c00f450 con 0x7f977c102ef0 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9783e5e740 1 Processor -- start 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9783e5e740 1 -- start start 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 0x5628ca286090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca2865d0 0x5628ca286a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca293760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9783e5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5628ca1ec2f0 con 0x5628ca291370 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9783e5e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5628ca193cf0 con 0x5628ca2865d0 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9783e5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5628ca1e82b0 con 0x5628ca287b40 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca293760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca293760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47830/0 (socket says 172.21.15.78:47830) 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977affd640 1 -- 172.21.15.78:0/3163211514 learned_addr learned my addr 172.21.15.78:0/3163211514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977b7fe640 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca2865d0 0x5628ca286a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977affd640 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 0x5628ca286090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977b7fe640 1 -- 172.21.15.78:0/3163211514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 msgr2=0x5628ca286090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977b7fe640 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 0x5628ca286090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977b7fe640 1 -- 172.21.15.78:0/3163211514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 msgr2=0x5628ca293760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977b7fe640 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca293760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977b7fe640 1 -- 172.21.15.78:0/3163211514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ca1783a0 con 0x5628ca2865d0 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f9778ff9640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97680a32d0 0x7f97680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.928+0000 7f977affd640 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca293760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f977affd640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97680a32d0 0x7f97680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f977b7fe640 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca2865d0 0x5628ca286a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9774009fa0 tx=0x7f97740118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d82946579b6b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f9778ff9640 1 -- 172.21.15.78:0/3771542427 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f976c0cfcd0 con 0x7f977c102ef0 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f9771ffb640 1 -- 172.21.15.78:0/3163211514 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774003c30 con 0x5628ca2865d0 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f9771ffb640 1 -- 172.21.15.78:0/3163211514 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9774003dd0 con 0x5628ca2865d0 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f9771ffb640 1 -- 172.21.15.78:0/3163211514 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977400fb20 con 0x5628ca2865d0 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f9783e5e740 1 -- 172.21.15.78:0/3163211514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca2865d0 msgr2=0x5628ca286a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f9783e5e740 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca2865d0 0x5628ca286a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9774009fa0 tx=0x7f97740118d0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.929+0000 7f977affd640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97680a32d0 0x7f97680a5790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9764005770 tx=0x7f976402a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 -- 172.21.15.78:0/3163211514 shutdown_connections 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca293760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca2865d0 0x5628ca286a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 --2- 172.21.15.78:0/3163211514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 0x5628ca286090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 -- 172.21.15.78:0/3163211514 >> 172.21.15.78:0/3163211514 conn(0x5628ca24f310 msgr2=0x5628ca282c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 -- 172.21.15.78:0/3163211514 shutdown_connections 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 -- 172.21.15.78:0/3163211514 wait complete. 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 -- start start 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.930+0000 7f9783e5e740 1 Processor -- start 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 0x5628ca290ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca287b40 0x5628ca28df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f9783e5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5628ca294070 con 0x5628ca287b40 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f9783e5e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5628ca1a7060 con 0x5628ca291370 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f9783e5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5628ca2941f0 con 0x5628ca2865d0 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f9783e5e740 10 client.0 _ll_register_callbacks cb 0x5628c9efa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:26.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca287b40 0x5628ca28df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca287b40 0x5628ca28df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47846/0 (socket says 172.21.15.78:47846) 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 -- 172.21.15.78:0/441101566 learned_addr learned my addr 172.21.15.78:0/441101566 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977b7fe640 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 0x5628ca290ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977bfff640 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 -- 172.21.15.78:0/441101566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 msgr2=0x5628ca290ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 0x5628ca290ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 -- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 msgr2=0x5628ca28e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 -- 172.21.15.78:0/441101566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c9f5d3c0 con 0x5628ca287b40 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977bfff640 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f977affd640 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca287b40 0x5628ca28df90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9764005ab0 tx=0x7f9764005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8b86934635e443 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f975affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f975affd640 1 -- 172.21.15.78:0/441101566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976404c450 con 0x5628ca287b40 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f975affd640 1 -- 172.21.15.78:0/441101566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f976404c5f0 con 0x5628ca287b40 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.931+0000 7f9783e5e740 1 -- 172.21.15.78:0/441101566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9774003dd0 con 0x5628ca287b40 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f975affd640 1 -- 172.21.15.78:0/441101566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f976405d050 con 0x5628ca287b40 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f9783e5e740 10 client.15597 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f9783e5e740 1 -- 172.21.15.78:0/441101566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628ca29fae0 con 0x5628ca287b40 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f975affd640 1 -- 172.21.15.78:0/441101566 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9774003dd0 con 0x5628ca287b40 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f975affd640 1 -- 172.21.15.78:0/441101566 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f9764049040 con 0x5628ca287b40 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f9783e5e740 10 client.15597 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:26.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f9783e5e740 10 client.15597 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f9783e5e740 20 client.15597 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f9783e5e740 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5628ca2a1040 0x5628ca2a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f9783e5e740 4 client.15597 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.932+0000 7f9783e5e740 1 -- 172.21.15.78:0/441101566 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628ca1f9860 con 0x5628ca2a1040 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.933+0000 7f977b7fe640 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5628ca2a1040 0x5628ca2a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.933+0000 7f977b7fe640 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5628ca2a1040 0x5628ca2a3450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=703213030e35c332 server_cookie=9afb5493e8f0994c in_seq=0 out_seq=0 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.933+0000 7f975affd640 10 client.15597 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f975affd640 1 -- 172.21.15.78:0/441101566 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628ca1f9860 con 0x5628ca2a1040 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f975affd640 10 client.15597 handle_command_reply: tid=0 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 client.15597 shutdown 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 -- 172.21.15.78:0/441101566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca287b40 msgr2=0x5628ca28df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca287b40 0x5628ca28df90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9764005ab0 tx=0x7f9764005a20 comp rx=0 tx=0).stop 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 -- 172.21.15.78:0/441101566 shutdown_connections 2024-09-16T00:14:26.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5628ca2a1040 0x5628ca2a3450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca287b40 0x5628ca28df90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 --2- 172.21.15.78:0/441101566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 0x5628ca290ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 -- 172.21.15.78:0/441101566 >> 172.21.15.78:0/441101566 conn(0x5628ca24f310 msgr2=0x5628ca288960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f975affd640 0 client.15597 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 -- 172.21.15.78:0/441101566 shutdown_connections 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 1 -- 172.21.15.78:0/441101566 wait complete. 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.934+0000 7f9783e5e740 20 client.15597 trim_cache size 0 max 16384 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f9783e5e740 1 Processor -- start 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f9783e5e740 1 -- start start 2024-09-16T00:14:26.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 0x5628ca29d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca287b40 0x5628ca28cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca2a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f9783e5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b0d20 con 0x5628ca291370 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f9783e5e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b28f0 con 0x5628ca287b40 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f9783e5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b13f0 con 0x5628ca2865d0 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f977bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca2a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f977bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca2a8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47850/0 (socket says 172.21.15.78:47850) 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.947+0000 7f977bfff640 1 -- 172.21.15.78:0/2882160344 learned_addr learned my addr 172.21.15.78:0/2882160344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f977bfff640 1 -- 172.21.15.78:0/2882160344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 msgr2=0x5628ca29d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f977bfff640 1 --2- 172.21.15.78:0/2882160344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 0x5628ca29d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f977affd640 1 --2- 172.21.15.78:0/2882160344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca287b40 0x5628ca28cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f977bfff640 1 -- 172.21.15.78:0/2882160344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca287b40 msgr2=0x5628ca28cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:26.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f977bfff640 1 --2- 172.21.15.78:0/2882160344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca287b40 0x5628ca28cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f977bfff640 1 -- 172.21.15.78:0/2882160344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628ca289120 con 0x5628ca291370 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f977bfff640 1 --2- 172.21.15.78:0/2882160344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca2a8d60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f976c098290 tx=0x7f976c09b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f727a8bebcdd4d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f97717fa640 1 -- 172.21.15.78:0/2882160344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c0a01e0 con 0x5628ca291370 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f97717fa640 1 -- 172.21.15.78:0/2882160344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c086040 con 0x5628ca291370 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.948+0000 7f97717fa640 1 -- 172.21.15.78:0/2882160344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c00f740 con 0x5628ca291370 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 -- 172.21.15.78:0/2882160344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 msgr2=0x5628ca2a8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 --2- 172.21.15.78:0/2882160344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca2a8d60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f976c098290 tx=0x7f976c09b870 comp rx=0 tx=0).stop 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 -- 172.21.15.78:0/2882160344 shutdown_connections 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 --2- 172.21.15.78:0/2882160344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca291370 0x5628ca2a8d60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 --2- 172.21.15.78:0/2882160344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca287b40 0x5628ca28cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 --2- 172.21.15.78:0/2882160344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca2865d0 0x5628ca29d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 -- 172.21.15.78:0/2882160344 >> 172.21.15.78:0/2882160344 conn(0x5628ca24f310 msgr2=0x5628ca28c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 -- 172.21.15.78:0/2882160344 shutdown_connections 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 -- 172.21.15.78:0/2882160344 wait complete. 2024-09-16T00:14:26.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.949+0000 7f9783e5e740 1 -- start start 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f9783e5e740 1 Processor -- start 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca2865d0 0x5628ca299a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 0x5628ca28ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f9783e5e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f9783e5e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b21f0 con 0x5628ca2865d0 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f9783e5e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1b1b20 con 0x5628ca291370 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f9783e5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f977c1aed00 con 0x5628ca287b40 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f9783e5e740 10 client.0 _ll_register_callbacks cb 0x7f97640037f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca2865d0 0x5628ca299a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca2865d0 0x5628ca299a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47866/0 (socket says 172.21.15.78:47866) 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 -- 172.21.15.78:0/1696113900 learned_addr learned my addr 172.21.15.78:0/1696113900 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977affd640 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 0x5628ca28ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977bfff640 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 -- 172.21.15.78:0/1696113900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 msgr2=0x5628ca28ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 0x5628ca28ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 -- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 msgr2=0x5628ca28b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 -- 172.21.15.78:0/1696113900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628ca28b6e0 con 0x5628ca2865d0 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977bfff640 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f977b7fe640 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca2865d0 0x5628ca299a90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9774009fa0 tx=0x7f9774009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b003818dac519d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.950+0000 7f975affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f975affd640 1 -- 172.21.15.78:0/1696113900 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977400f690 con 0x5628ca2865d0 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f975affd640 1 -- 172.21.15.78:0/1696113900 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f977400f830 con 0x5628ca2865d0 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 1 -- 172.21.15.78:0/1696113900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976c086040 con 0x5628ca2865d0 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f975affd640 1 -- 172.21.15.78:0/1696113900 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f977400fa50 con 0x5628ca2865d0 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 10 client.15603 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 1 -- 172.21.15.78:0/1696113900 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628ca30bba0 con 0x5628ca2865d0 2024-09-16T00:14:26.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f975affd640 1 -- 172.21.15.78:0/1696113900 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c086040 con 0x5628ca2865d0 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f975affd640 1 -- 172.21.15.78:0/1696113900 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f97740057a0 con 0x5628ca2865d0 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 10 client.15603 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 10 client.15603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 20 client.15603 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5628ca30c160 0x5628ca30e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 4 client.15603 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.951+0000 7f9783e5e740 1 -- 172.21.15.78:0/1696113900 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628ca1f3d70 con 0x5628ca30c160 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f977affd640 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5628ca30c160 0x5628ca30e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f977affd640 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5628ca30c160 0x5628ca30e580 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e5cafb494b6853f server_cookie=b947e273d7ab9cc6 in_seq=0 out_seq=0 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f975affd640 10 client.15603 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f975affd640 1 -- 172.21.15.78:0/1696113900 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628ca1f3d70 con 0x5628ca30c160 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f975affd640 10 client.15603 handle_command_reply: tid=0 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 client.15603 shutdown 2024-09-16T00:14:26.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 -- 172.21.15.78:0/1696113900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca2865d0 msgr2=0x5628ca299a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca2865d0 0x5628ca299a90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9774009fa0 tx=0x7f9774009760 comp rx=0 tx=0).stop 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 -- 172.21.15.78:0/1696113900 shutdown_connections 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5628ca30c160 0x5628ca30e580 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5628ca291370 0x5628ca28b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f975affd640 0 client.15603 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628ca287b40 0x5628ca28ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 --2- 172.21.15.78:0/1696113900 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5628ca2865d0 0x5628ca299a90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 -- 172.21.15.78:0/1696113900 >> 172.21.15.78:0/1696113900 conn(0x5628ca24f310 msgr2=0x5628ca28d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 -- 172.21.15.78:0/1696113900 shutdown_connections 2024-09-16T00:14:26.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 1 -- 172.21.15.78:0/1696113900 wait complete. 2024-09-16T00:14:26.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.952+0000 7f9783e5e740 20 client.15603 trim_cache size 0 max 16384 2024-09-16T00:14:26.726 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:14:26.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 -- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97680a32d0 msgr2=0x7f97680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97680a32d0 0x7f97680a5790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9764005770 tx=0x7f976402a480 comp rx=0 tx=0).stop 2024-09-16T00:14:26.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 -- 172.21.15.78:0/3771542427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f977c102ef0 msgr2=0x7f977c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:26.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f977c102ef0 0x7f977c0fd700 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c970 tx=0x7f976c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:14:26.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 -- 172.21.15.78:0/3771542427 shutdown_connections 2024-09-16T00:14:26.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f97680a32d0 0x7f97680a5790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f977c102ef0 0x7f977c0fd700 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f977c102570 0x7f977c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 --2- 172.21.15.78:0/3771542427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f977c100e20 0x7f977c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:26.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 -- 172.21.15.78:0/3771542427 >> 172.21.15.78:0/3771542427 conn(0x7f977c0fc7d0 msgr2=0x7f977c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:26.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 -- 172.21.15.78:0/3771542427 shutdown_connections 2024-09-16T00:14:26.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:25.953+0000 7f9781a85640 1 -- 172.21.15.78:0/3771542427 wait complete. 2024-09-16T00:14:27.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:14:27.767 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:27 smithi078 ceph-mon[31157]: pgmap v392: 129 pgs: 129 active+clean; 962 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 894 op/s 2024-09-16T00:14:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:27 smithi193 ceph-mon[35484]: pgmap v392: 129 pgs: 129 active+clean; 962 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 894 op/s 2024-09-16T00:14:27.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:27 smithi148 ceph-mon[35599]: pgmap v392: 129 pgs: 129 active+clean; 962 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 20 MiB/s wr, 894 op/s 2024-09-16T00:14:28.087 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '56b5046b', quiesce took 14.8 seconds. Will release after: 15.399999999999999 2024-09-16T00:14:28.160 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 -- 172.21.15.78:0/3171844082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 msgr2=0x7fa52010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 --2- 172.21.15.78:0/3171844082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 0x7fa52010da40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa518009ca0 tx=0x7fa51802a950 comp rx=0 tx=0).stop 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 -- 172.21.15.78:0/3171844082 shutdown_connections 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 --2- 172.21.15.78:0/3171844082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 0x7fa52010da40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 --2- 172.21.15.78:0/3171844082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa520102570 0x7fa5201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 --2- 172.21.15.78:0/3171844082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520100e20 0x7fa520101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 -- 172.21.15.78:0/3171844082 >> 172.21.15.78:0/3171844082 conn(0x7fa5200fc7d0 msgr2=0x7fa5200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 -- 172.21.15.78:0/3171844082 shutdown_connections 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 -- 172.21.15.78:0/3171844082 wait complete. 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 Processor -- start 2024-09-16T00:14:28.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.677+0000 7fa5265f2640 1 -- start start 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa5265f2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520100e20 0x7fa5200fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa5265f2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa520102570 0x7fa5200fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa5265f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 0x7fa5200fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa5265f2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa520110f80 con 0x7fa520102ef0 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa5265f2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa520110e00 con 0x7fa520100e20 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa5265f2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111100 con 0x7fa520102570 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 0x7fa5200fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 0x7fa5200fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58120/0 (socket says 172.21.15.78:58120) 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 -- 172.21.15.78:0/425071978 learned_addr learned my addr 172.21.15.78:0/425071978 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa51ffff640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520100e20 0x7fa5200fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa520102570 0x7fa5200fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 -- 172.21.15.78:0/425071978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa520102570 msgr2=0x7fa5200fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa520102570 0x7fa5200fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 -- 172.21.15.78:0/425071978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520100e20 msgr2=0x7fa5200fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520100e20 0x7fa5200fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 -- 172.21.15.78:0/425071978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5201a6890 con 0x7fa520102ef0 2024-09-16T00:14:28.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.678+0000 7fa524b8f640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 0x7fa5200fd700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa518006010 tx=0x7fa51802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:28.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.679+0000 7fa51d7fa640 1 -- 172.21.15.78:0/425071978 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518034040 con 0x7fa520102ef0 2024-09-16T00:14:28.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.679+0000 7fa5265f2640 1 -- 172.21.15.78:0/425071978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5201a6b20 con 0x7fa520102ef0 2024-09-16T00:14:28.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.679+0000 7fa51d7fa640 1 -- 172.21.15.78:0/425071978 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa518003e00 con 0x7fa520102ef0 2024-09-16T00:14:28.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.679+0000 7fa51d7fa640 1 -- 172.21.15.78:0/425071978 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518045480 con 0x7fa520102ef0 2024-09-16T00:14:28.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.679+0000 7fa5265f2640 1 -- 172.21.15.78:0/425071978 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201a7050 con 0x7fa520102ef0 2024-09-16T00:14:28.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa51d7fa640 1 -- 172.21.15.78:0/425071978 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fa518037020 con 0x7fa520102ef0 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa5289cb740 1 Processor -- start 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa5289cb740 1 -- start start 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 0x5562f3c64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c71760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa5289cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5562f3bca2f0 con 0x5562f3c6f370 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa5289cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562f3b71cf0 con 0x5562f3c645d0 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa5289cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5562f3bc62b0 con 0x5562f3c65b40 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c71760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c71760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58132/0 (socket says 172.21.15.78:58132) 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa51f7fe640 1 -- 172.21.15.78:0/1099375588 learned_addr learned my addr 172.21.15.78:0/1099375588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa51d7fa640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa50c0a32d0 0x7fa50c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa51ffff640 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 0x5562f3c64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.682+0000 7fa524b8f640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa50c0a32d0 0x7fa50c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa51f7fe640 1 -- 172.21.15.78:0/1099375588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 msgr2=0x5562f3c64090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c64090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa51f7fe640 1 -- 172.21.15.78:0/1099375588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 msgr2=0x5562f3c64a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 0x5562f3c64a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa51f7fe640 1 -- 172.21.15.78:0/1099375588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f3b563a0 con 0x5562f3c6f370 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa51d7fa640 1 -- 172.21.15.78:0/425071978 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa5180f0dc0 con 0x7fa520102ef0 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c71760 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa508010370 tx=0x7fa50800c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6802af5370801b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa524b8f640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa50c0a32d0 0x7fa50c0a5790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5180b9490 tx=0x7fa51808cca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa5167fc640 1 -- 172.21.15.78:0/1099375588 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508018030 con 0x5562f3c6f370 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa5167fc640 1 -- 172.21.15.78:0/1099375588 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa508005500 con 0x5562f3c6f370 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.683+0000 7fa5167fc640 1 -- 172.21.15.78:0/1099375588 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508012660 con 0x5562f3c6f370 2024-09-16T00:14:28.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 -- 172.21.15.78:0/1099375588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 msgr2=0x5562f3c71760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c71760 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa508010370 tx=0x7fa50800c540 comp rx=0 tx=0).stop 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 -- 172.21.15.78:0/1099375588 shutdown_connections 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c71760 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 0x5562f3c64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 --2- 172.21.15.78:0/1099375588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c64090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 -- 172.21.15.78:0/1099375588 >> 172.21.15.78:0/1099375588 conn(0x5562f3c2d310 msgr2=0x5562f3c60c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 -- 172.21.15.78:0/1099375588 shutdown_connections 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 -- 172.21.15.78:0/1099375588 wait complete. 2024-09-16T00:14:28.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 -- start start 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.684+0000 7fa5289cb740 1 Processor -- start 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c645d0 0x5562f3c6ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c6c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c6f370 0x5562f3c6c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa5289cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5562f3c72070 con 0x5562f3c645d0 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa5289cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562f3b85060 con 0x5562f3c6f370 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa5289cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5562f3c721f0 con 0x5562f3c65b40 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa5289cb740 10 client.0 _ll_register_callbacks cb 0x5562f38d88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c645d0 0x5562f3c6ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c645d0 0x5562f3c6ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58142/0 (socket says 172.21.15.78:58142) 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 -- 172.21.15.78:0/155554197 learned_addr learned my addr 172.21.15.78:0/155554197 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:28.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51ffff640 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c6c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c6f370 0x5562f3c6c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 -- 172.21.15.78:0/155554197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 msgr2=0x5562f3c6c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c6c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 -- 172.21.15.78:0/155554197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c6f370 msgr2=0x5562f3c6c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c6f370 0x5562f3c6c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 -- 172.21.15.78:0/155554197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f393b3c0 con 0x5562f3c645d0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c645d0 0x5562f3c6ed60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa50800d010 tx=0x7fa508005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6325f6d5167142c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa4ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.685+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/155554197 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508018040 con 0x5562f3c645d0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/155554197 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa508005200 con 0x5562f3c645d0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 1 -- 172.21.15.78:0/155554197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa508005500 con 0x5562f3c645d0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/155554197 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa508004020 con 0x5562f3c645d0 2024-09-16T00:14:28.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 10 client.15615 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 1 -- 172.21.15.78:0/155554197 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5562f3c7db40 con 0x5562f3c645d0 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/155554197 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa508005500 con 0x5562f3c645d0 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/155554197 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fa508012cd0 con 0x5562f3c645d0 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 10 client.15615 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 10 client.15615 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 20 client.15615 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5562f3c7ee90 0x5562f3c81280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 4 client.15615 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa5289cb740 1 -- 172.21.15.78:0/155554197 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562f3bd7860 con 0x5562f3c7ee90 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.686+0000 7fa51ffff640 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5562f3c7ee90 0x5562f3c81280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.687+0000 7fa4ff7fe640 10 client.15615 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.687+0000 7fa51ffff640 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5562f3c7ee90 0x5562f3c81280 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8698a1e802fadaab server_cookie=748ad6d3bc125512 in_seq=0 out_seq=0 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/155554197 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562f3bd7860 con 0x5562f3c7ee90 2024-09-16T00:14:28.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa4ff7fe640 10 client.15615 handle_command_reply: tid=0 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 client.15615 shutdown 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 -- 172.21.15.78:0/155554197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c645d0 msgr2=0x5562f3c6ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c645d0 0x5562f3c6ed60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa50800d010 tx=0x7fa508005ce0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 -- 172.21.15.78:0/155554197 shutdown_connections 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5562f3c7ee90 0x5562f3c81280 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c6f370 0x5562f3c6c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c6c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 --2- 172.21.15.78:0/155554197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c645d0 0x5562f3c6ed60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 -- 172.21.15.78:0/155554197 >> 172.21.15.78:0/155554197 conn(0x5562f3c2d310 msgr2=0x5562f3c66960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa4ff7fe640 0 client.15615 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 -- 172.21.15.78:0/155554197 shutdown_connections 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 1 -- 172.21.15.78:0/155554197 wait complete. 2024-09-16T00:14:28.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.688+0000 7fa5289cb740 20 client.15615 trim_cache size 0 max 16384 2024-09-16T00:14:28.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa5289cb740 1 Processor -- start 2024-09-16T00:14:28.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa5289cb740 1 -- start start 2024-09-16T00:14:28.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c645d0 0x5562f3c7b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c65b40 0x5562f3c6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c86a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa5289cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201b0cc0 con 0x5562f3c6f370 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa5289cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201b2890 con 0x5562f3c65b40 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa5289cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201b1390 con 0x5562f3c645d0 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c86a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c86a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58146/0 (socket says 172.21.15.78:58146) 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 -- 172.21.15.78:0/1737088448 learned_addr learned my addr 172.21.15.78:0/1737088448 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51ffff640 1 --2- 172.21.15.78:0/1737088448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c65b40 0x5562f3c6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 -- 172.21.15.78:0/1737088448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c645d0 msgr2=0x5562f3c7b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/1737088448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c645d0 0x5562f3c7b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 -- 172.21.15.78:0/1737088448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c65b40 msgr2=0x5562f3c6b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/1737088448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c65b40 0x5562f3c6b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 -- 172.21.15.78:0/1737088448 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f3c65300 con 0x5562f3c6f370 2024-09-16T00:14:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.701+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/1737088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c86a10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa508001eb0 tx=0x7fa50800c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aab5958644bc2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa515ffb640 1 -- 172.21.15.78:0/1737088448 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508018070 con 0x5562f3c6f370 2024-09-16T00:14:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa515ffb640 1 -- 172.21.15.78:0/1737088448 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5080041c0 con 0x5562f3c6f370 2024-09-16T00:14:28.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa515ffb640 1 -- 172.21.15.78:0/1737088448 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508012660 con 0x5562f3c6f370 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 -- 172.21.15.78:0/1737088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 msgr2=0x5562f3c86a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 --2- 172.21.15.78:0/1737088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c86a10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa508001eb0 tx=0x7fa50800c780 comp rx=0 tx=0).stop 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 -- 172.21.15.78:0/1737088448 shutdown_connections 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 --2- 172.21.15.78:0/1737088448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c86a10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 --2- 172.21.15.78:0/1737088448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c65b40 0x5562f3c6b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 --2- 172.21.15.78:0/1737088448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c645d0 0x5562f3c7b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 -- 172.21.15.78:0/1737088448 >> 172.21.15.78:0/1737088448 conn(0x5562f3c2d310 msgr2=0x5562f3c6a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 -- 172.21.15.78:0/1737088448 shutdown_connections 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.702+0000 7fa5289cb740 1 -- 172.21.15.78:0/1737088448 wait complete. 2024-09-16T00:14:28.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 1 -- start start 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 1 Processor -- start 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 0x5562f3c774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201b2190 con 0x5562f3c6f370 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201b1ac0 con 0x5562f3c645d0 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201aeca0 con 0x5562f3c65b40 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa5289cb740 10 client.0 _ll_register_callbacks cb 0x7fa50800ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58156/0 (socket says 172.21.15.78:58156) 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa51f7fe640 1 -- 172.21.15.78:0/259097511 learned_addr learned my addr 172.21.15.78:0/259097511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:28.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa51ffff640 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.703+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 0x5562f3c774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa51f7fe640 1 -- 172.21.15.78:0/259097511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 msgr2=0x5562f3c779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c779f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa51f7fe640 1 -- 172.21.15.78:0/259097511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 msgr2=0x5562f3c774b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 0x5562f3c774b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa51f7fe640 1 -- 172.21.15.78:0/259097511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f3c68be0 con 0x5562f3c6f370 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa51f7fe640 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c686a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa508003610 tx=0x7fa508037040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02774d71c35939f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa4ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/259097511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508018040 con 0x5562f3c6f370 2024-09-16T00:14:28.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/259097511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa50803e040 con 0x5562f3c6f370 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 1 -- 172.21.15.78:0/259097511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562f3bd1d70 con 0x5562f3c6f370 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/259097511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa508003d50 con 0x5562f3c6f370 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 10 client.15621 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 1 -- 172.21.15.78:0/259097511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5562f3ce9220 con 0x5562f3c6f370 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/259097511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5562f3bd1d70 con 0x5562f3c6f370 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/259097511 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fa508042480 con 0x5562f3c6f370 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 10 client.15621 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 10 client.15621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 20 client.15621 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5562f3ce9880 0x5562f3cebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:28.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 4 client.15621 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.704+0000 7fa5289cb740 1 -- 172.21.15.78:0/259097511 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562f3b4e170 con 0x5562f3ce9880 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa51ffff640 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5562f3ce9880 0x5562f3cebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa51ffff640 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5562f3ce9880 0x5562f3cebc70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1ea39d2e94f077 server_cookie=4539f4c82b04676b in_seq=0 out_seq=0 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa4ff7fe640 10 client.15621 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa4ff7fe640 1 -- 172.21.15.78:0/259097511 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562f3b4e170 con 0x5562f3ce9880 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa4ff7fe640 10 client.15621 handle_command_reply: tid=0 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa5289cb740 1 client.15621 shutdown 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa5289cb740 1 -- 172.21.15.78:0/259097511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 msgr2=0x5562f3c686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa5289cb740 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c686a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa508003610 tx=0x7fa508037040 comp rx=0 tx=0).stop 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.705+0000 7fa5289cb740 1 -- 172.21.15.78:0/259097511 shutdown_connections 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5289cb740 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5562f3ce9880 0x5562f3cebc70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5289cb740 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5562f3c6f370 0x5562f3c686a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5289cb740 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562f3c65b40 0x5562f3c779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa4ff7fe640 0 client.15621 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:14:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5289cb740 1 --2- 172.21.15.78:0/259097511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562f3c645d0 0x5562f3c774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5289cb740 1 -- 172.21.15.78:0/259097511 >> 172.21.15.78:0/259097511 conn(0x5562f3c2d310 msgr2=0x5562f3c6b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5289cb740 1 -- 172.21.15.78:0/259097511 shutdown_connections 2024-09-16T00:14:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5289cb740 1 -- 172.21.15.78:0/259097511 wait complete. 2024-09-16T00:14:28.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5289cb740 20 client.15621 trim_cache size 0 max 16384 2024-09-16T00:14:28.718 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:14:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 -- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa50c0a32d0 msgr2=0x7fa50c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa50c0a32d0 0x7fa50c0a5790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa5180b9490 tx=0x7fa51808cca0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 -- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 msgr2=0x7fa5200fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 0x7fa5200fd700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa518006010 tx=0x7fa51802f6f0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 -- 172.21.15.78:0/425071978 shutdown_connections 2024-09-16T00:14:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa50c0a32d0 0x7fa50c0a5790 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa520102ef0 0x7fa5200fd700 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa520102570 0x7fa5200fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 --2- 172.21.15.78:0/425071978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520100e20 0x7fa5200fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 -- 172.21.15.78:0/425071978 >> 172.21.15.78:0/425071978 conn(0x7fa5200fc7d0 msgr2=0x7fa52006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 -- 172.21.15.78:0/425071978 shutdown_connections 2024-09-16T00:14:28.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:28.706+0000 7fa5265f2640 1 -- 172.21.15.78:0/425071978 wait complete. 2024-09-16T00:14:29.678 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:29 smithi078 ceph-mon[31157]: pgmap v393: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-09-16T00:14:29.708 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:14:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:29 smithi193 ceph-mon[35484]: pgmap v393: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-09-16T00:14:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:29 smithi148 ceph-mon[35599]: pgmap v393: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 22 MiB/s wr, 1.04k op/s 2024-09-16T00:14:29.955 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:30.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 -- 172.21.15.78:0/829852702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 msgr2=0x7f4b88100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 --2- 172.21.15.78:0/829852702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 0x7f4b88100ee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009930 tx=0x7f4b7002a420 comp rx=0 tx=0).stop 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 -- 172.21.15.78:0/829852702 shutdown_connections 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 --2- 172.21.15.78:0/829852702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88101420 0x7f4b8810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 --2- 172.21.15.78:0/829852702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 0x7f4b88100ee0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 --2- 172.21.15.78:0/829852702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b88106a80 0x7f4b88106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 -- 172.21.15.78:0/829852702 >> 172.21.15.78:0/829852702 conn(0x7f4b880fc7a0 msgr2=0x7f4b880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 -- 172.21.15.78:0/829852702 shutdown_connections 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 -- 172.21.15.78:0/829852702 wait complete. 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.420+0000 7f4b8d746640 1 Processor -- start 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b8d746640 1 -- start start 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b8d746640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 0x7f4b881984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b8d746640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b88101420 0x7f4b881989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b8d746640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88106a80 0x7f4b8819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b8d746640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88110ea0 con 0x7f4b88100a80 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b8d746640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88110d20 con 0x7f4b88106a80 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b8d746640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88111020 con 0x7f4b88101420 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 0x7f4b881984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 0x7f4b881984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58170/0 (socket says 172.21.15.78:58170) 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 -- 172.21.15.78:0/4203635574 learned_addr learned my addr 172.21.15.78:0/4203635574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b877fe640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88106a80 0x7f4b8819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b867fc640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b88101420 0x7f4b881989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 -- 172.21.15.78:0/4203635574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b88101420 msgr2=0x7f4b881989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b88101420 0x7f4b881989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 -- 172.21.15.78:0/4203635574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88106a80 msgr2=0x7f4b8819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88106a80 0x7f4b8819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 -- 172.21.15.78:0/4203635574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8819d460 con 0x7f4b88100a80 2024-09-16T00:14:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 0x7f4b881984b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800b4a0 tx=0x7f4b7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.421+0000 7f4b7ffff640 1 -- 172.21.15.78:0/4203635574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78014070 con 0x7f4b88100a80 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.422+0000 7f4b8d746640 1 -- 172.21.15.78:0/4203635574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8819d750 con 0x7f4b88100a80 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.422+0000 7f4b7ffff640 1 -- 172.21.15.78:0/4203635574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b78010af0 con 0x7f4b88100a80 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.422+0000 7f4b7ffff640 1 -- 172.21.15.78:0/4203635574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78017d10 con 0x7f4b88100a80 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.422+0000 7f4b8d746640 1 -- 172.21.15.78:0/4203635574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a5030 con 0x7f4b88100a80 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b7ffff640 1 -- 172.21.15.78:0/4203635574 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f4b78017070 con 0x7f4b88100a80 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b8fb1f740 1 Processor -- start 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b8fb1f740 1 -- start start 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bbd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbd5d0 0x562c53bbda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 0x562c53bca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c53b232f0 con 0x562c53bbeb40 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562c53acacf0 con 0x562c53bc8370 2024-09-16T00:14:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562c53b1f2b0 con 0x562c53bbd5d0 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.424+0000 7f4b877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bbd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bbd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58174/0 (socket says 172.21.15.78:58174) 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 -- 172.21.15.78:0/3101320810 learned_addr learned my addr 172.21.15.78:0/3101320810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbd5d0 0x562c53bbda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b867fc640 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 0x562c53bca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 -- 172.21.15.78:0/3101320810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbd5d0 msgr2=0x562c53bbda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbd5d0 0x562c53bbda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 -- 172.21.15.78:0/3101320810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 msgr2=0x562c53bca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 0x562c53bca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b7ffff640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4b740a32d0 0x7f4b740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 -- 172.21.15.78:0/3101320810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c53aaf3a0 con 0x562c53bbeb40 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4b740a32d0 0x7f4b740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b7ffff640 1 -- 172.21.15.78:0/4203635574 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4b780cf0c0 con 0x7f4b88100a80 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b877fe640 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bbd090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b80010370 tx=0x7f4b80010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192adb39b492683c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b7d7fa640 1 -- 172.21.15.78:0/3101320810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80022070 con 0x562c53bbeb40 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b7d7fa640 1 -- 172.21.15.78:0/3101320810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b800113e0 con 0x562c53bbeb40 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.425+0000 7f4b7d7fa640 1 -- 172.21.15.78:0/3101320810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80016070 con 0x562c53bbeb40 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/3101320810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 msgr2=0x562c53bbd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bbd090 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b80010370 tx=0x7f4b80010830 comp rx=0 tx=0).stop 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b877fe640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4b740a32d0 0x7f4b740a5790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4b8001f000 tx=0x7f4b80022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/3101320810 shutdown_connections 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 0x562c53bca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbd5d0 0x562c53bbda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/3101320810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bbd090 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/3101320810 >> 172.21.15.78:0/3101320810 conn(0x562c53b86310 msgr2=0x562c53bb9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/3101320810 shutdown_connections 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.426+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/3101320810 wait complete. 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 1 -- start start 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 1 Processor -- start 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bc7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 0x562c53bc5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bc8370 0x562c53bc5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c53bcb070 con 0x562c53bc8370 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562c53ade060 con 0x562c53bbd5d0 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562c53bcb1f0 con 0x562c53bbeb40 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b8fb1f740 10 client.0 _ll_register_callbacks cb 0x562c538318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bc8370 0x562c53bc5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b877fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bc8370 0x562c53bc5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58190/0 (socket says 172.21.15.78:58190) 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b877fe640 1 -- 172.21.15.78:0/2918431958 learned_addr learned my addr 172.21.15.78:0/2918431958 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bc7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b867fc640 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 0x562c53bc5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b877fe640 1 -- 172.21.15.78:0/2918431958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 msgr2=0x562c53bc5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b877fe640 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 0x562c53bc5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b877fe640 1 -- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 msgr2=0x562c53bc7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b877fe640 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bc7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b877fe640 1 -- 172.21.15.78:0/2918431958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c538943c0 con 0x562c53bc8370 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.427+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bc7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b877fe640 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bc8370 0x562c53bc5640 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b80011e30 tx=0x7f4b80012aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda4e672d39ad9ac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:30.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/2918431958 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80016050 con 0x562c53bc8370 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/2918431958 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4b80011590 con 0x562c53bc8370 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/2918431958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b800113e0 con 0x562c53bc8370 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/2918431958 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4b8003a050 con 0x562c53bc8370 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 10 client.15633 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/2918431958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c53bd6b40 con 0x562c53bc8370 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/2918431958 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b800113e0 con 0x562c53bc8370 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/2918431958 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f4b8002e030 con 0x562c53bc8370 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 10 client.15633 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 20 client.15633 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c53bd7f40 0x562c53bda330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 4 client.15633 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:30.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.428+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/2918431958 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c53b30860 con 0x562c53bd7f40 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.429+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c53bd7f40 0x562c53bda330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.429+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c53bd7f40 0x562c53bda330 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432009304bf78353 server_cookie=dbe0d5791c3e53a7 in_seq=0 out_seq=0 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.429+0000 7f4b6e7fc640 10 client.15633 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/2918431958 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c53b30860 con 0x562c53bd7f40 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b6e7fc640 10 client.15633 handle_command_reply: tid=0 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 client.15633 shutdown 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/2918431958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bc8370 msgr2=0x562c53bc5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bc8370 0x562c53bc5640 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b80011e30 tx=0x7f4b80012aa0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/2918431958 shutdown_connections 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c53bd7f40 0x562c53bda330 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bc8370 0x562c53bc5640 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 0x562c53bc5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/2918431958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bc7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b6e7fc640 0 client.15633 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:30.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/2918431958 >> 172.21.15.78:0/2918431958 conn(0x562c53b86310 msgr2=0x562c53bbf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:30.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/2918431958 shutdown_connections 2024-09-16T00:14:30.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/2918431958 wait complete. 2024-09-16T00:14:30.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.430+0000 7f4b8fb1f740 20 client.15633 trim_cache size 0 max 16384 2024-09-16T00:14:30.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b8fb1f740 1 Processor -- start 2024-09-16T00:14:30.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b8fb1f740 1 -- start start 2024-09-16T00:14:30.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbd5d0 0x562c53bc4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 0x562c53bc6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 0x562c53bc0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881aecb0 con 0x562c53bbd5d0 2024-09-16T00:14:30.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881b0880 con 0x562c53bc8370 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881af380 con 0x562c53bbeb40 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbd5d0 0x562c53bc4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbd5d0 0x562c53bc4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58206/0 (socket says 172.21.15.78:58206) 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 -- 172.21.15.78:0/1421753823 learned_addr learned my addr 172.21.15.78:0/1421753823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b877fe640 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 0x562c53bc0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b867fc640 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 0x562c53bc6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 -- 172.21.15.78:0/1421753823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 msgr2=0x562c53bc6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 0x562c53bc6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 -- 172.21.15.78:0/1421753823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 msgr2=0x562c53bc0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 0x562c53bc0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 -- 172.21.15.78:0/1421753823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c53bc7740 con 0x562c53bbd5d0 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbd5d0 0x562c53bc4910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4b780cc850 tx=0x7f4b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ba5d472f9e5c63 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b7cff9640 1 -- 172.21.15.78:0/1421753823 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7801e040 con 0x562c53bbd5d0 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b7cff9640 1 -- 172.21.15.78:0/1421753823 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7809f6e0 con 0x562c53bbd5d0 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.444+0000 7f4b7cff9640 1 -- 172.21.15.78:0/1421753823 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b780979a0 con 0x562c53bbd5d0 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/1421753823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbd5d0 msgr2=0x562c53bc4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbd5d0 0x562c53bc4910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4b780cc850 tx=0x7f4b780046c0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/1421753823 shutdown_connections 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bc8370 0x562c53bc0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bbeb40 0x562c53bc6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/1421753823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbd5d0 0x562c53bc4910 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/1421753823 >> 172.21.15.78:0/1421753823 conn(0x562c53b86310 msgr2=0x562c53bb8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/1421753823 shutdown_connections 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/1421753823 wait complete. 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 -- start start 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.445+0000 7f4b8fb1f740 1 Processor -- start 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bd0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bcbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b8fb1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bc8370 0x562c53bcc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881b0180 con 0x562c53bbeb40 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881afab0 con 0x562c53bbd5d0 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b8fb1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881acc90 con 0x562c53bc8370 2024-09-16T00:14:30.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b8fb1f740 10 client.0 _ll_register_callbacks cb 0x7f4b80020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bcbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bcbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58214/0 (socket says 172.21.15.78:58214) 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 -- 172.21.15.78:0/473945324 learned_addr learned my addr 172.21.15.78:0/473945324 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bd0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b877fe640 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bc8370 0x562c53bcc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 -- 172.21.15.78:0/473945324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bc8370 msgr2=0x562c53bcc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bc8370 0x562c53bcc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 -- 172.21.15.78:0/473945324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 msgr2=0x562c53bd0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bd0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 -- 172.21.15.78:0/473945324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c53bccbf0 con 0x562c53bbeb40 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b867fc640 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bcbe80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4b70031550 tx=0x7f4b700317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c957614dea298b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.446+0000 7f4b6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:30.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/473945324 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70035050 con 0x562c53bbeb40 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/473945324 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4b70004250 con 0x562c53bbeb40 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/473945324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7809f6e0 con 0x562c53bbeb40 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/473945324 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4b7004d050 con 0x562c53bbeb40 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 10 client.15639 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/473945324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562c53c46e80 con 0x562c53bbeb40 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/473945324 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b7809f6e0 con 0x562c53bbeb40 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/473945324 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f4b70004ac0 con 0x562c53bbeb40 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 10 client.15639 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 20 client.15639 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c53c47660 0x562c53c49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 4 client.15639 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:14:30.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.447+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/473945324 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c53b2ad70 con 0x562c53c47660 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c53c47660 0x562c53c49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b86ffd640 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c53c47660 0x562c53c49a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e0710256b5ffe0 server_cookie=e4979aafa7fe41d8 in_seq=0 out_seq=0 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b6e7fc640 10 client.15639 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b6e7fc640 1 -- 172.21.15.78:0/473945324 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c53b2ad70 con 0x562c53c47660 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b6e7fc640 10 client.15639 handle_command_reply: tid=0 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b8fb1f740 1 client.15639 shutdown 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/473945324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 msgr2=0x562c53bcbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bcbe80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4b70031550 tx=0x7f4b700317e0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/473945324 shutdown_connections 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c53c47660 0x562c53c49a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.448+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c53bc8370 0x562c53bcc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c53bbeb40 0x562c53bcbe80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8fb1f740 1 --2- 172.21.15.78:0/473945324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c53bbd5d0 0x562c53bd0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/473945324 >> 172.21.15.78:0/473945324 conn(0x562c53b86310 msgr2=0x562c53bc3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:30.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b6e7fc640 0 client.15639 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/473945324 shutdown_connections 2024-09-16T00:14:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8fb1f740 1 -- 172.21.15.78:0/473945324 wait complete. 2024-09-16T00:14:30.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8fb1f740 20 client.15639 trim_cache size 0 max 16384 2024-09-16T00:14:30.462 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 -- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4b740a32d0 msgr2=0x7f4b740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4b740a32d0 0x7f4b740a5790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4b8001f000 tx=0x7f4b80022000 comp rx=0 tx=0).stop 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 -- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 msgr2=0x7f4b881984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 0x7f4b881984b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800b4a0 tx=0x7f4b7800b960 comp rx=0 tx=0).stop 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 -- 172.21.15.78:0/4203635574 shutdown_connections 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4b740a32d0 0x7f4b740a5790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4b88106a80 0x7f4b8819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b88101420 0x7f4b881989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 --2- 172.21.15.78:0/4203635574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b88100a80 0x7f4b881984b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:30.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 -- 172.21.15.78:0/4203635574 >> 172.21.15.78:0/4203635574 conn(0x7f4b880fc7a0 msgr2=0x7f4b88104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:30.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.449+0000 7f4b8d746640 1 -- 172.21.15.78:0/4203635574 shutdown_connections 2024-09-16T00:14:30.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:30.450+0000 7f4b8d746640 1 -- 172.21.15.78:0/4203635574 wait complete. 2024-09-16T00:14:30.860 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 30.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:14:31.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:31 smithi193 ceph-mon[35484]: pgmap v394: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 16 MiB/s wr, 748 op/s 2024-09-16T00:14:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:31 smithi148 ceph-mon[35599]: pgmap v394: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 16 MiB/s wr, 748 op/s 2024-09-16T00:14:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:31 smithi078 ceph-mon[31157]: pgmap v394: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 16 MiB/s wr, 748 op/s 2024-09-16T00:14:32.190 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:14:32.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:14:32.444 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 -- 172.21.15.78:0/2964872236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4101450 msgr2=0x7f86a4105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 --2- 172.21.15.78:0/2964872236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4101450 0x7f86a4105920 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8694009900 tx=0x7f869402a470 comp rx=0 tx=0).stop 2024-09-16T00:14:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 -- 172.21.15.78:0/2964872236 shutdown_connections 2024-09-16T00:14:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 --2- 172.21.15.78:0/2964872236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a4105f40 0x7f86a41063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 --2- 172.21.15.78:0/2964872236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4101450 0x7f86a4105920 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 --2- 172.21.15.78:0/2964872236 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86a4100b30 0x7f86a4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 -- 172.21.15.78:0/2964872236 >> 172.21.15.78:0/2964872236 conn(0x7f86a40fc7a0 msgr2=0x7f86a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 -- 172.21.15.78:0/2964872236 shutdown_connections 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.924+0000 7f86abf97640 1 -- 172.21.15.78:0/2964872236 wait complete. 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86abf97640 1 Processor -- start 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86abf97640 1 -- start start 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86abf97640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86a4100b30 0x7f86a41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86abf97640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a4101450 0x7f86a41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86abf97640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4105f40 0x7f86a419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86abf97640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4118560 con 0x7f86a4105f40 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86abf97640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41183e0 con 0x7f86a4100b30 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86abf97640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41186e0 con 0x7f86a4101450 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86aa534640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4105f40 0x7f86a419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86aa534640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4105f40 0x7f86a419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58238/0 (socket says 172.21.15.78:58238) 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86aa534640 1 -- 172.21.15.78:0/1848234674 learned_addr learned my addr 172.21.15.78:0/1848234674 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86a9d33640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86a4100b30 0x7f86a41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86aa534640 1 -- 172.21.15.78:0/1848234674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a4101450 msgr2=0x7f86a41a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86aa534640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a4101450 0x7f86a41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86aa534640 1 -- 172.21.15.78:0/1848234674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86a4100b30 msgr2=0x7f86a41a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86aa534640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86a4100b30 0x7f86a41a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86aa534640 1 -- 172.21.15.78:0/1848234674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a419efe0 con 0x7f86a4105f40 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.925+0000 7f86a9d33640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86a4100b30 0x7f86a41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.926+0000 7f86aa534640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4105f40 0x7f86a419e8b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f869c00cad0 tx=0x7f869c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.926+0000 7f869affd640 1 -- 172.21.15.78:0/1848234674 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c00ae20 con 0x7f86a4105f40 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.926+0000 7f869affd640 1 -- 172.21.15.78:0/1848234674 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c005020 con 0x7f86a4105f40 2024-09-16T00:14:32.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.926+0000 7f86abf97640 1 -- 172.21.15.78:0/1848234674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a419f2d0 con 0x7f86a4105f40 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.926+0000 7f869affd640 1 -- 172.21.15.78:0/1848234674 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c00f6f0 con 0x7f86a4105f40 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.926+0000 7f86abf97640 1 -- 172.21.15.78:0/1848234674 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a41ab5c0 con 0x7f86a4105f40 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.928+0000 7f869affd640 1 -- 172.21.15.78:0/1848234674 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f869c0051d0 con 0x7f86a4105f40 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.928+0000 7f86ae370740 1 Processor -- start 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86ae370740 1 -- start start 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86ae370740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c0b60 0x5614929c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86ae370740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 0x5614929cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86ae370740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929cbd40 0x5614929ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86ae370740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561492926280 con 0x5614929cbd40 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86ae370740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5614928cdb10 con 0x5614929c1480 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86ae370740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561492922240 con 0x5614929c0b60 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f869affd640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86900a32c0 0x7f86900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929cbd40 0x5614929ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929cbd40 0x5614929ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58252/0 (socket says 172.21.15.78:58252) 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 -- 172.21.15.78:0/3795688907 learned_addr learned my addr 172.21.15.78:0/3795688907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9d33640 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 0x5614929cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86aa534640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86900a32c0 0x7f86900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c0b60 0x5614929c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 -- 172.21.15.78:0/3795688907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c0b60 msgr2=0x5614929c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c0b60 0x5614929c0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 -- 172.21.15.78:0/3795688907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 msgr2=0x5614929cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 0x5614929cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f869affd640 1 -- 172.21.15.78:0/1848234674 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f869c0cffc0 con 0x7f86a4105f40 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 -- 172.21.15.78:0/3795688907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614928da1f0 con 0x5614929cbd40 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.929+0000 7f86a9532640 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929cbd40 0x5614929ce130 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8694031550 tx=0x7f8694005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dfcb37f7af1056 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:32.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f86aa534640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86900a32c0 0x7f86900a5780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f869c0923c0 tx=0x7f869c0670b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f868ffff640 1 -- 172.21.15.78:0/3795688907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694035070 con 0x5614929cbd40 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f868ffff640 1 -- 172.21.15.78:0/3795688907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8694044070 con 0x5614929cbd40 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f868ffff640 1 -- 172.21.15.78:0/3795688907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869403d680 con 0x5614929cbd40 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f86ae370740 1 -- 172.21.15.78:0/3795688907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929cbd40 msgr2=0x5614929ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f86ae370740 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929cbd40 0x5614929ce130 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8694031550 tx=0x7f8694005a20 comp rx=0 tx=0).stop 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f86ae370740 1 -- 172.21.15.78:0/3795688907 shutdown_connections 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f86ae370740 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929cbd40 0x5614929ce130 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f86ae370740 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 0x5614929cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f86ae370740 1 --2- 172.21.15.78:0/3795688907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c0b60 0x5614929c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.930+0000 7f86ae370740 1 -- 172.21.15.78:0/3795688907 >> 172.21.15.78:0/3795688907 conn(0x5614929bd0f0 msgr2=0x5614929bd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 -- 172.21.15.78:0/3795688907 shutdown_connections 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 -- 172.21.15.78:0/3795688907 wait complete. 2024-09-16T00:14:32.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 -- start start 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 Processor -- start 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c1480 0x5614929c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929cbd40 0x5614929c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5614929ce910 con 0x5614929c0b60 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5614928e0f10 con 0x5614929cbd40 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5614929cea90 con 0x5614929c1480 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86ae370740 10 client.0 _ll_register_callbacks cb 0x5614928a9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86a9532640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86a9532640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58260/0 (socket says 172.21.15.78:58260) 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.931+0000 7f86a9532640 1 -- 172.21.15.78:0/508118807 learned_addr learned my addr 172.21.15.78:0/508118807 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86a9d33640 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c1480 0x5614929c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86a9532640 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929cbd40 0x5614929c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86a9532640 1 -- 172.21.15.78:0/508118807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c1480 msgr2=0x5614929c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86a9532640 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c1480 0x5614929c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86a9532640 1 -- 172.21.15.78:0/508118807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929cbd40 msgr2=0x5614929c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86a9532640 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929cbd40 0x5614929c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86a9532640 1 -- 172.21.15.78:0/508118807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614926976a0 con 0x5614929c0b60 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86a9532640 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929c8c10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f86940056d0 tx=0x7f8694003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71864a3308cb486f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f868d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f868d7fa640 1 -- 172.21.15.78:0/508118807 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694035070 con 0x5614929c0b60 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f868d7fa640 1 -- 172.21.15.78:0/508118807 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8694004090 con 0x5614929c0b60 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86ae370740 1 -- 172.21.15.78:0/508118807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8694044070 con 0x5614929c0b60 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f868d7fa640 1 -- 172.21.15.78:0/508118807 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8694004290 con 0x5614929c0b60 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86ae370740 10 client.15651 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:32.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f86ae370740 1 -- 172.21.15.78:0/508118807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5614929da540 con 0x5614929c0b60 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.932+0000 7f868d7fa640 1 -- 172.21.15.78:0/508118807 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8694044070 con 0x5614929c0b60 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.933+0000 7f868d7fa640 1 -- 172.21.15.78:0/508118807 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f869403a040 con 0x5614929c0b60 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.933+0000 7f86ae370740 10 client.15651 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.933+0000 7f86ae370740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.933+0000 7f86ae370740 20 client.15651 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.933+0000 7f86ae370740 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5614929db890 0x5614929ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.933+0000 7f86ae370740 4 client.15651 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.933+0000 7f86ae370740 1 -- 172.21.15.78:0/508118807 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614929337f0 con 0x5614929db890 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.933+0000 7f86a9d33640 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5614929db890 0x5614929ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86a9d33640 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5614929db890 0x5614929ddc80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1cc762665ccc40 server_cookie=30038b4fb150a807 in_seq=0 out_seq=0 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f868d7fa640 10 client.15651 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f868d7fa640 1 -- 172.21.15.78:0/508118807 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614929337f0 con 0x5614929db890 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f868d7fa640 10 client.15651 handle_command_reply: tid=0 2024-09-16T00:14:32.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 client.15651 shutdown 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 -- 172.21.15.78:0/508118807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 msgr2=0x5614929c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929c8c10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f86940056d0 tx=0x7f8694003cf0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 -- 172.21.15.78:0/508118807 shutdown_connections 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5614929db890 0x5614929ddc80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929cbd40 0x5614929c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f868d7fa640 0 client.15651 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929c1480 0x5614929c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 --2- 172.21.15.78:0/508118807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929c8c10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.934+0000 7f86ae370740 1 -- 172.21.15.78:0/508118807 >> 172.21.15.78:0/508118807 conn(0x5614929bd0f0 msgr2=0x5614929c2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.935+0000 7f86ae370740 1 -- 172.21.15.78:0/508118807 shutdown_connections 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.935+0000 7f86ae370740 1 -- 172.21.15.78:0/508118807 wait complete. 2024-09-16T00:14:32.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.935+0000 7f86ae370740 20 client.15651 trim_cache size 0 max 16384 2024-09-16T00:14:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.951+0000 7f86ae370740 1 Processor -- start 2024-09-16T00:14:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86ae370740 1 -- start start 2024-09-16T00:14:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86ae370740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c0b60 0x5614929e3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86ae370740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c1480 0x5614929e3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86ae370740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 0x5614929bba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86ae370740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b5250 con 0x5614929c1480 2024-09-16T00:14:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86ae370740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b6e20 con 0x5614929c0b60 2024-09-16T00:14:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86ae370740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b5920 con 0x5614929cbd40 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c1480 0x5614929e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c1480 0x5614929e3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58264/0 (socket says 172.21.15.78:58264) 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 -- 172.21.15.78:0/2359148862 learned_addr learned my addr 172.21.15.78:0/2359148862 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9532640 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 0x5614929bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9532640 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c0b60 0x5614929e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 -- 172.21.15.78:0/2359148862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 msgr2=0x5614929bba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 0x5614929bba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 -- 172.21.15.78:0/2359148862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c0b60 msgr2=0x5614929e3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c0b60 0x5614929e3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 -- 172.21.15.78:0/2359148862 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614929c12b0 con 0x5614929c1480 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9532640 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c0b60 0x5614929e3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f86a9d33640 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c1480 0x5614929e3790 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b900 tx=0x7f86a000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3858653b879aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:32.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f868f7fe640 1 -- 172.21.15.78:0/2359148862 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a001b620 con 0x5614929c1480 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.952+0000 7f868f7fe640 1 -- 172.21.15.78:0/2359148862 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a001bca0 con 0x5614929c1480 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f868f7fe640 1 -- 172.21.15.78:0/2359148862 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a001acc0 con 0x5614929c1480 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 -- 172.21.15.78:0/2359148862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c1480 msgr2=0x5614929e3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c1480 0x5614929e3790 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f86a000b900 tx=0x7f86a000f7f0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 -- 172.21.15.78:0/2359148862 shutdown_connections 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 0x5614929bba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c1480 0x5614929e3790 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 --2- 172.21.15.78:0/2359148862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c0b60 0x5614929e3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 -- 172.21.15.78:0/2359148862 >> 172.21.15.78:0/2359148862 conn(0x5614929bd0f0 msgr2=0x5614929cab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 -- 172.21.15.78:0/2359148862 shutdown_connections 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 -- 172.21.15.78:0/2359148862 wait complete. 2024-09-16T00:14:32.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.953+0000 7f86ae370740 1 -- start start 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86ae370740 1 Processor -- start 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86ae370740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86ae370740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 0x5614929cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86ae370740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 0x5614929cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86ae370740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b6720 con 0x5614929c0b60 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86ae370740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b6050 con 0x5614929c1480 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86ae370740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86a41b3230 con 0x5614929cbd40 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86ae370740 10 client.0 _ll_register_callbacks cb 0x7f86a000b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9532640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9532640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929d4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58278/0 (socket says 172.21.15.78:58278) 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9532640 1 -- 172.21.15.78:0/1407030036 learned_addr learned my addr 172.21.15.78:0/1407030036 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:32.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9d33640 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 0x5614929cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9532640 1 -- 172.21.15.78:0/1407030036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 msgr2=0x5614929cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9532640 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 0x5614929cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9532640 1 -- 172.21.15.78:0/1407030036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 msgr2=0x5614929cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9532640 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 0x5614929cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.954+0000 7f86a9532640 1 -- 172.21.15.78:0/1407030036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614929d05f0 con 0x5614929c0b60 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86a9532640 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929d4510 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f869403dfd0 tx=0x7f8694003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dc9b9a7768c500 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f868d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f868d7fa640 1 -- 172.21.15.78:0/1407030036 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694035070 con 0x5614929c0b60 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f868d7fa640 1 -- 172.21.15.78:0/1407030036 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f869404ad20 con 0x5614929c0b60 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 1 -- 172.21.15.78:0/1407030036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a001bca0 con 0x5614929c0b60 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f868d7fa640 1 -- 172.21.15.78:0/1407030036 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8694003ab0 con 0x5614929c0b60 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 10 client.15657 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 1 -- 172.21.15.78:0/1407030036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561492a46270 con 0x5614929c0b60 2024-09-16T00:14:32.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f868d7fa640 1 -- 172.21.15.78:0/1407030036 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a001bca0 con 0x5614929c0b60 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f868d7fa640 1 -- 172.21.15.78:0/1407030036 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f869403d960 con 0x5614929c0b60 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 10 client.15657 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 20 client.15657 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561492a467d0 0x561492a48bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 4 client.15657 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.955+0000 7f86ae370740 1 -- 172.21.15.78:0/1407030036 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56149292dd00 con 0x561492a467d0 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.956+0000 7f86a9d33640 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561492a467d0 0x561492a48bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.956+0000 7f868d7fa640 10 client.15657 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.956+0000 7f86a9d33640 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561492a467d0 0x561492a48bf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b805915c0b79ed server_cookie=3f6e3b9ebe605d53 in_seq=0 out_seq=0 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.957+0000 7f868d7fa640 1 -- 172.21.15.78:0/1407030036 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56149292dd00 con 0x561492a467d0 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.957+0000 7f868d7fa640 10 client.15657 handle_command_reply: tid=0 2024-09-16T00:14:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.957+0000 7f86ae370740 1 client.15657 shutdown 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.957+0000 7f86ae370740 1 -- 172.21.15.78:0/1407030036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 msgr2=0x5614929d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.957+0000 7f86ae370740 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929d4510 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f869403dfd0 tx=0x7f8694003800 comp rx=0 tx=0).stop 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.957+0000 7f86ae370740 1 -- 172.21.15.78:0/1407030036 shutdown_connections 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.957+0000 7f86ae370740 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561492a467d0 0x561492a48bf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86ae370740 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614929cbd40 0x5614929cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86ae370740 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614929c1480 0x5614929cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86ae370740 1 --2- 172.21.15.78:0/1407030036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5614929c0b60 0x5614929d4510 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86ae370740 1 -- 172.21.15.78:0/1407030036 >> 172.21.15.78:0/1407030036 conn(0x5614929bd0f0 msgr2=0x5614929ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f868d7fa640 0 client.15657 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86ae370740 1 -- 172.21.15.78:0/1407030036 shutdown_connections 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86ae370740 1 -- 172.21.15.78:0/1407030036 wait complete. 2024-09-16T00:14:32.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86ae370740 20 client.15657 trim_cache size 0 max 16384 2024-09-16T00:14:32.972 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:14:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:14:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c47e5dd4-da4f-4cc6-acc0-e687f5d0ff2a", 2024-09-16T00:14:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:14:32.972 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:14:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 -- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86900a32c0 msgr2=0x7f86900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86900a32c0 0x7f86900a5780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f869c0923c0 tx=0x7f869c0670b0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 -- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4105f40 msgr2=0x7f86a419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4105f40 0x7f86a419e8b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f869c00cad0 tx=0x7f869c00cf90 comp rx=0 tx=0).stop 2024-09-16T00:14:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 -- 172.21.15.78:0/1848234674 shutdown_connections 2024-09-16T00:14:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f86900a32c0 0x7f86900a5780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86a4105f40 0x7f86a419e8b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86a4101450 0x7f86a41a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 --2- 172.21.15.78:0/1848234674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f86a4100b30 0x7f86a41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.958+0000 7f86abf97640 1 -- 172.21.15.78:0/1848234674 >> 172.21.15.78:0/1848234674 conn(0x7f86a40fc7a0 msgr2=0x7f86a40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.959+0000 7f86abf97640 1 -- 172.21.15.78:0/1848234674 shutdown_connections 2024-09-16T00:14:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:32.959+0000 7f86abf97640 1 -- 172.21.15.78:0/1848234674 wait complete. 2024-09-16T00:14:33.758 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:14:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:33 smithi193 ceph-mon[35484]: pgmap v395: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-16T00:14:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:33 smithi193 ceph-mon[35484]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T00:14:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:33 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:14:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:33 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:14:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:33 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:33 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:33 smithi148 ceph-mon[35599]: pgmap v395: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-16T00:14:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:33 smithi148 ceph-mon[35599]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T00:14:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:33 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:14:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:33 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:14:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:33 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:33 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:34.003 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:33 smithi078 ceph-mon[31157]: pgmap v395: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 15 MiB/s wr, 696 op/s 2024-09-16T00:14:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:33 smithi078 ceph-mon[31157]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T00:14:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:33 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:14:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:33 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:14:34.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:33 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:34.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:33 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 -- 172.21.15.78:0/988828682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc101de0 msgr2=0x7f05cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 --2- 172.21.15.78:0/988828682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc101de0 0x7f05cc1021c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009900 tx=0x7f05bc02a3f0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 -- 172.21.15.78:0/988828682 shutdown_connections 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 --2- 172.21.15.78:0/988828682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05cc106f10 0x7f05cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 --2- 172.21.15.78:0/988828682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05cc102790 0x7f05cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 --2- 172.21.15.78:0/988828682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc101de0 0x7f05cc1021c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 -- 172.21.15.78:0/988828682 >> 172.21.15.78:0/988828682 conn(0x7f05cc0fd8c0 msgr2=0x7f05cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 -- 172.21.15.78:0/988828682 shutdown_connections 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 -- 172.21.15.78:0/988828682 wait complete. 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.534+0000 7f05d19c4640 1 Processor -- start 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05d19c4640 1 -- start start 2024-09-16T00:14:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05d19c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05cc101de0 0x7f05cc198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05d19c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc102790 0x7f05cc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05d19c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05cc106f10 0x7f05cc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05d19c4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc111f30 con 0x7f05cc102790 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05d19c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc111db0 con 0x7f05cc106f10 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05d19c4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1120b0 con 0x7f05cc101de0 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc102790 0x7f05cc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc102790 0x7f05cc198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58286/0 (socket says 172.21.15.78:58286) 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05c27fc640 1 -- 172.21.15.78:0/2892852492 learned_addr learned my addr 172.21.15.78:0/2892852492 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05cc106f10 0x7f05cc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05c27fc640 1 -- 172.21.15.78:0/2892852492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05cc101de0 msgr2=0x7f05cc198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05c27fc640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05cc101de0 0x7f05cc198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05c27fc640 1 -- 172.21.15.78:0/2892852492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05cc106f10 msgr2=0x7f05cc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05c27fc640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05cc106f10 0x7f05cc19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05c27fc640 1 -- 172.21.15.78:0/2892852492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05cc19d2c0 con 0x7f05cc102790 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.535+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05cc106f10 0x7f05cc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.536+0000 7f05c27fc640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc102790 0x7f05cc198850 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c920 tx=0x7f05b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.536+0000 7f05d09c2640 1 -- 172.21.15.78:0/2892852492 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4007c70 con 0x7f05cc102790 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.536+0000 7f05d09c2640 1 -- 172.21.15.78:0/2892852492 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b4007e10 con 0x7f05cc102790 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.536+0000 7f05d19c4640 1 -- 172.21.15.78:0/2892852492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05cc19d5b0 con 0x7f05cc102790 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.536+0000 7f05d09c2640 1 -- 172.21.15.78:0/2892852492 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4018890 con 0x7f05cc102790 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.536+0000 7f05d19c4640 1 -- 172.21.15.78:0/2892852492 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05cc1a4e90 con 0x7f05cc102790 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.538+0000 7f05d09c2640 1 -- 172.21.15.78:0/2892852492 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f05b4018a70 con 0x7f05cc102790 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.538+0000 7f05d3d9d740 1 Processor -- start 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.538+0000 7f05d3d9d740 1 -- start start 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.538+0000 7f05d09c2640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05b80a3240 0x7f05b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.538+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 0x562190986760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.538+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190986ca0 0x562190995550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.538+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190995a90 0x562190997e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.538+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621908ec2e0 con 0x562190995a90 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562190893ce0 con 0x562190986ca0 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621908e82a0 con 0x562190986380 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05b80a3240 0x7f05b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05d09c2640 1 -- 172.21.15.78:0/2892852492 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f05b40cfe70 con 0x7f05cc102790 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190995a90 0x562190997e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190995a90 0x562190997e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58294/0 (socket says 172.21.15.78:58294) 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 -- 172.21.15.78:0/1139895009 learned_addr learned my addr 172.21.15.78:0/1139895009 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05c27fc640 1 --2- 172.21.15.78:0/1139895009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190986ca0 0x562190995550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 -- 172.21.15.78:0/1139895009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 msgr2=0x562190986760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/1139895009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 0x562190986760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 -- 172.21.15.78:0/1139895009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190986ca0 msgr2=0x562190995550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:34.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/1139895009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190986ca0 0x562190995550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 -- 172.21.15.78:0/1139895009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562190878440 con 0x562190995a90 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05b80a3240 0x7f05b80a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05c4012ae0 tx=0x7f05c400ad30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/1139895009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190995a90 0x562190997e80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f05c400fc60 tx=0x7f05c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51643bb9d18acbda server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05c1ffb640 1 -- 172.21.15.78:0/1139895009 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4022070 con 0x562190995a90 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05c1ffb640 1 -- 172.21.15.78:0/1139895009 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c400dde0 con 0x562190995a90 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.539+0000 7f05c1ffb640 1 -- 172.21.15.78:0/1139895009 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c40105d0 con 0x562190995a90 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1139895009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190995a90 msgr2=0x562190997e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1139895009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190995a90 0x562190997e80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f05c400fc60 tx=0x7f05c4019000 comp rx=0 tx=0).stop 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1139895009 shutdown_connections 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1139895009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190995a90 0x562190997e80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1139895009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190986ca0 0x562190995550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1139895009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 0x562190986760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1139895009 >> 172.21.15.78:0/1139895009 conn(0x562190982a30 msgr2=0x562190982e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:34.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1139895009 shutdown_connections 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.540+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1139895009 wait complete. 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 1 -- start start 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 1 Processor -- start 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909926d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x562190992c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562190998660 con 0x562190986380 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5621908a7050 con 0x562190995a90 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621909987e0 con 0x562190986ca0 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05d3d9d740 10 client.0 _ll_register_callbacks cb 0x5621908e7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58306/0 (socket says 172.21.15.78:58306) 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05caffd640 1 -- 172.21.15.78:0/1548520635 learned_addr learned my addr 172.21.15.78:0/1548520635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x562190992c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.541+0000 7f05c27fc640 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909926d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05caffd640 1 -- 172.21.15.78:0/1548520635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 msgr2=0x5621909926d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05caffd640 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909926d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05caffd640 1 -- 172.21.15.78:0/1548520635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 msgr2=0x562190992c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05caffd640 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x562190992c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05caffd640 1 -- 172.21.15.78:0/1548520635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56219065d3c0 con 0x562190986380 2024-09-16T00:14:34.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05c27fc640 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909926d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05caffd640 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909953d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f05bc032fd0 tx=0x7f05bc0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f504ca68470b395 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05aaffd640 1 -- 172.21.15.78:0/1548520635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc040070 con 0x562190986380 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05aaffd640 1 -- 172.21.15.78:0/1548520635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f05bc004240 con 0x562190986380 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1548520635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c400dde0 con 0x562190986380 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05aaffd640 1 -- 172.21.15.78:0/1548520635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f05bc04c050 con 0x562190986380 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05d3d9d740 10 client.15669 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1548520635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5621909a40e0 con 0x562190986380 2024-09-16T00:14:34.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05aaffd640 1 -- 172.21.15.78:0/1548520635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05c400dde0 con 0x562190986380 2024-09-16T00:14:34.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05aaffd640 1 -- 172.21.15.78:0/1548520635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f05bc03a4f0 con 0x562190986380 2024-09-16T00:14:34.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05d3d9d740 10 client.15669 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:34.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05d3d9d740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:14:34.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05d3d9d740 20 client.15669 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:34.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5621909a5540 0x5621909a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.542+0000 7f05d3d9d740 4 client.15669 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:34.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.543+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1548520635 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621908f9850 con 0x5621909a5540 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.543+0000 7f05c27fc640 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5621909a5540 0x5621909a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.543+0000 7f05aaffd640 10 client.15669 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.543+0000 7f05c27fc640 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5621909a5540 0x5621909a7950 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ee9566456af72d server_cookie=63cdbc9620faf3fa in_seq=0 out_seq=0 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05aaffd640 1 -- 172.21.15.78:0/1548520635 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621908f9850 con 0x5621909a5540 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05aaffd640 10 client.15669 handle_command_reply: tid=0 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05d3d9d740 1 client.15669 shutdown 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1548520635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 msgr2=0x5621909953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909953d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f05bc032fd0 tx=0x7f05bc0317e0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1548520635 shutdown_connections 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5621909a5540 0x5621909a7950 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x562190992c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909926d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.544+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/1548520635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909953d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.545+0000 7f05aaffd640 0 client.15669 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:34.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.545+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1548520635 >> 172.21.15.78:0/1548520635 conn(0x562190982a30 msgr2=0x562190988790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:34.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.545+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1548520635 shutdown_connections 2024-09-16T00:14:34.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.545+0000 7f05d3d9d740 1 -- 172.21.15.78:0/1548520635 wait complete. 2024-09-16T00:14:34.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.545+0000 7f05d3d9d740 20 client.15669 trim_cache size 0 max 16384 2024-09-16T00:14:34.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05d3d9d740 1 Processor -- start 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05d3d9d740 1 -- start start 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909a1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x5621909ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1ac850 con 0x562190986380 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1ae420 con 0x562190995a90 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1acf20 con 0x562190986ca0 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58316/0 (socket says 172.21.15.78:58316) 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 -- 172.21.15.78:0/3691854168 learned_addr learned my addr 172.21.15.78:0/3691854168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:34.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x5621909ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05c27fc640 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 -- 172.21.15.78:0/3691854168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 msgr2=0x5621909ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 -- 172.21.15.78:0/3691854168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 msgr2=0x5621909ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x5621909ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 -- 172.21.15.78:0/3691854168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621909a7fd0 con 0x562190986380 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.562+0000 7f05caffd640 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909a1950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f05bc03a8f0 tx=0x7f05bc031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235430bc4b46f1e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05c17fa640 1 -- 172.21.15.78:0/3691854168 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc055070 con 0x562190986380 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05c17fa640 1 -- 172.21.15.78:0/3691854168 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05bc005550 con 0x562190986380 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05c17fa640 1 -- 172.21.15.78:0/3691854168 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc049070 con 0x562190986380 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05d3d9d740 1 -- 172.21.15.78:0/3691854168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 msgr2=0x5621909a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909a1950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f05bc03a8f0 tx=0x7f05bc031b60 comp rx=0 tx=0).stop 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05d3d9d740 1 -- 172.21.15.78:0/3691854168 shutdown_connections 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x5621909ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986ca0 0x5621909ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/3691854168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986380 0x5621909a1950 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05d3d9d740 1 -- 172.21.15.78:0/3691854168 >> 172.21.15.78:0/3691854168 conn(0x562190982a30 msgr2=0x56219098dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.563+0000 7f05d3d9d740 1 -- 172.21.15.78:0/3691854168 shutdown_connections 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 -- 172.21.15.78:0/3691854168 wait complete. 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 -- start start 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 Processor -- start 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 0x562190999990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986ca0 0x562190999ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x56219099a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1add20 con 0x562190986ca0 2024-09-16T00:14:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1ad650 con 0x562190995a90 2024-09-16T00:14:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05cc1aa830 con 0x562190986380 2024-09-16T00:14:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05d3d9d740 10 client.0 _ll_register_callbacks cb 0x7f05bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986ca0 0x562190999ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986ca0 0x562190999ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58332/0 (socket says 172.21.15.78:58332) 2024-09-16T00:14:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.564+0000 7f05c27fc640 1 -- 172.21.15.78:0/2332800848 learned_addr learned my addr 172.21.15.78:0/2332800848 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05caffd640 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 0x562190999990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05cb7fe640 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x56219099a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05c27fc640 1 -- 172.21.15.78:0/2332800848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 msgr2=0x562190999990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05c27fc640 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 0x562190999990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05c27fc640 1 -- 172.21.15.78:0/2332800848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 msgr2=0x56219099a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05c27fc640 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x56219099a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05c27fc640 1 -- 172.21.15.78:0/2332800848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562190a0f390 con 0x562190986ca0 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05c27fc640 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986ca0 0x562190999ed0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f05b4098b00 tx=0x7f05b409c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f594f64439fa97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.565+0000 7f05aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.567+0000 7f05aaffd640 1 -- 172.21.15.78:0/2332800848 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40b4020 con 0x562190986ca0 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05aaffd640 1 -- 172.21.15.78:0/2332800848 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f05b409a430 con 0x562190986ca0 2024-09-16T00:14:34.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 1 -- 172.21.15.78:0/2332800848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05bc005550 con 0x562190986ca0 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05aaffd640 1 -- 172.21.15.78:0/2332800848 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f05b4018d60 con 0x562190986ca0 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 10 client.15675 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 1 -- 172.21.15.78:0/2332800848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562190a106a0 con 0x562190986ca0 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05aaffd640 1 -- 172.21.15.78:0/2332800848 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05bc005550 con 0x562190986ca0 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05aaffd640 1 -- 172.21.15.78:0/2332800848 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f05b40b41c0 con 0x562190986ca0 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 10 client.15675 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 20 client.15675 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562190a10e00 0x562190a13220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 4 client.15675 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05d3d9d740 1 -- 172.21.15.78:0/2332800848 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5621908f3d60 con 0x562190a10e00 2024-09-16T00:14:34.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.568+0000 7f05caffd640 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562190a10e00 0x562190a13220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05caffd640 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562190a10e00 0x562190a13220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24bc8a0d195259e server_cookie=6088579dac40974a in_seq=0 out_seq=0 2024-09-16T00:14:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05aaffd640 10 client.15675 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05aaffd640 1 -- 172.21.15.78:0/2332800848 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5621908f3d60 con 0x562190a10e00 2024-09-16T00:14:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05aaffd640 10 client.15675 handle_command_reply: tid=0 2024-09-16T00:14:34.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05d3d9d740 1 client.15675 shutdown 2024-09-16T00:14:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05d3d9d740 1 -- 172.21.15.78:0/2332800848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986ca0 msgr2=0x562190999ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986ca0 0x562190999ed0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f05b4098b00 tx=0x7f05b409c1f0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05d3d9d740 1 -- 172.21.15.78:0/2332800848 shutdown_connections 2024-09-16T00:14:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562190a10e00 0x562190a13220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562190995a90 0x56219099a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562190986ca0 0x562190999ed0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.569+0000 7f05d3d9d740 1 --2- 172.21.15.78:0/2332800848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562190986380 0x562190999990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d3d9d740 1 -- 172.21.15.78:0/2332800848 >> 172.21.15.78:0/2332800848 conn(0x562190982a30 msgr2=0x562190981350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05aaffd640 0 client.15675 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d3d9d740 1 -- 172.21.15.78:0/2332800848 shutdown_connections 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d3d9d740 1 -- 172.21.15.78:0/2332800848 wait complete. 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d3d9d740 20 client.15675 trim_cache size 0 max 16384 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "c47e5dd4-da4f-4cc6-acc0-e687f5d0ff2a": { 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c47e5dd4-da4f-4cc6-acc0-e687f5d0ff2a", 2024-09-16T00:14:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:14:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:34.579 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:14:34.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 -- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05b80a3240 msgr2=0x7f05b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05b80a3240 0x7f05b80a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05c4012ae0 tx=0x7f05c400ad30 comp rx=0 tx=0).stop 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 -- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc102790 msgr2=0x7f05cc198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc102790 0x7f05cc198850 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c920 tx=0x7f05b400cde0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 -- 172.21.15.78:0/2892852492 shutdown_connections 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05b80a3240 0x7f05b80a5700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05cc106f10 0x7f05cc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05cc102790 0x7f05cc198850 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 --2- 172.21.15.78:0/2892852492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05cc101de0 0x7f05cc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.570+0000 7f05d19c4640 1 -- 172.21.15.78:0/2892852492 >> 172.21.15.78:0/2892852492 conn(0x7f05cc0fd8c0 msgr2=0x7f05cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.571+0000 7f05d19c4640 1 -- 172.21.15.78:0/2892852492 shutdown_connections 2024-09-16T00:14:34.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:34.571+0000 7f05d19c4640 1 -- 172.21.15.78:0/2892852492 wait complete. 2024-09-16T00:14:34.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:34 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:14:34.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:34 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:34.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:34 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:14:34.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:34 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:34.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:34 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:14:34.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:34 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:34.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:34 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:14:34.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:34 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:34 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:14:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:34 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:34 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:14:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:34 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:14:35.263 INFO:tasks.cephfs.filesystem:scrub status for tag:c47e5dd4-da4f-4cc6-acc0-e687f5d0ff2a - {'path': '/', 'tag': 'c47e5dd4-da4f-4cc6-acc0-e687f5d0ff2a', 'options': 'recursive,force'} 2024-09-16T00:14:35.865 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 35.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:14:35.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:35 smithi193 ceph-mon[35484]: pgmap v396: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 12 MiB/s wr, 535 op/s 2024-09-16T00:14:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:35 smithi148 ceph-mon[35599]: pgmap v396: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 12 MiB/s wr, 535 op/s 2024-09-16T00:14:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:35 smithi078 ceph-mon[31157]: pgmap v396: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 12 MiB/s wr, 535 op/s 2024-09-16T00:14:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:37 smithi193 ceph-mon[35484]: pgmap v397: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 10 MiB/s wr, 456 op/s 2024-09-16T00:14:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:37 smithi148 ceph-mon[35599]: pgmap v397: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 10 MiB/s wr, 456 op/s 2024-09-16T00:14:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:37 smithi078 ceph-mon[31157]: pgmap v397: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 10 MiB/s wr, 456 op/s 2024-09-16T00:14:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:39 smithi193 ceph-mon[35484]: pgmap v398: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 7.3 MiB/s wr, 334 op/s 2024-09-16T00:14:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:39 smithi148 ceph-mon[35599]: pgmap v398: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 7.3 MiB/s wr, 334 op/s 2024-09-16T00:14:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:39 smithi078 ceph-mon[31157]: pgmap v398: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 7.3 MiB/s wr, 334 op/s 2024-09-16T00:14:40.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 40.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:14:41.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:41 smithi193 ceph-mon[35484]: pgmap v399: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-09-16T00:14:41.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:41 smithi148 ceph-mon[35599]: pgmap v399: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-09-16T00:14:42.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:41 smithi078 ceph-mon[31157]: pgmap v399: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-09-16T00:14:43.489 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:14:43.746 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:43.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:43 smithi078 ceph-mon[31157]: pgmap v400: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-09-16T00:14:43.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:43 smithi193 ceph-mon[35484]: pgmap v400: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-09-16T00:14:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:43 smithi148 ceph-mon[35599]: pgmap v400: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 -- 172.21.15.78:0/599218154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102570 msgr2=0x7feef01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 --2- 172.21.15.78:0/599218154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102570 0x7feef01029b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feee0009900 tx=0x7feee002a480 comp rx=0 tx=0).stop 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 -- 172.21.15.78:0/599218154 shutdown_connections 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 --2- 172.21.15.78:0/599218154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef0102ef0 0x7feef010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 --2- 172.21.15.78:0/599218154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102570 0x7feef01029b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 --2- 172.21.15.78:0/599218154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef0100e20 0x7feef0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 -- 172.21.15.78:0/599218154 >> 172.21.15.78:0/599218154 conn(0x7feef00fc7d0 msgr2=0x7feef00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 -- 172.21.15.78:0/599218154 shutdown_connections 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.279+0000 7feef864f640 1 -- 172.21.15.78:0/599218154 wait complete. 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef864f640 1 Processor -- start 2024-09-16T00:14:44.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef864f640 1 -- start start 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef864f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef0100e20 0x7feef01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef864f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef0102570 0x7feef0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef864f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102ef0 0x7feef019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef864f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feef0110f50 con 0x7feef0102ef0 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef864f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feef0110dd0 con 0x7feef0100e20 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef864f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feef01110d0 con 0x7feef0102570 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef6bec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102ef0 0x7feef019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef6bec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102ef0 0x7feef019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33500/0 (socket says 172.21.15.78:33500) 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef6bec640 1 -- 172.21.15.78:0/1733174770 learned_addr learned my addr 172.21.15.78:0/1733174770 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef63eb640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef0100e20 0x7feef01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef5bea640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef0102570 0x7feef0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef6bec640 1 -- 172.21.15.78:0/1733174770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef0102570 msgr2=0x7feef0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:44.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef6bec640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef0102570 0x7feef0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef6bec640 1 -- 172.21.15.78:0/1733174770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef0100e20 msgr2=0x7feef01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef6bec640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef0100e20 0x7feef01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.280+0000 7feef6bec640 1 -- 172.21.15.78:0/1733174770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef019d550 con 0x7feef0102ef0 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.281+0000 7feef6bec640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102ef0 0x7feef019ce70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feee800eaa0 tx=0x7feee800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.281+0000 7feee77fe640 1 -- 172.21.15.78:0/1733174770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee800cdb0 con 0x7feef0102ef0 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.281+0000 7feee77fe640 1 -- 172.21.15.78:0/1733174770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee8005020 con 0x7feef0102ef0 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.281+0000 7feee77fe640 1 -- 172.21.15.78:0/1733174770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee80116b0 con 0x7feef0102ef0 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.281+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef019d840 con 0x7feef0102ef0 2024-09-16T00:14:44.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.281+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01a5120 con 0x7feef0102ef0 2024-09-16T00:14:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.282+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef0101200 con 0x7feef0102ef0 2024-09-16T00:14:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.283+0000 7feee77fe640 1 -- 172.21.15.78:0/1733174770 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7feee8005560 con 0x7feef0102ef0 2024-09-16T00:14:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.284+0000 7feee77fe640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feedc0a3240 0x7feedc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.284+0000 7feee77fe640 1 -- 172.21.15.78:0/1733174770 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feee80cfe60 con 0x7feef0102ef0 2024-09-16T00:14:44.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.284+0000 7feef63eb640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feedc0a3240 0x7feedc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:44.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.285+0000 7feef63eb640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feedc0a3240 0x7feedc0a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feeec005d00 tx=0x7feeec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:44.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.289+0000 7feee77fe640 1 -- 172.21.15.78:0/1733174770 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feee8016030 con 0x7feef0102ef0 2024-09-16T00:14:44.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.497+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feef010a0a0 con 0x7feef0102ef0 2024-09-16T00:14:44.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.498+0000 7feee77fe640 1 -- 172.21.15.78:0/1733174770 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7feee809c820 con 0x7feef0102ef0 2024-09-16T00:14:44.499 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:14:44.499 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:14:44.499 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feedc0a3240 msgr2=0x7feedc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feedc0a3240 0x7feedc0a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feeec005d00 tx=0x7feeec00b410 comp rx=0 tx=0).stop 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102ef0 msgr2=0x7feef019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102ef0 0x7feef019ce70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feee800eaa0 tx=0x7feee800ef60 comp rx=0 tx=0).stop 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 shutdown_connections 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feedc0a3240 0x7feedc0a5700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feef0102ef0 0x7feef019ce70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feef0102570 0x7feef0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 --2- 172.21.15.78:0/1733174770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feef0100e20 0x7feef01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 >> 172.21.15.78:0/1733174770 conn(0x7feef00fc7d0 msgr2=0x7feef006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 shutdown_connections 2024-09-16T00:14:44.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:44.500+0000 7feef864f640 1 -- 172.21.15.78:0/1733174770 wait complete. 2024-09-16T00:14:44.875 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:44 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:14:44.876 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:44 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:14:44.876 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:44 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:14:44.876 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:44 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1733174770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:44 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:14:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:44 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:14:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:44 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:14:44.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:44 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1733174770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:44 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:14:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:44 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:14:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:44 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:14:44.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:44 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1733174770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:14:45.228 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 56b5046b --release --await' 2024-09-16T00:14:45.228 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 56b5046b --release --await 2024-09-16T00:14:45.475 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:14:45.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 45.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:14:45.905 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:45 smithi078 ceph-mon[31157]: pgmap v401: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 525 KiB/s wr, 47 op/s 2024-09-16T00:14:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:45 smithi193 ceph-mon[35484]: pgmap v401: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 525 KiB/s wr, 47 op/s 2024-09-16T00:14:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:45 smithi148 ceph-mon[35599]: pgmap v401: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 525 KiB/s wr, 47 op/s 2024-09-16T00:14:46.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 -- 172.21.15.78:0/1785765022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4106f10 msgr2=0x7febb410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 --2- 172.21.15.78:0/1785765022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4106f10 0x7febb410ea90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7febac009900 tx=0x7febac02a410 comp rx=0 tx=0).stop 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 -- 172.21.15.78:0/1785765022 shutdown_connections 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 --2- 172.21.15.78:0/1785765022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4106f10 0x7febb410ea90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 --2- 172.21.15.78:0/1785765022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb4102790 0x7febb41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 --2- 172.21.15.78:0/1785765022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb4101de0 0x7febb41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 -- 172.21.15.78:0/1785765022 >> 172.21.15.78:0/1785765022 conn(0x7febb40fd8c0 msgr2=0x7febb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 -- 172.21.15.78:0/1785765022 shutdown_connections 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.992+0000 7febb9df4640 1 -- 172.21.15.78:0/1785765022 wait complete. 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb9df4640 1 Processor -- start 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb9df4640 1 -- start start 2024-09-16T00:14:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb9df4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb4101de0 0x7febb4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb9df4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4102790 0x7febb4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb9df4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb4106f10 0x7febb419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb9df4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febb4112010 con 0x7febb4102790 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb9df4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7febb4111e90 con 0x7febb4101de0 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb9df4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febb4112190 con 0x7febb4106f10 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4102790 0x7febb4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4102790 0x7febb4198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33514/0 (socket says 172.21.15.78:33514) 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb2ffd640 1 -- 172.21.15.78:0/1124622906 learned_addr learned my addr 172.21.15.78:0/1124622906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb37fe640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb4101de0 0x7febb4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb3fff640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb4106f10 0x7febb419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb2ffd640 1 -- 172.21.15.78:0/1124622906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb4106f10 msgr2=0x7febb419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb4106f10 0x7febb419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb2ffd640 1 -- 172.21.15.78:0/1124622906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb4101de0 msgr2=0x7febb4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb4101de0 0x7febb4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb2ffd640 1 -- 172.21.15.78:0/1124622906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb419d520 con 0x7febb4102790 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb3fff640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb4106f10 0x7febb419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.993+0000 7febb37fe640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb4101de0 0x7febb4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.994+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4102790 0x7febb4198ab0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feba800b4a0 tx=0x7feba800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.994+0000 7febb0ff9640 1 -- 172.21.15.78:0/1124622906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8014070 con 0x7febb4102790 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.994+0000 7febb9df4640 1 -- 172.21.15.78:0/1124622906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb419d810 con 0x7febb4102790 2024-09-16T00:14:46.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.994+0000 7febb0ff9640 1 -- 172.21.15.78:0/1124622906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba8010af0 con 0x7febb4102790 2024-09-16T00:14:46.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.994+0000 7febb0ff9640 1 -- 172.21.15.78:0/1124622906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8017d10 con 0x7febb4102790 2024-09-16T00:14:46.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.994+0000 7febb9df4640 1 -- 172.21.15.78:0/1124622906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb41a50f0 con 0x7febb4102790 2024-09-16T00:14:46.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.996+0000 7febb0ff9640 1 -- 172.21.15.78:0/1124622906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7feba8017070 con 0x7febb4102790 2024-09-16T00:14:46.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.996+0000 7febbc1cd740 1 Processor -- start 2024-09-16T00:14:46.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febbc1cd740 1 -- start start 2024-09-16T00:14:46.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ec440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febbc1cd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561d1e451620 con 0x561d1e4ec980 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febbc1cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561d1e3f8e50 con 0x561d1e4ec060 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febbc1cd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561d1e44d5e0 con 0x561d1e4f7240 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb0ff9640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb9c0a3240 0x7feb9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33528/0 (socket says 172.21.15.78:33528) 2024-09-16T00:14:46.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb37fe640 1 -- 172.21.15.78:0/1369879210 learned_addr learned my addr 172.21.15.78:0/1369879210 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb3fff640 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb0ff9640 1 -- 172.21.15.78:0/1124622906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feba80cf150 con 0x7febb4102790 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb3fff640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb9c0a3240 0x7feb9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb37fe640 1 -- 172.21.15.78:0/1369879210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 msgr2=0x561d1e4f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb37fe640 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb37fe640 1 -- 172.21.15.78:0/1369879210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 msgr2=0x561d1e4ec440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb37fe640 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ec440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb37fe640 1 -- 172.21.15.78:0/1369879210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d1e3e9d40 con 0x561d1e4ec980 2024-09-16T00:14:46.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.997+0000 7febb3fff640 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.998+0000 7febb37fe640 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f6d00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7feba4007fb0 tx=0x7feba400c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8bbb289f2a76dd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.998+0000 7feb99ffb640 1 -- 172.21.15.78:0/1369879210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4014070 con 0x561d1e4ec980 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.998+0000 7febb3fff640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb9c0a3240 0x7feb9c0a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febac02a930 tx=0x7febac005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.998+0000 7feb99ffb640 1 -- 172.21.15.78:0/1369879210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba400fd30 con 0x561d1e4ec980 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.998+0000 7feb99ffb640 1 -- 172.21.15.78:0/1369879210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba400ed70 con 0x561d1e4ec980 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.998+0000 7febbc1cd740 1 -- 172.21.15.78:0/1369879210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 msgr2=0x561d1e4f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.998+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f6d00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7feba4007fb0 tx=0x7feba400c750 comp rx=0 tx=0).stop 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 -- 172.21.15.78:0/1369879210 shutdown_connections 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f6d00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1369879210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 -- 172.21.15.78:0/1369879210 >> 172.21.15.78:0/1369879210 conn(0x561d1e4e85f0 msgr2=0x561d1e4e89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 -- 172.21.15.78:0/1369879210 shutdown_connections 2024-09-16T00:14:46.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 -- 172.21.15.78:0/1369879210 wait complete. 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 -- start start 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 Processor -- start 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4ec060 0x561d1e4f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4f7240 0x561d1e4f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561d1e4f9e10 con 0x561d1e4ec980 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561d1e3d9f10 con 0x561d1e4f7240 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561d1e4f9f90 con 0x561d1e4ec060 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febbc1cd740 10 client.0 _ll_register_callbacks cb 0x561d1e3fddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33544/0 (socket says 172.21.15.78:33544) 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:45.999+0000 7febb2ffd640 1 -- 172.21.15.78:0/1507024804 learned_addr learned my addr 172.21.15.78:0/1507024804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:46.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febb37fe640 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4ec060 0x561d1e4f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febb2ffd640 1 -- 172.21.15.78:0/1507024804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4ec060 msgr2=0x561d1e4f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4ec060 0x561d1e4f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febb2ffd640 1 -- 172.21.15.78:0/1507024804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4f7240 msgr2=0x561d1e4f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4f7240 0x561d1e4f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febb2ffd640 1 -- 172.21.15.78:0/1507024804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d1e160580 con 0x561d1e4ec980 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f4600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feba800be60 tx=0x7feba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903302e6493d2996 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7feb92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7feb92ffd640 1 -- 172.21.15.78:0/1507024804 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba801e050 con 0x561d1e4ec980 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7feb92ffd640 1 -- 172.21.15.78:0/1507024804 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feba8010ca0 con 0x561d1e4ec980 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febbc1cd740 1 -- 172.21.15.78:0/1507024804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba400fd30 con 0x561d1e4ec980 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7feb92ffd640 1 -- 172.21.15.78:0/1507024804 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feba8097a20 con 0x561d1e4ec980 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febbc1cd740 10 client.15693 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:46.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febbc1cd740 1 -- 172.21.15.78:0/1507024804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561d1e22a960 con 0x561d1e4ec980 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7feb92ffd640 1 -- 172.21.15.78:0/1507024804 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba400fd30 con 0x561d1e4ec980 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7feb92ffd640 1 -- 172.21.15.78:0/1507024804 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7feba809a920 con 0x561d1e4ec980 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febbc1cd740 10 client.15693 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febbc1cd740 10 client.15693 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.000+0000 7febbc1cd740 20 client.15693 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.001+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561d1e506d00 0x561d1e509190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.001+0000 7febbc1cd740 4 client.15693 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.001+0000 7febbc1cd740 1 -- 172.21.15.78:0/1507024804 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d1e45eb90 con 0x561d1e506d00 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.001+0000 7febb37fe640 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561d1e506d00 0x561d1e509190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.001+0000 7feb92ffd640 10 client.15693 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.001+0000 7febb37fe640 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561d1e506d00 0x561d1e509190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b218e4c03bc8c8 server_cookie=5ac28d9dcab62d5e in_seq=0 out_seq=0 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7feb92ffd640 1 -- 172.21.15.78:0/1507024804 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d1e45eb90 con 0x561d1e506d00 2024-09-16T00:14:46.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7feb92ffd640 10 client.15693 handle_command_reply: tid=0 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 client.15693 shutdown 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 -- 172.21.15.78:0/1507024804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 msgr2=0x561d1e4f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f4600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feba800be60 tx=0x7feba80046c0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 -- 172.21.15.78:0/1507024804 shutdown_connections 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561d1e506d00 0x561d1e509190 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4f7240 0x561d1e4f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4f4600 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7feb92ffd640 0 client.15693 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1507024804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4ec060 0x561d1e4f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 -- 172.21.15.78:0/1507024804 >> 172.21.15.78:0/1507024804 conn(0x561d1e4e85f0 msgr2=0x561d1e4ee320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 -- 172.21.15.78:0/1507024804 shutdown_connections 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 1 -- 172.21.15.78:0/1507024804 wait complete. 2024-09-16T00:14:46.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.002+0000 7febbc1cd740 20 client.15693 trim_cache size 0 max 16384 2024-09-16T00:14:46.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.019+0000 7febbc1cd740 1 Processor -- start 2024-09-16T00:14:46.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.019+0000 7febbc1cd740 1 -- start start 2024-09-16T00:14:46.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e50ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e50efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febbc1cd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febb41aece0 con 0x561d1e4ec980 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febbc1cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7febb41b08b0 con 0x561d1e4ec060 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febbc1cd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febb41af3b0 con 0x561d1e4f7240 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e50efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e50efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33556/0 (socket says 172.21.15.78:33556) 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 -- 172.21.15.78:0/1871030672 learned_addr learned my addr 172.21.15.78:0/1871030672 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb37fe640 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e50ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb3fff640 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 -- 172.21.15.78:0/1871030672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 msgr2=0x561d1e4f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:46.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 -- 172.21.15.78:0/1871030672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 msgr2=0x561d1e50ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e50ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 -- 172.21.15.78:0/1871030672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d1e4ec7b0 con 0x561d1e4ec980 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7febb2ffd640 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e50efa0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7feba80a3af0 tx=0x7feba80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ed6aa55e343ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7feb997fa640 1 -- 172.21.15.78:0/1871030672 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba801e070 con 0x561d1e4ec980 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7feb997fa640 1 -- 172.21.15.78:0/1871030672 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba80b4240 con 0x561d1e4ec980 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.020+0000 7feb997fa640 1 -- 172.21.15.78:0/1871030672 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba809a800 con 0x561d1e4ec980 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 -- 172.21.15.78:0/1871030672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 msgr2=0x561d1e50efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e50efa0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7feba80a3af0 tx=0x7feba80047d0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 -- 172.21.15.78:0/1871030672 shutdown_connections 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e50efa0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 --2- 172.21.15.78:0/1871030672 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e50ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 -- 172.21.15.78:0/1871030672 >> 172.21.15.78:0/1871030672 conn(0x561d1e4e85f0 msgr2=0x561d1e4f2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 -- 172.21.15.78:0/1871030672 shutdown_connections 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.021+0000 7febbc1cd740 1 -- 172.21.15.78:0/1871030672 wait complete. 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 1 -- start start 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 1 Processor -- start 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4ff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7febb41b01b0 con 0x561d1e4ec980 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7febb41afae0 con 0x561d1e4ec060 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febb41accc0 con 0x561d1e4f7240 2024-09-16T00:14:46.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febbc1cd740 10 client.0 _ll_register_callbacks cb 0x7feba809bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4ff720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33564/0 (socket says 172.21.15.78:33564) 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb2ffd640 1 -- 172.21.15.78:0/471839962 learned_addr learned my addr 172.21.15.78:0/471839962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb3fff640 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb37fe640 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb2ffd640 1 -- 172.21.15.78:0/471839962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 msgr2=0x561d1e4fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb2ffd640 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb2ffd640 1 -- 172.21.15.78:0/471839962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 msgr2=0x561d1e4ff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb2ffd640 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb2ffd640 1 -- 172.21.15.78:0/471839962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d1e4fc7a0 con 0x561d1e4ec980 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.022+0000 7febb3fff640 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febb2ffd640 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4ff720 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feba8097120 tx=0x7feba809ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983afb66afb4c878 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7feb92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:14:46.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7feb92ffd640 1 -- 172.21.15.78:0/471839962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba801e050 con 0x561d1e4ec980 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7feb92ffd640 1 -- 172.21.15.78:0/471839962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feba8010610 con 0x561d1e4ec980 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 1 -- 172.21.15.78:0/471839962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba80b4240 con 0x561d1e4ec980 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7feb92ffd640 1 -- 172.21.15.78:0/471839962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feba8010810 con 0x561d1e4ec980 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 10 client.15699 fetch_fsmap learned FSMap version 31 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 1 -- 172.21.15.78:0/471839962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561d1e572340 con 0x561d1e4ec980 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7feb92ffd640 1 -- 172.21.15.78:0/471839962 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba80b4240 con 0x561d1e4ec980 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7feb92ffd640 1 -- 172.21.15.78:0/471839962 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7feba809f120 con 0x561d1e4ec980 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 10 client.15699 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 10 client.15699 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 20 client.15699 populate_metadata read hostname 'smithi078' 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561d1e506d00 0x561d1e5747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 4 client.15699 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "56b5046b", "release": true, "await": true}] 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.023+0000 7febbc1cd740 1 -- 172.21.15.78:0/471839962 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "56b5046b", "release": true, "await": true}) -- 0x561d1e4590a0 con 0x561d1e506d00 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.024+0000 7febb37fe640 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561d1e506d00 0x561d1e5747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:14:46.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.024+0000 7febb37fe640 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561d1e506d00 0x561d1e5747c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c0eb011608df16 server_cookie=7603e7129c610425 in_seq=0 out_seq=0 2024-09-16T00:14:46.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.024+0000 7feb92ffd640 10 client.15699 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7feb92ffd640 1 -- 172.21.15.78:0/471839962 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561d1e4590a0 con 0x561d1e506d00 2024-09-16T00:14:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7feb92ffd640 10 client.15699 handle_command_reply: tid=0 2024-09-16T00:14:46.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 client.15699 shutdown 2024-09-16T00:14:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 -- 172.21.15.78:0/471839962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 msgr2=0x561d1e4ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4ff720 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7feba8097120 tx=0x7feba809ae40 comp rx=0 tx=0).stop 2024-09-16T00:14:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 -- 172.21.15.78:0/471839962 shutdown_connections 2024-09-16T00:14:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561d1e506d00 0x561d1e5747c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d1e4f7240 0x561d1e4fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561d1e4ec980 0x561d1e4ff720 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7feb92ffd640 0 client.15699 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:14:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 --2- 172.21.15.78:0/471839962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561d1e4ec060 0x561d1e4ff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.075+0000 7febbc1cd740 1 -- 172.21.15.78:0/471839962 >> 172.21.15.78:0/471839962 conn(0x561d1e4e85f0 msgr2=0x561d1e4f5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.076+0000 7febbc1cd740 1 -- 172.21.15.78:0/471839962 shutdown_connections 2024-09-16T00:14:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.076+0000 7febbc1cd740 1 -- 172.21.15.78:0/471839962 wait complete. 2024-09-16T00:14:46.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.076+0000 7febbc1cd740 20 client.15699 trim_cache size 0 max 16384 2024-09-16T00:14:46.078 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:14:46.079 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 31, 2024-09-16T00:14:46.079 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:14:46.079 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 4, 2024-09-16T00:14:46.079 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:14:46.079 INFO:teuthology.orchestra.run.smithi078.stdout: "56b5046b": { 2024-09-16T00:14:46.080 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 4, 2024-09-16T00:14:46.080 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:14:46.080 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:14:46.080 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:14:46.080 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:14:46.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:14:46.081 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:14:46.081 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 150.2, 2024-09-16T00:14:46.081 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:14:46.081 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:14:46.082 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 -- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb9c0a3240 msgr2=0x7feb9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb9c0a3240 0x7feb9c0a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febac02a930 tx=0x7febac005fd0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 -- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4102790 msgr2=0x7febb4198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4102790 0x7febb4198ab0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feba800b4a0 tx=0x7feba800b960 comp rx=0 tx=0).stop 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 -- 172.21.15.78:0/1124622906 shutdown_connections 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feb9c0a3240 0x7feb9c0a5700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febb4106f10 0x7febb419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7febb4102790 0x7febb4198ab0 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feba800b4a0 tx=0x7feba800b960 comp rx=0 tx=0).stop 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 --2- 172.21.15.78:0/1124622906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7febb4101de0 0x7febb4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:14:46.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.077+0000 7febb9df4640 1 -- 172.21.15.78:0/1124622906 >> 172.21.15.78:0/1124622906 conn(0x7febb40fd8c0 msgr2=0x7febb40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:14:46.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.078+0000 7febb9df4640 1 -- 172.21.15.78:0/1124622906 shutdown_connections 2024-09-16T00:14:46.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:14:46.078+0000 7febb9df4640 1 -- 172.21.15.78:0/1124622906 wait complete. 2024-09-16T00:14:46.819 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '56b5046b', total seconds elapsed: 33.6 2024-09-16T00:14:47.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:47 smithi193 ceph-mon[35484]: pgmap v402: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 524 KiB/s wr, 47 op/s 2024-09-16T00:14:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:47 smithi148 ceph-mon[35599]: pgmap v402: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 524 KiB/s wr, 47 op/s 2024-09-16T00:14:48.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:47 smithi078 ceph-mon[31157]: pgmap v402: 129 pgs: 129 active+clean; 964 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 524 KiB/s wr, 47 op/s 2024-09-16T00:14:49.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:49 smithi193 ceph-mon[35484]: pgmap v403: 129 pgs: 129 active+clean; 965 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 741 KiB/s wr, 74 op/s 2024-09-16T00:14:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:49 smithi148 ceph-mon[35599]: pgmap v403: 129 pgs: 129 active+clean; 965 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 741 KiB/s wr, 74 op/s 2024-09-16T00:14:50.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:49 smithi078 ceph-mon[31157]: pgmap v403: 129 pgs: 129 active+clean; 965 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 741 KiB/s wr, 74 op/s 2024-09-16T00:14:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 50.0 s, 3742.7 tps, lat 141.197 ms stddev 1578.030 2024-09-16T00:14:51.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:51 smithi193 ceph-mon[35484]: pgmap v404: 129 pgs: 129 active+clean; 965 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-16T00:14:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:51 smithi148 ceph-mon[35599]: pgmap v404: 129 pgs: 129 active+clean; 965 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-16T00:14:52.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:51 smithi078 ceph-mon[31157]: pgmap v404: 129 pgs: 129 active+clean; 965 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-16T00:14:53.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:53 smithi193 ceph-mon[35484]: pgmap v405: 129 pgs: 129 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-16T00:14:53.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:53 smithi148 ceph-mon[35599]: pgmap v405: 129 pgs: 129 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-16T00:14:54.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:53 smithi078 ceph-mon[31157]: pgmap v405: 129 pgs: 129 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 MiB/s wr, 250 op/s 2024-09-16T00:14:55.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 55.0 s, 4275.9 tps, lat 23.347 ms stddev 25.746 2024-09-16T00:14:55.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:55 smithi193 ceph-mon[35484]: pgmap v406: 129 pgs: 129 active+clean; 983 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 373 op/s 2024-09-16T00:14:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:55 smithi148 ceph-mon[35599]: pgmap v406: 129 pgs: 129 active+clean; 983 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 373 op/s 2024-09-16T00:14:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:55 smithi078 ceph-mon[31157]: pgmap v406: 129 pgs: 129 active+clean; 983 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.1 MiB/s wr, 373 op/s 2024-09-16T00:14:57.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:57 smithi193 ceph-mon[35484]: pgmap v407: 129 pgs: 129 active+clean; 983 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 451 op/s 2024-09-16T00:14:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:57 smithi148 ceph-mon[35599]: pgmap v407: 129 pgs: 129 active+clean; 983 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 451 op/s 2024-09-16T00:14:58.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:57 smithi078 ceph-mon[31157]: pgmap v407: 129 pgs: 129 active+clean; 983 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.7 MiB/s wr, 451 op/s 2024-09-16T00:14:59.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:14:59 smithi193 ceph-mon[35484]: pgmap v408: 129 pgs: 129 active+clean; 985 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.1 MiB/s wr, 741 op/s 2024-09-16T00:14:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:14:59 smithi148 ceph-mon[35599]: pgmap v408: 129 pgs: 129 active+clean; 985 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.1 MiB/s wr, 741 op/s 2024-09-16T00:15:00.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:14:59 smithi078 ceph-mon[31157]: pgmap v408: 129 pgs: 129 active+clean; 985 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 9.1 MiB/s wr, 741 op/s 2024-09-16T00:15:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 60.0 s, 4366.0 tps, lat 22.942 ms stddev 29.152 2024-09-16T00:15:01.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:01 smithi193 ceph-mon[35484]: pgmap v409: 129 pgs: 129 active+clean; 994 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-09-16T00:15:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:01 smithi148 ceph-mon[35599]: pgmap v409: 129 pgs: 129 active+clean; 994 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-09-16T00:15:02.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:01 smithi078 ceph-mon[31157]: pgmap v409: 129 pgs: 129 active+clean; 994 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-09-16T00:15:03.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:03 smithi193 ceph-mon[35484]: pgmap v410: 129 pgs: 129 active+clean; 998 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 15 MiB/s wr, 962 op/s 2024-09-16T00:15:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:03 smithi148 ceph-mon[35599]: pgmap v410: 129 pgs: 129 active+clean; 998 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 15 MiB/s wr, 962 op/s 2024-09-16T00:15:04.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:03 smithi078 ceph-mon[31157]: pgmap v410: 129 pgs: 129 active+clean; 998 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 15 MiB/s wr, 962 op/s 2024-09-16T00:15:05.264 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:15:05.684 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:05.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 65.0 s, 3428.0 tps, lat 29.049 ms stddev 32.182 2024-09-16T00:15:05.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:05 smithi193 ceph-mon[35484]: pgmap v411: 129 pgs: 129 active+clean; 1003 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 17 MiB/s wr, 950 op/s 2024-09-16T00:15:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:05 smithi148 ceph-mon[35599]: pgmap v411: 129 pgs: 129 active+clean; 1003 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 17 MiB/s wr, 950 op/s 2024-09-16T00:15:05.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:05 smithi078 ceph-mon[31157]: pgmap v411: 129 pgs: 129 active+clean; 1003 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 17 MiB/s wr, 950 op/s 2024-09-16T00:15:06.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.235+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1675557729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac100e20 msgr2=0x7fb0ac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.235+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1675557729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac100e20 0x7fb0ac101200 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009930 tx=0x7fb09c02a4d0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.237+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1675557729 shutdown_connections 2024-09-16T00:15:06.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.237+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1675557729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0ac102ef0 0x7fb0ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.237+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1675557729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0ac102570 0x7fb0ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.237+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1675557729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac100e20 0x7fb0ac101200 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.237+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1675557729 >> 172.21.15.78:0/1675557729 conn(0x7fb0ac0fc7d0 msgr2=0x7fb0ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:06.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.237+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1675557729 shutdown_connections 2024-09-16T00:15:06.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.237+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1675557729 wait complete. 2024-09-16T00:15:06.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.238+0000 7fb0b1f38640 1 Processor -- start 2024-09-16T00:15:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.238+0000 7fb0b1f38640 1 -- start start 2024-09-16T00:15:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.238+0000 7fb0b1f38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0ac100e20 0x7fb0ac19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0b1f38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac102570 0x7fb0ac19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0b1f38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0ac102ef0 0x7fb0ac10c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0b1f38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac112050 con 0x7fb0ac102ef0 2024-09-16T00:15:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0b1f38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac111ed0 con 0x7fb0ac100e20 2024-09-16T00:15:06.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0b1f38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1121d0 con 0x7fb0ac102570 2024-09-16T00:15:06.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0aaffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac102570 0x7fb0ac19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0ac102ef0 0x7fb0ac10c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0ab7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0ac100e20 0x7fb0ac19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0aaffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac102570 0x7fb0ac19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:55126/0 (socket says 172.21.15.78:55126) 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0ac102ef0 0x7fb0ac10c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45964/0 (socket says 172.21.15.78:45964) 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1931439412 learned_addr learned my addr 172.21.15.78:0/1931439412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0ab7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0ac100e20 0x7fb0ac19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37470/0 (socket says 172.21.15.78:37470) 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0abfff640 1 -- 172.21.15.78:0/1931439412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac102570 msgr2=0x7fb0ac19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0abfff640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac102570 0x7fb0ac19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0abfff640 1 -- 172.21.15.78:0/1931439412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0ac100e20 msgr2=0x7fb0ac19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0abfff640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0ac100e20 0x7fb0ac19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0abfff640 1 -- 172.21.15.78:0/1931439412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac10d150 con 0x7fb0ac102ef0 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac102570 0x7fb0ac19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:06.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.239+0000 7fb0abfff640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0ac102ef0 0x7fb0ac10c890 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e940 tx=0x7fb0a400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:06.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.241+0000 7fb0a8ff9640 1 -- 172.21.15.78:0/1931439412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4004d20 con 0x7fb0ac102ef0 2024-09-16T00:15:06.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.241+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1931439412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac10d440 con 0x7fb0ac102ef0 2024-09-16T00:15:06.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.242+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1931439412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac19ed30 con 0x7fb0ac102ef0 2024-09-16T00:15:06.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.242+0000 7fb0a8ff9640 1 -- 172.21.15.78:0/1931439412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a4011470 con 0x7fb0ac102ef0 2024-09-16T00:15:06.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.242+0000 7fb0a8ff9640 1 -- 172.21.15.78:0/1931439412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a401f060 con 0x7fb0ac102ef0 2024-09-16T00:15:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.245+0000 7fb0b4311740 1 Processor -- start 2024-09-16T00:15:06.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.245+0000 7fb0a8ff9640 1 -- 172.21.15.78:0/1931439412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb0a40105f0 con 0x7fb0ac102ef0 2024-09-16T00:15:06.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.246+0000 7fb0b4311740 1 -- start start 2024-09-16T00:15:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.246+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3dc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.246+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.246+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3e7680 0x55db3b3e9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.246+0000 7fb0b4311740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55db3b3422e0 con 0x55db3b3e7680 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.246+0000 7fb0b4311740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55db3b2e9ce0 con 0x55db3b3dcdc0 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.246+0000 7fb0b4311740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55db3b33e2a0 con 0x55db3b3dc4a0 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.246+0000 7fb0aaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3e7680 0x55db3b3e9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3e7680 0x55db3b3e9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34884/0 (socket says 172.21.15.78:34884) 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1258340459 learned_addr learned my addr 172.21.15.78:0/1258340459 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1258340459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 msgr2=0x55db3b3dc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3dc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1258340459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3dcdc0 msgr2=0x55db3b3e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1258340459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db3b2ce440 con 0x55db3b3e7680 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0a8ff9640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb0940a3340 0x7fb0940a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb0940a3340 0x7fb0940a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3e7680 0x55db3b3e9a70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005d00 tx=0x7fb0a0005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6704920e9e11559b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.247+0000 7fb0a8ff9640 1 -- 172.21.15.78:0/1931439412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb0a40cfd10 con 0x7fb0ac102ef0 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.248+0000 7fb091ffb640 1 -- 172.21.15.78:0/1258340459 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0012cf0 con 0x55db3b3e7680 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.248+0000 7fb091ffb640 1 -- 172.21.15.78:0/1258340459 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a0010c00 con 0x55db3b3e7680 2024-09-16T00:15:06.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.248+0000 7fb091ffb640 1 -- 172.21.15.78:0/1258340459 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0003c10 con 0x55db3b3e7680 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 -- 172.21.15.78:0/1258340459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3e7680 msgr2=0x55db3b3e9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3e7680 0x55db3b3e9a70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005d00 tx=0x7fb0a0005c70 comp rx=0 tx=0).stop 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb0940a3340 0x7fb0940a5800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55db3b3d9890 tx=0x7fb09c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 -- 172.21.15.78:0/1258340459 shutdown_connections 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3e7680 0x55db3b3e9a70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1258340459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3dc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 -- 172.21.15.78:0/1258340459 >> 172.21.15.78:0/1258340459 conn(0x55db3b3d8a30 msgr2=0x55db3b3d8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 -- 172.21.15.78:0/1258340459 shutdown_connections 2024-09-16T00:15:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 -- 172.21.15.78:0/1258340459 wait complete. 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.249+0000 7fb0b4311740 1 -- start start 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0b4311740 1 Processor -- start 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3e7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3e4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0b4311740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55db3b3ea250 con 0x55db3b3dcdc0 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0b4311740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55db3b2fd050 con 0x55db3b3e7680 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0b4311740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55db3b3ea3d0 con 0x55db3b3dc4a0 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0b4311740 10 client.0 _ll_register_callbacks cb 0x55db3b33d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0aaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0aaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34890/0 (socket says 172.21.15.78:34890) 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1457169917 learned_addr learned my addr 172.21.15.78:0/1457169917 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.250+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.251+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1457169917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 msgr2=0x55db3b3e7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.251+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3e7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.251+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1457169917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 msgr2=0x55db3b3e4870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.251+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3e4870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.251+0000 7fb0aaffd640 1 -- 172.21.15.78:0/1457169917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db3b0b33c0 con 0x55db3b3dcdc0 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.251+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e42d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00037b0 tx=0x7fb0a0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d674338b77b25b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.251+0000 7fb08affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.251+0000 7fb08affd640 1 -- 172.21.15.78:0/1457169917 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a001f820 con 0x55db3b3dcdc0 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.252+0000 7fb0b4311740 1 -- 172.21.15.78:0/1457169917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a0010c00 con 0x55db3b3dcdc0 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.252+0000 7fb08affd640 1 -- 172.21.15.78:0/1457169917 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb0a001f9c0 con 0x55db3b3dcdc0 2024-09-16T00:15:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.252+0000 7fb08affd640 1 -- 172.21.15.78:0/1457169917 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb0a002e050 con 0x55db3b3dcdc0 2024-09-16T00:15:06.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.252+0000 7fb0b4311740 10 client.15711 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:06.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.252+0000 7fb0b4311740 1 -- 172.21.15.78:0/1457169917 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55db3b3f5cc0 con 0x55db3b3dcdc0 2024-09-16T00:15:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.252+0000 7fb08affd640 1 -- 172.21.15.78:0/1457169917 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a0010c00 con 0x55db3b3dcdc0 2024-09-16T00:15:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.253+0000 7fb08affd640 1 -- 172.21.15.78:0/1457169917 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb0a00273a0 con 0x55db3b3dcdc0 2024-09-16T00:15:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.253+0000 7fb0b4311740 10 client.15711 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.253+0000 7fb0b4311740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:15:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.253+0000 7fb0b4311740 20 client.15711 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.253+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55db3b3f72b0 0x55db3b3f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.253+0000 7fb0b4311740 4 client.15711 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:06.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.253+0000 7fb0b4311740 1 -- 172.21.15.78:0/1457169917 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db3b34f850 con 0x55db3b3f72b0 2024-09-16T00:15:06.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.255+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55db3b3f72b0 0x55db3b3f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.256+0000 7fb08affd640 10 client.15711 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:06.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.256+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55db3b3f72b0 0x55db3b3f96f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea268c9d6950cd6 server_cookie=11c0430e83e4218d in_seq=0 out_seq=0 2024-09-16T00:15:06.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.256+0000 7fb08affd640 1 -- 172.21.15.78:0/1457169917 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db3b34f850 con 0x55db3b3f72b0 2024-09-16T00:15:06.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.256+0000 7fb08affd640 10 client.15711 handle_command_reply: tid=0 2024-09-16T00:15:06.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.256+0000 7fb0b4311740 1 client.15711 shutdown 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 -- 172.21.15.78:0/1457169917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3dcdc0 msgr2=0x55db3b3e42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e42d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00037b0 tx=0x7fb0a0001e50 comp rx=0 tx=0).stop 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 -- 172.21.15.78:0/1457169917 shutdown_connections 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55db3b3f72b0 0x55db3b3f96f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3dcdc0 0x55db3b3e42d0 secure :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00037b0 tx=0x7fb0a0001e50 comp rx=0 tx=0).stop 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 --2- 172.21.15.78:0/1457169917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3e7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 -- 172.21.15.78:0/1457169917 >> 172.21.15.78:0/1457169917 conn(0x55db3b3d8a30 msgr2=0x55db3b3de710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb08affd640 0 client.15711 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.257+0000 7fb0b4311740 1 -- 172.21.15.78:0/1457169917 shutdown_connections 2024-09-16T00:15:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.259+0000 7fb0b4311740 1 -- 172.21.15.78:0/1457169917 wait complete. 2024-09-16T00:15:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.259+0000 7fb0b4311740 20 client.15711 trim_cache size 0 max 16384 2024-09-16T00:15:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.296+0000 7fb0b4311740 1 Processor -- start 2024-09-16T00:15:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.298+0000 7fb0b4311740 1 -- start start 2024-09-16T00:15:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.298+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.298+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3ff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.298+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3ff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.298+0000 7fb0b4311740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1bdaa0 con 0x55db3b3df8b0 2024-09-16T00:15:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.298+0000 7fb0b4311740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1bf670 con 0x55db3b3e7680 2024-09-16T00:15:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.298+0000 7fb0b4311740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1be170 con 0x55db3b3dc4a0 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.299+0000 7fb0abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.299+0000 7fb0abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3ff540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34898/0 (socket says 172.21.15.78:34898) 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.299+0000 7fb0abfff640 1 -- 172.21.15.78:0/201998532 learned_addr learned my addr 172.21.15.78:0/201998532 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.299+0000 7fb0abfff640 1 -- 172.21.15.78:0/201998532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 msgr2=0x55db3b3f3510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.299+0000 7fb0abfff640 1 --2- 172.21.15.78:0/201998532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.299+0000 7fb0abfff640 1 -- 172.21.15.78:0/201998532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 msgr2=0x55db3b3ff000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.299+0000 7fb0abfff640 1 --2- 172.21.15.78:0/201998532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3ff000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.299+0000 7fb0abfff640 1 -- 172.21.15.78:0/201998532 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db3b3e4270 con 0x55db3b3df8b0 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.300+0000 7fb0abfff640 1 --2- 172.21.15.78:0/201998532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3ff540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb0a409bd20 tx=0x7fb0a409bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71786c416c368c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.300+0000 7fb0917fa640 1 -- 172.21.15.78:0/201998532 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40b6020 con 0x55db3b3df8b0 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.300+0000 7fb0917fa640 1 -- 172.21.15.78:0/201998532 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a40a4be0 con 0x55db3b3df8b0 2024-09-16T00:15:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.300+0000 7fb0917fa640 1 -- 172.21.15.78:0/201998532 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4098820 con 0x55db3b3df8b0 2024-09-16T00:15:06.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.301+0000 7fb0b4311740 1 -- 172.21.15.78:0/201998532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 msgr2=0x55db3b3ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.301+0000 7fb0b4311740 1 --2- 172.21.15.78:0/201998532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3ff540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb0a409bd20 tx=0x7fb0a409bab0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.302+0000 7fb0b4311740 1 -- 172.21.15.78:0/201998532 shutdown_connections 2024-09-16T00:15:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.303+0000 7fb0b4311740 1 --2- 172.21.15.78:0/201998532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3ff540 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.303+0000 7fb0b4311740 1 --2- 172.21.15.78:0/201998532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.303+0000 7fb0b4311740 1 --2- 172.21.15.78:0/201998532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.303+0000 7fb0b4311740 1 -- 172.21.15.78:0/201998532 >> 172.21.15.78:0/201998532 conn(0x55db3b3d8a30 msgr2=0x55db3b3e2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:06.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.303+0000 7fb0b4311740 1 -- 172.21.15.78:0/201998532 shutdown_connections 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.303+0000 7fb0b4311740 1 -- 172.21.15.78:0/201998532 wait complete. 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.303+0000 7fb0b4311740 1 -- start start 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0b4311740 1 Processor -- start 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0b4311740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3ef100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0b4311740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1bef70 con 0x55db3b3df8b0 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0b4311740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1be8a0 con 0x55db3b3e7680 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0b4311740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac1bba80 con 0x55db3b3dc4a0 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0ab7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0b4311740 10 client.0 _ll_register_callbacks cb 0x7fb09c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0ab7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3ee680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42202/0 (socket says 172.21.15.78:42202) 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0ab7fe640 1 -- 172.21.15.78:0/128953239 learned_addr learned my addr 172.21.15.78:0/128953239 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.304+0000 7fb0abfff640 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb0ab7fe640 1 -- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 msgr2=0x55db3b3ef100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3ef100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb0ab7fe640 1 -- 172.21.15.78:0/128953239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 msgr2=0x55db3b3eebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3eebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb0ab7fe640 1 -- 172.21.15.78:0/128953239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db3b3e7390 con 0x55db3b3dc4a0 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb0abfff640 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb0ab7fe640 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3ee680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009930 tx=0x7fb09c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9245217aa81fbe7d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb08affd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:06.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb08affd640 1 -- 172.21.15.78:0/128953239 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c035040 con 0x55db3b3dc4a0 2024-09-16T00:15:06.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb0b4311740 1 -- 172.21.15.78:0/128953239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a40a4be0 con 0x55db3b3dc4a0 2024-09-16T00:15:06.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.305+0000 7fb08affd640 1 -- 172.21.15.78:0/128953239 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb09c03ddb0 con 0x55db3b3dc4a0 2024-09-16T00:15:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.306+0000 7fb08affd640 1 -- 172.21.15.78:0/128953239 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb09c061050 con 0x55db3b3dc4a0 2024-09-16T00:15:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.306+0000 7fb0b4311740 10 client.25099 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.306+0000 7fb0b4311740 1 -- 172.21.15.78:0/128953239 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55db3b4734f0 con 0x55db3b3dc4a0 2024-09-16T00:15:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.306+0000 7fb08affd640 1 -- 172.21.15.78:0/128953239 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a40a4be0 con 0x55db3b3dc4a0 2024-09-16T00:15:06.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.307+0000 7fb08affd640 1 -- 172.21.15.78:0/128953239 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fb09c044040 con 0x55db3b3dc4a0 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.307+0000 7fb0b4311740 10 client.25099 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.307+0000 7fb0b4311740 10 client.25099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.307+0000 7fb0b4311740 20 client.25099 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.307+0000 7fb0b4311740 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55db3b473c80 0x55db3b4760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.307+0000 7fb0b4311740 4 client.25099 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.307+0000 7fb0b4311740 1 -- 172.21.15.78:0/128953239 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db3b349d60 con 0x55db3b473c80 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.307+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55db3b473c80 0x55db3b4760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.308+0000 7fb0aaffd640 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55db3b473c80 0x55db3b4760a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed4b7f3e3a2cfa6 server_cookie=162996d9b0378167 in_seq=0 out_seq=0 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.308+0000 7fb08affd640 10 client.25099 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.308+0000 7fb08affd640 1 -- 172.21.15.78:0/128953239 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55db3b349d60 con 0x55db3b473c80 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.308+0000 7fb08affd640 10 client.25099 handle_command_reply: tid=0 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.308+0000 7fb0b4311740 1 client.25099 shutdown 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.308+0000 7fb0b4311740 1 -- 172.21.15.78:0/128953239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 msgr2=0x55db3b3ee680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.308+0000 7fb0b4311740 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3ee680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009930 tx=0x7fb09c005c00 comp rx=0 tx=0).stop 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 1 -- 172.21.15.78:0/128953239 shutdown_connections 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55db3b473c80 0x55db3b4760a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb08affd640 0 client.25099 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55db3b3e7680 0x55db3b3ef100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55db3b3df8b0 0x55db3b3eebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 1 --2- 172.21.15.78:0/128953239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55db3b3dc4a0 0x55db3b3ee680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 1 -- 172.21.15.78:0/128953239 >> 172.21.15.78:0/128953239 conn(0x55db3b3d8a30 msgr2=0x55db3b3e6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 1 -- 172.21.15.78:0/128953239 shutdown_connections 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 1 -- 172.21.15.78:0/128953239 wait complete. 2024-09-16T00:15:06.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.309+0000 7fb0b4311740 20 client.25099 trim_cache size 0 max 16384 2024-09-16T00:15:06.313 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:15:06.313 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:15:06.313 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:15:06.313 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:15:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.310+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb0940a3340 msgr2=0x7fb0940a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.310+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb0940a3340 0x7fb0940a5800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55db3b3d9890 tx=0x7fb09c00b040 comp rx=0 tx=0).stop 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.310+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0ac102ef0 msgr2=0x7fb0ac10c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.310+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0ac102ef0 0x7fb0ac10c890 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e940 tx=0x7fb0a400ee00 comp rx=0 tx=0).stop 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.311+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1931439412 shutdown_connections 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.311+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb0940a3340 0x7fb0940a5800 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.311+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb0ac102ef0 0x7fb0ac10c890 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.311+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0ac102570 0x7fb0ac19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.311+0000 7fb0b1f38640 1 --2- 172.21.15.78:0/1931439412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb0ac100e20 0x7fb0ac19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.311+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1931439412 >> 172.21.15.78:0/1931439412 conn(0x7fb0ac0fc7d0 msgr2=0x7fb0ac103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.311+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1931439412 shutdown_connections 2024-09-16T00:15:06.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:06.311+0000 7fb0b1f38640 1 -- 172.21.15.78:0/1931439412 wait complete. 2024-09-16T00:15:06.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:06 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:15:06.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:06 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:15:07.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:06 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:15:07.189 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:15:07.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:15:07.537 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:07 smithi193 ceph-mon[35484]: pgmap v412: 129 pgs: 129 active+clean; 1003 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 17 MiB/s wr, 1.00k op/s 2024-09-16T00:15:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:15:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:15:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:07 smithi148 ceph-mon[35599]: pgmap v412: 129 pgs: 129 active+clean; 1003 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 17 MiB/s wr, 1.00k op/s 2024-09-16T00:15:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:07.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:15:07.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:15:07.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:07 smithi078 ceph-mon[31157]: pgmap v412: 129 pgs: 129 active+clean; 1003 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 17 MiB/s wr, 1.00k op/s 2024-09-16T00:15:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:15:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:15:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:15:08.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.158+0000 7ff088a8a640 1 -- 172.21.15.78:0/3326589360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109450 msgr2=0x7ff084110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:08.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.158+0000 7ff088a8a640 1 --2- 172.21.15.78:0/3326589360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109450 0x7ff084110250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff074009ca0 tx=0x7ff07402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.160+0000 7ff088a8a640 1 -- 172.21.15.78:0/3326589360 shutdown_connections 2024-09-16T00:15:08.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.160+0000 7ff088a8a640 1 --2- 172.21.15.78:0/3326589360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08411b140 0x7ff08411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.160+0000 7ff088a8a640 1 --2- 172.21.15.78:0/3326589360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff084109830 0x7ff084109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.160+0000 7ff088a8a640 1 --2- 172.21.15.78:0/3326589360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109450 0x7ff084110250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.160+0000 7ff088a8a640 1 -- 172.21.15.78:0/3326589360 >> 172.21.15.78:0/3326589360 conn(0x7ff08406e140 msgr2=0x7ff08406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:08.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.161+0000 7ff088a8a640 1 -- 172.21.15.78:0/3326589360 shutdown_connections 2024-09-16T00:15:08.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.161+0000 7ff088a8a640 1 -- 172.21.15.78:0/3326589360 wait complete. 2024-09-16T00:15:08.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.162+0000 7ff088a8a640 1 Processor -- start 2024-09-16T00:15:08.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.162+0000 7ff088a8a640 1 -- start start 2024-09-16T00:15:08.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.162+0000 7ff088a8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff084109450 0x7ff0841a7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:08.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.162+0000 7ff088a8a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109830 0x7ff0841a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:08.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.162+0000 7ff088a8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08411b140 0x7ff0841abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:08.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.162+0000 7ff088a8a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff084120e30 con 0x7ff084109450 2024-09-16T00:15:08.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.162+0000 7ff088a8a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff084120cb0 con 0x7ff08411b140 2024-09-16T00:15:08.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.162+0000 7ff088a8a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff084120fb0 con 0x7ff084109830 2024-09-16T00:15:08.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.163+0000 7ff08259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff084109450 0x7ff0841a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.163+0000 7ff08259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff084109450 0x7ff0841a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34916/0 (socket says 172.21.15.78:34916) 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.163+0000 7ff08259c640 1 -- 172.21.15.78:0/9113263 learned_addr learned my addr 172.21.15.78:0/9113263 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.163+0000 7ff082d9d640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08411b140 0x7ff0841abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.163+0000 7ff081d9b640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109830 0x7ff0841a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.164+0000 7ff08259c640 1 -- 172.21.15.78:0/9113263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109830 msgr2=0x7ff0841a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.164+0000 7ff08259c640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109830 0x7ff0841a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.164+0000 7ff08259c640 1 -- 172.21.15.78:0/9113263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08411b140 msgr2=0x7ff0841abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.164+0000 7ff08259c640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08411b140 0x7ff0841abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.164+0000 7ff08259c640 1 -- 172.21.15.78:0/9113263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0841ac300 con 0x7ff084109450 2024-09-16T00:15:08.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.164+0000 7ff082d9d640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08411b140 0x7ff0841abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:08.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.164+0000 7ff08259c640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff084109450 0x7ff0841a7350 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff07402aed0 tx=0x7ff074032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:08.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.165+0000 7ff07b7fe640 1 -- 172.21.15.78:0/9113263 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074003d30 con 0x7ff084109450 2024-09-16T00:15:08.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.165+0000 7ff07b7fe640 1 -- 172.21.15.78:0/9113263 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff074004520 con 0x7ff084109450 2024-09-16T00:15:08.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.165+0000 7ff07b7fe640 1 -- 172.21.15.78:0/9113263 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074030d60 con 0x7ff084109450 2024-09-16T00:15:08.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.165+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0841ac590 con 0x7ff084109450 2024-09-16T00:15:08.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.165+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0841b3ed0 con 0x7ff084109450 2024-09-16T00:15:08.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.167+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0841a86b0 con 0x7ff084109450 2024-09-16T00:15:08.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.171+0000 7ff07b7fe640 1 -- 172.21.15.78:0/9113263 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff074004040 con 0x7ff084109450 2024-09-16T00:15:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.172+0000 7ff07b7fe640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0700a3380 0x7ff0700a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.172+0000 7ff081d9b640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0700a3380 0x7ff0700a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.172+0000 7ff07b7fe640 1 -- 172.21.15.78:0/9113263 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff0740f1250 con 0x7ff084109450 2024-09-16T00:15:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.173+0000 7ff081d9b640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0700a3380 0x7ff0700a5840 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff08411d8c0 tx=0x7ff06c0090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.176+0000 7ff07b7fe640 1 -- 172.21.15.78:0/9113263 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0740801b0 con 0x7ff084109450 2024-09-16T00:15:08.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.514+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff08410b3a0 con 0x7ff084109450 2024-09-16T00:15:08.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.515+0000 7ff07b7fe640 1 -- 172.21.15.78:0/9113263 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7ff0740bdcd0 con 0x7ff084109450 2024-09-16T00:15:08.517 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:15:08.517 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:15:08.517 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:15:08.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0700a3380 msgr2=0x7ff0700a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:08.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0700a3380 0x7ff0700a5840 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff08411d8c0 tx=0x7ff06c0090e0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff084109450 msgr2=0x7ff0841a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:08.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff084109450 0x7ff0841a7350 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff07402aed0 tx=0x7ff074032fb0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 shutdown_connections 2024-09-16T00:15:08.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0700a3380 0x7ff0700a5840 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff08411b140 0x7ff0841abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff084109830 0x7ff0841a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 --2- 172.21.15.78:0/9113263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff084109450 0x7ff0841a7350 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:08.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.518+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 >> 172.21.15.78:0/9113263 conn(0x7ff08406e140 msgr2=0x7ff08411bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:08.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.519+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 shutdown_connections 2024-09-16T00:15:08.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:08.519+0000 7ff088a8a640 1 -- 172.21.15.78:0/9113263 wait complete. 2024-09-16T00:15:08.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:08 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/9113263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:15:08.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:08 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/9113263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:15:08.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:08 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/9113263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:15:09.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:15:09.694 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:09.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:09 smithi193 ceph-mon[35484]: pgmap v413: 129 pgs: 129 active+clean; 1004 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-16T00:15:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:09 smithi148 ceph-mon[35599]: pgmap v413: 129 pgs: 129 active+clean; 1004 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-16T00:15:09.981 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:09 smithi078 ceph-mon[31157]: pgmap v413: 129 pgs: 129 active+clean; 1004 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 1.10k op/s 2024-09-16T00:15:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.228+0000 7f7c9f034640 1 -- 172.21.15.78:0/4258845869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 msgr2=0x7f7c98101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.228+0000 7f7c9f034640 1 --2- 172.21.15.78:0/4258845869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 0x7f7c98101620 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009900 tx=0x7f7c8002a410 comp rx=0 tx=0).stop 2024-09-16T00:15:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.229+0000 7f7c9f034640 1 -- 172.21.15.78:0/4258845869 shutdown_connections 2024-09-16T00:15:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.229+0000 7f7c9f034640 1 --2- 172.21.15.78:0/4258845869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98101b60 0x7f7c9810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.229+0000 7f7c9f034640 1 --2- 172.21.15.78:0/4258845869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 0x7f7c98101620 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.229+0000 7f7c9f034640 1 --2- 172.21.15.78:0/4258845869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98105ad0 0x7f7c98105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.229+0000 7f7c9f034640 1 -- 172.21.15.78:0/4258845869 >> 172.21.15.78:0/4258845869 conn(0x7f7c98077210 msgr2=0x7f7c980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.230+0000 7f7c9f034640 1 -- 172.21.15.78:0/4258845869 shutdown_connections 2024-09-16T00:15:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.230+0000 7f7c9f034640 1 -- 172.21.15.78:0/4258845869 wait complete. 2024-09-16T00:15:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.230+0000 7f7c9f034640 1 Processor -- start 2024-09-16T00:15:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9f034640 1 -- start start 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9f034640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 0x7f7c981992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9f034640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98101b60 0x7f7c981997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9f034640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98105ad0 0x7f7c981a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9f034640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98110eb0 con 0x7f7c981011c0 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9f034640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98110d30 con 0x7f7c98101b60 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9f034640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98111030 con 0x7f7c98105ad0 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9cdd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 0x7f7c981992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9cdd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 0x7f7c981992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34926/0 (socket says 172.21.15.78:34926) 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9cdd0640 1 -- 172.21.15.78:0/1579461384 learned_addr learned my addr 172.21.15.78:0/1579461384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98105ad0 0x7f7c981a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c97fff640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98101b60 0x7f7c981997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/1579461384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98101b60 msgr2=0x7f7c981997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98101b60 0x7f7c981997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 msgr2=0x7f7c981992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 0x7f7c981992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/1579461384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c981a4870 con 0x7f7c98105ad0 2024-09-16T00:15:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.231+0000 7f7c9cdd0640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 0x7f7c981992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.232+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98105ad0 0x7f7c981a4330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009c50 tx=0x7f7c9000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.232+0000 7f7c95ffb640 1 -- 172.21.15.78:0/1579461384 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90005220 con 0x7f7c98105ad0 2024-09-16T00:15:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.232+0000 7f7c95ffb640 1 -- 172.21.15.78:0/1579461384 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c90012530 con 0x7f7c98105ad0 2024-09-16T00:15:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.232+0000 7f7c9f034640 1 -- 172.21.15.78:0/1579461384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c981970a0 con 0x7f7c98105ad0 2024-09-16T00:15:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.232+0000 7f7c95ffb640 1 -- 172.21.15.78:0/1579461384 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90004c20 con 0x7f7c98105ad0 2024-09-16T00:15:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.232+0000 7f7c9f034640 1 -- 172.21.15.78:0/1579461384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c98197560 con 0x7f7c98105ad0 2024-09-16T00:15:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.236+0000 7f7c95ffb640 1 -- 172.21.15.78:0/1579461384 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7c90021020 con 0x7f7c98105ad0 2024-09-16T00:15:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.236+0000 7f7c95ffb640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c840a32c0 0x7f7c840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7c9cdd0640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c840a32c0 0x7f7c840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7c95ffb640 1 -- 172.21.15.78:0/1579461384 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7c900d3910 con 0x7f7c98105ad0 2024-09-16T00:15:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7ca140d740 1 Processor -- start 2024-09-16T00:15:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7c9cdd0640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c840a32c0 0x7f7c840a5780 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800e740 tx=0x7f7c88009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7ca140d740 1 -- start start 2024-09-16T00:15:10.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 0x56075215c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075215ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x560752169760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7ca140d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607520c22f0 con 0x560752167370 2024-09-16T00:15:10.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7ca140d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560752069cf0 con 0x56075215c5d0 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.237+0000 7f7ca140d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607520be2b0 con 0x56075215db40 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075215ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075215ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:51798/0 (socket says 172.21.15.78:51798) 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c97fff640 1 -- 172.21.15.78:0/2639258325 learned_addr learned my addr 172.21.15.78:0/2639258325 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x560752169760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c9cdd0640 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 0x56075215c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c97fff640 1 -- 172.21.15.78:0/2639258325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 msgr2=0x56075215c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c97fff640 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 0x56075215c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c97fff640 1 -- 172.21.15.78:0/2639258325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 msgr2=0x560752169760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c97fff640 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x560752169760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c97fff640 1 -- 172.21.15.78:0/2639258325 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075204e3a0 con 0x56075215c5d0 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.238+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x560752169760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:15:10.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.239+0000 7f7c97fff640 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075215ca10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c8002a930 tx=0x7f7c80005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ecd56f53adc8dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.239+0000 7f7c8effd640 1 -- 172.21.15.78:0/2639258325 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80035030 con 0x56075215c5d0 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.239+0000 7f7c8effd640 1 -- 172.21.15.78:0/2639258325 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c80040070 con 0x56075215c5d0 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.239+0000 7f7c8effd640 1 -- 172.21.15.78:0/2639258325 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80031930 con 0x56075215c5d0 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 -- 172.21.15.78:0/2639258325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 msgr2=0x56075215ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075215ca10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c8002a930 tx=0x7f7c80005e80 comp rx=0 tx=0).stop 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 -- 172.21.15.78:0/2639258325 shutdown_connections 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x560752169760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075215ca10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 --2- 172.21.15.78:0/2639258325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 0x56075215c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 -- 172.21.15.78:0/2639258325 >> 172.21.15.78:0/2639258325 conn(0x560752125310 msgr2=0x560752158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 -- 172.21.15.78:0/2639258325 shutdown_connections 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 -- 172.21.15.78:0/2639258325 wait complete. 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 -- start start 2024-09-16T00:15:10.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.240+0000 7f7ca140d740 1 Processor -- start 2024-09-16T00:15:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215c5d0 0x560752166d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215db40 0x560752164040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x5607521646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7ca140d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56075216a010 con 0x560752167370 2024-09-16T00:15:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7ca140d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56075207d060 con 0x56075215db40 2024-09-16T00:15:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7ca140d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56075216a190 con 0x56075215c5d0 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7ca140d740 10 client.0 _ll_register_callbacks cb 0x560751dd08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7c9cdd0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215c5d0 0x560752166d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7c9cdd0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215c5d0 0x560752166d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42268/0 (socket says 172.21.15.78:42268) 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.241+0000 7f7c9cdd0640 1 -- 172.21.15.78:0/3663371256 learned_addr learned my addr 172.21.15.78:0/3663371256 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.242+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x5607521646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.242+0000 7f7c97fff640 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215db40 0x560752164040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.242+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/3663371256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215c5d0 msgr2=0x560752166d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.242+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215c5d0 0x560752166d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.242+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/3663371256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215db40 msgr2=0x560752164040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.242+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215db40 0x560752164040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.242+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/3663371256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560751e333c0 con 0x560752167370 2024-09-16T00:15:10.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x5607521646a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x56075215d070 tx=0x7f7c90004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1e58ff865328c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7c77fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7c77fff640 1 -- 172.21.15.78:0/3663371256 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900a3f20 con 0x560752167370 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7c77fff640 1 -- 172.21.15.78:0/3663371256 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7c900a8350 con 0x560752167370 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7ca140d740 1 -- 172.21.15.78:0/3663371256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c80040070 con 0x560752167370 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7c77fff640 1 -- 172.21.15.78:0/3663371256 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c900a87e0 con 0x560752167370 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7ca140d740 10 client.15729 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7ca140d740 1 -- 172.21.15.78:0/3663371256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560752175a90 con 0x560752167370 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.243+0000 7f7c77fff640 1 -- 172.21.15.78:0/3663371256 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c80040070 con 0x560752167370 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.244+0000 7f7c77fff640 1 -- 172.21.15.78:0/3663371256 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f7c9008a070 con 0x560752167370 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.244+0000 7f7ca140d740 10 client.15729 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.244+0000 7f7ca140d740 10 client.15729 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.244+0000 7f7ca140d740 20 client.15729 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.244+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x560752176ea0 0x5607521792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.244+0000 7f7ca140d740 4 client.15729 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:10.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.244+0000 7f7ca140d740 1 -- 172.21.15.78:0/3663371256 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607520cf860 con 0x560752176ea0 2024-09-16T00:15:10.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.245+0000 7f7c9cdd0640 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x560752176ea0 0x5607521792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.246+0000 7f7c77fff640 10 client.15729 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:15:10.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.246+0000 7f7c9cdd0640 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x560752176ea0 0x5607521792b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aca0aefca00088c0 server_cookie=88928547749074cd in_seq=0 out_seq=0 2024-09-16T00:15:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.246+0000 7f7c77fff640 1 -- 172.21.15.78:0/3663371256 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607520cf860 con 0x560752176ea0 2024-09-16T00:15:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.246+0000 7f7c77fff640 10 client.15729 handle_command_reply: tid=0 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.246+0000 7f7ca140d740 1 client.15729 shutdown 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 -- 172.21.15.78:0/3663371256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 msgr2=0x5607521646a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x5607521646a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x56075215d070 tx=0x7f7c90004f00 comp rx=0 tx=0).stop 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 -- 172.21.15.78:0/3663371256 shutdown_connections 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x5607521646a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215db40 0x560752164040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x560752176ea0 0x5607521792b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3663371256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215c5d0 0x560752166d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7c77fff640 0 client.15729 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 -- 172.21.15.78:0/3663371256 >> 172.21.15.78:0/3663371256 conn(0x560752125310 msgr2=0x56075215e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.247+0000 7f7ca140d740 1 -- 172.21.15.78:0/3663371256 shutdown_connections 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.248+0000 7f7ca140d740 1 -- 172.21.15.78:0/3663371256 wait complete. 2024-09-16T00:15:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.248+0000 7f7ca140d740 20 client.15729 trim_cache size 0 max 16384 2024-09-16T00:15:10.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.267+0000 7f7ca140d740 1 Processor -- start 2024-09-16T00:15:10.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.267+0000 7f7ca140d740 1 -- start start 2024-09-16T00:15:10.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.267+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x5607521732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.267+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 0x560752163080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.268+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x56075217eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.268+0000 7f7ca140d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98073b10 con 0x560752167370 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.268+0000 7f7ca140d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981a5d10 con 0x56075215c5d0 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.268+0000 7f7ca140d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c980741e0 con 0x56075215db40 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.268+0000 7f7c9d5d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x56075217eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.268+0000 7f7c9d5d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x56075217eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34962/0 (socket says 172.21.15.78:34962) 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.268+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/4172748814 learned_addr learned my addr 172.21.15.78:0/4172748814 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.269+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/4172748814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 msgr2=0x560752163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:10.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.269+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/4172748814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 0x560752163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.269+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/4172748814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 msgr2=0x5607521732b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:15:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.269+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/4172748814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x5607521732b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.269+0000 7f7c9d5d1640 1 -- 172.21.15.78:0/4172748814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607521646e0 con 0x560752167370 2024-09-16T00:15:10.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.273+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/4172748814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x56075217eb20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7c900a89a0 tx=0x7f7c90012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eae995757075f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:10.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.278+0000 7f7c8e7fc640 1 -- 172.21.15.78:0/4172748814 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9009f940 con 0x560752167370 2024-09-16T00:15:10.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.278+0000 7f7c8e7fc640 1 -- 172.21.15.78:0/4172748814 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c900b7060 con 0x560752167370 2024-09-16T00:15:10.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.279+0000 7f7ca140d740 1 -- 172.21.15.78:0/4172748814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 msgr2=0x56075217eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.279+0000 7f7ca140d740 1 --2- 172.21.15.78:0/4172748814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x56075217eb20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7c900a89a0 tx=0x7f7c90012810 comp rx=0 tx=0).stop 2024-09-16T00:15:10.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.281+0000 7f7ca140d740 1 -- 172.21.15.78:0/4172748814 shutdown_connections 2024-09-16T00:15:10.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.281+0000 7f7ca140d740 1 --2- 172.21.15.78:0/4172748814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560752167370 0x56075217eb20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.281+0000 7f7ca140d740 1 --2- 172.21.15.78:0/4172748814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56075215db40 0x560752163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.281+0000 7f7ca140d740 1 --2- 172.21.15.78:0/4172748814 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x5607521732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.281+0000 7f7ca140d740 1 -- 172.21.15.78:0/4172748814 >> 172.21.15.78:0/4172748814 conn(0x560752125310 msgr2=0x560752162970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:10.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.282+0000 7f7ca140d740 1 -- 172.21.15.78:0/4172748814 shutdown_connections 2024-09-16T00:15:10.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.283+0000 7f7ca140d740 1 -- 172.21.15.78:0/4172748814 wait complete. 2024-09-16T00:15:10.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 1 -- start start 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 1 Processor -- start 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075216fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56075215db40 0x560752160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560752167370 0x560752161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981a5610 con 0x56075215db40 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c981a4f40 con 0x56075215c5d0 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c98071af0 con 0x560752167370 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.284+0000 7f7ca140d740 10 client.0 _ll_register_callbacks cb 0x7f7c9009be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:10.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.285+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56075215db40 0x560752160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.285+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56075215db40 0x560752160ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34966/0 (socket says 172.21.15.78:34966) 2024-09-16T00:15:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.285+0000 7f7c97fff640 1 -- 172.21.15.78:0/3810917638 learned_addr learned my addr 172.21.15.78:0/3810917638 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.286+0000 7f7c9d5d1640 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560752167370 0x560752161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.286+0000 7f7c97fff640 1 -- 172.21.15.78:0/3810917638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560752167370 msgr2=0x560752161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.286+0000 7f7c97fff640 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560752167370 0x560752161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.286+0000 7f7c97fff640 1 -- 172.21.15.78:0/3810917638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 msgr2=0x56075216fc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:15:10.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.286+0000 7f7c97fff640 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075216fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.286+0000 7f7c97fff640 1 -- 172.21.15.78:0/3810917638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560752161cc0 con 0x56075215db40 2024-09-16T00:15:10.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.287+0000 7f7c97fff640 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56075215db40 0x560752160ec0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7c8002a450 tx=0x7f7c800044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577fe0fdbb1a63c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:10.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.287+0000 7f7c77fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:10.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.288+0000 7f7c77fff640 1 -- 172.21.15.78:0/3810917638 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80035070 con 0x56075215db40 2024-09-16T00:15:10.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.288+0000 7f7ca140d740 1 -- 172.21.15.78:0/3810917638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c900b7060 con 0x56075215db40 2024-09-16T00:15:10.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.288+0000 7f7c77fff640 1 -- 172.21.15.78:0/3810917638 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7c80005440 con 0x56075215db40 2024-09-16T00:15:10.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.288+0000 7f7c77fff640 1 -- 172.21.15.78:0/3810917638 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7c80057050 con 0x56075215db40 2024-09-16T00:15:10.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.290+0000 7f7ca140d740 10 client.15735 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:10.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.290+0000 7f7ca140d740 1 -- 172.21.15.78:0/3810917638 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5607521e1c20 con 0x56075215db40 2024-09-16T00:15:10.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.291+0000 7f7c77fff640 1 -- 172.21.15.78:0/3810917638 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c900b7060 con 0x56075215db40 2024-09-16T00:15:10.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.291+0000 7f7c77fff640 1 -- 172.21.15.78:0/3810917638 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f7c800039f0 con 0x56075215db40 2024-09-16T00:15:10.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.292+0000 7f7ca140d740 10 client.15735 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:10.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.292+0000 7f7ca140d740 10 client.15735 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:15:10.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.292+0000 7f7ca140d740 20 client.15735 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:10.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.292+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x560752176ea0 0x5607521e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:10.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.292+0000 7f7ca140d740 4 client.15735 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:15:10.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.292+0000 7f7ca140d740 1 -- 172.21.15.78:0/3810917638 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607520c9d70 con 0x560752176ea0 2024-09-16T00:15:10.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.295+0000 7f7c9cdd0640 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x560752176ea0 0x5607521e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:10.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.297+0000 7f7c9cdd0640 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x560752176ea0 0x5607521e3f10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55946781addecff6 server_cookie=c5903e6c6ba6ec01 in_seq=0 out_seq=0 2024-09-16T00:15:10.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.297+0000 7f7c77fff640 10 client.15735 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:15:10.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.298+0000 7f7c77fff640 1 -- 172.21.15.78:0/3810917638 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607520c9d70 con 0x560752176ea0 2024-09-16T00:15:10.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.298+0000 7f7c77fff640 10 client.15735 handle_command_reply: tid=0 2024-09-16T00:15:10.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.299+0000 7f7ca140d740 1 client.15735 shutdown 2024-09-16T00:15:10.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.302+0000 7f7ca140d740 1 -- 172.21.15.78:0/3810917638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56075215db40 msgr2=0x560752160ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.302+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56075215db40 0x560752160ec0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7c8002a450 tx=0x7f7c800044b0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.303+0000 7f7ca140d740 1 -- 172.21.15.78:0/3810917638 shutdown_connections 2024-09-16T00:15:10.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.303+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x560752176ea0 0x5607521e3f10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.303+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560752167370 0x560752161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.303+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56075215db40 0x560752160ec0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.303+0000 7f7ca140d740 1 --2- 172.21.15.78:0/3810917638 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56075215c5d0 0x56075216fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.303+0000 7f7ca140d740 1 -- 172.21.15.78:0/3810917638 >> 172.21.15.78:0/3810917638 conn(0x560752125310 msgr2=0x560752163190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:10.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.303+0000 7f7c77fff640 0 client.15735 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:15:10.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.306+0000 7f7ca140d740 1 -- 172.21.15.78:0/3810917638 shutdown_connections 2024-09-16T00:15:10.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.306+0000 7f7ca140d740 1 -- 172.21.15.78:0/3810917638 wait complete. 2024-09-16T00:15:10.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.306+0000 7f7ca140d740 20 client.15735 trim_cache size 0 max 16384 2024-09-16T00:15:10.308 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:15:10.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 -- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c840a32c0 msgr2=0x7f7c840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c840a32c0 0x7f7c840a5780 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800e740 tx=0x7f7c88009800 comp rx=0 tx=0).stop 2024-09-16T00:15:10.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 -- 172.21.15.78:0/1579461384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98105ad0 msgr2=0x7f7c981a4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:10.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98105ad0 0x7f7c981a4330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009c50 tx=0x7f7c9000c6a0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 -- 172.21.15.78:0/1579461384 shutdown_connections 2024-09-16T00:15:10.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c840a32c0 0x7f7c840a5780 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800e740 tx=0x7f7c88009800 comp rx=0 tx=0).stop 2024-09-16T00:15:10.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c98105ad0 0x7f7c981a4330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c98101b60 0x7f7c981997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 --2- 172.21.15.78:0/1579461384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c981011c0 0x7f7c981992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:10.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 -- 172.21.15.78:0/1579461384 >> 172.21.15.78:0/1579461384 conn(0x7f7c98077210 msgr2=0x7f7c98103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:10.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 -- 172.21.15.78:0/1579461384 shutdown_connections 2024-09-16T00:15:10.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:10.309+0000 7f7c9f034640 1 -- 172.21.15.78:0/1579461384 wait complete. 2024-09-16T00:15:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 70.0 s, 2952.6 tps, lat 33.829 ms stddev 42.875 2024-09-16T00:15:11.328 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:15:11.765 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:11.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:11 smithi193 ceph-mon[35484]: pgmap v414: 129 pgs: 129 active+clean; 1008 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 851 op/s 2024-09-16T00:15:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:11 smithi148 ceph-mon[35599]: pgmap v414: 129 pgs: 129 active+clean; 1008 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 851 op/s 2024-09-16T00:15:12.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:11 smithi078 ceph-mon[31157]: pgmap v414: 129 pgs: 129 active+clean; 1008 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 851 op/s 2024-09-16T00:15:12.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.584+0000 7f109d472640 1 -- 172.21.15.78:0/4063736786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102ef0 msgr2=0x7f109810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.584+0000 7f109d472640 1 --2- 172.21.15.78:0/4063736786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102ef0 0x7f109810da40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1090009900 tx=0x7f109002a490 comp rx=0 tx=0).stop 2024-09-16T00:15:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.585+0000 7f109d472640 1 -- 172.21.15.78:0/4063736786 shutdown_connections 2024-09-16T00:15:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.585+0000 7f109d472640 1 --2- 172.21.15.78:0/4063736786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102ef0 0x7f109810da40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.585+0000 7f109d472640 1 --2- 172.21.15.78:0/4063736786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1098102570 0x7f10981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.585+0000 7f109d472640 1 --2- 172.21.15.78:0/4063736786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098100e20 0x7f1098101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.585+0000 7f109d472640 1 -- 172.21.15.78:0/4063736786 >> 172.21.15.78:0/4063736786 conn(0x7f10980fc7d0 msgr2=0x7f10980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.585+0000 7f109d472640 1 -- 172.21.15.78:0/4063736786 shutdown_connections 2024-09-16T00:15:12.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.585+0000 7f109d472640 1 -- 172.21.15.78:0/4063736786 wait complete. 2024-09-16T00:15:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.586+0000 7f109d472640 1 Processor -- start 2024-09-16T00:15:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.586+0000 7f109d472640 1 -- start start 2024-09-16T00:15:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.586+0000 7f109d472640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098100e20 0x7f109810bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f109d472640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102570 0x7f109811b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f109d472640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1098102ef0 0x7f109810c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f109d472640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f109810f530 con 0x7f1098102570 2024-09-16T00:15:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f109d472640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f109810f3b0 con 0x7f1098100e20 2024-09-16T00:15:12.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f109d472640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f109810f6b0 con 0x7f1098102ef0 2024-09-16T00:15:12.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f10977fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1098102ef0 0x7f109810c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098100e20 0x7f109810bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102570 0x7f109811b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102570 0x7f109811b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34998/0 (socket says 172.21.15.78:34998) 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f108e7fc640 1 -- 172.21.15.78:0/2554818892 learned_addr learned my addr 172.21.15.78:0/2554818892 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.587+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098100e20 0x7f109810bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:51824/0 (socket says 172.21.15.78:51824) 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.588+0000 7f1096ffd640 1 -- 172.21.15.78:0/2554818892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1098102ef0 msgr2=0x7f109810c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.588+0000 7f1096ffd640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1098102ef0 0x7f109810c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.588+0000 7f1096ffd640 1 -- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102570 msgr2=0x7f109811b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f1096ffd640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102570 0x7f109811b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f1096ffd640 1 -- 172.21.15.78:0/2554818892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109810cc40 con 0x7f1098100e20 2024-09-16T00:15:12.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f108e7fc640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102570 0x7f109811b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f10977fe640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1098102ef0 0x7f109810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f1096ffd640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098100e20 0x7f109810bf80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f108800eb00 tx=0x7f108800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f1094ff9640 1 -- 172.21.15.78:0/2554818892 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088010430 con 0x7f1098100e20 2024-09-16T00:15:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f109d472640 1 -- 172.21.15.78:0/2554818892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981b38d0 con 0x7f1098100e20 2024-09-16T00:15:12.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f1094ff9640 1 -- 172.21.15.78:0/2554818892 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1088010ab0 con 0x7f1098100e20 2024-09-16T00:15:12.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.589+0000 7f1094ff9640 1 -- 172.21.15.78:0/2554818892 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088017b70 con 0x7f1098100e20 2024-09-16T00:15:12.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.590+0000 7f109d472640 1 -- 172.21.15.78:0/2554818892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981b3dd0 con 0x7f1098100e20 2024-09-16T00:15:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.593+0000 7f1094ff9640 1 -- 172.21.15.78:0/2554818892 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1088017d10 con 0x7f1098100e20 2024-09-16T00:15:12.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.594+0000 7f109f84b740 1 Processor -- start 2024-09-16T00:15:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.594+0000 7f109f84b740 1 -- start start 2024-09-16T00:15:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.594+0000 7f109f84b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd5750e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.594+0000 7f109f84b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750e5d0 0x55cd5750ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.594+0000 7f109f84b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd5751b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f109f84b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cd574742f0 con 0x55cd5750fb40 2024-09-16T00:15:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f109f84b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cd5741bcf0 con 0x55cd57519370 2024-09-16T00:15:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f109f84b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd574702b0 con 0x55cd5750e5d0 2024-09-16T00:15:12.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f1094ff9640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10840a3310 0x7f10840a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd5751b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd5751b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:51836/0 (socket says 172.21.15.78:51836) 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750e5d0 0x55cd5750ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f108e7fc640 1 -- 172.21.15.78:0/3110773481 learned_addr learned my addr 172.21.15.78:0/3110773481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f1094ff9640 1 -- 172.21.15.78:0/2554818892 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f10880d0550 con 0x7f1098100e20 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f10977fe640 1 --2- 172.21.15.78:0/3110773481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd5750e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f108e7fc640 1 -- 172.21.15.78:0/3110773481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750e5d0 msgr2=0x55cd5750ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f108e7fc640 1 --2- 172.21.15.78:0/3110773481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750e5d0 0x55cd5750ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f108e7fc640 1 -- 172.21.15.78:0/3110773481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 msgr2=0x55cd5750e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f108e7fc640 1 --2- 172.21.15.78:0/3110773481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd5750e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f10977fe640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10840a3310 0x7f10840a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.595+0000 7f108e7fc640 1 -- 172.21.15.78:0/3110773481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd574003a0 con 0x55cd57519370 2024-09-16T00:15:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.596+0000 7f108e7fc640 1 --2- 172.21.15.78:0/3110773481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd5751b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1080011510 tx=0x7f10800119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a0d9f8680b8c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.596+0000 7f10977fe640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10840a3310 0x7f10840a57d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1090005ab0 tx=0x7f109002a990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.596+0000 7f108d7fa640 1 -- 172.21.15.78:0/3110773481 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108001e070 con 0x55cd57519370 2024-09-16T00:15:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.596+0000 7f108d7fa640 1 -- 172.21.15.78:0/3110773481 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108000faa0 con 0x55cd57519370 2024-09-16T00:15:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.596+0000 7f108d7fa640 1 -- 172.21.15.78:0/3110773481 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108001e070 con 0x55cd57519370 2024-09-16T00:15:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.597+0000 7f109f84b740 1 -- 172.21.15.78:0/3110773481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 msgr2=0x55cd5751b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.597+0000 7f109f84b740 1 --2- 172.21.15.78:0/3110773481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd5751b760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1080011510 tx=0x7f10800119d0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.598+0000 7f109f84b740 1 -- 172.21.15.78:0/3110773481 shutdown_connections 2024-09-16T00:15:12.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.598+0000 7f109f84b740 1 --2- 172.21.15.78:0/3110773481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd5751b760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.598+0000 7f109f84b740 1 --2- 172.21.15.78:0/3110773481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750e5d0 0x55cd5750ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.598+0000 7f109f84b740 1 --2- 172.21.15.78:0/3110773481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd5750e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.598+0000 7f109f84b740 1 -- 172.21.15.78:0/3110773481 >> 172.21.15.78:0/3110773481 conn(0x55cd574d7310 msgr2=0x55cd5750ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.598+0000 7f109f84b740 1 -- 172.21.15.78:0/3110773481 shutdown_connections 2024-09-16T00:15:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.598+0000 7f109f84b740 1 -- 172.21.15.78:0/3110773481 wait complete. 2024-09-16T00:15:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f109f84b740 1 -- start start 2024-09-16T00:15:12.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f109f84b740 1 Processor -- start 2024-09-16T00:15:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f109f84b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750e5d0 0x55cd57518d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f109f84b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd57515fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f109f84b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd575165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f109f84b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55cd5751c0c0 con 0x55cd5750e5d0 2024-09-16T00:15:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f109f84b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cd5742f060 con 0x55cd57519370 2024-09-16T00:15:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f109f84b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd5751c240 con 0x55cd5750fb40 2024-09-16T00:15:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.599+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750e5d0 0x55cd57518d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd57515fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f1096ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750e5d0 0x55cd57518d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35010/0 (socket says 172.21.15.78:35010) 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f109f84b740 10 client.0 _ll_register_callbacks cb 0x55cd571828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f1096ffd640 1 -- 172.21.15.78:0/4027733217 learned_addr learned my addr 172.21.15.78:0/4027733217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f10977fe640 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd575165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f108e7fc640 1 -- 172.21.15.78:0/4027733217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 msgr2=0x55cd575165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f108e7fc640 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd575165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f108e7fc640 1 -- 172.21.15.78:0/4027733217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750e5d0 msgr2=0x55cd57518d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f108e7fc640 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750e5d0 0x55cd57518d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f108e7fc640 1 -- 172.21.15.78:0/4027733217 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd571e53c0 con 0x55cd5750fb40 2024-09-16T00:15:12.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.600+0000 7f10977fe640 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd575165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:15:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.602+0000 7f108e7fc640 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd57515fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1080011ed0 tx=0x7f1080009d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b15e0fe66779f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.602+0000 7f107e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.603+0000 7f107e7fc640 1 -- 172.21.15.78:0/4027733217 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108001e040 con 0x55cd5750fb40 2024-09-16T00:15:12.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.603+0000 7f107e7fc640 1 -- 172.21.15.78:0/4027733217 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1080027020 con 0x55cd5750fb40 2024-09-16T00:15:12.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.603+0000 7f109f84b740 1 -- 172.21.15.78:0/4027733217 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108000faa0 con 0x55cd5750fb40 2024-09-16T00:15:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.603+0000 7f107e7fc640 1 -- 172.21.15.78:0/4027733217 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f108001e040 con 0x55cd5750fb40 2024-09-16T00:15:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.603+0000 7f109f84b740 10 client.25132 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.603+0000 7f109f84b740 1 -- 172.21.15.78:0/4027733217 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cd57527b20 con 0x55cd5750fb40 2024-09-16T00:15:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.603+0000 7f107e7fc640 1 -- 172.21.15.78:0/4027733217 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108000faa0 con 0x55cd5750fb40 2024-09-16T00:15:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.604+0000 7f107e7fc640 1 -- 172.21.15.78:0/4027733217 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f1080021070 con 0x55cd5750fb40 2024-09-16T00:15:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.604+0000 7f109f84b740 10 client.25132 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:12.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.604+0000 7f109f84b740 10 client.25132 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:15:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.604+0000 7f109f84b740 20 client.25132 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.604+0000 7f109f84b740 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55cd57528f20 0x55cd5752b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.604+0000 7f109f84b740 4 client.25132 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.604+0000 7f109f84b740 1 -- 172.21.15.78:0/4027733217 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd57481860 con 0x55cd57528f20 2024-09-16T00:15:12.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.604+0000 7f1096ffd640 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55cd57528f20 0x55cd5752b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.605+0000 7f107e7fc640 10 client.25132 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:15:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.605+0000 7f1096ffd640 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55cd57528f20 0x55cd5752b310 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5aaeb1ff9bd2561 server_cookie=c50d1fa9ad023e1c in_seq=0 out_seq=0 2024-09-16T00:15:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.606+0000 7f107e7fc640 1 -- 172.21.15.78:0/4027733217 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd57481860 con 0x55cd57528f20 2024-09-16T00:15:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.606+0000 7f107e7fc640 10 client.25132 handle_command_reply: tid=0 2024-09-16T00:15:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.606+0000 7f109f84b740 1 client.25132 shutdown 2024-09-16T00:15:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.606+0000 7f109f84b740 1 -- 172.21.15.78:0/4027733217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 msgr2=0x55cd57515fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.606+0000 7f109f84b740 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd57515fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1080011ed0 tx=0x7f1080009d40 comp rx=0 tx=0).stop 2024-09-16T00:15:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 1 -- 172.21.15.78:0/4027733217 shutdown_connections 2024-09-16T00:15:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55cd57528f20 0x55cd5752b310 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f107e7fc640 0 client.25132 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:15:12.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd57519370 0x55cd575165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd57515fe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 1 --2- 172.21.15.78:0/4027733217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750e5d0 0x55cd57518d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 1 -- 172.21.15.78:0/4027733217 >> 172.21.15.78:0/4027733217 conn(0x55cd574d7310 msgr2=0x55cd57510960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 1 -- 172.21.15.78:0/4027733217 shutdown_connections 2024-09-16T00:15:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 1 -- 172.21.15.78:0/4027733217 wait complete. 2024-09-16T00:15:12.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.607+0000 7f109f84b740 20 client.25132 trim_cache size 0 max 16384 2024-09-16T00:15:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f109f84b740 1 Processor -- start 2024-09-16T00:15:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f109f84b740 1 -- start start 2024-09-16T00:15:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f109f84b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd57525360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f109f84b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd5751a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f109f84b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd57519370 0x55cd5751aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f109f84b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10981bdcc0 con 0x55cd57519370 2024-09-16T00:15:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f109f84b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10981bf890 con 0x55cd5750e5d0 2024-09-16T00:15:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd5751a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd5751a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42352/0 (socket says 172.21.15.78:42352) 2024-09-16T00:15:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f108e7fc640 1 -- 172.21.15.78:0/134884588 learned_addr learned my addr 172.21.15.78:0/134884588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.626+0000 7f109f84b740 1 -- 172.21.15.78:0/134884588 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10981be390 con 0x55cd5750fb40 2024-09-16T00:15:12.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.627+0000 7f1096ffd640 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd57525360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.627+0000 7f10977fe640 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd57519370 0x55cd5751aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.627+0000 7f108e7fc640 1 -- 172.21.15.78:0/134884588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 msgr2=0x55cd57525360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.627+0000 7f108e7fc640 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd57525360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.627+0000 7f108e7fc640 1 -- 172.21.15.78:0/134884588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd57519370 msgr2=0x55cd5751aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.627+0000 7f108e7fc640 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd57519370 0x55cd5751aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.627+0000 7f108e7fc640 1 -- 172.21.15.78:0/134884588 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd57519060 con 0x55cd5750fb40 2024-09-16T00:15:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.628+0000 7f1096ffd640 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd57525360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:15:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.628+0000 7f10977fe640 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd57519370 0x55cd5751aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:15:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.628+0000 7f108e7fc640 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd5751a760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1080005380 tx=0x7f1080005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e733b982cce52683 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.628+0000 7f108cff9640 1 -- 172.21.15.78:0/134884588 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108001e040 con 0x55cd5750fb40 2024-09-16T00:15:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.628+0000 7f108cff9640 1 -- 172.21.15.78:0/134884588 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108000faa0 con 0x55cd5750fb40 2024-09-16T00:15:12.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.628+0000 7f108cff9640 1 -- 172.21.15.78:0/134884588 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080019cd0 con 0x55cd5750fb40 2024-09-16T00:15:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.630+0000 7f109f84b740 1 -- 172.21.15.78:0/134884588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 msgr2=0x55cd5751a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.630+0000 7f109f84b740 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd5751a760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1080005380 tx=0x7f1080005d30 comp rx=0 tx=0).stop 2024-09-16T00:15:12.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.630+0000 7f109f84b740 1 -- 172.21.15.78:0/134884588 shutdown_connections 2024-09-16T00:15:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.630+0000 7f109f84b740 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd57519370 0x55cd5751aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.630+0000 7f109f84b740 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd5750fb40 0x55cd5751a760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.630+0000 7f109f84b740 1 --2- 172.21.15.78:0/134884588 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd57525360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.630+0000 7f109f84b740 1 -- 172.21.15.78:0/134884588 >> 172.21.15.78:0/134884588 conn(0x55cd574d7310 msgr2=0x55cd57518060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.630+0000 7f109f84b740 1 -- 172.21.15.78:0/134884588 shutdown_connections 2024-09-16T00:15:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.631+0000 7f109f84b740 1 -- 172.21.15.78:0/134884588 wait complete. 2024-09-16T00:15:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.631+0000 7f109f84b740 1 -- start start 2024-09-16T00:15:12.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.631+0000 7f109f84b740 1 Processor -- start 2024-09-16T00:15:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.631+0000 7f109f84b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd575305f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.631+0000 7f109f84b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd57530b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.631+0000 7f109f84b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd57519370 0x55cd57531070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.632+0000 7f109f84b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10981bf190 con 0x55cd5750fb40 2024-09-16T00:15:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.632+0000 7f109f84b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10981beac0 con 0x55cd5750e5d0 2024-09-16T00:15:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.632+0000 7f109f84b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10981bbc60 con 0x55cd57519370 2024-09-16T00:15:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.632+0000 7f109f84b740 10 client.0 _ll_register_callbacks cb 0x7f108001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:12.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.632+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd57530b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.632+0000 7f108e7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd57530b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35028/0 (socket says 172.21.15.78:35028) 2024-09-16T00:15:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.632+0000 7f108e7fc640 1 -- 172.21.15.78:0/3858722534 learned_addr learned my addr 172.21.15.78:0/3858722534 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.632+0000 7f10977fe640 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd57519370 0x55cd57531070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.633+0000 7f1096ffd640 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd575305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.633+0000 7f10977fe640 1 -- 172.21.15.78:0/3858722534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 msgr2=0x55cd575305f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.633+0000 7f10977fe640 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd575305f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.633+0000 7f10977fe640 1 -- 172.21.15.78:0/3858722534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 msgr2=0x55cd57530b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.633+0000 7f10977fe640 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd57530b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.633+0000 7f10977fe640 1 -- 172.21.15.78:0/3858722534 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5751f950 con 0x55cd57519370 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f10977fe640 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd57519370 0x55cd57531070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1090009850 tx=0x7f1090009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0b60944fb606385 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f107e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f107e7fc640 1 -- 172.21.15.78:0/3858722534 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109004e070 con 0x55cd57519370 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f107e7fc640 1 -- 172.21.15.78:0/3858722534 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1090031a10 con 0x55cd57519370 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f109f84b740 1 -- 172.21.15.78:0/3858722534 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108000faa0 con 0x55cd57519370 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f107e7fc640 1 -- 172.21.15.78:0/3858722534 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1090042040 con 0x55cd57519370 2024-09-16T00:15:12.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f109f84b740 10 client.25138 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f109f84b740 1 -- 172.21.15.78:0/3858722534 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cd57593c80 con 0x55cd57519370 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.634+0000 7f107e7fc640 1 -- 172.21.15.78:0/3858722534 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f108000faa0 con 0x55cd57519370 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.635+0000 7f107e7fc640 1 -- 172.21.15.78:0/3858722534 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f1090048040 con 0x55cd57519370 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.635+0000 7f109f84b740 10 client.25138 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.635+0000 7f109f84b740 10 client.25138 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.635+0000 7f109f84b740 20 client.25138 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.635+0000 7f109f84b740 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55cd57528f20 0x55cd575963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.635+0000 7f109f84b740 4 client.25138 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.635+0000 7f109f84b740 1 -- 172.21.15.78:0/3858722534 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd5747bd70 con 0x55cd57528f20 2024-09-16T00:15:12.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.635+0000 7f1096ffd640 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55cd57528f20 0x55cd575963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:12.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.636+0000 7f1096ffd640 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55cd57528f20 0x55cd575963b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc712fa0041a0672 server_cookie=8953321e44b42f13 in_seq=0 out_seq=0 2024-09-16T00:15:12.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.636+0000 7f107e7fc640 10 client.25138 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:15:12.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.636+0000 7f107e7fc640 1 -- 172.21.15.78:0/3858722534 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd5747bd70 con 0x55cd57528f20 2024-09-16T00:15:12.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.636+0000 7f107e7fc640 10 client.25138 handle_command_reply: tid=0 2024-09-16T00:15:12.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.637+0000 7f109f84b740 1 client.25138 shutdown 2024-09-16T00:15:12.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 -- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd57519370 msgr2=0x55cd57531070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd57519370 0x55cd57531070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1090009850 tx=0x7f1090009260 comp rx=0 tx=0).stop 2024-09-16T00:15:12.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 -- 172.21.15.78:0/3858722534 shutdown_connections 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd57519370 0x55cd57531070 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55cd5750fb40 0x55cd57530b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55cd57528f20 0x55cd575963b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 --2- 172.21.15.78:0/3858722534 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cd5750e5d0 0x55cd575305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 -- 172.21.15.78:0/3858722534 >> 172.21.15.78:0/3858722534 conn(0x55cd574d7310 msgr2=0x55cd57514780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f107e7fc640 0 client.25138 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 -- 172.21.15.78:0/3858722534 shutdown_connections 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 1 -- 172.21.15.78:0/3858722534 wait complete. 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.638+0000 7f109f84b740 20 client.25138 trim_cache size 0 max 16384 2024-09-16T00:15:12.640 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:15:12.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 -- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10840a3310 msgr2=0x7f10840a57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10840a3310 0x7f10840a57d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1090005ab0 tx=0x7f109002a990 comp rx=0 tx=0).stop 2024-09-16T00:15:12.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 -- 172.21.15.78:0/2554818892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098100e20 msgr2=0x7f109810bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:12.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098100e20 0x7f109810bf80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f108800eb00 tx=0x7f108800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 -- 172.21.15.78:0/2554818892 shutdown_connections 2024-09-16T00:15:12.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10840a3310 0x7f10840a57d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1098102ef0 0x7f109810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1098102570 0x7f109811b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 --2- 172.21.15.78:0/2554818892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098100e20 0x7f109810bf80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:12.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 -- 172.21.15.78:0/2554818892 >> 172.21.15.78:0/2554818892 conn(0x7f10980fc7d0 msgr2=0x7f1098106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:12.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.639+0000 7f109d472640 1 -- 172.21.15.78:0/2554818892 shutdown_connections 2024-09-16T00:15:12.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:12.640+0000 7f109d472640 1 -- 172.21.15.78:0/2554818892 wait complete. 2024-09-16T00:15:13.503 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:15:13.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:13 smithi078 ceph-mon[31157]: pgmap v415: 129 pgs: 129 active+clean; 1012 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 821 op/s 2024-09-16T00:15:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:13 smithi193 ceph-mon[35484]: pgmap v415: 129 pgs: 129 active+clean; 1012 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 821 op/s 2024-09-16T00:15:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:13 smithi148 ceph-mon[35599]: pgmap v415: 129 pgs: 129 active+clean; 1012 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 821 op/s 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 -- 172.21.15.78:0/4284595850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30108da0 msgr2=0x7fdc3010cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 --2- 172.21.15.78:0/4284595850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30108da0 0x7fdc3010cdf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009900 tx=0x7fdc2002a410 comp rx=0 tx=0).stop 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 -- 172.21.15.78:0/4284595850 shutdown_connections 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 --2- 172.21.15.78:0/4284595850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc3010d520 0x7fdc301150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 --2- 172.21.15.78:0/4284595850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30108da0 0x7fdc3010cdf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 --2- 172.21.15.78:0/4284595850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc301083f0 0x7fdc301087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 -- 172.21.15.78:0/4284595850 >> 172.21.15.78:0/4284595850 conn(0x7fdc30077af0 msgr2=0x7fdc30079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 -- 172.21.15.78:0/4284595850 shutdown_connections 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.522+0000 7fdc3825d640 1 -- 172.21.15.78:0/4284595850 wait complete. 2024-09-16T00:15:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.524+0000 7fdc3825d640 1 Processor -- start 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.524+0000 7fdc3825d640 1 -- start start 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.524+0000 7fdc3825d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc301083f0 0x7fdc301a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.524+0000 7fdc3825d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30108da0 0x7fdc301a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.524+0000 7fdc3825d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc3010d520 0x7fdc3019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc3825d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30118590 con 0x7fdc30108da0 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc3825d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30118410 con 0x7fdc301083f0 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc3825d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30118710 con 0x7fdc3010d520 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc367fa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc3010d520 0x7fdc3019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc367fa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc3010d520 0x7fdc3019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42372/0 (socket says 172.21.15.78:42372) 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc367fa640 1 -- 172.21.15.78:0/1438620081 learned_addr learned my addr 172.21.15.78:0/1438620081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc367fa640 1 -- 172.21.15.78:0/1438620081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc301083f0 msgr2=0x7fdc301a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc367fa640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc301083f0 0x7fdc301a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc367fa640 1 -- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30108da0 msgr2=0x7fdc301a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc367fa640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30108da0 0x7fdc301a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.525+0000 7fdc367fa640 1 -- 172.21.15.78:0/1438620081 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3019efd0 con 0x7fdc3010d520 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.526+0000 7fdc367fa640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc3010d520 0x7fdc3019e8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800eb10 tx=0x7fdc2800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.526+0000 7fdc26ffd640 1 -- 172.21.15.78:0/1438620081 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28004b20 con 0x7fdc3010d520 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.526+0000 7fdc26ffd640 1 -- 172.21.15.78:0/1438620081 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc28004cc0 con 0x7fdc3010d520 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.526+0000 7fdc3825d640 1 -- 172.21.15.78:0/1438620081 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3019f2c0 con 0x7fdc3010d520 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.526+0000 7fdc3825d640 1 -- 172.21.15.78:0/1438620081 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc301ab5b0 con 0x7fdc3010d520 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.526+0000 7fdc26ffd640 1 -- 172.21.15.78:0/1438620081 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc280107b0 con 0x7fdc3010d520 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.529+0000 7fdc26ffd640 1 -- 172.21.15.78:0/1438620081 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fdc28010950 con 0x7fdc3010d520 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.529+0000 7fdc3a636740 1 Processor -- start 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc3a636740 1 -- start start 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58124090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58124a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 0x560b58131760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc3a636740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560b5808a2f0 con 0x560b5812f370 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc3a636740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560b58031cf0 con 0x560b58125b40 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc3a636740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560b580862b0 con 0x560b581245d0 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc26ffd640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc1c0a3280 0x7fdc1c0a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc357f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58124090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.530+0000 7fdc357f8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58124090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:51884/0 (socket says 172.21.15.78:51884) 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc357f8640 1 -- 172.21.15.78:0/2530560037 learned_addr learned my addr 172.21.15.78:0/2530560037 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc357f8640 1 -- 172.21.15.78:0/2530560037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 msgr2=0x560b58124a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc357f8640 1 --2- 172.21.15.78:0/2530560037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58124a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc357f8640 1 -- 172.21.15.78:0/2530560037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 msgr2=0x560b58131760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc357f8640 1 --2- 172.21.15.78:0/2530560037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 0x560b58131760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc357f8640 1 -- 172.21.15.78:0/2530560037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b580163a0 con 0x560b58125b40 2024-09-16T00:15:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc357f8640 1 --2- 172.21.15.78:0/2530560037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58124090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005ab0 tx=0x7fdc20031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70daf44faa51097c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc1bfff640 1 -- 172.21.15.78:0/2530560037 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20040070 con 0x560b58125b40 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.531+0000 7fdc26ffd640 1 -- 172.21.15.78:0/1438620081 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdc280d0290 con 0x7fdc3010d520 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.532+0000 7fdc1bfff640 1 -- 172.21.15.78:0/2530560037 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc20031c90 con 0x560b58125b40 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.533+0000 7fdc3a636740 1 -- 172.21.15.78:0/2530560037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 msgr2=0x560b58124090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.533+0000 7fdc3a636740 1 --2- 172.21.15.78:0/2530560037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58124090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005ab0 tx=0x7fdc20031400 comp rx=0 tx=0).stop 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.533+0000 7fdc367fa640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc1c0a3280 0x7fdc1c0a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.533+0000 7fdc3a636740 1 -- 172.21.15.78:0/2530560037 shutdown_connections 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.533+0000 7fdc3a636740 1 --2- 172.21.15.78:0/2530560037 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 0x560b58131760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.533+0000 7fdc3a636740 1 --2- 172.21.15.78:0/2530560037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58124a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.533+0000 7fdc3a636740 1 --2- 172.21.15.78:0/2530560037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58124090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.533+0000 7fdc3a636740 1 -- 172.21.15.78:0/2530560037 >> 172.21.15.78:0/2530560037 conn(0x560b580ed310 msgr2=0x560b58120c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.534+0000 7fdc367fa640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc1c0a3280 0x7fdc1c0a5740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800efd0 tx=0x7fdc2800c950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.534+0000 7fdc3a636740 1 -- 172.21.15.78:0/2530560037 shutdown_connections 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.534+0000 7fdc3a636740 1 -- 172.21.15.78:0/2530560037 wait complete. 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.534+0000 7fdc3a636740 1 -- start start 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.534+0000 7fdc3a636740 1 Processor -- start 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b581245d0 0x560b5812ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b58125b40 0x560b5812bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 0x560b5812c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc3a636740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560b581320b0 con 0x560b581245d0 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc3a636740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560b58045060 con 0x560b5812f370 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc3a636740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560b58132230 con 0x560b58125b40 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc3a636740 10 client.0 _ll_register_callbacks cb 0x560b57d988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc357f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b581245d0 0x560b5812ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc35ff9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b58125b40 0x560b5812bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc35ff9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b58125b40 0x560b5812bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42390/0 (socket says 172.21.15.78:42390) 2024-09-16T00:15:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc357f8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b581245d0 0x560b5812ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35056/0 (socket says 172.21.15.78:35056) 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.536+0000 7fdc35ff9640 1 -- 172.21.15.78:0/3869814818 learned_addr learned my addr 172.21.15.78:0/3869814818 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc357f8640 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 0x560b5812c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc35ff9640 1 -- 172.21.15.78:0/3869814818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 msgr2=0x560b5812c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc35ff9640 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 0x560b5812c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc35ff9640 1 -- 172.21.15.78:0/3869814818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b581245d0 msgr2=0x560b5812ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc35ff9640 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b581245d0 0x560b5812ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc35ff9640 1 -- 172.21.15.78:0/3869814818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b57dfb3c0 con 0x560b58125b40 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc35ff9640 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b58125b40 0x560b5812bf30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00e5d0 tx=0x7fdc2c00ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eada8e2dba0b464 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc197fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.537+0000 7fdc197fa640 1 -- 172.21.15.78:0/3869814818 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c005220 con 0x560b58125b40 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.538+0000 7fdc197fa640 1 -- 172.21.15.78:0/3869814818 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdc2c0053c0 con 0x560b58125b40 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.538+0000 7fdc3a636740 1 -- 172.21.15.78:0/3869814818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc20031c90 con 0x560b58125b40 2024-09-16T00:15:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.538+0000 7fdc197fa640 1 -- 172.21.15.78:0/3869814818 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc2c0055c0 con 0x560b58125b40 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.538+0000 7fdc3a636740 10 client.25147 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.538+0000 7fdc3a636740 1 -- 172.21.15.78:0/3869814818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b5813dbb0 con 0x560b58125b40 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc197fa640 1 -- 172.21.15.78:0/3869814818 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc20031c90 con 0x560b58125b40 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc197fa640 1 -- 172.21.15.78:0/3869814818 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fdc2c018430 con 0x560b58125b40 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc3a636740 10 client.25147 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc3a636740 10 client.25147 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc3a636740 20 client.25147 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc3a636740 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x560b5813f5f0 0x560b58141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc3a636740 4 client.25147 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc3a636740 1 -- 172.21.15.78:0/3869814818 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b58097860 con 0x560b5813f5f0 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc357f8640 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x560b5813f5f0 0x560b58141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc357f8640 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x560b5813f5f0 0x560b58141a90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ea884f158bfcc79 server_cookie=6faba0771abb0230 in_seq=0 out_seq=0 2024-09-16T00:15:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.539+0000 7fdc197fa640 10 client.25147 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.540+0000 7fdc197fa640 1 -- 172.21.15.78:0/3869814818 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b58097860 con 0x560b5813f5f0 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.540+0000 7fdc197fa640 10 client.25147 handle_command_reply: tid=0 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.540+0000 7fdc3a636740 1 client.25147 shutdown 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc3a636740 1 -- 172.21.15.78:0/3869814818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b58125b40 msgr2=0x560b5812bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc3a636740 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b58125b40 0x560b5812bf30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00e5d0 tx=0x7fdc2c00ea90 comp rx=0 tx=0).stop 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc3a636740 1 -- 172.21.15.78:0/3869814818 shutdown_connections 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc3a636740 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x560b5813f5f0 0x560b58141a90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc3a636740 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 0x560b5812c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc3a636740 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b58125b40 0x560b5812bf30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc3a636740 1 --2- 172.21.15.78:0/3869814818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b581245d0 0x560b5812ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc197fa640 0 client.25147 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.541+0000 7fdc3a636740 1 -- 172.21.15.78:0/3869814818 >> 172.21.15.78:0/3869814818 conn(0x560b580ed310 msgr2=0x560b58126960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:14.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.542+0000 7fdc3a636740 1 -- 172.21.15.78:0/3869814818 shutdown_connections 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.542+0000 7fdc3a636740 1 -- 172.21.15.78:0/3869814818 wait complete. 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.542+0000 7fdc3a636740 20 client.25147 trim_cache size 0 max 16384 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.561+0000 7fdc3a636740 1 Processor -- start 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.561+0000 7fdc3a636740 1 -- start start 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.561+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58120e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.562+0000 7fdc357f8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58120e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.562+0000 7fdc357f8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58120e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42400/0 (socket says 172.21.15.78:42400) 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.562+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b58125b40 0x560b5813b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.562+0000 7fdc357f8640 1 -- 172.21.15.78:0/264544011 learned_addr learned my addr 172.21.15.78:0/264544011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.562+0000 7fdc3a636740 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 0x560b58147360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.562+0000 7fdc3a636740 1 -- 172.21.15.78:0/264544011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1400fae0 con 0x560b58125b40 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.562+0000 7fdc3a636740 1 -- 172.21.15.78:0/264544011 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc140116b0 con 0x560b5812f370 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.562+0000 7fdc3a636740 1 -- 172.21.15.78:0/264544011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc140101b0 con 0x560b581245d0 2024-09-16T00:15:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc35ff9640 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b58125b40 0x560b5813b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc357f8640 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 0x560b58147360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc357f8640 1 -- 172.21.15.78:0/264544011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 msgr2=0x560b58147360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc357f8640 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 0x560b58147360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc357f8640 1 -- 172.21.15.78:0/264544011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b58125b40 msgr2=0x560b5813b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc357f8640 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b58125b40 0x560b5813b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc357f8640 1 -- 172.21.15.78:0/264544011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5813d9e0 con 0x560b581245d0 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc357f8640 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58120e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc20033000 tx=0x7fdc2003ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e534c15e404a7b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc1b7fe640 1 -- 172.21.15.78:0/264544011 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20040040 con 0x560b581245d0 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc1b7fe640 1 -- 172.21.15.78:0/264544011 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2004b070 con 0x560b581245d0 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.563+0000 7fdc1b7fe640 1 -- 172.21.15.78:0/264544011 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20031ad0 con 0x560b581245d0 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 -- 172.21.15.78:0/264544011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 msgr2=0x560b58120e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58120e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc20033000 tx=0x7fdc2003ca30 comp rx=0 tx=0).stop 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 -- 172.21.15.78:0/264544011 shutdown_connections 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b5812f370 0x560b58147360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b58125b40 0x560b5813b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 --2- 172.21.15.78:0/264544011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58120e10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 -- 172.21.15.78:0/264544011 >> 172.21.15.78:0/264544011 conn(0x560b580ed310 msgr2=0x560b58120700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 -- 172.21.15.78:0/264544011 shutdown_connections 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.564+0000 7fdc3a636740 1 -- 172.21.15.78:0/264544011 wait complete. 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.565+0000 7fdc3a636740 1 -- start start 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.565+0000 7fdc3a636740 1 Processor -- start 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.565+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58137750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.565+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58137c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.565+0000 7fdc357f8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58137750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.565+0000 7fdc357f8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58137750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42406/0 (socket says 172.21.15.78:42406) 2024-09-16T00:15:14.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc3a636740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 0x560b58128780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc357f8640 1 -- 172.21.15.78:0/285163245 learned_addr learned my addr 172.21.15.78:0/285163245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14010fb0 con 0x560b5812f370 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc35ff9640 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58137c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdc140108e0 con 0x560b58125b40 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1400dac0 con 0x560b581245d0 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc3a636740 10 client.0 _ll_register_callbacks cb 0x7fdc2c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc357f8640 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 0x560b58128780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc357f8640 1 -- 172.21.15.78:0/285163245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 msgr2=0x560b58137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc357f8640 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58137c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc357f8640 1 -- 172.21.15.78:0/285163245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 msgr2=0x560b58128780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc357f8640 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 0x560b58128780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.566+0000 7fdc357f8640 1 -- 172.21.15.78:0/285163245 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b58127120 con 0x560b581245d0 2024-09-16T00:15:14.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.567+0000 7fdc357f8640 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58137750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdc200319b0 tx=0x7fdc200318b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d172883426e6699 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.567+0000 7fdc197fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.567+0000 7fdc197fa640 1 -- 172.21.15.78:0/285163245 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20064060 con 0x560b581245d0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.567+0000 7fdc197fa640 1 -- 172.21.15.78:0/285163245 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdc20054e80 con 0x560b581245d0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.567+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc2004b070 con 0x560b581245d0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.567+0000 7fdc197fa640 1 -- 172.21.15.78:0/285163245 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdc2006b050 con 0x560b581245d0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.567+0000 7fdc3a636740 10 client.25153 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b58121960 con 0x560b581245d0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc197fa640 1 -- 172.21.15.78:0/285163245 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2004b070 con 0x560b581245d0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc197fa640 1 -- 172.21.15.78:0/285163245 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fdc2003a040 con 0x560b581245d0 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc3a636740 10 client.25153 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc3a636740 10 client.25153 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc3a636740 20 client.25153 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:14.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc3a636740 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x560b581a9bb0 0x560b581abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc3a636740 4 client.25153 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b58091d70 con 0x560b581a9bb0 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.568+0000 7fdc35ff9640 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x560b581a9bb0 0x560b581abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.569+0000 7fdc197fa640 10 client.25153 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.569+0000 7fdc35ff9640 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x560b581a9bb0 0x560b581abfa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=811c4c8acd5b4f83 server_cookie=53881f81cd79fddc in_seq=0 out_seq=0 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.569+0000 7fdc197fa640 1 -- 172.21.15.78:0/285163245 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b58091d70 con 0x560b581a9bb0 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.569+0000 7fdc197fa640 10 client.25153 handle_command_reply: tid=0 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.569+0000 7fdc3a636740 1 client.25153 shutdown 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 msgr2=0x560b58137750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58137750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdc200319b0 tx=0x7fdc200318b0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 shutdown_connections 2024-09-16T00:15:14.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560b5812f370 0x560b58128780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x560b581a9bb0 0x560b581abfa0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560b58125b40 0x560b58137c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.613 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:15:14.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 --2- 172.21.15.78:0/285163245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560b581245d0 0x560b58137750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc197fa640 0 client.25153 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 >> 172.21.15.78:0/285163245 conn(0x560b580ed310 msgr2=0x560b5812a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.570+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 shutdown_connections 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.571+0000 7fdc3a636740 1 -- 172.21.15.78:0/285163245 wait complete. 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.571+0000 7fdc3a636740 20 client.25153 trim_cache size 0 max 16384 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 -- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc1c0a3280 msgr2=0x7fdc1c0a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc1c0a3280 0x7fdc1c0a5740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800efd0 tx=0x7fdc2800c950 comp rx=0 tx=0).stop 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 -- 172.21.15.78:0/1438620081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc3010d520 msgr2=0x7fdc3019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc3010d520 0x7fdc3019e8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800eb10 tx=0x7fdc2800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 -- 172.21.15.78:0/1438620081 shutdown_connections 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdc1c0a3280 0x7fdc1c0a5740 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc3010d520 0x7fdc3019e8a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc30108da0 0x7fdc301a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 --2- 172.21.15.78:0/1438620081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdc301083f0 0x7fdc301a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:14.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 -- 172.21.15.78:0/1438620081 >> 172.21.15.78:0/1438620081 conn(0x7fdc30077af0 msgr2=0x7fdc30078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:14.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 -- 172.21.15.78:0/1438620081 shutdown_connections 2024-09-16T00:15:14.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:14.572+0000 7fdc3825d640 1 -- 172.21.15.78:0/1438620081 wait complete. 2024-09-16T00:15:15.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 75.0 s, 2649.6 tps, lat 37.834 ms stddev 42.193 2024-09-16T00:15:15.862 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:15:15.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:15 smithi193 ceph-mon[35484]: pgmap v416: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 803 op/s 2024-09-16T00:15:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:15 smithi148 ceph-mon[35599]: pgmap v416: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 803 op/s 2024-09-16T00:15:16.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:15 smithi078 ceph-mon[31157]: pgmap v416: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 803 op/s 2024-09-16T00:15:16.266 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:17.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:17 smithi078 ceph-mon[31157]: pgmap v417: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 744 op/s 2024-09-16T00:15:17.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:17 smithi193 ceph-mon[35484]: pgmap v417: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 744 op/s 2024-09-16T00:15:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:17 smithi148 ceph-mon[35599]: pgmap v417: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 744 op/s 2024-09-16T00:15:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:19 smithi193 ceph-mon[35484]: pgmap v418: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 754 op/s 2024-09-16T00:15:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:19 smithi148 ceph-mon[35599]: pgmap v418: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 754 op/s 2024-09-16T00:15:19.967 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:19 smithi078 ceph-mon[31157]: pgmap v418: 129 pgs: 129 active+clean; 1021 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 MiB/s wr, 754 op/s 2024-09-16T00:15:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 80.0 s, 3019.0 tps, lat 33.212 ms stddev 47.293 2024-09-16T00:15:21.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:20 smithi078 ceph-mon[31157]: pgmap v419: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.0 MiB/s wr, 704 op/s 2024-09-16T00:15:21.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:20 smithi193 ceph-mon[35484]: pgmap v419: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.0 MiB/s wr, 704 op/s 2024-09-16T00:15:21.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.964+0000 7f1c5f388640 1 -- 172.21.15.78:0/1366038098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c5810d520 msgr2=0x7f1c581150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.964+0000 7f1c5f388640 1 --2- 172.21.15.78:0/1366038098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c5810d520 0x7f1c581150a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009be0 tx=0x7f1c4802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.965+0000 7f1c5f388640 1 -- 172.21.15.78:0/1366038098 shutdown_connections 2024-09-16T00:15:21.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.965+0000 7f1c5f388640 1 --2- 172.21.15.78:0/1366038098 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c5810d520 0x7f1c581150a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.965+0000 7f1c5f388640 1 --2- 172.21.15.78:0/1366038098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c58108da0 0x7f1c5810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.965+0000 7f1c5f388640 1 --2- 172.21.15.78:0/1366038098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c581083f0 0x7f1c581087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.965+0000 7f1c5f388640 1 -- 172.21.15.78:0/1366038098 >> 172.21.15.78:0/1366038098 conn(0x7f1c58077af0 msgr2=0x7f1c58079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.966+0000 7f1c5f388640 1 -- 172.21.15.78:0/1366038098 shutdown_connections 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.966+0000 7f1c5f388640 1 -- 172.21.15.78:0/1366038098 wait complete. 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.967+0000 7f1c5f388640 1 Processor -- start 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.967+0000 7f1c5f388640 1 -- start start 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.968+0000 7f1c5f388640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c581083f0 0x7f1c5819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.968+0000 7f1c5f388640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c58108da0 0x7f1c5819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.968+0000 7f1c5f388640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c5810d520 0x7f1c581a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.968+0000 7f1c5f388640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c581185c0 con 0x7f1c581083f0 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.968+0000 7f1c5f388640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58118440 con 0x7f1c58108da0 2024-09-16T00:15:21.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.968+0000 7f1c5f388640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58118740 con 0x7f1c5810d520 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5c923640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c58108da0 0x7f1c5819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5c923640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c58108da0 0x7f1c5819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:56606/0 (socket says 172.21.15.78:56606) 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5c923640 1 -- 172.21.15.78:0/737793161 learned_addr learned my addr 172.21.15.78:0/737793161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5d925640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c5810d520 0x7f1c581a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5c923640 1 -- 172.21.15.78:0/737793161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c5810d520 msgr2=0x7f1c581a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5c923640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c5810d520 0x7f1c581a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5c923640 1 -- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c581083f0 msgr2=0x7f1c5819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5c923640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c581083f0 0x7f1c5819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.969+0000 7f1c5c923640 1 -- 172.21.15.78:0/737793161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c581a3b70 con 0x7f1c58108da0 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.970+0000 7f1c5c923640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c58108da0 0x7f1c5819f080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000b470 tx=0x7f1c5000b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.971+0000 7f1c567fc640 1 -- 172.21.15.78:0/737793161 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50015020 con 0x7f1c58108da0 2024-09-16T00:15:21.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.971+0000 7f1c5f388640 1 -- 172.21.15.78:0/737793161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c581a3e60 con 0x7f1c58108da0 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.971+0000 7f1c5f388640 1 -- 172.21.15.78:0/737793161 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c581ab6f0 con 0x7f1c58108da0 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.971+0000 7f1c567fc640 1 -- 172.21.15.78:0/737793161 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c50011d70 con 0x7f1c58108da0 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.971+0000 7f1c567fc640 1 -- 172.21.15.78:0/737793161 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000f340 con 0x7f1c58108da0 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.975+0000 7f1c61761740 1 Processor -- start 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.975+0000 7f1c61761740 1 -- start start 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.975+0000 7f1c61761740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0deb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.975+0000 7f1c61761740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0deba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.975+0000 7f1c61761740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 0x5613a0df8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.975+0000 7f1c61761740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5613a0d512f0 con 0x5613a0decb40 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.975+0000 7f1c61761740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5613a0cf8cf0 con 0x5613a0df6370 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.975+0000 7f1c61761740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5613a0d4d2b0 con 0x5613a0deb5d0 2024-09-16T00:15:21.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d124640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0deba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d124640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0deba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42670/0 (socket says 172.21.15.78:42670) 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d925640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0deb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d124640 1 -- 172.21.15.78:0/3699251445 learned_addr learned my addr 172.21.15.78:0/3699251445 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d925640 1 -- 172.21.15.78:0/3699251445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 msgr2=0x5613a0deba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d925640 1 --2- 172.21.15.78:0/3699251445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0deba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d925640 1 -- 172.21.15.78:0/3699251445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 msgr2=0x5613a0df8760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d925640 1 --2- 172.21.15.78:0/3699251445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 0x5613a0df8760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d925640 1 -- 172.21.15.78:0/3699251445 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613a0cdd3a0 con 0x5613a0decb40 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d124640 1 --2- 172.21.15.78:0/3699251445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0deba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.976+0000 7f1c5d925640 1 --2- 172.21.15.78:0/3699251445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0deb090 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009b30 tx=0x7f1c48030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881d36454b39561b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.977+0000 7f1c567fc640 1 -- 172.21.15.78:0/737793161 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1c50018020 con 0x7f1c58108da0 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.977+0000 7f1c3f7fe640 1 -- 172.21.15.78:0/3699251445 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48040030 con 0x5613a0decb40 2024-09-16T00:15:21.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.977+0000 7f1c3f7fe640 1 -- 172.21.15.78:0/3699251445 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c48030db0 con 0x5613a0decb40 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.978+0000 7f1c3f7fe640 1 -- 172.21.15.78:0/3699251445 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48036040 con 0x5613a0decb40 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.978+0000 7f1c567fc640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1c400a33d0 0x7f1c400a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.978+0000 7f1c567fc640 1 -- 172.21.15.78:0/737793161 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1c500cf600 con 0x7f1c58108da0 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c61761740 1 -- 172.21.15.78:0/3699251445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 msgr2=0x5613a0deb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c61761740 1 --2- 172.21.15.78:0/3699251445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0deb090 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009b30 tx=0x7f1c48030cc0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c5d124640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1c400a33d0 0x7f1c400a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c61761740 1 -- 172.21.15.78:0/3699251445 shutdown_connections 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c61761740 1 --2- 172.21.15.78:0/3699251445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 0x5613a0df8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c61761740 1 --2- 172.21.15.78:0/3699251445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0deba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c61761740 1 --2- 172.21.15.78:0/3699251445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0deb090 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c61761740 1 -- 172.21.15.78:0/3699251445 >> 172.21.15.78:0/3699251445 conn(0x5613a0db4310 msgr2=0x5613a0de7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.979+0000 7f1c61761740 1 -- 172.21.15.78:0/3699251445 shutdown_connections 2024-09-16T00:15:21.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.980+0000 7f1c61761740 1 -- 172.21.15.78:0/3699251445 wait complete. 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.980+0000 7f1c61761740 1 -- start start 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.980+0000 7f1c61761740 1 Processor -- start 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c5d124640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1c400a33d0 0x7f1c400a5890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c005fd0 tx=0x7f1c4c00b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c61761740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0df5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c61761740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0df2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c61761740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 0x5613a0df3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c61761740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5613a0df90b0 con 0x5613a0decb40 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c61761740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5613a0d0c060 con 0x5613a0df6370 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c61761740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5613a0df9230 con 0x5613a0deb5d0 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c61761740 10 client.0 _ll_register_callbacks cb 0x5613a0a5f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c5d124640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0df5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c5d124640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0df5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42680/0 (socket says 172.21.15.78:42680) 2024-09-16T00:15:21.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c5d124640 1 -- 172.21.15.78:0/1006002883 learned_addr learned my addr 172.21.15.78:0/1006002883 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c5c923640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0df2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.981+0000 7f1c5d925640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 0x5613a0df3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.982+0000 7f1c5c923640 1 -- 172.21.15.78:0/1006002883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 msgr2=0x5613a0df5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.982+0000 7f1c5c923640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0df5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.982+0000 7f1c5c923640 1 -- 172.21.15.78:0/1006002883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 msgr2=0x5613a0df3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.982+0000 7f1c5c923640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 0x5613a0df3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.982+0000 7f1c5c923640 1 -- 172.21.15.78:0/1006002883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613a0ac23c0 con 0x5613a0decb40 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.982+0000 7f1c5d925640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 0x5613a0df3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.982+0000 7f1c5d124640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0df5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.983+0000 7f1c5c923640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0df2fb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x5613a0dec070 tx=0x7f1c5009bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658daa56591eef45 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.983+0000 7f1c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.983+0000 7f1c3cff9640 1 -- 172.21.15.78:0/1006002883 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5009fc70 con 0x5613a0decb40 2024-09-16T00:15:21.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.983+0000 7f1c61761740 1 -- 172.21.15.78:0/1006002883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c48030db0 con 0x5613a0decb40 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.983+0000 7f1c3cff9640 1 -- 172.21.15.78:0/1006002883 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1c500cf370 con 0x5613a0decb40 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.983+0000 7f1c3cff9640 1 -- 172.21.15.78:0/1006002883 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c500a4530 con 0x5613a0decb40 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.984+0000 7f1c61761740 10 client.15771 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.984+0000 7f1c61761740 1 -- 172.21.15.78:0/1006002883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5613a0e04c20 con 0x5613a0decb40 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c3cff9640 1 -- 172.21.15.78:0/1006002883 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c48030db0 con 0x5613a0decb40 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c3cff9640 1 -- 172.21.15.78:0/1006002883 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f1c50097610 con 0x5613a0decb40 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c61761740 10 client.15771 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c61761740 10 client.15771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c61761740 20 client.15771 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c61761740 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5613a0e06170 0x5613a0e08560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c61761740 4 client.15771 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c61761740 1 -- 172.21.15.78:0/1006002883 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613a0d5e860 con 0x5613a0e06170 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.985+0000 7f1c5d124640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5613a0e06170 0x5613a0e08560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.986+0000 7f1c3cff9640 10 client.15771 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.986+0000 7f1c5d124640 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5613a0e06170 0x5613a0e08560 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcae5371bf1b4a47 server_cookie=4f4d052655e78892 in_seq=0 out_seq=0 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c3cff9640 1 -- 172.21.15.78:0/1006002883 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613a0d5e860 con 0x5613a0e06170 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c3cff9640 10 client.15771 handle_command_reply: tid=0 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 client.15771 shutdown 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 -- 172.21.15.78:0/1006002883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 msgr2=0x5613a0df2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0df2fb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x5613a0dec070 tx=0x7f1c5009bfc0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 -- 172.21.15.78:0/1006002883 shutdown_connections 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5613a0e06170 0x5613a0e08560 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0df6370 0x5613a0df3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0df2fb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 --2- 172.21.15.78:0/1006002883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0deb5d0 0x5613a0df5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 -- 172.21.15.78:0/1006002883 >> 172.21.15.78:0/1006002883 conn(0x5613a0db4310 msgr2=0x5613a0ded930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:21.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c3cff9640 0 client.15771 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 -- 172.21.15.78:0/1006002883 shutdown_connections 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.987+0000 7f1c61761740 1 -- 172.21.15.78:0/1006002883 wait complete. 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:19.988+0000 7f1c61761740 20 client.15771 trim_cache size 0 max 16384 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c61761740 1 Processor -- start 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c61761740 1 -- start start 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c61761740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0df4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c61761740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0deb5d0 0x5613a0dee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c61761740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0decb40 0x5613a0dee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c61761740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4400fa00 con 0x5613a0deb5d0 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c61761740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440115d0 con 0x5613a0decb40 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c61761740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c440100d0 con 0x5613a0db4310 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c5d925640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0decb40 0x5613a0dee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c5d925640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0decb40 0x5613a0dee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:56624/0 (socket says 172.21.15.78:56624) 2024-09-16T00:15:21.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c5d124640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0df4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c5d925640 1 -- 172.21.15.78:0/2109460504 learned_addr learned my addr 172.21.15.78:0/2109460504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.011+0000 7f1c5c923640 1 --2- 172.21.15.78:0/2109460504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0deb5d0 0x5613a0dee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.012+0000 7f1c5c923640 1 -- 172.21.15.78:0/2109460504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 msgr2=0x5613a0df4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.012+0000 7f1c5c923640 1 --2- 172.21.15.78:0/2109460504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0df4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.012+0000 7f1c5c923640 1 -- 172.21.15.78:0/2109460504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0decb40 msgr2=0x5613a0dee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.012+0000 7f1c5c923640 1 --2- 172.21.15.78:0/2109460504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0decb40 0x5613a0dee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.012+0000 7f1c5c923640 1 -- 172.21.15.78:0/2109460504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613a0df8f50 con 0x5613a0deb5d0 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.012+0000 7f1c5c923640 1 --2- 172.21.15.78:0/2109460504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0deb5d0 0x5613a0dee120 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5613a0df4560 tx=0x7f1c5009c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0bdf97eedd3e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.013+0000 7f1c3effd640 1 -- 172.21.15.78:0/2109460504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500ba2b0 con 0x5613a0deb5d0 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.013+0000 7f1c3effd640 1 -- 172.21.15.78:0/2109460504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c500b4070 con 0x5613a0deb5d0 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.013+0000 7f1c3effd640 1 -- 172.21.15.78:0/2109460504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500a40a0 con 0x5613a0deb5d0 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 -- 172.21.15.78:0/2109460504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0deb5d0 msgr2=0x5613a0dee120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 --2- 172.21.15.78:0/2109460504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0deb5d0 0x5613a0dee120 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5613a0df4560 tx=0x7f1c5009c0f0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 -- 172.21.15.78:0/2109460504 shutdown_connections 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 --2- 172.21.15.78:0/2109460504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0decb40 0x5613a0dee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 --2- 172.21.15.78:0/2109460504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0deb5d0 0x5613a0dee120 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 --2- 172.21.15.78:0/2109460504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0df4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 -- 172.21.15.78:0/2109460504 >> 172.21.15.78:0/2109460504 conn(0x5613a0e08860 msgr2=0x5613a0df4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 -- 172.21.15.78:0/2109460504 shutdown_connections 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 -- 172.21.15.78:0/2109460504 wait complete. 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.014+0000 7f1c61761740 1 -- start start 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.015+0000 7f1c61761740 1 Processor -- start 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.015+0000 7f1c61761740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0dfa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.015+0000 7f1c61761740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0deb5d0 0x5613a0dfa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.015+0000 7f1c61761740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0dfae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.015+0000 7f1c61761740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44010ed0 con 0x5613a0decb40 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.015+0000 7f1c61761740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c44010800 con 0x5613a0deb5d0 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.015+0000 7f1c61761740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4400d9e0 con 0x5613a0db4310 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c61761740 10 client.0 _ll_register_callbacks cb 0x7f1c500092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d124640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0dfa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d124640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0dfa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42698/0 (socket says 172.21.15.78:42698) 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d124640 1 -- 172.21.15.78:0/3255158319 learned_addr learned my addr 172.21.15.78:0/3255158319 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d925640 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0dfae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5c923640 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0deb5d0 0x5613a0dfa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d124640 1 -- 172.21.15.78:0/3255158319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0deb5d0 msgr2=0x5613a0dfa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d124640 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0deb5d0 0x5613a0dfa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d124640 1 -- 172.21.15.78:0/3255158319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 msgr2=0x5613a0dfae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d124640 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0dfae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.016+0000 7f1c5d124640 1 -- 172.21.15.78:0/3255158319 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613a0de8960 con 0x5613a0db4310 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.017+0000 7f1c5d124640 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0dfa3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5613a0e08ca0 tx=0x7f1c4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5bcda122c324f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.017+0000 7f1c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.017+0000 7f1c3cff9640 1 -- 172.21.15.78:0/3255158319 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c028500 con 0x5613a0db4310 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.017+0000 7f1c3cff9640 1 -- 172.21.15.78:0/3255158319 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1c4c0286a0 con 0x5613a0db4310 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.017+0000 7f1c61761740 1 -- 172.21.15.78:0/3255158319 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c500b4070 con 0x5613a0db4310 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.018+0000 7f1c3cff9640 1 -- 172.21.15.78:0/3255158319 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c4c039050 con 0x5613a0db4310 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.018+0000 7f1c61761740 10 client.25171 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.018+0000 7f1c61761740 1 -- 172.21.15.78:0/3255158319 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5613a0de8dd0 con 0x5613a0db4310 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.019+0000 7f1c3cff9640 1 -- 172.21.15.78:0/3255158319 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c500b4070 con 0x5613a0db4310 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.019+0000 7f1c3cff9640 1 -- 172.21.15.78:0/3255158319 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f1c4c01fcb0 con 0x5613a0db4310 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.019+0000 7f1c61761740 10 client.25171 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.019+0000 7f1c61761740 10 client.25171 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:15:21.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.019+0000 7f1c61761740 20 client.25171 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.019+0000 7f1c61761740 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5613a0df6370 0x5613a0e73500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.019+0000 7f1c61761740 4 client.25171 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.019+0000 7f1c61761740 1 -- 172.21.15.78:0/3255158319 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613a0d58d70 con 0x5613a0df6370 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.020+0000 7f1c5c923640 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5613a0df6370 0x5613a0e73500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.020+0000 7f1c3cff9640 10 client.25171 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.020+0000 7f1c5c923640 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5613a0df6370 0x5613a0e73500 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b493ea4de3aa6af7 server_cookie=7cccc9f7de8e11f2 in_seq=0 out_seq=0 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c3cff9640 1 -- 172.21.15.78:0/3255158319 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613a0d58d70 con 0x5613a0df6370 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c3cff9640 10 client.25171 handle_command_reply: tid=0 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 client.25171 shutdown 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 -- 172.21.15.78:0/3255158319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 msgr2=0x5613a0dfa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0dfa3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5613a0e08ca0 tx=0x7f1c4c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 -- 172.21.15.78:0/3255158319 shutdown_connections 2024-09-16T00:15:21.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5613a0df6370 0x5613a0e73500 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.218 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:15:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5613a0decb40 0x5613a0dfae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5613a0deb5d0 0x5613a0dfa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 --2- 172.21.15.78:0/3255158319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5613a0db4310 0x5613a0dfa3d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c61761740 1 -- 172.21.15.78:0/3255158319 >> 172.21.15.78:0/3255158319 conn(0x5613a0e08860 msgr2=0x5613a0df4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.021+0000 7f1c3cff9640 0 client.25171 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:15:21.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.023+0000 7f1c61761740 1 -- 172.21.15.78:0/3255158319 shutdown_connections 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.023+0000 7f1c61761740 1 -- 172.21.15.78:0/3255158319 wait complete. 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.023+0000 7f1c61761740 20 client.25171 trim_cache size 0 max 16384 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 -- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1c400a33d0 msgr2=0x7f1c400a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1c400a33d0 0x7f1c400a5890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c005fd0 tx=0x7f1c4c00b2c0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 -- 172.21.15.78:0/737793161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c58108da0 msgr2=0x7f1c5819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c58108da0 0x7f1c5819f080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000b470 tx=0x7f1c5000b930 comp rx=0 tx=0).stop 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 -- 172.21.15.78:0/737793161 shutdown_connections 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1c400a33d0 0x7f1c400a5890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c5810d520 0x7f1c581a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c58108da0 0x7f1c5819f080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 --2- 172.21.15.78:0/737793161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c581083f0 0x7f1c5819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 -- 172.21.15.78:0/737793161 >> 172.21.15.78:0/737793161 conn(0x7f1c58077af0 msgr2=0x7f1c58078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 -- 172.21.15.78:0/737793161 shutdown_connections 2024-09-16T00:15:21.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:20.024+0000 7f1c5f388640 1 -- 172.21.15.78:0/737793161 wait complete. 2024-09-16T00:15:21.220 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:20 smithi148 ceph-mon[35599]: pgmap v419: 129 pgs: 129 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.0 MiB/s wr, 704 op/s 2024-09-16T00:15:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:23 smithi193 ceph-mon[35484]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 887 op/s 2024-09-16T00:15:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:23 smithi148 ceph-mon[35599]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 887 op/s 2024-09-16T00:15:24.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:23 smithi078 ceph-mon[31157]: pgmap v420: 129 pgs: 129 active+clean; 1.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 887 op/s 2024-09-16T00:15:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 85.0 s, 2716.5 tps, lat 34.756 ms stddev 48.405 2024-09-16T00:15:25.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:25 smithi193 ceph-mon[35484]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 944 op/s 2024-09-16T00:15:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:25 smithi148 ceph-mon[35599]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 944 op/s 2024-09-16T00:15:26.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:25 smithi078 ceph-mon[31157]: pgmap v421: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 944 op/s 2024-09-16T00:15:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:27 smithi193 ceph-mon[35484]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 971 op/s 2024-09-16T00:15:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:27 smithi148 ceph-mon[35599]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 971 op/s 2024-09-16T00:15:28.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:27 smithi078 ceph-mon[31157]: pgmap v422: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 971 op/s 2024-09-16T00:15:29.019 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:15:29.446 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:29.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:29 smithi193 ceph-mon[35484]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-16T00:15:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:29 smithi148 ceph-mon[35599]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-16T00:15:30.000 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:29 smithi078 ceph-mon[31157]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-16T00:15:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 90.0 s, 3277.7 tps, lat 32.156 ms stddev 50.514 2024-09-16T00:15:30.993 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:15:30.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.133+0000 7fc533096640 1 -- 172.21.15.78:0/2691266450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 msgr2=0x7fc52c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:30.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.133+0000 7fc533096640 1 --2- 172.21.15.78:0/2691266450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 0x7fc52c10da40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc514009900 tx=0x7fc51402a480 comp rx=0 tx=0).stop 2024-09-16T00:15:30.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.134+0000 7fc533096640 1 -- 172.21.15.78:0/2691266450 shutdown_connections 2024-09-16T00:15:30.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.134+0000 7fc533096640 1 --2- 172.21.15.78:0/2691266450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 0x7fc52c10da40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:30.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.134+0000 7fc533096640 1 --2- 172.21.15.78:0/2691266450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc52c102570 0x7fc52c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:30.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.134+0000 7fc533096640 1 --2- 172.21.15.78:0/2691266450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c100e20 0x7fc52c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.134+0000 7fc533096640 1 -- 172.21.15.78:0/2691266450 >> 172.21.15.78:0/2691266450 conn(0x7fc52c0fc7d0 msgr2=0x7fc52c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.134+0000 7fc533096640 1 -- 172.21.15.78:0/2691266450 shutdown_connections 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.134+0000 7fc533096640 1 -- 172.21.15.78:0/2691266450 wait complete. 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.135+0000 7fc533096640 1 Processor -- start 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.135+0000 7fc533096640 1 -- start start 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.135+0000 7fc533096640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c100e20 0x7fc52c116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.135+0000 7fc533096640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc52c102570 0x7fc52c116780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.135+0000 7fc533096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 0x7fc52c10c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.135+0000 7fc533096640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c10fe30 con 0x7fc52c102ef0 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc533096640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c10fcb0 con 0x7fc52c100e20 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc533096640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c10ffb0 con 0x7fc52c102570 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 0x7fc52c10c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 0x7fc52c10c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44240/0 (socket says 172.21.15.78:44240) 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 -- 172.21.15.78:0/1140960966 learned_addr learned my addr 172.21.15.78:0/1140960966 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc52bfff640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc52c102570 0x7fc52c116780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc530e32640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c100e20 0x7fc52c116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 -- 172.21.15.78:0/1140960966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc52c102570 msgr2=0x7fc52c116780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc52c102570 0x7fc52c116780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 -- 172.21.15.78:0/1140960966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c100e20 msgr2=0x7fc52c116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c100e20 0x7fc52c116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 -- 172.21.15.78:0/1140960966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c10d0a0 con 0x7fc52c102ef0 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc530e32640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c100e20 0x7fc52c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.136+0000 7fc531633640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 0x7fc52c10c7e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc51402a9a0 tx=0x7fc514003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.137+0000 7fc529ffb640 1 -- 172.21.15.78:0/1140960966 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51403f070 con 0x7fc52c102ef0 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.137+0000 7fc533096640 1 -- 172.21.15.78:0/1140960966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c10d330 con 0x7fc52c102ef0 2024-09-16T00:15:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.137+0000 7fc529ffb640 1 -- 172.21.15.78:0/1140960966 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc514031c20 con 0x7fc52c102ef0 2024-09-16T00:15:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.137+0000 7fc529ffb640 1 -- 172.21.15.78:0/1140960966 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514035030 con 0x7fc52c102ef0 2024-09-16T00:15:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.137+0000 7fc533096640 1 -- 172.21.15.78:0/1140960966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c11a120 con 0x7fc52c102ef0 2024-09-16T00:15:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.141+0000 7fc53546f740 1 Processor -- start 2024-09-16T00:15:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.143+0000 7fc529ffb640 1 -- 172.21.15.78:0/1140960966 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc514042070 con 0x7fc52c102ef0 2024-09-16T00:15:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.143+0000 7fc53546f740 1 -- start start 2024-09-16T00:15:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.143+0000 7fc53546f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a183090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.143+0000 7fc53546f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a183a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:30.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc53546f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a190760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc53546f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a72a0e92f0 con 0x55a72a184b40 2024-09-16T00:15:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc53546f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a72a090cf0 con 0x55a72a18e370 2024-09-16T00:15:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc53546f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a72a0e52b0 con 0x55a72a1835d0 2024-09-16T00:15:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc530e32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a183a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc530e32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a183a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:59308/0 (socket says 172.21.15.78:59308) 2024-09-16T00:15:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc530e32640 1 -- 172.21.15.78:0/1772552776 learned_addr learned my addr 172.21.15.78:0/1772552776 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc529ffb640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5180a32b0 0x7fc5180a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:30.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc530e32640 1 -- 172.21.15.78:0/1772552776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 msgr2=0x55a72a190760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc530e32640 1 --2- 172.21.15.78:0/1772552776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a190760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc530e32640 1 -- 172.21.15.78:0/1772552776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 msgr2=0x55a72a183090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc530e32640 1 --2- 172.21.15.78:0/1772552776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a183090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc530e32640 1 -- 172.21.15.78:0/1772552776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a72a0753a0 con 0x55a72a1835d0 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc529ffb640 1 -- 172.21.15.78:0/1140960966 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc5140effc0 con 0x7fc52c102ef0 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.144+0000 7fc531633640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5180a32b0 0x7fc5180a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.145+0000 7fc530e32640 1 --2- 172.21.15.78:0/1772552776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a183a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc524009d50 tx=0x7fc5240117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54a73bae8be2e95f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.145+0000 7fc531633640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5180a32b0 0x7fc5180a5770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc5140bc1b0 tx=0x7fc514005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.145+0000 7fc522ffd640 1 -- 172.21.15.78:0/1772552776 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524012650 con 0x55a72a1835d0 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.145+0000 7fc522ffd640 1 -- 172.21.15.78:0/1772552776 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc524009e40 con 0x55a72a1835d0 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.145+0000 7fc522ffd640 1 -- 172.21.15.78:0/1772552776 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52400fd10 con 0x55a72a1835d0 2024-09-16T00:15:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 -- 172.21.15.78:0/1772552776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 msgr2=0x55a72a183a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 --2- 172.21.15.78:0/1772552776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a183a10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc524009d50 tx=0x7fc5240117b0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 -- 172.21.15.78:0/1772552776 shutdown_connections 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 --2- 172.21.15.78:0/1772552776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a190760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 --2- 172.21.15.78:0/1772552776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a183a10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 --2- 172.21.15.78:0/1772552776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a183090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 -- 172.21.15.78:0/1772552776 >> 172.21.15.78:0/1772552776 conn(0x55a72a14c310 msgr2=0x55a72a17fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 -- 172.21.15.78:0/1772552776 shutdown_connections 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.146+0000 7fc53546f740 1 -- 172.21.15.78:0/1772552776 wait complete. 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 1 -- start start 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 1 Processor -- start 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a1835d0 0x55a72a18dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a184b40 0x55a72a18b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a18b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a72a191070 con 0x55a72a1835d0 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a72a0a4060 con 0x55a72a18e370 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a72a1911f0 con 0x55a72a184b40 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.147+0000 7fc53546f740 10 client.0 _ll_register_callbacks cb 0x55a729df78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a184b40 0x55a72a18b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a184b40 0x55a72a18b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:59312/0 (socket says 172.21.15.78:59312) 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 -- 172.21.15.78:0/1987943105 learned_addr learned my addr 172.21.15.78:0/1987943105 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc52bfff640 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a18b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc52bfff640 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a1835d0 0x55a72a18dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 -- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 msgr2=0x55a72a18b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a18b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 -- 172.21.15.78:0/1987943105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a1835d0 msgr2=0x55a72a18dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:31.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a1835d0 0x55a72a18dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 -- 172.21.15.78:0/1987943105 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a729e5a3c0 con 0x55a72a184b40 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc52bfff640 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a18b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc530e32640 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a184b40 0x55a72a18b020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc524011cb0 tx=0x7fc5240045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12270c79baac5d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.148+0000 7fc50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc50bfff640 1 -- 172.21.15.78:0/1987943105 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524028020 con 0x55a72a184b40 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc50bfff640 1 -- 172.21.15.78:0/1987943105 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc52400fe20 con 0x55a72a184b40 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 1 -- 172.21.15.78:0/1987943105 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a72a0f6860 con 0x55a72a184b40 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc50bfff640 1 -- 172.21.15.78:0/1987943105 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc524033050 con 0x55a72a184b40 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 10 client.25183 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 1 -- 172.21.15.78:0/1987943105 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a72a19cb10 con 0x55a72a184b40 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc50bfff640 1 -- 172.21.15.78:0/1987943105 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc524003ad0 con 0x55a72a184b40 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc50bfff640 1 -- 172.21.15.78:0/1987943105 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc524012830 con 0x55a72a184b40 2024-09-16T00:15:31.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 10 client.25183 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 10 client.25183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 20 client.25183 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a72a19df70 0x55a72a1a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 4 client.25183 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.149+0000 7fc53546f740 1 -- 172.21.15.78:0/1987943105 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a72a0f6860 con 0x55a72a19df70 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.150+0000 7fc52bfff640 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a72a19df70 0x55a72a1a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.151+0000 7fc52bfff640 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a72a19df70 0x55a72a1a0360 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d12aa4974b48bd server_cookie=518c2bbacce6af4d in_seq=0 out_seq=0 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.151+0000 7fc50bfff640 10 client.25183 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc50bfff640 1 -- 172.21.15.78:0/1987943105 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a72a0f6860 con 0x55a72a19df70 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc50bfff640 10 client.25183 handle_command_reply: tid=0 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 client.25183 shutdown 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 -- 172.21.15.78:0/1987943105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a184b40 msgr2=0x55a72a18b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:31.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a184b40 0x55a72a18b020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc524011cb0 tx=0x7fc5240045f0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 -- 172.21.15.78:0/1987943105 shutdown_connections 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a72a19df70 0x55a72a1a0360 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a18b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a184b40 0x55a72a18b020 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 --2- 172.21.15.78:0/1987943105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a1835d0 0x55a72a18dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc50bfff640 0 client.25183 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 -- 172.21.15.78:0/1987943105 >> 172.21.15.78:0/1987943105 conn(0x55a72a14c310 msgr2=0x55a72a185960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 -- 172.21.15.78:0/1987943105 shutdown_connections 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.152+0000 7fc53546f740 1 -- 172.21.15.78:0/1987943105 wait complete. 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.153+0000 7fc53546f740 20 client.25183 trim_cache size 0 max 16384 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.167+0000 7fc53546f740 1 Processor -- start 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.167+0000 7fc53546f740 1 -- start start 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.167+0000 7fc53546f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a1835d0 0x55a72a19a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.167+0000 7fc53546f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a186120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc53546f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a18e370 0x55a72a186660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc53546f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1c62a0 con 0x55a72a184b40 2024-09-16T00:15:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc53546f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1c7e70 con 0x55a72a1835d0 2024-09-16T00:15:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc53546f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1c6970 con 0x55a72a18e370 2024-09-16T00:15:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a18e370 0x55a72a186660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a1835d0 0x55a72a19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a1835d0 0x55a72a19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:55102/0 (socket says 172.21.15.78:55102) 2024-09-16T00:15:31.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 -- 172.21.15.78:0/3443091563 learned_addr learned my addr 172.21.15.78:0/3443091563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 -- 172.21.15.78:0/3443091563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a18e370 msgr2=0x55a72a186660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 --2- 172.21.15.78:0/3443091563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a18e370 0x55a72a186660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 -- 172.21.15.78:0/3443091563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 msgr2=0x55a72a186120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 --2- 172.21.15.78:0/3443091563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a186120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.168+0000 7fc52bfff640 1 -- 172.21.15.78:0/3443091563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a72a18b940 con 0x55a72a1835d0 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.169+0000 7fc52bfff640 1 --2- 172.21.15.78:0/3443091563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a1835d0 0x55a72a19a310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc52c112b20 tx=0x7fc51c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163a8b13e217c83c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.169+0000 7fc5227fc640 1 -- 172.21.15.78:0/3443091563 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c023070 con 0x55a72a1835d0 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.169+0000 7fc5227fc640 1 -- 172.21.15.78:0/3443091563 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51c024040 con 0x55a72a1835d0 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.169+0000 7fc5227fc640 1 -- 172.21.15.78:0/3443091563 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00fcc0 con 0x55a72a1835d0 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.170+0000 7fc53546f740 1 -- 172.21.15.78:0/3443091563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a1835d0 msgr2=0x55a72a19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.170+0000 7fc53546f740 1 --2- 172.21.15.78:0/3443091563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a1835d0 0x55a72a19a310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc52c112b20 tx=0x7fc51c004ab0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.171+0000 7fc53546f740 1 -- 172.21.15.78:0/3443091563 shutdown_connections 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.171+0000 7fc53546f740 1 --2- 172.21.15.78:0/3443091563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a18e370 0x55a72a186660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.171+0000 7fc53546f740 1 --2- 172.21.15.78:0/3443091563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a186120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.171+0000 7fc53546f740 1 --2- 172.21.15.78:0/3443091563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a1835d0 0x55a72a19a310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.171+0000 7fc53546f740 1 -- 172.21.15.78:0/3443091563 >> 172.21.15.78:0/3443091563 conn(0x55a72a14c310 msgr2=0x55a72a18d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.171+0000 7fc53546f740 1 -- 172.21.15.78:0/3443091563 shutdown_connections 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.171+0000 7fc53546f740 1 -- 172.21.15.78:0/3443091563 wait complete. 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.172+0000 7fc53546f740 1 -- start start 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.172+0000 7fc53546f740 1 Processor -- start 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.173+0000 7fc53546f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.173+0000 7fc53546f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a187e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.173+0000 7fc53546f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a188340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.173+0000 7fc53546f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1c7770 con 0x55a72a184b40 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.173+0000 7fc53546f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1c70a0 con 0x55a72a18e370 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.173+0000 7fc53546f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c0725f0 con 0x55a72a1835d0 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.173+0000 7fc53546f740 10 client.0 _ll_register_callbacks cb 0x7fc524009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.174+0000 7fc530e32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a187e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.174+0000 7fc530e32640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a187e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44292/0 (socket says 172.21.15.78:44292) 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.174+0000 7fc530e32640 1 -- 172.21.15.78:0/3260692553 learned_addr learned my addr 172.21.15.78:0/3260692553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.174+0000 7fc530e32640 1 -- 172.21.15.78:0/3260692553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 msgr2=0x55a72a196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.174+0000 7fc530e32640 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.175+0000 7fc530e32640 1 -- 172.21.15.78:0/3260692553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 msgr2=0x55a72a188340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.175+0000 7fc530e32640 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a188340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.175+0000 7fc530e32640 1 -- 172.21.15.78:0/3260692553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a72a188b70 con 0x55a72a184b40 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.175+0000 7fc530e32640 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a187e00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55a72a18c660 tx=0x7fc524009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fa94472050e052 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.175+0000 7fc50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.178+0000 7fc50bfff640 1 -- 172.21.15.78:0/3260692553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52403b070 con 0x55a72a184b40 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.178+0000 7fc50bfff640 1 -- 172.21.15.78:0/3260692553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc52400f700 con 0x55a72a184b40 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.179+0000 7fc53546f740 1 -- 172.21.15.78:0/3260692553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51c024040 con 0x55a72a184b40 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.182+0000 7fc50bfff640 1 -- 172.21.15.78:0/3260692553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc524000e60 con 0x55a72a184b40 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.182+0000 7fc53546f740 10 client.15789 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.182+0000 7fc53546f740 1 -- 172.21.15.78:0/3260692553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a72a2089d0 con 0x55a72a184b40 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.183+0000 7fc50bfff640 1 -- 172.21.15.78:0/3260692553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc524005500 con 0x55a72a184b40 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.183+0000 7fc50bfff640 1 -- 172.21.15.78:0/3260692553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc52403e020 con 0x55a72a184b40 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.184+0000 7fc53546f740 10 client.15789 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.184+0000 7fc53546f740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.184+0000 7fc53546f740 20 client.15789 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:31.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.184+0000 7fc53546f740 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a72a209120 0x55a72a20b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.184+0000 7fc53546f740 4 client.15789 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.184+0000 7fc53546f740 1 -- 172.21.15.78:0/3260692553 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc51c024040 con 0x55a72a209120 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.185+0000 7fc52bfff640 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a72a209120 0x55a72a20b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.186+0000 7fc52bfff640 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a72a209120 0x55a72a20b540 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffea840bfb60281c server_cookie=eeed3480f272f781 in_seq=0 out_seq=0 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.187+0000 7fc50bfff640 10 client.15789 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.189+0000 7fc50bfff640 1 -- 172.21.15.78:0/3260692553 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc51c024040 con 0x55a72a209120 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.189+0000 7fc50bfff640 10 client.15789 handle_command_reply: tid=0 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.189+0000 7fc53546f740 1 client.15789 shutdown 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.189+0000 7fc53546f740 1 -- 172.21.15.78:0/3260692553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 msgr2=0x55a72a187e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.189+0000 7fc53546f740 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a187e00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55a72a18c660 tx=0x7fc524009530 comp rx=0 tx=0).stop 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.190+0000 7fc53546f740 1 -- 172.21.15.78:0/3260692553 shutdown_connections 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.190+0000 7fc53546f740 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a72a209120 0x55a72a20b540 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.190+0000 7fc53546f740 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a72a18e370 0x55a72a188340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.190+0000 7fc53546f740 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a72a184b40 0x55a72a187e00 secure :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55a72a18c660 tx=0x7fc524009530 comp rx=0 tx=0).stop 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.190+0000 7fc53546f740 1 --2- 172.21.15.78:0/3260692553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a72a1835d0 0x55a72a196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.190+0000 7fc53546f740 1 -- 172.21.15.78:0/3260692553 >> 172.21.15.78:0/3260692553 conn(0x55a72a14c310 msgr2=0x55a72a18c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.190+0000 7fc50bfff640 0 client.15789 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.191+0000 7fc53546f740 1 -- 172.21.15.78:0/3260692553 shutdown_connections 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.191+0000 7fc53546f740 1 -- 172.21.15.78:0/3260692553 wait complete. 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.193+0000 7fc53546f740 20 client.15789 trim_cache size 0 max 16384 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.198+0000 7fc533096640 1 -- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5180a32b0 msgr2=0x7fc5180a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.198+0000 7fc533096640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5180a32b0 0x7fc5180a5770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc5140bc1b0 tx=0x7fc514005d50 comp rx=0 tx=0).stop 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.198+0000 7fc533096640 1 -- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 msgr2=0x7fc52c10c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.198+0000 7fc533096640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 0x7fc52c10c7e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc51402a9a0 tx=0x7fc514003b90 comp rx=0 tx=0).stop 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.199+0000 7fc533096640 1 -- 172.21.15.78:0/1140960966 shutdown_connections 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.199+0000 7fc533096640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5180a32b0 0x7fc5180a5770 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.199+0000 7fc533096640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc52c102ef0 0x7fc52c10c7e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.199+0000 7fc533096640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc52c102570 0x7fc52c116780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.199+0000 7fc533096640 1 --2- 172.21.15.78:0/1140960966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc52c100e20 0x7fc52c116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.199+0000 7fc533096640 1 -- 172.21.15.78:0/1140960966 >> 172.21.15.78:0/1140960966 conn(0x7fc52c0fc7d0 msgr2=0x7fc52c106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.201+0000 7fc533096640 1 -- 172.21.15.78:0/1140960966 shutdown_connections 2024-09-16T00:15:31.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:30.201+0000 7fc533096640 1 -- 172.21.15.78:0/1140960966 wait complete. 2024-09-16T00:15:31.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:31 smithi078 ceph-mon[31157]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 920 op/s 2024-09-16T00:15:31.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:31 smithi193 ceph-mon[35484]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 920 op/s 2024-09-16T00:15:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:31 smithi148 ceph-mon[35599]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 16 MiB/s wr, 920 op/s 2024-09-16T00:15:33.009 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:15:33.009 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:15:33.432 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:33.792 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:33 smithi078 ceph-mon[31157]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-09-16T00:15:33.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:33 smithi193 ceph-mon[35484]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-09-16T00:15:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:33 smithi148 ceph-mon[35599]: pgmap v425: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-09-16T00:15:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.040+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1073834177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 msgr2=0x7fc3c41150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.040+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1073834177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 0x7fc3c41150a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009be0 tx=0x7fc3bc02a6c0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.041+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1073834177 shutdown_connections 2024-09-16T00:15:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.042+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1073834177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 0x7fc3c41150a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.042+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1073834177 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3c4108da0 0x7fc3c410cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.042+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1073834177 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3c41083f0 0x7fc3c41087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.042+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1073834177 >> 172.21.15.78:0/1073834177 conn(0x7fc3c4077af0 msgr2=0x7fc3c4079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.043+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1073834177 shutdown_connections 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.044+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1073834177 wait complete. 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.044+0000 7fc3cbc61640 1 Processor -- start 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.044+0000 7fc3cbc61640 1 -- start start 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3cbc61640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3c41083f0 0x7fc3c41a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3cbc61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3c4108da0 0x7fc3c41a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3cbc61640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 0x7fc3c419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3cbc61640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4118500 con 0x7fc3c41083f0 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3cbc61640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4118380 con 0x7fc3c4108da0 2024-09-16T00:15:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3cbc61640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4118680 con 0x7fc3c410d520 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3ca1fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 0x7fc3c419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3ca1fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 0x7fc3c419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:59338/0 (socket says 172.21.15.78:59338) 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3ca1fe640 1 -- 172.21.15.78:0/1589288489 learned_addr learned my addr 172.21.15.78:0/1589288489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3c4108da0 0x7fc3c41a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3c41083f0 0x7fc3c41a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.045+0000 7fc3ca1fe640 1 -- 172.21.15.78:0/1589288489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3c4108da0 msgr2=0x7fc3c41a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3ca1fe640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3c4108da0 0x7fc3c41a49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3ca1fe640 1 -- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3c41083f0 msgr2=0x7fc3c41a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3ca1fe640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3c41083f0 0x7fc3c41a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3ca1fe640 1 -- 172.21.15.78:0/1589288489 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c419ed50 con 0x7fc3c410d520 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3ca1fe640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 0x7fc3c419e620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc034040 tx=0x7fc3bc0094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3baffd640 1 -- 172.21.15.78:0/1589288489 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc041020 con 0x7fc3c410d520 2024-09-16T00:15:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3baffd640 1 -- 172.21.15.78:0/1589288489 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3bc037040 con 0x7fc3c410d520 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3baffd640 1 -- 172.21.15.78:0/1589288489 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc036070 con 0x7fc3c410d520 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1589288489 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c419efe0 con 0x7fc3c410d520 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.046+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1589288489 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c419f3b0 con 0x7fc3c410d520 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.048+0000 7fc3baffd640 1 -- 172.21.15.78:0/1589288489 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc3bc044020 con 0x7fc3c410d520 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3ce03a740 1 Processor -- start 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3ce03a740 1 -- start start 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 0x557d1a508f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a509480 0x557d1a513800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a513d40 0x557d1a516130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557d1a46e280 con 0x557d1a513d40 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557d1a415b10 con 0x557d1a508b60 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557d1a46a240 con 0x557d1a509480 2024-09-16T00:15:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.049+0000 7fc3baffd640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3b00a32c0 0x7fc3b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3ca1fe640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3b00a32c0 0x7fc3b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c99fd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a509480 0x557d1a513800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a513d40 0x557d1a516130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a513d40 0x557d1a516130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44326/0 (socket says 172.21.15.78:44326) 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 -- 172.21.15.78:0/4022000036 learned_addr learned my addr 172.21.15.78:0/4022000036 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 0x557d1a508f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3baffd640 1 -- 172.21.15.78:0/1589288489 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc3bc0f0570 con 0x7fc3c410d520 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 -- 172.21.15.78:0/4022000036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a509480 msgr2=0x557d1a513800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a509480 0x557d1a513800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 -- 172.21.15.78:0/4022000036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 msgr2=0x557d1a508f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:34.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 0x557d1a508f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 -- 172.21.15.78:0/4022000036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1a4221f0 con 0x557d1a513d40 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a509480 0x557d1a513800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3ca1fe640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3b00a32c0 0x7fc3b00a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc0c4f10 tx=0x7fc3bc0e75a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a513d40 0x557d1a516130 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40126c0 tx=0x7fc3b400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea69c78be3b977b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3affff640 1 -- 172.21.15.78:0/4022000036 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4012c40 con 0x557d1a513d40 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3affff640 1 -- 172.21.15.78:0/4022000036 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b4012de0 con 0x557d1a513d40 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.050+0000 7fc3affff640 1 -- 172.21.15.78:0/4022000036 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4004a00 con 0x557d1a513d40 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 -- 172.21.15.78:0/4022000036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a513d40 msgr2=0x557d1a516130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a513d40 0x557d1a516130 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40126c0 tx=0x7fc3b400b6e0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 -- 172.21.15.78:0/4022000036 shutdown_connections 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a513d40 0x557d1a516130 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a509480 0x557d1a513800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/4022000036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 0x557d1a508f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 -- 172.21.15.78:0/4022000036 >> 172.21.15.78:0/4022000036 conn(0x557d1a5050f0 msgr2=0x557d1a5054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 -- 172.21.15.78:0/4022000036 shutdown_connections 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.051+0000 7fc3ce03a740 1 -- 172.21.15.78:0/4022000036 wait complete. 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 1 -- start start 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 1 Processor -- start 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a508b60 0x557d1a510cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a511210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a513d40 0x557d1a511750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557d1a516910 con 0x557d1a509480 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557d1a428f10 con 0x557d1a513d40 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557d1a516a90 con 0x557d1a508b60 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3ce03a740 10 client.0 _ll_register_callbacks cb 0x557d1a3f1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3c99fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a511210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3c99fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a511210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44336/0 (socket says 172.21.15.78:44336) 2024-09-16T00:15:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3c99fd640 1 -- 172.21.15.78:0/6338025 learned_addr learned my addr 172.21.15.78:0/6338025 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a508b60 0x557d1a510cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.052+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a513d40 0x557d1a511750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3c99fd640 1 -- 172.21.15.78:0/6338025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a508b60 msgr2=0x557d1a510cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a508b60 0x557d1a510cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3c99fd640 1 -- 172.21.15.78:0/6338025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a513d40 msgr2=0x557d1a511750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:34.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a513d40 0x557d1a511750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3c99fd640 1 -- 172.21.15.78:0/6338025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1a1df6a0 con 0x557d1a509480 2024-09-16T00:15:34.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a511210 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000ba30 tx=0x7fc3c000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f21b3cadebf8d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:34.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:34.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/6338025 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0011440 con 0x557d1a509480 2024-09-16T00:15:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/6338025 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc3c00115e0 con 0x557d1a509480 2024-09-16T00:15:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ce03a740 1 -- 172.21.15.78:0/6338025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3b4012de0 con 0x557d1a509480 2024-09-16T00:15:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/6338025 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc3c00117e0 con 0x557d1a509480 2024-09-16T00:15:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ce03a740 10 client.15801 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ce03a740 1 -- 172.21.15.78:0/6338025 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557d1a522620 con 0x557d1a509480 2024-09-16T00:15:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/6338025 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b4012de0 con 0x557d1a509480 2024-09-16T00:15:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/6338025 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc3c001ba00 con 0x557d1a509480 2024-09-16T00:15:34.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ce03a740 10 client.15801 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ce03a740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.053+0000 7fc3ce03a740 20 client.15801 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.054+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557d1a523b80 0x557d1a525f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.054+0000 7fc3ce03a740 4 client.15801 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.054+0000 7fc3ce03a740 1 -- 172.21.15.78:0/6338025 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d1a47b7f0 con 0x557d1a523b80 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.054+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557d1a523b80 0x557d1a525f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.054+0000 7fc3ad7fa640 10 client.15801 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.054+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557d1a523b80 0x557d1a525f70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445a54f27b95f24f server_cookie=970a2528d9f51f53 in_seq=0 out_seq=0 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/6338025 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d1a47b7f0 con 0x557d1a523b80 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ad7fa640 10 client.15801 handle_command_reply: tid=0 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 client.15801 shutdown 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 -- 172.21.15.78:0/6338025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 msgr2=0x557d1a511210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a511210 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000ba30 tx=0x7fc3c000bef0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 -- 172.21.15.78:0/6338025 shutdown_connections 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557d1a523b80 0x557d1a525f70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ad7fa640 0 client.15801 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a513d40 0x557d1a511750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a511210 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/6338025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a508b60 0x557d1a510cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 -- 172.21.15.78:0/6338025 >> 172.21.15.78:0/6338025 conn(0x557d1a5050f0 msgr2=0x557d1a50add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 -- 172.21.15.78:0/6338025 shutdown_connections 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 1 -- 172.21.15.78:0/6338025 wait complete. 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.055+0000 7fc3ce03a740 20 client.15801 trim_cache size 0 max 16384 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.072+0000 7fc3ce03a740 1 Processor -- start 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.072+0000 7fc3ce03a740 1 -- start start 2024-09-16T00:15:34.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.072+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a508b60 0x557d1a52b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.072+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a509480 0x557d1a52bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.072+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 0x557d1a503a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.072+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b51a0 con 0x557d1a508b60 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.072+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b6d70 con 0x557d1a509480 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.072+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b5870 con 0x557d1a513d40 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a508b60 0x557d1a52b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a508b60 0x557d1a52b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44342/0 (socket says 172.21.15.78:44342) 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 -- 172.21.15.78:0/3578995946 learned_addr learned my addr 172.21.15.78:0/3578995946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a509480 0x557d1a52bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 0x557d1a503a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 -- 172.21.15.78:0/3578995946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 msgr2=0x557d1a503a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 0x557d1a503a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 -- 172.21.15.78:0/3578995946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a509480 msgr2=0x557d1a52bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a509480 0x557d1a52bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 -- 172.21.15.78:0/3578995946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d1a50f220 con 0x557d1a508b60 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a508b60 0x557d1a52b860 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4005590 tx=0x7fc3b4012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9cffa56636038e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3af7fe640 1 -- 172.21.15.78:0/3578995946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40108a0 con 0x557d1a508b60 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3af7fe640 1 -- 172.21.15.78:0/3578995946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b4016030 con 0x557d1a508b60 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.073+0000 7fc3af7fe640 1 -- 172.21.15.78:0/3578995946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b402ec00 con 0x557d1a508b60 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.074+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3578995946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a508b60 msgr2=0x557d1a52b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.074+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a508b60 0x557d1a52b860 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4005590 tx=0x7fc3b4012f90 comp rx=0 tx=0).stop 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.074+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3578995946 shutdown_connections 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.074+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 0x557d1a503a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.074+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a509480 0x557d1a52bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.074+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3578995946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a508b60 0x557d1a52b860 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.074+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3578995946 >> 172.21.15.78:0/3578995946 conn(0x557d1a5050f0 msgr2=0x557d1a50fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.075+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3578995946 shutdown_connections 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.075+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3578995946 wait complete. 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.075+0000 7fc3ce03a740 1 -- start start 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.075+0000 7fc3ce03a740 1 Processor -- start 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.076+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 0x557d1a50dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.076+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a50e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.076+0000 7fc3ce03a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 0x557d1a50e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.076+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b6670 con 0x557d1a509480 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.076+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b5fa0 con 0x557d1a508b60 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.077+0000 7fc3ce03a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41b3180 con 0x557d1a513d40 2024-09-16T00:15:34.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.077+0000 7fc3ce03a740 10 client.0 _ll_register_callbacks cb 0x7fc3c00042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.077+0000 7fc3c99fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a50e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.077+0000 7fc3c99fd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a50e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44346/0 (socket says 172.21.15.78:44346) 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.077+0000 7fc3c99fd640 1 -- 172.21.15.78:0/3388660291 learned_addr learned my addr 172.21.15.78:0/3388660291 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.077+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 0x557d1a50dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3c99fd640 1 -- 172.21.15.78:0/3388660291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 msgr2=0x557d1a50e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 0x557d1a50e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3c99fd640 1 -- 172.21.15.78:0/3388660291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 msgr2=0x557d1a50dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 0x557d1a50dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3c99fd640 1 -- 172.21.15.78:0/3388660291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d1a58d8e0 con 0x557d1a509480 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3c99fd640 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a50e2e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc3c001bef0 tx=0x7fc3c0003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905f19ea65c2dceb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/3388660291 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c002d070 con 0x557d1a509480 2024-09-16T00:15:34.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/3388660291 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc3c0011b90 con 0x557d1a509480 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3388660291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3b4016030 con 0x557d1a509480 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/3388660291 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc3c0011db0 con 0x557d1a509480 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3ce03a740 10 client.15807 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.078+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3388660291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557d1a58ec00 con 0x557d1a509480 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.079+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/3388660291 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b4016030 con 0x557d1a509480 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.079+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/3388660291 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fc3c0004b30 con 0x557d1a509480 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.079+0000 7fc3ce03a740 10 client.15807 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.079+0000 7fc3ce03a740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.079+0000 7fc3ce03a740 20 client.15807 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.079+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557d1a523b80 0x557d1a591420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.079+0000 7fc3ce03a740 4 client.15807 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:15:34.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.079+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3388660291 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557d1a475d00 con 0x557d1a523b80 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.080+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557d1a523b80 0x557d1a591420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.081+0000 7fc3c91fc640 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557d1a523b80 0x557d1a591420 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec12d48de052bf4 server_cookie=cd83410b123cd179 in_seq=0 out_seq=0 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.081+0000 7fc3ad7fa640 10 client.15807 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ad7fa640 1 -- 172.21.15.78:0/3388660291 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557d1a475d00 con 0x557d1a523b80 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ad7fa640 10 client.15807 handle_command_reply: tid=0 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 client.15807 shutdown 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3388660291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 msgr2=0x557d1a50e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a50e2e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc3c001bef0 tx=0x7fc3c0003b00 comp rx=0 tx=0).stop 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3388660291 shutdown_connections 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557d1a523b80 0x557d1a591420 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d1a513d40 0x557d1a50e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557d1a509480 0x557d1a50e2e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 --2- 172.21.15.78:0/3388660291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557d1a508b60 0x557d1a50dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3388660291 >> 172.21.15.78:0/3388660291 conn(0x557d1a5050f0 msgr2=0x557d1a50a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:34.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ad7fa640 0 client.15807 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:34.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3388660291 shutdown_connections 2024-09-16T00:15:34.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 1 -- 172.21.15.78:0/3388660291 wait complete. 2024-09-16T00:15:34.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.083+0000 7fc3ce03a740 20 client.15807 trim_cache size 0 max 16384 2024-09-16T00:15:34.697 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:15:34.698 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:15:34.698 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "6ee681e6-1355-4a44-8e8a-560c9f5eefb7", 2024-09-16T00:15:34.698 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:15:34.698 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:15:34.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3b00a32c0 msgr2=0x7fc3b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3b00a32c0 0x7fc3b00a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc0c4f10 tx=0x7fc3bc0e75a0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1589288489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 msgr2=0x7fc3c419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 0x7fc3c419e620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc034040 tx=0x7fc3bc0094f0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1589288489 shutdown_connections 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc3b00a32c0 0x7fc3b00a5780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3c410d520 0x7fc3c419e620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3c4108da0 0x7fc3c41a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 --2- 172.21.15.78:0/1589288489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3c41083f0 0x7fc3c41a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1589288489 >> 172.21.15.78:0/1589288489 conn(0x7fc3c4077af0 msgr2=0x7fc3c40783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1589288489 shutdown_connections 2024-09-16T00:15:34.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:34.084+0000 7fc3cbc61640 1 -- 172.21.15.78:0/1589288489 wait complete. 2024-09-16T00:15:34.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:34 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:15:34.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:34 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:34 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:15:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:34 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:35.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:34 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:15:35.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:34 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:35.597 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:15:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 95.0 s, 2847.8 tps, lat 35.080 ms stddev 39.644 2024-09-16T00:15:35.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:35 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:15:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:35 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:35 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:15:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:35 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:15:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:35 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:35 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:35.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:35 smithi193 ceph-mon[35484]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 984 op/s 2024-09-16T00:15:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:35 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:15:35.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:35 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:35.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:35 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:15:35.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:35 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:15:35.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:35 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:35.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:35 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:35.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:35 smithi148 ceph-mon[35599]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 984 op/s 2024-09-16T00:15:36.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:35 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:15:36.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:35 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:36.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:35 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:15:36.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:35 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:15:36.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:35 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:36.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:35 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:15:36.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:35 smithi078 ceph-mon[31157]: pgmap v426: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 984 op/s 2024-09-16T00:15:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.758+0000 7f6a29d0a640 1 -- 172.21.15.78:0/1365269168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24101e20 msgr2=0x7f6a241056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.758+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/1365269168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24101e20 0x7f6a241056b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009900 tx=0x7f6a1802a410 comp rx=0 tx=0).stop 2024-09-16T00:15:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.758+0000 7f6a29d0a640 1 -- 172.21.15.78:0/1365269168 shutdown_connections 2024-09-16T00:15:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.758+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/1365269168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a24105de0 0x7f6a2410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.758+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/1365269168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24101e20 0x7f6a241056b0 secure :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009900 tx=0x7f6a1802a410 comp rx=0 tx=0).stop 2024-09-16T00:15:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.758+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/1365269168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a24101470 0x7f6a24101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.758+0000 7f6a29d0a640 1 -- 172.21.15.78:0/1365269168 >> 172.21.15.78:0/1365269168 conn(0x7f6a24077230 msgr2=0x7f6a240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.760+0000 7f6a29d0a640 1 -- 172.21.15.78:0/1365269168 shutdown_connections 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.760+0000 7f6a29d0a640 1 -- 172.21.15.78:0/1365269168 wait complete. 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.761+0000 7f6a29d0a640 1 Processor -- start 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.761+0000 7f6a29d0a640 1 -- start start 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.761+0000 7f6a29d0a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a24101470 0x7f6a2419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.761+0000 7f6a29d0a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24105de0 0x7f6a2419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.761+0000 7f6a29d0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a2419c510 0x7f6a2419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.761+0000 7f6a29d0a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24110e50 con 0x7f6a24105de0 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.761+0000 7f6a29d0a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24110cd0 con 0x7f6a2419c510 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.761+0000 7f6a29d0a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24110fd0 con 0x7f6a24101470 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a24101470 0x7f6a2419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24105de0 0x7f6a2419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a23fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a2419c510 0x7f6a2419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a24101470 0x7f6a2419e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:58454/0 (socket says 172.21.15.78:58454) 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a237fe640 1 -- 172.21.15.78:0/2701376393 learned_addr learned my addr 172.21.15.78:0/2701376393 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a23fff640 1 -- 172.21.15.78:0/2701376393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a24101470 msgr2=0x7f6a2419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a23fff640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a24101470 0x7f6a2419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a23fff640 1 -- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24105de0 msgr2=0x7f6a2419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a23fff640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24105de0 0x7f6a2419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a23fff640 1 -- 172.21.15.78:0/2701376393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a24196300 con 0x7f6a2419c510 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.762+0000 7f6a23fff640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a2419c510 0x7f6a2419c9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009fd0 tx=0x7f6a1c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.763+0000 7f6a20ff9640 1 -- 172.21.15.78:0/2701376393 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c00ce20 con 0x7f6a2419c510 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.763+0000 7f6a29d0a640 1 -- 172.21.15.78:0/2701376393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241965f0 con 0x7f6a2419c510 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.763+0000 7f6a29d0a640 1 -- 172.21.15.78:0/2701376393 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a24196b00 con 0x7f6a2419c510 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.763+0000 7f6a20ff9640 1 -- 172.21.15.78:0/2701376393 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c005020 con 0x7f6a2419c510 2024-09-16T00:15:37.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.763+0000 7f6a20ff9640 1 -- 172.21.15.78:0/2701376393 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c010720 con 0x7f6a2419c510 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.766+0000 7f6a2c0e3740 1 Processor -- start 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a20ff9640 1 -- 172.21.15.78:0/2701376393 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6a1c0108c0 con 0x7f6a2419c510 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a2c0e3740 1 -- start start 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c8576b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 0x558c85776140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85776680 0x558c85778a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c856d12e0 con 0x558c8576b4a0 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558c85678ce0 con 0x558c85776680 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558c856cd2a0 con 0x558c8576bdc0 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c8576b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c8576b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33214/0 (socket says 172.21.15.78:33214) 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.767+0000 7f6a22ffd640 1 -- 172.21.15.78:0/510797453 learned_addr learned my addr 172.21.15.78:0/510797453 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.768+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/510797453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85776680 0x558c85778a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.768+0000 7f6a22ffd640 1 -- 172.21.15.78:0/510797453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 msgr2=0x558c85776140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.768+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/510797453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 0x558c85776140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.768+0000 7f6a22ffd640 1 -- 172.21.15.78:0/510797453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85776680 msgr2=0x558c85778a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.768+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/510797453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85776680 0x558c85778a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.768+0000 7f6a22ffd640 1 -- 172.21.15.78:0/510797453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8565d440 con 0x558c8576b4a0 2024-09-16T00:15:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.768+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/510797453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c8576b880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005c40 tx=0x7f6a18000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424fd4a404e4ae06 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.769+0000 7f6a09ffb640 1 -- 172.21.15.78:0/510797453 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18035040 con 0x558c8576b4a0 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.769+0000 7f6a20ff9640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a0c0a32c0 0x7f6a0c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.769+0000 7f6a09ffb640 1 -- 172.21.15.78:0/510797453 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a18005520 con 0x558c8576b4a0 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.769+0000 7f6a09ffb640 1 -- 172.21.15.78:0/510797453 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a180315b0 con 0x558c8576b4a0 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.769+0000 7f6a20ff9640 1 -- 172.21.15.78:0/2701376393 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6a1c0d0190 con 0x7f6a2419c510 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.769+0000 7f6a237fe640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a0c0a32c0 0x7f6a0c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/510797453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 msgr2=0x558c8576b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/510797453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c8576b880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005c40 tx=0x7f6a18000bc0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/510797453 shutdown_connections 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a237fe640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a0c0a32c0 0x7f6a0c0a5780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x558c85768890 tx=0x7f6a14004140 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/510797453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85776680 0x558c85778a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/510797453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 0x558c85776140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/510797453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c8576b880 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/510797453 >> 172.21.15.78:0/510797453 conn(0x558c85767a30 msgr2=0x558c85767e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/510797453 shutdown_connections 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.770+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/510797453 wait complete. 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.771+0000 7f6a2c0e3740 1 -- start start 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.771+0000 7f6a2c0e3740 1 Processor -- start 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.772+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85775f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c8576bdc0 0x558c857731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c85776680 0x558c85773700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558c85779250 con 0x558c8576b4a0 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558c8568c050 con 0x558c8576bdc0 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558c857793d0 con 0x558c85776680 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a2c0e3740 10 client.0 _ll_register_callbacks cb 0x558c856cc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85775f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85775f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33216/0 (socket says 172.21.15.78:33216) 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a237fe640 1 -- 172.21.15.78:0/3683748193 learned_addr learned my addr 172.21.15.78:0/3683748193 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c8576bdc0 0x558c857731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a23fff640 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c85776680 0x558c85773700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a237fe640 1 -- 172.21.15.78:0/3683748193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c85776680 msgr2=0x558c85773700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a237fe640 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c85776680 0x558c85773700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a237fe640 1 -- 172.21.15.78:0/3683748193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c8576bdc0 msgr2=0x558c857731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.773+0000 7f6a237fe640 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c8576bdc0 0x558c857731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.774+0000 7f6a237fe640 1 -- 172.21.15.78:0/3683748193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c854423c0 con 0x558c8576b4a0 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.774+0000 7f6a237fe640 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85775f10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003e70 tx=0x7f6a1400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac439d5d6515136b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.774+0000 7f6a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.774+0000 7f6a02ffd640 1 -- 172.21.15.78:0/3683748193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1401e620 con 0x558c8576b4a0 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.775+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/3683748193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a18005520 con 0x558c8576b4a0 2024-09-16T00:15:37.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.775+0000 7f6a02ffd640 1 -- 172.21.15.78:0/3683748193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6a1401e7c0 con 0x558c8576b4a0 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.775+0000 7f6a02ffd640 1 -- 172.21.15.78:0/3683748193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6a1402f050 con 0x558c8576b4a0 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.775+0000 7f6a2c0e3740 10 client.15819 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/3683748193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c85784c20 con 0x558c8576b4a0 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a02ffd640 1 -- 172.21.15.78:0/3683748193 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a1401c3b0 con 0x558c8576b4a0 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a02ffd640 1 -- 172.21.15.78:0/3683748193 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f6a1401d3b0 con 0x558c8576b4a0 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a2c0e3740 10 client.15819 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a2c0e3740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a2c0e3740 20 client.15819 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558c85786500 0x558c85788960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a2c0e3740 4 client.15819 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.776+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/3683748193 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a18005520 con 0x558c85786500 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.777+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558c85786500 0x558c85788960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.777+0000 7f6a02ffd640 10 client.15819 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.777+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558c85786500 0x558c85788960 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4fcfda3072061a server_cookie=91b7078dc7c83a0f in_seq=0 out_seq=0 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.778+0000 7f6a02ffd640 1 -- 172.21.15.78:0/3683748193 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6a18005520 con 0x558c85786500 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.778+0000 7f6a02ffd640 10 client.15819 handle_command_reply: tid=0 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.778+0000 7f6a2c0e3740 1 client.15819 shutdown 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/3683748193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 msgr2=0x558c85775f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85775f10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003e70 tx=0x7f6a1400b4a0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/3683748193 shutdown_connections 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558c85786500 0x558c85788960 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c85776680 0x558c85773700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c8576bdc0 0x558c857731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/3683748193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85775f10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/3683748193 >> 172.21.15.78:0/3683748193 conn(0x558c85767a30 msgr2=0x558c85766b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:37.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a02ffd640 0 client.15819 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.779+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/3683748193 shutdown_connections 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.780+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/3683748193 wait complete. 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.782+0000 7f6a2c0e3740 20 client.15819 trim_cache size 0 max 16384 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.811+0000 7f6a2c0e3740 1 Processor -- start 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.811+0000 7f6a2c0e3740 1 -- start start 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.811+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8576e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.811+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85773180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.811+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 0x558c85766a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.811+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1000fb70 con 0x558c8576b4a0 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.811+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10011740 con 0x558c85767a30 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10010240 con 0x558c8576bdc0 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a23fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 0x558c85766a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a237fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8576e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a23fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 0x558c85766a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:58484/0 (socket says 172.21.15.78:58484) 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a23fff640 1 -- 172.21.15.78:0/504046971 learned_addr learned my addr 172.21.15.78:0/504046971 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/504046971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85773180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a237fe640 1 -- 172.21.15.78:0/504046971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 msgr2=0x558c85766a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a237fe640 1 --2- 172.21.15.78:0/504046971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 0x558c85766a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a237fe640 1 -- 172.21.15.78:0/504046971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 msgr2=0x558c85773180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a237fe640 1 --2- 172.21.15.78:0/504046971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85773180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.812+0000 7f6a237fe640 1 -- 172.21.15.78:0/504046971 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c85778db0 con 0x558c85767a30 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.813+0000 7f6a237fe640 1 --2- 172.21.15.78:0/504046971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8576e900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003e70 tx=0x7f6a14005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c5750dabfb6e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.813+0000 7f6a097fa640 1 -- 172.21.15.78:0/504046971 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1402c040 con 0x558c85767a30 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.813+0000 7f6a097fa640 1 -- 172.21.15.78:0/504046971 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a14038030 con 0x558c85767a30 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.813+0000 7f6a097fa640 1 -- 172.21.15.78:0/504046971 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14033e10 con 0x558c85767a30 2024-09-16T00:15:37.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.814+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/504046971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 msgr2=0x558c8576e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.814+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/504046971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8576e900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a14003e70 tx=0x7f6a14005fd0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.814+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/504046971 shutdown_connections 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.814+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/504046971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576bdc0 0x558c85766a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.814+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/504046971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576b4a0 0x558c85773180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.814+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/504046971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8576e900 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.814+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/504046971 >> 172.21.15.78:0/504046971 conn(0x558c85773860 msgr2=0x558c85766350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.814+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/504046971 shutdown_connections 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.815+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/504046971 wait complete. 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.815+0000 7f6a2c0e3740 1 -- start start 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.815+0000 7f6a2c0e3740 1 Processor -- start 2024-09-16T00:15:37.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8578bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576b4a0 0x558c8578c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a2c0e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576bdc0 0x558c8578c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10011040 con 0x558c8576bdc0 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10010970 con 0x558c85767a30 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a2c0e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1000db50 con 0x558c8576b4a0 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a2c0e3740 10 client.0 _ll_register_callbacks cb 0x7f6a14004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576b4a0 0x558c8578c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a22ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576b4a0 0x558c8578c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:58500/0 (socket says 172.21.15.78:58500) 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a22ffd640 1 -- 172.21.15.78:0/729430818 learned_addr learned my addr 172.21.15.78:0/729430818 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a237fe640 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8578bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a22ffd640 1 -- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 msgr2=0x558c8578bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8578bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a22ffd640 1 -- 172.21.15.78:0/729430818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576bdc0 msgr2=0x558c8578c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576bdc0 0x558c8578c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.816+0000 7f6a22ffd640 1 -- 172.21.15.78:0/729430818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c858011d0 con 0x558c8576b4a0 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.817+0000 7f6a237fe640 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8578bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.817+0000 7f6a22ffd640 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576b4a0 0x558c8578c4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009900 tx=0x7f6a18009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8cc05af11e7adb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.817+0000 7f6a02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.817+0000 7f6a02ffd640 1 -- 172.21.15.78:0/729430818 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a18035070 con 0x558c8576b4a0 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.817+0000 7f6a02ffd640 1 -- 172.21.15.78:0/729430818 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6a18031bf0 con 0x558c8576b4a0 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.817+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/729430818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a14038030 con 0x558c8576b4a0 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a02ffd640 1 -- 172.21.15.78:0/729430818 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6a18031e10 con 0x558c8576b4a0 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a2c0e3740 10 client.25213 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/729430818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c858024a0 con 0x558c8576b4a0 2024-09-16T00:15:37.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a02ffd640 1 -- 172.21.15.78:0/729430818 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a14038030 con 0x558c8576b4a0 2024-09-16T00:15:37.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a02ffd640 1 -- 172.21.15.78:0/729430818 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f6a18051020 con 0x558c8576b4a0 2024-09-16T00:15:37.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a2c0e3740 10 client.25213 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:37.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a2c0e3740 10 client.25213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:15:37.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a2c0e3740 20 client.25213 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:37.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558c85776680 0x558c85804c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:37.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a2c0e3740 4 client.25213 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:15:37.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.818+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/729430818 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c856de850 con 0x558c85776680 2024-09-16T00:15:37.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.819+0000 7f6a237fe640 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558c85776680 0x558c85804c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:37.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a237fe640 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558c85776680 0x558c85804c90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f463ab93a59fdff4 server_cookie=a4580a700ecd0a24 in_seq=0 out_seq=0 2024-09-16T00:15:37.327 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:15:37.327 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:15:37.327 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:15:37.327 INFO:teuthology.orchestra.run.smithi078.stdout: "6ee681e6-1355-4a44-8e8a-560c9f5eefb7": { 2024-09-16T00:15:37.327 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:15:37.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "6ee681e6-1355-4a44-8e8a-560c9f5eefb7", 2024-09-16T00:15:37.327 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a02ffd640 10 client.25213 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a02ffd640 1 -- 172.21.15.78:0/729430818 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558c856de850 con 0x558c85776680 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a02ffd640 10 client.25213 handle_command_reply: tid=0 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 client.25213 shutdown 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/729430818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576b4a0 msgr2=0x558c8578c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576b4a0 0x558c8578c4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009900 tx=0x7f6a18009300 comp rx=0 tx=0).stop 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/729430818 shutdown_connections 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558c85776680 0x558c85804c90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558c8576bdc0 0x558c8578c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8576b4a0 0x558c8578c4b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 --2- 172.21.15.78:0/729430818 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c85767a30 0x558c8578bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/729430818 >> 172.21.15.78:0/729430818 conn(0x558c85773860 msgr2=0x558c85766b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.821+0000 7f6a02ffd640 0 client.25213 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.822+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/729430818 shutdown_connections 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.822+0000 7f6a2c0e3740 1 -- 172.21.15.78:0/729430818 wait complete. 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.822+0000 7f6a2c0e3740 20 client.25213 trim_cache size 0 max 16384 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.822+0000 7f6a29d0a640 1 -- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a0c0a32c0 msgr2=0x7f6a0c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.822+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a0c0a32c0 0x7f6a0c0a5780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x558c85768890 tx=0x7f6a14004140 comp rx=0 tx=0).stop 2024-09-16T00:15:37.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 -- 172.21.15.78:0/2701376393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a2419c510 msgr2=0x7f6a2419c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a2419c510 0x7f6a2419c9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009fd0 tx=0x7f6a1c00ef10 comp rx=0 tx=0).stop 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 -- 172.21.15.78:0/2701376393 shutdown_connections 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6a0c0a32c0 0x7f6a0c0a5780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a2419c510 0x7f6a2419c9c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a24105de0 0x7f6a2419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 --2- 172.21.15.78:0/2701376393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a24101470 0x7f6a2419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 -- 172.21.15.78:0/2701376393 >> 172.21.15.78:0/2701376393 conn(0x7f6a24077230 msgr2=0x7f6a24104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 -- 172.21.15.78:0/2701376393 shutdown_connections 2024-09-16T00:15:37.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:36.823+0000 7f6a29d0a640 1 -- 172.21.15.78:0/2701376393 wait complete. 2024-09-16T00:15:37.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:37 smithi193 ceph-mon[35484]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 975 op/s 2024-09-16T00:15:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:37 smithi148 ceph-mon[35599]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 975 op/s 2024-09-16T00:15:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:37 smithi078 ceph-mon[31157]: pgmap v427: 129 pgs: 129 active+clean; 1.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 975 op/s 2024-09-16T00:15:38.089 INFO:tasks.cephfs.filesystem:scrub status for tag:6ee681e6-1355-4a44-8e8a-560c9f5eefb7 - {'path': '/', 'tag': '6ee681e6-1355-4a44-8e8a-560c9f5eefb7', 'options': 'recursive,force'} 2024-09-16T00:15:39.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:39 smithi193 ceph-mon[35484]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 934 op/s 2024-09-16T00:15:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:39 smithi148 ceph-mon[35599]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 934 op/s 2024-09-16T00:15:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:39 smithi078 ceph-mon[31157]: pgmap v428: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 934 op/s 2024-09-16T00:15:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 100.0 s, 3419.2 tps, lat 29.188 ms stddev 35.814 2024-09-16T00:15:41.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:41 smithi193 ceph-mon[35484]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 798 op/s 2024-09-16T00:15:41.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:41 smithi148 ceph-mon[35599]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 798 op/s 2024-09-16T00:15:42.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:41 smithi078 ceph-mon[31157]: pgmap v429: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 798 op/s 2024-09-16T00:15:42.905 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:15:42.905 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T00:15:42.906 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:15:43.299 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:43.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:43 smithi078 ceph-mon[31157]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 890 op/s 2024-09-16T00:15:43.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:43 smithi193 ceph-mon[35484]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 890 op/s 2024-09-16T00:15:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:43 smithi148 ceph-mon[35599]: pgmap v430: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 890 op/s 2024-09-16T00:15:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 -- 172.21.15.78:0/1446434041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148101de0 msgr2=0x7ff1481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 --2- 172.21.15.78:0/1446434041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148101de0 0x7ff1481021c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff138009900 tx=0x7ff13802a410 comp rx=0 tx=0).stop 2024-09-16T00:15:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 -- 172.21.15.78:0/1446434041 shutdown_connections 2024-09-16T00:15:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 --2- 172.21.15.78:0/1446434041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff148106f10 0x7ff14810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 --2- 172.21.15.78:0/1446434041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff148102790 0x7ff1481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 --2- 172.21.15.78:0/1446434041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148101de0 0x7ff1481021c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 -- 172.21.15.78:0/1446434041 >> 172.21.15.78:0/1446434041 conn(0x7ff1480fd880 msgr2=0x7ff1480ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 -- 172.21.15.78:0/1446434041 shutdown_connections 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.516+0000 7ff14e260640 1 -- 172.21.15.78:0/1446434041 wait complete. 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff14e260640 1 Processor -- start 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff14e260640 1 -- start start 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff14e260640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff148101de0 0x7ff148198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff14e260640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148102790 0x7ff1481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff14e260640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff148106f10 0x7ff14819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff14e260640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff148111f90 con 0x7ff148102790 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff14e260640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff148111e10 con 0x7ff148101de0 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff14e260640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff148112110 con 0x7ff148106f10 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff146ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148102790 0x7ff1481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff146ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148102790 0x7ff1481989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33266/0 (socket says 172.21.15.78:33266) 2024-09-16T00:15:45.423 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:15:45.423 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:15:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff146ffd640 1 -- 172.21.15.78:0/632287154 learned_addr learned my addr 172.21.15.78:0/632287154 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.517+0000 7ff1477fe640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff148101de0 0x7ff148198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff147fff640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff148106f10 0x7ff14819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff146ffd640 1 -- 172.21.15.78:0/632287154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff148106f10 msgr2=0x7ff14819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff146ffd640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff148106f10 0x7ff14819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff146ffd640 1 -- 172.21.15.78:0/632287154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff148101de0 msgr2=0x7ff148198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff146ffd640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff148101de0 0x7ff148198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff146ffd640 1 -- 172.21.15.78:0/632287154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14819d420 con 0x7ff148102790 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff146ffd640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148102790 0x7ff1481989b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00b4a0 tx=0x7ff13c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff144ff9640 1 -- 172.21.15.78:0/632287154 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c014070 con 0x7ff148102790 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff144ff9640 1 -- 172.21.15.78:0/632287154 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff13c010af0 con 0x7ff148102790 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff144ff9640 1 -- 172.21.15.78:0/632287154 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c017d10 con 0x7ff148102790 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14819d710 con 0x7ff148102790 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.518+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff148074420 con 0x7ff148102790 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.519+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1481032c0 con 0x7ff148102790 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.520+0000 7ff144ff9640 1 -- 172.21.15.78:0/632287154 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff13c017070 con 0x7ff148102790 2024-09-16T00:15:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.521+0000 7ff144ff9640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff1300a76d0 0x7ff1300a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.521+0000 7ff1477fe640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff1300a76d0 0x7ff1300a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.521+0000 7ff144ff9640 1 -- 172.21.15.78:0/632287154 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff13c0cef60 con 0x7ff148102790 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.521+0000 7ff1477fe640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff1300a76d0 0x7ff1300a9b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff138009230 tx=0x7ff1380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.525+0000 7ff144ff9640 1 -- 172.21.15.78:0/632287154 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff13c09b920 con 0x7ff148102790 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.808+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff14810a0f0 con 0x7ff148102790 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.809+0000 7ff144ff9640 1 -- 172.21.15.78:0/632287154 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7ff13c09b220 con 0x7ff148102790 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff1300a76d0 msgr2=0x7ff1300a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff1300a76d0 0x7ff1300a9b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff138009230 tx=0x7ff1380058d0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148102790 msgr2=0x7ff1481989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148102790 0x7ff1481989b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00b4a0 tx=0x7ff13c00b960 comp rx=0 tx=0).stop 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 shutdown_connections 2024-09-16T00:15:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff1300a76d0 0x7ff1300a9b90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff148106f10 0x7ff14819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff148102790 0x7ff1481989b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.812+0000 7ff14e260640 1 --2- 172.21.15.78:0/632287154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff148101de0 0x7ff148198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.813+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 >> 172.21.15.78:0/632287154 conn(0x7ff1480fd880 msgr2=0x7ff1480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.813+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 shutdown_connections 2024-09-16T00:15:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:44.813+0000 7ff14e260640 1 -- 172.21.15.78:0/632287154 wait complete. 2024-09-16T00:15:45.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:45 smithi078 ceph-mon[31157]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 791 op/s 2024-09-16T00:15:45.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:45 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/632287154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:15:45.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 105.0 s, 1855.0 tps, lat 53.028 ms stddev 60.458 2024-09-16T00:15:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:45 smithi193 ceph-mon[35484]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 791 op/s 2024-09-16T00:15:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:45 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/632287154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:15:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:45 smithi148 ceph-mon[35599]: pgmap v431: 129 pgs: 129 active+clean; 1.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 791 op/s 2024-09-16T00:15:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:45 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/632287154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:15:46.710 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-16T00:15:46.710 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-16T00:15:47.096 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:15:47.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.733+0000 7f526788f640 1 -- 172.21.15.78:0/2799712208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260100e20 msgr2=0x7f5260101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.733+0000 7f526788f640 1 --2- 172.21.15.78:0/2799712208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260100e20 0x7f5260101200 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f525c009900 tx=0x7f525c02a480 comp rx=0 tx=0).stop 2024-09-16T00:15:47.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.733+0000 7f526788f640 1 -- 172.21.15.78:0/2799712208 shutdown_connections 2024-09-16T00:15:47.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.733+0000 7f526788f640 1 --2- 172.21.15.78:0/2799712208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5260102ef0 0x7f526010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.733+0000 7f526788f640 1 --2- 172.21.15.78:0/2799712208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5260102570 0x7f52601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.733+0000 7f526788f640 1 --2- 172.21.15.78:0/2799712208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260100e20 0x7f5260101200 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.733+0000 7f526788f640 1 -- 172.21.15.78:0/2799712208 >> 172.21.15.78:0/2799712208 conn(0x7f52600fc7d0 msgr2=0x7f52600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:47.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.734+0000 7f526788f640 1 -- 172.21.15.78:0/2799712208 shutdown_connections 2024-09-16T00:15:47.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.734+0000 7f526788f640 1 -- 172.21.15.78:0/2799712208 wait complete. 2024-09-16T00:15:47.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.734+0000 7f526788f640 1 Processor -- start 2024-09-16T00:15:47.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f526788f640 1 -- start start 2024-09-16T00:15:47.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f526788f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5260100e20 0x7f52601984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f526788f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5260102570 0x7f5260198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f526788f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260102ef0 0x7f526019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f526788f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5260110fa0 con 0x7f5260102ef0 2024-09-16T00:15:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f526788f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5260110e20 con 0x7f5260102570 2024-09-16T00:15:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f526788f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5260111120 con 0x7f5260100e20 2024-09-16T00:15:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f5265e2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260102ef0 0x7f526019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f5265e2c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260102ef0 0x7f526019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50544/0 (socket says 172.21.15.78:50544) 2024-09-16T00:15:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f5265e2c640 1 -- 172.21.15.78:0/1958384866 learned_addr learned my addr 172.21.15.78:0/1958384866 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f526562b640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5260100e20 0x7f52601984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.735+0000 7f5264e2a640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5260102570 0x7f5260198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.736+0000 7f5265e2c640 1 -- 172.21.15.78:0/1958384866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5260100e20 msgr2=0x7f52601984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.736+0000 7f5265e2c640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5260100e20 0x7f52601984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.736+0000 7f5265e2c640 1 -- 172.21.15.78:0/1958384866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5260102570 msgr2=0x7f5260198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.736+0000 7f5265e2c640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5260102570 0x7f5260198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.736+0000 7f5265e2c640 1 -- 172.21.15.78:0/1958384866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526019d4a0 con 0x7f5260102ef0 2024-09-16T00:15:47.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.736+0000 7f5265e2c640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260102ef0 0x7f526019cdc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f525800eb10 tx=0x7f525800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:47.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.737+0000 7f52567fc640 1 -- 172.21.15.78:0/1958384866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258004b20 con 0x7f5260102ef0 2024-09-16T00:15:47.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.737+0000 7f52567fc640 1 -- 172.21.15.78:0/1958384866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258004cc0 con 0x7f5260102ef0 2024-09-16T00:15:47.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.737+0000 7f52567fc640 1 -- 172.21.15.78:0/1958384866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258005720 con 0x7f5260102ef0 2024-09-16T00:15:47.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.737+0000 7f526788f640 1 -- 172.21.15.78:0/1958384866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526019d790 con 0x7f5260102ef0 2024-09-16T00:15:47.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.737+0000 7f526788f640 1 -- 172.21.15.78:0/1958384866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526019db90 con 0x7f5260102ef0 2024-09-16T00:15:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.740+0000 7f5269c68740 1 Processor -- start 2024-09-16T00:15:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.741+0000 7f52567fc640 1 -- 172.21.15.78:0/1958384866 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f52580058c0 con 0x7f5260102ef0 2024-09-16T00:15:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.741+0000 7f5269c68740 1 -- start start 2024-09-16T00:15:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.741+0000 7f5269c68740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992f9850 0x5586992f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.741+0000 7f5269c68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x5586993044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.741+0000 7f5269c68740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x558699306e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.741+0000 7f5269c68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55869925e160 con 0x5586992fa170 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.741+0000 7f5269c68740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558699205720 con 0x558699304a30 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5269c68740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55869925a120 con 0x5586992f9850 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992f9850 0x5586992f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x558699306e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992f9850 0x5586992f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53598/0 (socket says 172.21.15.78:53598) 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 -- 172.21.15.78:0/3245411964 learned_addr learned my addr 172.21.15.78:0/3245411964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 -- 172.21.15.78:0/3245411964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 msgr2=0x558699306e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 --2- 172.21.15.78:0/3245411964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x558699306e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 -- 172.21.15.78:0/3245411964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 msgr2=0x5586993044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 --2- 172.21.15.78:0/3245411964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x5586993044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.742+0000 7f5264e2a640 1 -- 172.21.15.78:0/3245411964 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558698fcfba0 con 0x5586992f9850 2024-09-16T00:15:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.743+0000 7f5264e2a640 1 --2- 172.21.15.78:0/3245411964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992f9850 0x5586992f9c30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f52500093e0 tx=0x7f5250009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d024f83115db31c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.743+0000 7f524b7fe640 1 -- 172.21.15.78:0/3245411964 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250004020 con 0x5586992f9850 2024-09-16T00:15:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.743+0000 7f524b7fe640 1 -- 172.21.15.78:0/3245411964 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52500041c0 con 0x5586992f9850 2024-09-16T00:15:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.743+0000 7f524b7fe640 1 -- 172.21.15.78:0/3245411964 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52500039b0 con 0x5586992f9850 2024-09-16T00:15:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.744+0000 7f52567fc640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f524c0a33d0 0x7f524c0a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.744+0000 7f526562b640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f524c0a33d0 0x7f524c0a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.744+0000 7f52567fc640 1 -- 172.21.15.78:0/1958384866 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f52580d0200 con 0x7f5260102ef0 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.744+0000 7f5269c68740 1 -- 172.21.15.78:0/3245411964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992f9850 msgr2=0x5586992f9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.744+0000 7f5269c68740 1 --2- 172.21.15.78:0/3245411964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992f9850 0x5586992f9c30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f52500093e0 tx=0x7f5250009370 comp rx=0 tx=0).stop 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.744+0000 7f5269c68740 1 -- 172.21.15.78:0/3245411964 shutdown_connections 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.744+0000 7f5269c68740 1 --2- 172.21.15.78:0/3245411964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x558699306e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.745+0000 7f5269c68740 1 --2- 172.21.15.78:0/3245411964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x5586993044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.745+0000 7f5269c68740 1 --2- 172.21.15.78:0/3245411964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992f9850 0x5586992f9c30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.745+0000 7f5269c68740 1 -- 172.21.15.78:0/3245411964 >> 172.21.15.78:0/3245411964 conn(0x5586992f5de0 msgr2=0x5586992f61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.745+0000 7f5269c68740 1 -- 172.21.15.78:0/3245411964 shutdown_connections 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.745+0000 7f526562b640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f524c0a33d0 0x7f524c0a5890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f525c005ab0 tx=0x7f525c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.745+0000 7f5269c68740 1 -- 172.21.15.78:0/3245411964 wait complete. 2024-09-16T00:15:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.746+0000 7f5269c68740 1 -- start start 2024-09-16T00:15:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.746+0000 7f5269c68740 1 Processor -- start 2024-09-16T00:15:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.746+0000 7f5269c68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.746+0000 7f5269c68740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992fa170 0x558699301ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.746+0000 7f5269c68740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5269c68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558699307600 con 0x5586992f9850 2024-09-16T00:15:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5269c68740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5586991e6900 con 0x5586992fa170 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f526562b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5269c68740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558699307780 con 0x558699304a30 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f526562b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50548/0 (socket says 172.21.15.78:50548) 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f526562b640 1 -- 172.21.15.78:0/959967312 learned_addr learned my addr 172.21.15.78:0/959967312 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5264e2a640 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992fa170 0x558699301ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5265e2c640 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5269c68740 10 client.0 _ll_register_callbacks cb 0x5586991e4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5265e2c640 1 -- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992fa170 msgr2=0x558699301ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5265e2c640 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992fa170 0x558699301ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5265e2c640 1 -- 172.21.15.78:0/959967312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 msgr2=0x558699301960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5265e2c640 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5265e2c640 1 -- 172.21.15.78:0/959967312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558699226c50 con 0x558699304a30 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5264e2a640 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992fa170 0x558699301ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5265e2c640 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993023e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5258096fd0 tx=0x7f525809c060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc86970b5e3660fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.747+0000 7f5248ff9640 1 -- 172.21.15.78:0/959967312 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580a0810 con 0x558699304a30 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.748+0000 7f5248ff9640 1 -- 172.21.15.78:0/959967312 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5258013d40 con 0x558699304a30 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.748+0000 7f5269c68740 1 -- 172.21.15.78:0/959967312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52500041c0 con 0x558699304a30 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.748+0000 7f5248ff9640 1 -- 172.21.15.78:0/959967312 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f52580a50d0 con 0x558699304a30 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.748+0000 7f5269c68740 10 client.25225 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.748+0000 7f5269c68740 1 -- 172.21.15.78:0/959967312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558699313320 con 0x558699304a30 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.748+0000 7f5248ff9640 1 -- 172.21.15.78:0/959967312 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525809c6c0 con 0x558699304a30 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.748+0000 7f5248ff9640 1 -- 172.21.15.78:0/959967312 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f52580b51a0 con 0x558699304a30 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.749+0000 7f5269c68740 10 client.25225 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:47.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.749+0000 7f5269c68740 10 client.25225 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.749+0000 7f5269c68740 20 client.25225 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.749+0000 7f5269c68740 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558699314830 0x558699316c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.749+0000 7f5269c68740 4 client.25225 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.749+0000 7f5269c68740 1 -- 172.21.15.78:0/959967312 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f52500041c0 con 0x558699314830 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.750+0000 7f526562b640 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558699314830 0x558699316c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.751+0000 7f5248ff9640 10 client.25225 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.751+0000 7f526562b640 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558699314830 0x558699316c60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aeab57602b4856e server_cookie=4bf0d5c4b102accb in_seq=0 out_seq=0 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.751+0000 7f5248ff9640 1 -- 172.21.15.78:0/959967312 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f52500041c0 con 0x558699314830 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.751+0000 7f5248ff9640 10 client.25225 handle_command_reply: tid=0 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.751+0000 7f5269c68740 1 client.25225 shutdown 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 -- 172.21.15.78:0/959967312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 msgr2=0x5586993023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993023e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5258096fd0 tx=0x7f525809c060 comp rx=0 tx=0).stop 2024-09-16T00:15:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 -- 172.21.15.78:0/959967312 shutdown_connections 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558699314830 0x558699316c60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5248ff9640 0 client.25225 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993023e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992fa170 0x558699301ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 --2- 172.21.15.78:0/959967312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 -- 172.21.15.78:0/959967312 >> 172.21.15.78:0/959967312 conn(0x5586992f5de0 msgr2=0x5586992fbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 -- 172.21.15.78:0/959967312 shutdown_connections 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 1 -- 172.21.15.78:0/959967312 wait complete. 2024-09-16T00:15:47.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.752+0000 7f5269c68740 20 client.25225 trim_cache size 0 max 16384 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.779+0000 7f5269c68740 1 Processor -- start 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.779+0000 7f5269c68740 1 -- start start 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.779+0000 7f5269c68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.779+0000 7f5269c68740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992fa170 0x5586993023f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.779+0000 7f5269c68740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x55869931c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5269c68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52601b5170 con 0x5586992f9850 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5269c68740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52601b6d40 con 0x558699304a30 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5269c68740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52601b5840 con 0x5586992fa170 2024-09-16T00:15:47.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f526562b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x55869931c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x55869931c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42258/0 (socket says 172.21.15.78:42258) 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 -- 172.21.15.78:0/601321380 learned_addr learned my addr 172.21.15.78:0/601321380 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5264e2a640 1 --2- 172.21.15.78:0/601321380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992fa170 0x5586993023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 -- 172.21.15.78:0/601321380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992fa170 msgr2=0x5586993023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 --2- 172.21.15.78:0/601321380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992fa170 0x5586993023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 -- 172.21.15.78:0/601321380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 msgr2=0x558699301eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 --2- 172.21.15.78:0/601321380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 -- 172.21.15.78:0/601321380 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558699300910 con 0x558699304a30 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.780+0000 7f5265e2c640 1 --2- 172.21.15.78:0/601321380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x55869931c530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5258098e30 tx=0x7f525809a970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37b763caf347bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.781+0000 7f524affd640 1 -- 172.21.15.78:0/601321380 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580b9070 con 0x558699304a30 2024-09-16T00:15:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.781+0000 7f524affd640 1 -- 172.21.15.78:0/601321380 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525809b8b0 con 0x558699304a30 2024-09-16T00:15:47.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.781+0000 7f524affd640 1 -- 172.21.15.78:0/601321380 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580a4c40 con 0x558699304a30 2024-09-16T00:15:47.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.782+0000 7f5269c68740 1 -- 172.21.15.78:0/601321380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 msgr2=0x55869931c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.782+0000 7f5269c68740 1 --2- 172.21.15.78:0/601321380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x55869931c530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5258098e30 tx=0x7f525809a970 comp rx=0 tx=0).stop 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.782+0000 7f5269c68740 1 -- 172.21.15.78:0/601321380 shutdown_connections 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.782+0000 7f5269c68740 1 --2- 172.21.15.78:0/601321380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558699304a30 0x55869931c530 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.782+0000 7f5269c68740 1 --2- 172.21.15.78:0/601321380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5586992fa170 0x5586993023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.782+0000 7f5269c68740 1 --2- 172.21.15.78:0/601321380 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992f9850 0x558699301eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.782+0000 7f5269c68740 1 -- 172.21.15.78:0/601321380 >> 172.21.15.78:0/601321380 conn(0x5586992f5de0 msgr2=0x5586993039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.783+0000 7f5269c68740 1 -- 172.21.15.78:0/601321380 shutdown_connections 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.783+0000 7f5269c68740 1 -- 172.21.15.78:0/601321380 wait complete. 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.783+0000 7f5269c68740 1 -- start start 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.783+0000 7f5269c68740 1 Processor -- start 2024-09-16T00:15:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5269c68740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992f9850 0x558699309340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5269c68740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x558699309880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5269c68740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993884d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5269c68740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52601b6640 con 0x5586992fa170 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5269c68740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f52601b5f70 con 0x5586992f9850 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5269c68740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52601b3150 con 0x558699304a30 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5264e2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x558699309880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5264e2a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x558699309880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50568/0 (socket says 172.21.15.78:50568) 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5264e2a640 1 -- 172.21.15.78:0/965456991 learned_addr learned my addr 172.21.15.78:0/965456991 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f526562b640 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992f9850 0x558699309340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5265e2c640 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5269c68740 10 client.0 _ll_register_callbacks cb 0x7f52580050c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:15:47.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5264e2a640 1 -- 172.21.15.78:0/965456991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 msgr2=0x5586993884d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:47.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5264e2a640 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993884d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5264e2a640 1 -- 172.21.15.78:0/965456991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992f9850 msgr2=0x558699309340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:15:47.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5264e2a640 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992f9850 0x558699309340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.784+0000 7f5264e2a640 1 -- 172.21.15.78:0/965456991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558699388ce0 con 0x5586992fa170 2024-09-16T00:15:47.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.785+0000 7f5269c68740 1 -- 172.21.15.78:0/965456991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55869926b890 con 0x5586992fa170 2024-09-16T00:15:47.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.786+0000 7f5264e2a640 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x558699309880 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f525000cfe0 tx=0x7f5250005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601731ffa54ef10 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:15:47.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.786+0000 7f5248ff9640 10 client.15843 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:15:47.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.786+0000 7f5248ff9640 1 -- 172.21.15.78:0/965456991 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525002f050 con 0x5586992fa170 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.786+0000 7f5248ff9640 1 -- 172.21.15.78:0/965456991 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5250004680 con 0x5586992fa170 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5248ff9640 1 -- 172.21.15.78:0/965456991 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f52500048c0 con 0x5586992fa170 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5269c68740 10 client.15843 fetch_fsmap learned FSMap version 31 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5269c68740 1 -- 172.21.15.78:0/965456991 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=31}) -- 0x558699390ec0 con 0x5586992fa170 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5248ff9640 1 -- 172.21.15.78:0/965456991 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f5250004a60 con 0x5586992fa170 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5269c68740 10 client.15843 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5269c68740 10 client.15843 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5269c68740 20 client.15843 populate_metadata read hostname 'smithi078' 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5269c68740 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558699391330 0x558699393750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5269c68740 4 client.15843 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-16T00:15:47.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f5269c68740 1 -- 172.21.15.78:0/965456991 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x558699265be0 con 0x558699391330 2024-09-16T00:15:47.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.787+0000 7f526562b640 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558699391330 0x558699393750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:15:47.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.788+0000 7f5248ff9640 10 client.15843 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:47.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.788+0000 7f526562b640 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558699391330 0x558699393750 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf022e13f438f29c server_cookie=69164efccae041e in_seq=0 out_seq=0 2024-09-16T00:15:47.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:47 smithi193 ceph-mon[35484]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 720 op/s 2024-09-16T00:15:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:47 smithi148 ceph-mon[35599]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 720 op/s 2024-09-16T00:15:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5248ff9640 1 -- 172.21.15.78:0/965456991 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558699265be0 con 0x558699391330 2024-09-16T00:15:47.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5248ff9640 10 client.15843 handle_command_reply: tid=0 2024-09-16T00:15:47.960 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:47 smithi078 ceph-mon[31157]: pgmap v432: 129 pgs: 129 active+clean; 1.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 720 op/s 2024-09-16T00:15:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 client.15843 shutdown 2024-09-16T00:15:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 -- 172.21.15.78:0/965456991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 msgr2=0x558699309880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x558699309880 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f525000cfe0 tx=0x7f5250005270 comp rx=0 tx=0).stop 2024-09-16T00:15:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 -- 172.21.15.78:0/965456991 shutdown_connections 2024-09-16T00:15:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558699391330 0x558699393750 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558699304a30 0x5586993884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5586992fa170 0x558699309880 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5248ff9640 0 client.15843 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 --2- 172.21.15.78:0/965456991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5586992f9850 0x558699309340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.959+0000 7f5269c68740 1 -- 172.21.15.78:0/965456991 >> 172.21.15.78:0/965456991 conn(0x5586992f5de0 msgr2=0x5586992f46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.960+0000 7f5269c68740 1 -- 172.21.15.78:0/965456991 shutdown_connections 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.960+0000 7f5269c68740 1 -- 172.21.15.78:0/965456991 wait complete. 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.960+0000 7f5269c68740 20 client.15843 trim_cache size 0 max 16384 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 31, 2024-09-16T00:15:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 6, 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "db9eea90": { 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 6, 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 139.6, 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:15:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:15:47.964 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:15:47.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.960+0000 7f526788f640 1 -- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f524c0a33d0 msgr2=0x7f524c0a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.960+0000 7f526788f640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f524c0a33d0 0x7f524c0a5890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f525c005ab0 tx=0x7f525c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.960+0000 7f526788f640 1 -- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260102ef0 msgr2=0x7f526019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:15:47.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.960+0000 7f526788f640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260102ef0 0x7f526019cdc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f525800eb10 tx=0x7f525800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.961+0000 7f526788f640 1 -- 172.21.15.78:0/1958384866 shutdown_connections 2024-09-16T00:15:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.961+0000 7f526788f640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f524c0a33d0 0x7f524c0a5890 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.961+0000 7f526788f640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5260102ef0 0x7f526019cdc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.961+0000 7f526788f640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5260102570 0x7f5260198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.961+0000 7f526788f640 1 --2- 172.21.15.78:0/1958384866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5260100e20 0x7f52601984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:15:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.961+0000 7f526788f640 1 -- 172.21.15.78:0/1958384866 >> 172.21.15.78:0/1958384866 conn(0x7f52600fc7d0 msgr2=0x7f526006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:15:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.961+0000 7f526788f640 1 -- 172.21.15.78:0/1958384866 shutdown_connections 2024-09-16T00:15:47.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:15:47.961+0000 7f526788f640 1 -- 172.21.15.78:0/1958384866 wait complete. 2024-09-16T00:15:48.782 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'db9eea90', quiesce took 5.9 seconds. Will release after: 13.65 2024-09-16T00:15:49.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:49 smithi193 ceph-mon[35484]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 MiB/s wr, 715 op/s 2024-09-16T00:15:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:15:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:15:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:15:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:49 smithi148 ceph-mon[35599]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 MiB/s wr, 715 op/s 2024-09-16T00:15:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:15:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:15:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:15:50.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:49 smithi078 ceph-mon[31157]: pgmap v433: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.3 MiB/s wr, 715 op/s 2024-09-16T00:15:50.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:15:50.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:15:50.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:15:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 110.0 s, 863.5 tps, lat 47.159 ms stddev 54.105 2024-09-16T00:15:51.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:51 smithi193 ceph-mon[35484]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 578 op/s 2024-09-16T00:15:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:51 smithi148 ceph-mon[35599]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 578 op/s 2024-09-16T00:15:52.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:51 smithi078 ceph-mon[31157]: pgmap v434: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.7 MiB/s wr, 578 op/s 2024-09-16T00:15:53.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:53 smithi193 ceph-mon[35484]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 590 op/s 2024-09-16T00:15:53.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:53 smithi148 ceph-mon[35599]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 590 op/s 2024-09-16T00:15:54.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:53 smithi078 ceph-mon[31157]: pgmap v435: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 590 op/s 2024-09-16T00:15:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 115.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:15:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:55 smithi193 ceph-mon[35484]: pgmap v436: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 463 op/s 2024-09-16T00:15:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:55 smithi148 ceph-mon[35599]: pgmap v436: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 463 op/s 2024-09-16T00:15:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:55 smithi078 ceph-mon[31157]: pgmap v436: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 463 op/s 2024-09-16T00:15:57.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:57 smithi193 ceph-mon[35484]: pgmap v437: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 348 op/s 2024-09-16T00:15:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:57 smithi148 ceph-mon[35599]: pgmap v437: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 348 op/s 2024-09-16T00:15:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:57 smithi078 ceph-mon[31157]: pgmap v437: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 348 op/s 2024-09-16T00:15:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:15:59 smithi193 ceph-mon[35484]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 496 op/s 2024-09-16T00:15:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:15:59 smithi148 ceph-mon[35599]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 496 op/s 2024-09-16T00:16:00.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:15:59 smithi078 ceph-mon[31157]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 496 op/s 2024-09-16T00:16:00.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 120.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:16:01.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:01 smithi193 ceph-mon[35484]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 428 op/s 2024-09-16T00:16:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:01 smithi148 ceph-mon[35599]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 428 op/s 2024-09-16T00:16:02.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:01 smithi078 ceph-mon[31157]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 428 op/s 2024-09-16T00:16:02.433 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:16:02.697 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 -- 172.21.15.78:0/3773563801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684101420 msgr2=0x7f968410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 --2- 172.21.15.78:0/3773563801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684101420 0x7f968410d950 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f967c009cd0 tx=0x7f967c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 -- 172.21.15.78:0/3773563801 shutdown_connections 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 --2- 172.21.15.78:0/3773563801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684101420 0x7f968410d950 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 --2- 172.21.15.78:0/3773563801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9684100a80 0x7f9684100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 --2- 172.21.15.78:0/3773563801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9684106a80 0x7f9684106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 -- 172.21.15.78:0/3773563801 >> 172.21.15.78:0/3773563801 conn(0x7f96840fc7a0 msgr2=0x7f96840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 -- 172.21.15.78:0/3773563801 shutdown_connections 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 -- 172.21.15.78:0/3773563801 wait complete. 2024-09-16T00:16:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 Processor -- start 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.420+0000 7f968ae72640 1 -- start start 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f968ae72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684100a80 0x7f96841a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f968ae72640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9684101420 0x7f96841a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f968ae72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9684106a80 0x7f968419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f968ae72640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9684110e80 con 0x7f9684100a80 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f968ae72640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9684110d00 con 0x7f9684106a80 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f968ae72640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9684111000 con 0x7f9684101420 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684100a80 0x7f96841a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684100a80 0x7f96841a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47158/0 (socket says 172.21.15.78:47158) 2024-09-16T00:16:03.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 -- 172.21.15.78:0/786968378 learned_addr learned my addr 172.21.15.78:0/786968378 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9683fff640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9684101420 0x7f96841a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f968940f640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9684106a80 0x7f968419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 -- 172.21.15.78:0/786968378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9684101420 msgr2=0x7f96841a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9684101420 0x7f96841a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 -- 172.21.15.78:0/786968378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9684106a80 msgr2=0x7f968419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9684106a80 0x7f968419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 -- 172.21.15.78:0/786968378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968419f2a0 con 0x7f9684100a80 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f968940f640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9684106a80 0x7f968419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9688c0e640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684100a80 0x7f96841a46d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f967800c970 tx=0x7f967800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.421+0000 7f9681ffb640 1 -- 172.21.15.78:0/786968378 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678007bb0 con 0x7f9684100a80 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.422+0000 7f9681ffb640 1 -- 172.21.15.78:0/786968378 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9678007d50 con 0x7f9684100a80 2024-09-16T00:16:03.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.422+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968419f590 con 0x7f9684100a80 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.422+0000 7f9681ffb640 1 -- 172.21.15.78:0/786968378 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967800f450 con 0x7f9684100a80 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.422+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96841ab530 con 0x7f9684100a80 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.423+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96841021c0 con 0x7f9684100a80 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.424+0000 7f9681ffb640 1 -- 172.21.15.78:0/786968378 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9678004f20 con 0x7f9684100a80 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.424+0000 7f9681ffb640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96740a32c0 0x7f96740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.425+0000 7f9683fff640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96740a32c0 0x7f96740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.425+0000 7f9681ffb640 1 -- 172.21.15.78:0/786968378 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f96780cfcc0 con 0x7f9684100a80 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.425+0000 7f9683fff640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96740a32c0 0x7f96740a5780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f968419fce0 tx=0x7f967000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:03.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.427+0000 7f9681ffb640 1 -- 172.21.15.78:0/786968378 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9678014030 con 0x7f9684100a80 2024-09-16T00:16:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.631+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96841ab7c0 con 0x7f9684100a80 2024-09-16T00:16:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.632+0000 7f9681ffb640 1 -- 172.21.15.78:0/786968378 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7f967809c700 con 0x7f9684100a80 2024-09-16T00:16:03.633 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:16:03.633 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:16:03.633 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:16:03.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96740a32c0 msgr2=0x7f96740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:03.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96740a32c0 0x7f96740a5780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f968419fce0 tx=0x7f967000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684100a80 msgr2=0x7f96841a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684100a80 0x7f96841a46d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f967800c970 tx=0x7f967800cd70 comp rx=0 tx=0).stop 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 shutdown_connections 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96740a32c0 0x7f96740a5780 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9684106a80 0x7f968419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9684101420 0x7f96841a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 --2- 172.21.15.78:0/786968378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9684100a80 0x7f96841a46d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 >> 172.21.15.78:0/786968378 conn(0x7f96840fc7a0 msgr2=0x7f968406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 shutdown_connections 2024-09-16T00:16:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:03.634+0000 7f968ae72640 1 -- 172.21.15.78:0/786968378 wait complete. 2024-09-16T00:16:03.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:03 smithi078 ceph-mon[31157]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 601 op/s 2024-09-16T00:16:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:03 smithi193 ceph-mon[35484]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 601 op/s 2024-09-16T00:16:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:03 smithi148 ceph-mon[35599]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 601 op/s 2024-09-16T00:16:04.562 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id db9eea90 --release --await' 2024-09-16T00:16:04.562 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id db9eea90 --release --await 2024-09-16T00:16:04.808 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:04.929 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:04 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/786968378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:04.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:04 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/786968378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:04 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/786968378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:05.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 -- 172.21.15.78:0/3950690087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 msgr2=0x7f92f010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 --2- 172.21.15.78:0/3950690087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 0x7f92f010ec70 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009cd0 tx=0x7f92e802a8d0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 -- 172.21.15.78:0/3950690087 shutdown_connections 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 --2- 172.21.15.78:0/3950690087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 0x7f92f010ec70 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 --2- 172.21.15.78:0/3950690087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f0103780 0x7f92f0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 --2- 172.21.15.78:0/3950690087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f0102dd0 0x7f92f01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 -- 172.21.15.78:0/3950690087 >> 172.21.15.78:0/3950690087 conn(0x7f92f00fc7d0 msgr2=0x7f92f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 -- 172.21.15.78:0/3950690087 shutdown_connections 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.338+0000 7f92f6865640 1 -- 172.21.15.78:0/3950690087 wait complete. 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f6865640 1 Processor -- start 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f6865640 1 -- start start 2024-09-16T00:16:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f6865640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f0102dd0 0x7f92f01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f6865640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f0103780 0x7f92f0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f6865640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 0x7f92f019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f6865640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01120d0 con 0x7f92f0108300 2024-09-16T00:16:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f6865640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0111f50 con 0x7f92f0103780 2024-09-16T00:16:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f6865640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0112250 con 0x7f92f0102dd0 2024-09-16T00:16:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f4e02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 0x7f92f019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f4e02640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 0x7f92f019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47166/0 (socket says 172.21.15.78:47166) 2024-09-16T00:16:05.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92f4e02640 1 -- 172.21.15.78:0/3798255001 learned_addr learned my addr 172.21.15.78:0/3798255001 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92effff640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f0102dd0 0x7f92f01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.339+0000 7f92e7fff640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f0103780 0x7f92f0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92f4e02640 1 -- 172.21.15.78:0/3798255001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f0102dd0 msgr2=0x7f92f01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92f4e02640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f0102dd0 0x7f92f01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92f4e02640 1 -- 172.21.15.78:0/3798255001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f0103780 msgr2=0x7f92f0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92f4e02640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f0103780 0x7f92f0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92f4e02640 1 -- 172.21.15.78:0/3798255001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f019d5a0 con 0x7f92f0108300 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92effff640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f0102dd0 0x7f92f01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92f4e02640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 0x7f92f019cec0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009ca0 tx=0x7f92e802f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92edffb640 1 -- 172.21.15.78:0/3798255001 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80317e0 con 0x7f92f0108300 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92edffb640 1 -- 172.21.15.78:0/3798255001 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92e8003e50 con 0x7f92f0108300 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92f6865640 1 -- 172.21.15.78:0/3798255001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f019d830 con 0x7f92f0108300 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92edffb640 1 -- 172.21.15.78:0/3798255001 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8030590 con 0x7f92f0108300 2024-09-16T00:16:05.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.340+0000 7f92f6865640 1 -- 172.21.15.78:0/3798255001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a5170 con 0x7f92f0108300 2024-09-16T00:16:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92edffb640 1 -- 172.21.15.78:0/3798255001 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f92e8004040 con 0x7f92f0108300 2024-09-16T00:16:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92f8c3e740 1 Processor -- start 2024-09-16T00:16:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92f8c3e740 1 -- start start 2024-09-16T00:16:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347060 0x5623bc347440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347980 0x5623bc351d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc354630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5623bc2ac620 con 0x5623bc352240 2024-09-16T00:16:05.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5623bc253e50 con 0x5623bc347060 2024-09-16T00:16:05.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.343+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5623bc2a85e0 con 0x5623bc347980 2024-09-16T00:16:05.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.344+0000 7f92e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc354630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.344+0000 7f92e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc354630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47168/0 (socket says 172.21.15.78:47168) 2024-09-16T00:16:05.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.344+0000 7f92e7fff640 1 -- 172.21.15.78:0/2706587911 learned_addr learned my addr 172.21.15.78:0/2706587911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:05.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.344+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347060 0x5623bc347440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.344+0000 7f92effff640 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347980 0x5623bc351d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.344+0000 7f92edffb640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f92dc0a32c0 0x7f92dc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e7fff640 1 -- 172.21.15.78:0/2706587911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347980 msgr2=0x5623bc351d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347980 0x5623bc351d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e7fff640 1 -- 172.21.15.78:0/2706587911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347060 msgr2=0x5623bc347440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347060 0x5623bc347440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92f4e02640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f92dc0a32c0 0x7f92dc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e7fff640 1 -- 172.21.15.78:0/2706587911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623bc244d40 con 0x5623bc352240 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92effff640 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347980 0x5623bc351d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92edffb640 1 -- 172.21.15.78:0/3798255001 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f92e80f0b80 con 0x7f92f0108300 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc354630 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f92d80124d0 tx=0x7f92d8009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d409aecabf41d1ab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e67fc640 1 -- 172.21.15.78:0/2706587911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8004250 con 0x5623bc352240 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92f4e02640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f92dc0a32c0 0x7f92dc0a5780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f92e80c5580 tx=0x7f92e808cab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e67fc640 1 -- 172.21.15.78:0/2706587911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d80043f0 con 0x5623bc352240 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.345+0000 7f92e67fc640 1 -- 172.21.15.78:0/2706587911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80109f0 con 0x5623bc352240 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2706587911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 msgr2=0x5623bc354630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc354630 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f92d80124d0 tx=0x7f92d8009600 comp rx=0 tx=0).stop 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2706587911 shutdown_connections 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc354630 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347980 0x5623bc351d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2706587911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347060 0x5623bc347440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2706587911 >> 172.21.15.78:0/2706587911 conn(0x5623bc3435f0 msgr2=0x5623bc3439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2706587911 shutdown_connections 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2706587911 wait complete. 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.346+0000 7f92f8c3e740 1 -- start start 2024-09-16T00:16:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92f8c3e740 1 Processor -- start 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc34f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc34f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 0x5623bc34fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5623bc354e10 con 0x5623bc347060 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5623bc234f10 con 0x5623bc347980 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5623bc354f90 con 0x5623bc352240 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92f8c3e740 10 client.0 _ll_register_callbacks cb 0x5623bc258db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc34f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc34f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47182/0 (socket says 172.21.15.78:47182) 2024-09-16T00:16:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 -- 172.21.15.78:0/2113733149 learned_addr learned my addr 172.21.15.78:0/2113733149 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92effff640 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc34f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 0x5623bc34fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 -- 172.21.15.78:0/2113733149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 msgr2=0x5623bc34fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 0x5623bc34fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 -- 172.21.15.78:0/2113733149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 msgr2=0x5623bc34f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc34f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.347+0000 7f92e7fff640 1 -- 172.21.15.78:0/2113733149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623bbfbb580 con 0x5623bc347060 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc34f210 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f92d80124a0 tx=0x7f92d80105d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c2adb0a729de58 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2113733149 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8028040 con 0x5623bc347060 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2113733149 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f92d8005050 con 0x5623bc347060 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2113733149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d80043f0 con 0x5623bc347060 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2113733149 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f92d8032050 con 0x5623bc347060 2024-09-16T00:16:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 10 client.15861 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2113733149 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5623bc085960 con 0x5623bc347060 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2113733149 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d80043f0 con 0x5623bc347060 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2113733149 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f92d801c020 con 0x5623bc347060 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 10 client.15861 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 10 client.15861 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 20 client.15861 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5623bc361d10 0x5623bc364100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 4 client.15861 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.348+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2113733149 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623bc2b9b90 con 0x5623bc361d10 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.349+0000 7f92effff640 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5623bc361d10 0x5623bc364100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.349+0000 7f92cf7fe640 10 client.15861 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.349+0000 7f92effff640 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5623bc361d10 0x5623bc364100 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd32f4bce3fad89b server_cookie=2d9647df9438d72a in_seq=0 out_seq=0 2024-09-16T00:16:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2113733149 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623bc2b9b90 con 0x5623bc361d10 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92cf7fe640 10 client.15861 handle_command_reply: tid=0 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 client.15861 shutdown 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2113733149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 msgr2=0x5623bc34f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc34f210 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f92d80124a0 tx=0x7f92d80105d0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2113733149 shutdown_connections 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5623bc361d10 0x5623bc364100 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 0x5623bc34fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc34f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92cf7fe640 0 client.15861 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2113733149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc34f210 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2113733149 >> 172.21.15.78:0/2113733149 conn(0x5623bc3435f0 msgr2=0x5623bc3492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2113733149 shutdown_connections 2024-09-16T00:16:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2113733149 wait complete. 2024-09-16T00:16:05.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.350+0000 7f92f8c3e740 20 client.15861 trim_cache size 0 max 16384 2024-09-16T00:16:05.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.366+0000 7f92f8c3e740 1 Processor -- start 2024-09-16T00:16:05.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.366+0000 7f92f8c3e740 1 -- start start 2024-09-16T00:16:05.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347060 0x5623bc34a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc342640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc3699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ac860 con 0x5623bc352240 2024-09-16T00:16:05.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ae430 con 0x5623bc347980 2024-09-16T00:16:05.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01acf30 con 0x5623bc347060 2024-09-16T00:16:05.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc3699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc3699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47186/0 (socket says 172.21.15.78:47186) 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 -- 172.21.15.78:0/1134760325 learned_addr learned my addr 172.21.15.78:0/1134760325 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347060 0x5623bc34a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92effff640 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc342640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 -- 172.21.15.78:0/1134760325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347060 msgr2=0x5623bc34a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347060 0x5623bc34a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 -- 172.21.15.78:0/1134760325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 msgr2=0x5623bc342640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc342640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 -- 172.21.15.78:0/1134760325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623bc34f8b0 con 0x5623bc352240 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e7fff640 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc3699a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f92d80049f0 tx=0x7f92d8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f30457eb784b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e5ffb640 1 -- 172.21.15.78:0/1134760325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80042f0 con 0x5623bc352240 2024-09-16T00:16:05.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e5ffb640 1 -- 172.21.15.78:0/1134760325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d8023070 con 0x5623bc352240 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.367+0000 7f92e5ffb640 1 -- 172.21.15.78:0/1134760325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d801e6a0 con 0x5623bc352240 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 -- 172.21.15.78:0/1134760325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 msgr2=0x5623bc3699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc3699a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f92d80049f0 tx=0x7f92d8015040 comp rx=0 tx=0).stop 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 -- 172.21.15.78:0/1134760325 shutdown_connections 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc352240 0x5623bc3699a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc342640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/1134760325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc347060 0x5623bc34a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 -- 172.21.15.78:0/1134760325 >> 172.21.15.78:0/1134760325 conn(0x5623bc3435f0 msgr2=0x5623bc341f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 -- 172.21.15.78:0/1134760325 shutdown_connections 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 -- 172.21.15.78:0/1134760325 wait complete. 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.368+0000 7f92f8c3e740 1 -- start start 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92f8c3e740 1 Processor -- start 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc35a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc35a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92f8c3e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 0x5623bc357180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01add30 con 0x5623bc347060 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01ad660 con 0x5623bc347980 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92f8c3e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01aa840 con 0x5623bc352240 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92f8c3e740 10 client.0 _ll_register_callbacks cb 0x7f92d8005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc35a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92e7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc35a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47190/0 (socket says 172.21.15.78:47190) 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92e7fff640 1 -- 172.21.15.78:0/2641749126 learned_addr learned my addr 172.21.15.78:0/2641749126 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92effff640 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc35a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92e7fff640 1 -- 172.21.15.78:0/2641749126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 msgr2=0x5623bc357180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 0x5623bc357180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92e7fff640 1 -- 172.21.15.78:0/2641749126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 msgr2=0x5623bc35a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc35a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.369+0000 7f92e7fff640 1 -- 172.21.15.78:0/2641749126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623bc3576c0 con 0x5623bc347060 2024-09-16T00:16:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92e7fff640 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc35a1b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f92d80124a0 tx=0x7f92d8009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6320b983e665986f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2641749126 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d803d040 con 0x5623bc347060 2024-09-16T00:16:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2641749126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d8023070 con 0x5623bc347060 2024-09-16T00:16:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2641749126 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f92d80097b0 con 0x5623bc347060 2024-09-16T00:16:05.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2641749126 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f92d80042f0 con 0x5623bc347060 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 10 client.15867 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2641749126 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5623bc3cd130 con 0x5623bc347060 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2641749126 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d8023070 con 0x5623bc347060 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2641749126 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f92d8040070 con 0x5623bc347060 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 10 client.15867 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 10 client.15867 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 20 client.15867 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5623bc3cd690 0x5623bc3cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 4 client.15867 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "db9eea90", "release": true, "await": true}] 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.370+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2641749126 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "db9eea90", "release": true, "await": true}) -- 0x5623bc350020 con 0x5623bc3cd690 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.371+0000 7f92effff640 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5623bc3cd690 0x5623bc3cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.371+0000 7f92cf7fe640 10 client.15867 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:05.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.371+0000 7f92effff640 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5623bc3cd690 0x5623bc3cfab0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5e7bf360bf38c1 server_cookie=909b639a50e53a09 in_seq=0 out_seq=0 2024-09-16T00:16:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.416+0000 7f92cf7fe640 1 -- 172.21.15.78:0/2641749126 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5623bc350020 con 0x5623bc3cd690 2024-09-16T00:16:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.416+0000 7f92cf7fe640 10 client.15867 handle_command_reply: tid=0 2024-09-16T00:16:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.416+0000 7f92f8c3e740 1 client.15867 shutdown 2024-09-16T00:16:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2641749126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 msgr2=0x5623bc35a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc35a1b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f92d80124a0 tx=0x7f92d8009d60 comp rx=0 tx=0).stop 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2641749126 shutdown_connections 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5623bc3cd690 0x5623bc3cfab0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92cf7fe640 0 client.15867 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5623bc352240 0x5623bc357180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623bc347980 0x5623bc35a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 --2- 172.21.15.78:0/2641749126 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5623bc347060 0x5623bc35a1b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2641749126 >> 172.21.15.78:0/2641749126 conn(0x5623bc3435f0 msgr2=0x5623bc342750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2641749126 shutdown_connections 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 1 -- 172.21.15.78:0/2641749126 wait complete. 2024-09-16T00:16:05.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.417+0000 7f92f8c3e740 20 client.15867 trim_cache size 0 max 16384 2024-09-16T00:16:05.420 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:16:05.420 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 31, 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 8, 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "db9eea90": { 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 8, 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 139.6, 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:16:05.421 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:05.422 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 -- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f92dc0a32c0 msgr2=0x7f92dc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f92dc0a32c0 0x7f92dc0a5780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f92e80c5580 tx=0x7f92e808cab0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 -- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 msgr2=0x7f92f019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 0x7f92f019cec0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009ca0 tx=0x7f92e802f3b0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 -- 172.21.15.78:0/3798255001 shutdown_connections 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f92dc0a32c0 0x7f92dc0a5780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f92f0108300 0x7f92f019cec0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92f0103780 0x7f92f0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 --2- 172.21.15.78:0/3798255001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92f0102dd0 0x7f92f01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 -- 172.21.15.78:0/3798255001 >> 172.21.15.78:0/3798255001 conn(0x7f92f00fc7d0 msgr2=0x7f92f00fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 -- 172.21.15.78:0/3798255001 shutdown_connections 2024-09-16T00:16:05.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:05.418+0000 7f92f6865640 1 -- 172.21.15.78:0/3798255001 wait complete. 2024-09-16T00:16:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 125.0 s, 190.0 tps, lat 1897.306 ms stddev 5417.981 2024-09-16T00:16:05.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:05 smithi193 ceph-mon[35484]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 738 op/s 2024-09-16T00:16:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:05 smithi148 ceph-mon[35599]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 738 op/s 2024-09-16T00:16:06.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:05 smithi078 ceph-mon[31157]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 738 op/s 2024-09-16T00:16:06.366 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'db9eea90', total seconds elapsed: 23.5 2024-09-16T00:16:06.366 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T00:16:06.367 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:16:06.780 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3583164973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b181011e0 msgr2=0x7f6b18105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3583164973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b181011e0 0x7f6b18105980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009860 tx=0x7f6b0c02a340 comp rx=0 tx=0).stop 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3583164973 shutdown_connections 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3583164973 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b18105ec0 0x7f6b18106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3583164973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b181011e0 0x7f6b18105980 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3583164973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b181008c0 0x7f6b18100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3583164973 >> 172.21.15.78:0/3583164973 conn(0x7f6b180fc7d0 msgr2=0x7f6b180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3583164973 shutdown_connections 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.382+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3583164973 wait complete. 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.383+0000 7f6b1d4f9640 1 Processor -- start 2024-09-16T00:16:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.383+0000 7f6b1d4f9640 1 -- start start 2024-09-16T00:16:07.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.383+0000 7f6b1d4f9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b181008c0 0x7f6b181a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:07.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.383+0000 7f6b1d4f9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b181011e0 0x7f6b181a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:07.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.383+0000 7f6b1d4f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b18105ec0 0x7f6b1819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.383+0000 7f6b1d4f9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18118610 con 0x7f6b181011e0 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.383+0000 7f6b1d4f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18118490 con 0x7f6b18105ec0 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.383+0000 7f6b1d4f9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18118790 con 0x7f6b181008c0 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.384+0000 7f6b16ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b181008c0 0x7f6b181a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.384+0000 7f6b16ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b181008c0 0x7f6b181a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:46276/0 (socket says 172.21.15.78:46276) 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.384+0000 7f6b16ffd640 1 -- 172.21.15.78:0/3366272327 learned_addr learned my addr 172.21.15.78:0/3366272327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.384+0000 7f6b16ffd640 1 -- 172.21.15.78:0/3366272327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b18105ec0 msgr2=0x7f6b1819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.384+0000 7f6b16ffd640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b18105ec0 0x7f6b1819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.384+0000 7f6b16ffd640 1 -- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b181011e0 msgr2=0x7f6b181a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.384+0000 7f6b16ffd640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b181011e0 0x7f6b181a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.384+0000 7f6b16ffd640 1 -- 172.21.15.78:0/3366272327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1819f050 con 0x7f6b181008c0 2024-09-16T00:16:07.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.385+0000 7f6b16ffd640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b181008c0 0x7f6b181a4760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800efb0 tx=0x7f6b0800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:07.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.385+0000 7f6afffff640 1 -- 172.21.15.78:0/3366272327 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0801b070 con 0x7f6b181008c0 2024-09-16T00:16:07.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.385+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1819f340 con 0x7f6b181008c0 2024-09-16T00:16:07.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.385+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b181ab630 con 0x7f6b181008c0 2024-09-16T00:16:07.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.385+0000 7f6afffff640 1 -- 172.21.15.78:0/3366272327 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0800ba50 con 0x7f6b181008c0 2024-09-16T00:16:07.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.385+0000 7f6afffff640 1 -- 172.21.15.78:0/3366272327 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08016c00 con 0x7f6b181008c0 2024-09-16T00:16:07.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.386+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b18105980 con 0x7f6b181008c0 2024-09-16T00:16:07.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.387+0000 7f6afffff640 1 -- 172.21.15.78:0/3366272327 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6b0801e070 con 0x7f6b181008c0 2024-09-16T00:16:07.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.388+0000 7f6afffff640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b000a32c0 0x7f6b000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:07.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.388+0000 7f6b167fc640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b000a32c0 0x7f6b000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:07.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.389+0000 7f6afffff640 1 -- 172.21.15.78:0/3366272327 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6b080cfcb0 con 0x7f6b181008c0 2024-09-16T00:16:07.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.391+0000 7f6b167fc640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b000a32c0 0x7f6b000a5780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009780 tx=0x7f6b0c030460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:07.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.393+0000 7f6afffff640 1 -- 172.21.15.78:0/3366272327 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b0809c610 con 0x7f6b181008c0 2024-09-16T00:16:07.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.762+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1819fe50 con 0x7f6b181008c0 2024-09-16T00:16:07.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.764+0000 7f6afffff640 1 -- 172.21.15.78:0/3366272327 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7f6b0809bf10 con 0x7f6b181008c0 2024-09-16T00:16:07.765 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:16:07.765 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:16:07.765 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b000a32c0 msgr2=0x7f6b000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b000a32c0 0x7f6b000a5780 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009780 tx=0x7f6b0c030460 comp rx=0 tx=0).stop 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b181008c0 msgr2=0x7f6b181a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b181008c0 0x7f6b181a4760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800efb0 tx=0x7f6b0800c800 comp rx=0 tx=0).stop 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 shutdown_connections 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6b000a32c0 0x7f6b000a5780 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b18105ec0 0x7f6b1819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b181011e0 0x7f6b181a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 --2- 172.21.15.78:0/3366272327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b181008c0 0x7f6b181a4760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:07.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 >> 172.21.15.78:0/3366272327 conn(0x7f6b180fc7d0 msgr2=0x7f6b18103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:07.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.770+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 shutdown_connections 2024-09-16T00:16:07.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:07.771+0000 7f6b1d4f9640 1 -- 172.21.15.78:0/3366272327 wait complete. 2024-09-16T00:16:07.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:07 smithi193 ceph-mon[35484]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 927 op/s 2024-09-16T00:16:07.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:07 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:16:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:07 smithi148 ceph-mon[35599]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 927 op/s 2024-09-16T00:16:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:07 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:16:08.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:07 smithi078 ceph-mon[31157]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 927 op/s 2024-09-16T00:16:08.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:07 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:16:08.089 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:16:08.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:08 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3366272327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:08.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:08 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3366272327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:09.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:08 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3366272327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:09.257 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-16T00:16:09.258 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-16T00:16:09.272 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:09.785 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:09 smithi078 ceph-mon[31157]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.02k op/s 2024-09-16T00:16:09.785 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:09 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:16:09.785 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:09 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:16:09.785 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:09 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:16:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.829+0000 7f29ff1a2640 1 -- 172.21.15.78:0/1414387350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f81091f0 msgr2=0x7f29f81095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.829+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/1414387350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f81091f0 0x7f29f81095d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009900 tx=0x7f29e802a4b0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.830+0000 7f29ff1a2640 1 -- 172.21.15.78:0/1414387350 shutdown_connections 2024-09-16T00:16:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.830+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/1414387350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29f8114ab0 0x7f29f8116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.830+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/1414387350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f8109b10 0x7f29f8114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.830+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/1414387350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f81091f0 0x7f29f81095d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.830+0000 7f29ff1a2640 1 -- 172.21.15.78:0/1414387350 >> 172.21.15.78:0/1414387350 conn(0x7f29f806e140 msgr2=0x7f29f806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.830+0000 7f29ff1a2640 1 -- 172.21.15.78:0/1414387350 shutdown_connections 2024-09-16T00:16:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.830+0000 7f29ff1a2640 1 -- 172.21.15.78:0/1414387350 wait complete. 2024-09-16T00:16:09.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.831+0000 7f29ff1a2640 1 Processor -- start 2024-09-16T00:16:09.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.832+0000 7f29ff1a2640 1 -- start start 2024-09-16T00:16:09.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.832+0000 7f29ff1a2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29f81091f0 0x7f29f81a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.832+0000 7f29ff1a2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f8109b10 0x7f29f819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.832+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29f81091f0 0x7f29f81a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.832+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29f81091f0 0x7f29f81a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43598/0 (socket says 172.21.15.78:43598) 2024-09-16T00:16:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.833+0000 7f29fcf3e640 1 -- 172.21.15.78:0/2182040901 learned_addr learned my addr 172.21.15.78:0/2182040901 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.833+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f8114ab0 0x7f29f819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.833+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8118610 con 0x7f29f81091f0 2024-09-16T00:16:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.833+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8118490 con 0x7f29f8114ab0 2024-09-16T00:16:09.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.833+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8118790 con 0x7f29f8109b10 2024-09-16T00:16:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.835+0000 7f29f7fff640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f8109b10 0x7f29f819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.835+0000 7f29fcf3e640 1 -- 172.21.15.78:0/2182040901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f8109b10 msgr2=0x7f29f819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.835+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f8109b10 0x7f29f819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.835+0000 7f29fcf3e640 1 -- 172.21.15.78:0/2182040901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f8114ab0 msgr2=0x7f29f819ee10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:16:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.835+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f8114ab0 0x7f29f819ee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.835+0000 7f29fcf3e640 1 -- 172.21.15.78:0/2182040901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f819f380 con 0x7f29f81091f0 2024-09-16T00:16:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.836+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29f81091f0 0x7f29f81a5840 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f29e802aa20 tx=0x7f29e8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.836+0000 7f29f5ffb640 1 -- 172.21.15.78:0/2182040901 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e803f070 con 0x7f29f81091f0 2024-09-16T00:16:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.837+0000 7f29f5ffb640 1 -- 172.21.15.78:0/2182040901 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e8031c20 con 0x7f29f81091f0 2024-09-16T00:16:09.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.837+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f819f610 con 0x7f29f81091f0 2024-09-16T00:16:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.837+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f81a3590 con 0x7f29f81091f0 2024-09-16T00:16:09.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.837+0000 7f29f5ffb640 1 -- 172.21.15.78:0/2182040901 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8035030 con 0x7f29f81091f0 2024-09-16T00:16:09.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.840+0000 7f29f5ffb640 1 -- 172.21.15.78:0/2182040901 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f29e8042070 con 0x7f29f81091f0 2024-09-16T00:16:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.841+0000 7f29f5ffb640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29e00a33e0 0x7f29e00a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.841+0000 7f29f5ffb640 1 -- 172.21.15.78:0/2182040901 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f29e80f0310 con 0x7f29f81091f0 2024-09-16T00:16:09.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.842+0000 7f29f7fff640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29e00a33e0 0x7f29e00a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.843+0000 7f29f7fff640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29e00a33e0 0x7f29e00a58a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29f81a0090 tx=0x7f29ec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.844+0000 7f2a0157b740 1 Processor -- start 2024-09-16T00:16:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.845+0000 7f2a0157b740 1 -- start start 2024-09-16T00:16:09.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.845+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e881880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.845+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e88c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.845+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e88ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.845+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e881880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.845+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e881880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:35020/0 (socket says 172.21.15.78:35020) 2024-09-16T00:16:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29f7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e88c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29f7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e88c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:46298/0 (socket says 172.21.15.78:46298) 2024-09-16T00:16:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29fcf3e640 1 -- 172.21.15.78:0/864339189 learned_addr learned my addr 172.21.15.78:0/864339189 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f2a0157b740 1 -- 172.21.15.78:0/864339189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561e5e7e72e0 con 0x561e5e88c680 2024-09-16T00:16:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f2a0157b740 1 -- 172.21.15.78:0/864339189 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561e5e78ece0 con 0x561e5e8814a0 2024-09-16T00:16:09.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f2a0157b740 1 -- 172.21.15.78:0/864339189 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561e5e7e32a0 con 0x561e5e881dc0 2024-09-16T00:16:09.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29fd73f640 1 --2- 172.21.15.78:0/864339189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e88ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29fcf3e640 1 -- 172.21.15.78:0/864339189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 msgr2=0x561e5e88c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:09.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/864339189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e88c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29fcf3e640 1 -- 172.21.15.78:0/864339189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 msgr2=0x561e5e88ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:09.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/864339189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e88ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.846+0000 7f29fcf3e640 1 -- 172.21.15.78:0/864339189 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5e773440 con 0x561e5e8814a0 2024-09-16T00:16:09.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.848+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/864339189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e881880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29e80b9040 tx=0x7f29e802a4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2753787b14c87031 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:09.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.848+0000 7f29deffd640 1 -- 172.21.15.78:0/864339189 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80d5070 con 0x561e5e8814a0 2024-09-16T00:16:09.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.849+0000 7f29deffd640 1 -- 172.21.15.78:0/864339189 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e80bc670 con 0x561e5e8814a0 2024-09-16T00:16:09.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.849+0000 7f29deffd640 1 -- 172.21.15.78:0/864339189 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8049040 con 0x561e5e8814a0 2024-09-16T00:16:09.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 -- 172.21.15.78:0/864339189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 msgr2=0x561e5e881880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:09.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 --2- 172.21.15.78:0/864339189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e881880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29e80b9040 tx=0x7f29e802a4b0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 -- 172.21.15.78:0/864339189 shutdown_connections 2024-09-16T00:16:09.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 --2- 172.21.15.78:0/864339189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e88ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 --2- 172.21.15.78:0/864339189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e88c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 --2- 172.21.15.78:0/864339189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e881880 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 -- 172.21.15.78:0/864339189 >> 172.21.15.78:0/864339189 conn(0x561e5e87da30 msgr2=0x561e5e87de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:09.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 -- 172.21.15.78:0/864339189 shutdown_connections 2024-09-16T00:16:09.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.850+0000 7f2a0157b740 1 -- 172.21.15.78:0/864339189 wait complete. 2024-09-16T00:16:09.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.851+0000 7f2a0157b740 1 -- start start 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.851+0000 7f2a0157b740 1 Processor -- start 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.852+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e88be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.852+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e889130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.852+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e88be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.852+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e88be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:35024/0 (socket says 172.21.15.78:35024) 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.852+0000 7f29fcf3e640 1 -- 172.21.15.78:0/1362650101 learned_addr learned my addr 172.21.15.78:0/1362650101 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.852+0000 7f29f7fff640 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e889130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e889670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561e5e88f010 con 0x561e5e88c680 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561e5e7a2050 con 0x561e5e8814a0 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561e5e88f190 con 0x561e5e881dc0 2024-09-16T00:16:09.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f29fd73f640 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e889670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f2a0157b740 10 client.0 _ll_register_callbacks cb 0x561e5e7e2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:09.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f29fcf3e640 1 -- 172.21.15.78:0/1362650101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 msgr2=0x561e5e889130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e889130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f29fcf3e640 1 -- 172.21.15.78:0/1362650101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 msgr2=0x561e5e889670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e889670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.853+0000 7f29fcf3e640 1 -- 172.21.15.78:0/1362650101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5e5583c0 con 0x561e5e8814a0 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.854+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e88be80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f29e80f06a0 tx=0x7f29e80edb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477146af9deb8d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.854+0000 7f29d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.854+0000 7f29d7fff640 1 -- 172.21.15.78:0/1362650101 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80d9020 con 0x561e5e8814a0 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.854+0000 7f29d7fff640 1 -- 172.21.15.78:0/1362650101 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f29e80b8ce0 con 0x561e5e8814a0 2024-09-16T00:16:09.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.854+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e80bc670 con 0x561e5e8814a0 2024-09-16T00:16:09.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f29d7fff640 1 -- 172.21.15.78:0/1362650101 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f29e8049070 con 0x561e5e8814a0 2024-09-16T00:16:09.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f2a0157b740 10 client.25346 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:09.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561e5e88f010 con 0x561e5e8814a0 2024-09-16T00:16:09.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f29d7fff640 1 -- 172.21.15.78:0/1362650101 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e80c52e0 con 0x561e5e8814a0 2024-09-16T00:16:09.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f29d7fff640 1 -- 172.21.15.78:0/1362650101 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f29e80d0b90 con 0x561e5e8814a0 2024-09-16T00:16:09.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f2a0157b740 10 client.25346 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:09.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f2a0157b740 10 client.25346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:09.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f2a0157b740 20 client.25346 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:09.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561e5e89bc30 0x561e5e89e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f2a0157b740 4 client.25346 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:09.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.855+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f29e80bc670 con 0x561e5e89bc30 2024-09-16T00:16:09.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.856+0000 7f29f7fff640 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561e5e89bc30 0x561e5e89e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.858+0000 7f29f7fff640 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561e5e89bc30 0x561e5e89e050 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499c9a966a61051c server_cookie=5ddba51106f6d0c9 in_seq=0 out_seq=0 2024-09-16T00:16:09.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.858+0000 7f29d7fff640 10 client.25346 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:09.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.858+0000 7f29d7fff640 1 -- 172.21.15.78:0/1362650101 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f29e80bc670 con 0x561e5e89bc30 2024-09-16T00:16:09.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.858+0000 7f29d7fff640 10 client.25346 handle_command_reply: tid=0 2024-09-16T00:16:09.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.859+0000 7f2a0157b740 1 client.25346 shutdown 2024-09-16T00:16:09.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.859+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 msgr2=0x561e5e88be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:09.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.859+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e88be80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f29e80f06a0 tx=0x7f29e80edb40 comp rx=0 tx=0).stop 2024-09-16T00:16:09.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.859+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 shutdown_connections 2024-09-16T00:16:09.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.859+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561e5e89bc30 0x561e5e89e050 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.860+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e889670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.860+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e889130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.860+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1362650101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e8814a0 0x561e5e88be80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.860+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 >> 172.21.15.78:0/1362650101 conn(0x561e5e87da30 msgr2=0x561e5e87cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:09.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.860+0000 7f29d7fff640 0 client.25346 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:09.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.860+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 shutdown_connections 2024-09-16T00:16:09.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.861+0000 7f2a0157b740 1 -- 172.21.15.78:0/1362650101 wait complete. 2024-09-16T00:16:09.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.861+0000 7f2a0157b740 20 client.25346 trim_cache size 0 max 16384 2024-09-16T00:16:09.877 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:09.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f2a0157b740 1 Processor -- start 2024-09-16T00:16:09.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f2a0157b740 1 -- start start 2024-09-16T00:16:09.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e8814a0 0x561e5e884730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e8a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e88c680 0x561e5e8a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f2a0157b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29e400faa0 con 0x561e5e8814a0 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f2a0157b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4011670 con 0x561e5e88c680 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f2a0157b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4010170 con 0x561e5e881dc0 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e8814a0 0x561e5e884730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e8814a0 0x561e5e884730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43630/0 (socket says 172.21.15.78:43630) 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f29fd73f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e88c680 0x561e5e8a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f29fcf3e640 1 -- 172.21.15.78:0/678523915 learned_addr learned my addr 172.21.15.78:0/678523915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.888+0000 7f29f7fff640 1 --2- 172.21.15.78:0/678523915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e8a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29fd73f640 1 -- 172.21.15.78:0/678523915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 msgr2=0x561e5e8a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29fd73f640 1 --2- 172.21.15.78:0/678523915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e8a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29fd73f640 1 -- 172.21.15.78:0/678523915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e8814a0 msgr2=0x561e5e884730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29fd73f640 1 --2- 172.21.15.78:0/678523915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e8814a0 0x561e5e884730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29fd73f640 1 -- 172.21.15.78:0/678523915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5e881bf0 con 0x561e5e88c680 2024-09-16T00:16:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29fd73f640 1 --2- 172.21.15.78:0/678523915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e88c680 0x561e5e8a3ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f29f806b430 tx=0x7f29f0013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2885f0a740ddb223 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29de7fc640 1 -- 172.21.15.78:0/678523915 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0004b20 con 0x561e5e88c680 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29de7fc640 1 -- 172.21.15.78:0/678523915 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29f0003e50 con 0x561e5e88c680 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.889+0000 7f29de7fc640 1 -- 172.21.15.78:0/678523915 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0004840 con 0x561e5e88c680 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 -- 172.21.15.78:0/678523915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e88c680 msgr2=0x561e5e8a3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 --2- 172.21.15.78:0/678523915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e88c680 0x561e5e8a3ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f29f806b430 tx=0x7f29f0013830 comp rx=0 tx=0).stop 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 -- 172.21.15.78:0/678523915 shutdown_connections 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 --2- 172.21.15.78:0/678523915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e88c680 0x561e5e8a3ea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 --2- 172.21.15.78:0/678523915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e881dc0 0x561e5e8a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 --2- 172.21.15.78:0/678523915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e8814a0 0x561e5e884730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 -- 172.21.15.78:0/678523915 >> 172.21.15.78:0/678523915 conn(0x561e5e87da30 msgr2=0x561e5e883710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 -- 172.21.15.78:0/678523915 shutdown_connections 2024-09-16T00:16:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 -- 172.21.15.78:0/678523915 wait complete. 2024-09-16T00:16:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 -- start start 2024-09-16T00:16:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.890+0000 7f2a0157b740 1 Processor -- start 2024-09-16T00:16:09.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e8814a0 0x561e5e894ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e881dc0 0x561e5e885f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f2a0157b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e886570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f2a0157b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4010f70 con 0x561e5e88c680 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f2a0157b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40108a0 con 0x561e5e881dc0 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f2a0157b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29e400da80 con 0x561e5e8814a0 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f2a0157b740 10 client.0 _ll_register_callbacks cb 0x7f29e80d3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e8814a0 0x561e5e894ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f29fcf3e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e8814a0 0x561e5e894ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:46330/0 (socket says 172.21.15.78:46330) 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f29fcf3e640 1 -- 172.21.15.78:0/1786998085 learned_addr learned my addr 172.21.15.78:0/1786998085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.891+0000 7f29f7fff640 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e881dc0 0x561e5e885f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29fd73f640 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e886570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29fcf3e640 1 -- 172.21.15.78:0/1786998085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e881dc0 msgr2=0x561e5e885f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e881dc0 0x561e5e885f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29fcf3e640 1 -- 172.21.15.78:0/1786998085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 msgr2=0x561e5e886570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e886570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29fcf3e640 1 -- 172.21.15.78:0/1786998085 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5e886d70 con 0x561e5e8814a0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29fcf3e640 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e8814a0 0x561e5e894ce0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f29e80ba920 tx=0x7f29e80c5080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa46dd9f7c211d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.892+0000 7f29d7fff640 1 -- 172.21.15.78:0/1786998085 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80d5040 con 0x561e5e8814a0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.893+0000 7f29d7fff640 1 -- 172.21.15.78:0/1786998085 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f29e80d0da0 con 0x561e5e8814a0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.893+0000 7f2a0157b740 1 -- 172.21.15.78:0/1786998085 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29f0003e50 con 0x561e5e8814a0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.893+0000 7f29d7fff640 1 -- 172.21.15.78:0/1786998085 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f29e803be60 con 0x561e5e8814a0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.893+0000 7f2a0157b740 10 client.25270 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.893+0000 7f2a0157b740 1 -- 172.21.15.78:0/1786998085 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561e5e906a20 con 0x561e5e8814a0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.893+0000 7f29d7fff640 1 -- 172.21.15.78:0/1786998085 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e80c52e0 con 0x561e5e8814a0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.893+0000 7f29d7fff640 1 -- 172.21.15.78:0/1786998085 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f29e80bb530 con 0x561e5e8814a0 2024-09-16T00:16:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.893+0000 7f2a0157b740 10 client.25270 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:09.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.894+0000 7f2a0157b740 10 client.25270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:09.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.894+0000 7f2a0157b740 20 client.25270 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:09.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.894+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561e5e906f50 0x561e5e909340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:09.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.894+0000 7f2a0157b740 4 client.25270 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:16:09.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.894+0000 7f2a0157b740 1 -- 172.21.15.78:0/1786998085 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f29f0003e50 con 0x561e5e906f50 2024-09-16T00:16:09.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.896+0000 7f29f7fff640 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561e5e906f50 0x561e5e909340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:09.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.896+0000 7f29f7fff640 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561e5e906f50 0x561e5e909340 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302107c250857a64 server_cookie=49981bab9c221429 in_seq=0 out_seq=0 2024-09-16T00:16:09.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.896+0000 7f29d7fff640 10 client.25270 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.896+0000 7f29d7fff640 1 -- 172.21.15.78:0/1786998085 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f29f0003e50 con 0x561e5e906f50 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.896+0000 7f29d7fff640 10 client.25270 handle_command_reply: tid=0 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 client.25270 shutdown 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 -- 172.21.15.78:0/1786998085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e8814a0 msgr2=0x561e5e894ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e8814a0 0x561e5e894ce0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f29e80ba920 tx=0x7f29e80c5080 comp rx=0 tx=0).stop 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 -- 172.21.15.78:0/1786998085 shutdown_connections 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561e5e906f50 0x561e5e909340 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561e5e88c680 0x561e5e886570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f29d7fff640 0 client.25270 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e5e881dc0 0x561e5e885f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 --2- 172.21.15.78:0/1786998085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561e5e8814a0 0x561e5e894ce0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 -- 172.21.15.78:0/1786998085 >> 172.21.15.78:0/1786998085 conn(0x561e5e87da30 msgr2=0x561e5e87cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 -- 172.21.15.78:0/1786998085 shutdown_connections 2024-09-16T00:16:09.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 1 -- 172.21.15.78:0/1786998085 wait complete. 2024-09-16T00:16:09.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.897+0000 7f2a0157b740 20 client.25270 trim_cache size 0 max 16384 2024-09-16T00:16:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:16:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:16:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:16:09.900 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29e00a33e0 msgr2=0x7f29e00a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29e00a33e0 0x7f29e00a58a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29f81a0090 tx=0x7f29ec00b390 comp rx=0 tx=0).stop 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29f81091f0 msgr2=0x7f29f81a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29f81091f0 0x7f29f81a5840 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f29e802aa20 tx=0x7f29e8003b90 comp rx=0 tx=0).stop 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 shutdown_connections 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f29e00a33e0 0x7f29e00a58a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f29f8114ab0 0x7f29f819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f8109b10 0x7f29f819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 --2- 172.21.15.78:0/2182040901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29f81091f0 0x7f29f81a5840 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 >> 172.21.15.78:0/2182040901 conn(0x7f29f806e140 msgr2=0x7f29f8070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 shutdown_connections 2024-09-16T00:16:09.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:09.899+0000 7f29ff1a2640 1 -- 172.21.15.78:0/2182040901 wait complete. 2024-09-16T00:16:09.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:09 smithi193 ceph-mon[35484]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.02k op/s 2024-09-16T00:16:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:09 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:16:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:09 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:16:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:09 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:16:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:09 smithi148 ceph-mon[35599]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.02k op/s 2024-09-16T00:16:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:09 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:16:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:09 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:16:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:09 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:16:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 130.0 s, 1803.6 tps, lat 55.684 ms stddev 60.801 2024-09-16T00:16:11.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.744+0000 7f6958800640 1 -- 172.21.15.78:0/1189451622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69541012a0 msgr2=0x7f6954105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:11.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.744+0000 7f6958800640 1 --2- 172.21.15.78:0/1189451622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69541012a0 0x7f6954105770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6948009900 tx=0x7f694802a4b0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.744+0000 7f6958800640 1 -- 172.21.15.78:0/1189451622 shutdown_connections 2024-09-16T00:16:11.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.744+0000 7f6958800640 1 --2- 172.21.15.78:0/1189451622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6954105d90 0x7f6954106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.744+0000 7f6958800640 1 --2- 172.21.15.78:0/1189451622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69541012a0 0x7f6954105770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.744+0000 7f6958800640 1 --2- 172.21.15.78:0/1189451622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6954100980 0x7f6954100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.744+0000 7f6958800640 1 -- 172.21.15.78:0/1189451622 >> 172.21.15.78:0/1189451622 conn(0x7f69540fc610 msgr2=0x7f69540fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:11.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.745+0000 7f6958800640 1 -- 172.21.15.78:0/1189451622 shutdown_connections 2024-09-16T00:16:11.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.747+0000 7f6958800640 1 -- 172.21.15.78:0/1189451622 wait complete. 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6958800640 1 Processor -- start 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6958800640 1 -- start start 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6958800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6954100980 0x7f6954073990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6958800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69541012a0 0x7f6954073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6958800640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6954105d90 0x7f695406cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6958800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69541184b0 con 0x7f6954100980 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6958800640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6954118330 con 0x7f6954105d90 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6958800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6954118630 con 0x7f69541012a0 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f6952d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6954100980 0x7f6954073990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.748+0000 7f695359e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6954105d90 0x7f695406cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f6952d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6954100980 0x7f6954073990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43660/0 (socket says 172.21.15.78:43660) 2024-09-16T00:16:11.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f6952d9d640 1 -- 172.21.15.78:0/774578858 learned_addr learned my addr 172.21.15.78:0/774578858 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f695259c640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69541012a0 0x7f6954073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f695359e640 1 -- 172.21.15.78:0/774578858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69541012a0 msgr2=0x7f6954073ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f695359e640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69541012a0 0x7f6954073ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f695359e640 1 -- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6954100980 msgr2=0x7f6954073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f695359e640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6954100980 0x7f6954073990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f695359e640 1 -- 172.21.15.78:0/774578858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695406d510 con 0x7f6954105d90 2024-09-16T00:16:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.749+0000 7f6952d9d640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6954100980 0x7f6954073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.750+0000 7f695359e640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6954105d90 0x7f695406cc50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f694c00be20 tx=0x7f694c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.750+0000 7f693bfff640 1 -- 172.21.15.78:0/774578858 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c019070 con 0x7f6954105d90 2024-09-16T00:16:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.750+0000 7f6958800640 1 -- 172.21.15.78:0/774578858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695406d800 con 0x7f6954105d90 2024-09-16T00:16:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.750+0000 7f693bfff640 1 -- 172.21.15.78:0/774578858 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f694c014920 con 0x7f6954105d90 2024-09-16T00:16:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.750+0000 7f6958800640 1 -- 172.21.15.78:0/774578858 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6954071750 con 0x7f6954105d90 2024-09-16T00:16:11.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.750+0000 7f693bfff640 1 -- 172.21.15.78:0/774578858 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c012940 con 0x7f6954105d90 2024-09-16T00:16:11.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.752+0000 7f693bfff640 1 -- 172.21.15.78:0/774578858 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f694c01c070 con 0x7f6954105d90 2024-09-16T00:16:11.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.753+0000 7f695b3da740 1 Processor -- start 2024-09-16T00:16:11.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.753+0000 7f695b3da740 1 -- start start 2024-09-16T00:16:11.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.753+0000 7f695b3da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed797e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.753+0000 7f695b3da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed798340 0x563eed7a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.753+0000 7f695b3da740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 0x563eed7a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695b3da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563eed6fc160 con 0x563eed798340 2024-09-16T00:16:11.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695b3da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563eed6a3720 con 0x563eed797a20 2024-09-16T00:16:11.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695b3da740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563eed6f8120 con 0x563eed7a2c00 2024-09-16T00:16:11.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f693bfff640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f693c0a32c0 0x7f693c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f6952d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed798340 0x563eed7a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f6952d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed798340 0x563eed7a26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43676/0 (socket says 172.21.15.78:43676) 2024-09-16T00:16:11.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f6952d9d640 1 -- 172.21.15.78:0/3811291206 learned_addr learned my addr 172.21.15.78:0/3811291206 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:11.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695259c640 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed797e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695259c640 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 0x563eed7a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f693bfff640 1 -- 172.21.15.78:0/774578858 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f694c0cf530 con 0x7f6954105d90 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695259c640 1 -- 172.21.15.78:0/3811291206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 msgr2=0x563eed7a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695259c640 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 0x563eed7a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695259c640 1 -- 172.21.15.78:0/3811291206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed798340 msgr2=0x563eed7a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695259c640 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed798340 0x563eed7a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f695259c640 1 -- 172.21.15.78:0/3811291206 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eed46dba0 con 0x563eed797a20 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.754+0000 7f6952d9d640 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed798340 0x563eed7a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.755+0000 7f695359e640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f693c0a32c0 0x7f693c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.755+0000 7f695259c640 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed797e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6948006010 tx=0x7f69480058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b78583525ed4e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.755+0000 7f69397fa640 1 -- 172.21.15.78:0/3811291206 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948035020 con 0x563eed797a20 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.755+0000 7f69397fa640 1 -- 172.21.15.78:0/3811291206 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69480047a0 con 0x563eed797a20 2024-09-16T00:16:11.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f69397fa640 1 -- 172.21.15.78:0/3811291206 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948035020 con 0x563eed797a20 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 -- 172.21.15.78:0/3811291206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 msgr2=0x563eed797e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed797e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6948006010 tx=0x7f69480058b0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695359e640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f693c0a32c0 0x7f693c0a5780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f694c098270 tx=0x7f694c012360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 -- 172.21.15.78:0/3811291206 shutdown_connections 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 0x563eed7a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed798340 0x563eed7a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 --2- 172.21.15.78:0/3811291206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed797e00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 -- 172.21.15.78:0/3811291206 >> 172.21.15.78:0/3811291206 conn(0x563eed793fb0 msgr2=0x563eed794390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 -- 172.21.15.78:0/3811291206 shutdown_connections 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.756+0000 7f695b3da740 1 -- 172.21.15.78:0/3811291206 wait complete. 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 1 -- start start 2024-09-16T00:16:11.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 1 Processor -- start 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed798340 0x563eed7a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 0x563eed7a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563eed7a57c0 con 0x563eed797a20 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563eed684900 con 0x563eed798340 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563eed7a5940 con 0x563eed7a2c00 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695b3da740 10 client.0 _ll_register_callbacks cb 0x563eed682780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43690/0 (socket says 172.21.15.78:43690) 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.757+0000 7f695259c640 1 -- 172.21.15.78:0/2909082129 learned_addr learned my addr 172.21.15.78:0/2909082129 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f6952d9d640 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed798340 0x563eed7a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f695259c640 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 0x563eed7a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f695259c640 1 -- 172.21.15.78:0/2909082129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 msgr2=0x563eed7a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:11.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f695259c640 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 0x563eed7a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f695259c640 1 -- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed798340 msgr2=0x563eed7a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f695259c640 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed798340 0x563eed7a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f695259c640 1 -- 172.21.15.78:0/2909082129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eed6c4c50 con 0x563eed797a20 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f6952d9d640 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed798340 0x563eed7a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f695259c640 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79fb10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6948005940 tx=0x7f6948005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3eb8d658b759c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f6936ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f6936ff5640 1 -- 172.21.15.78:0/2909082129 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948053060 con 0x563eed797a20 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f6936ff5640 1 -- 172.21.15.78:0/2909082129 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6948004950 con 0x563eed797a20 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.758+0000 7f695b3da740 1 -- 172.21.15.78:0/2909082129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69480047a0 con 0x563eed797a20 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.759+0000 7f6936ff5640 1 -- 172.21.15.78:0/2909082129 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6948042030 con 0x563eed797a20 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.760+0000 7f695b3da740 10 client.15897 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.760+0000 7f695b3da740 1 -- 172.21.15.78:0/2909082129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563eed7b14f0 con 0x563eed797a20 2024-09-16T00:16:11.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.760+0000 7f6936ff5640 1 -- 172.21.15.78:0/2909082129 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69480047a0 con 0x563eed797a20 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.760+0000 7f6936ff5640 1 -- 172.21.15.78:0/2909082129 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f694804c940 con 0x563eed797a20 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.760+0000 7f695b3da740 10 client.15897 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.760+0000 7f695b3da740 10 client.15897 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.760+0000 7f695b3da740 20 client.15897 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.760+0000 7f695b3da740 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563eed7b2840 0x563eed7b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.761+0000 7f695b3da740 4 client.15897 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.761+0000 7f695b3da740 1 -- 172.21.15.78:0/2909082129 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eed709890 con 0x563eed7b2840 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.761+0000 7f6952d9d640 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563eed7b2840 0x563eed7b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.762+0000 7f6952d9d640 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563eed7b2840 0x563eed7b4c30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f9b26ef47c69446 server_cookie=252a37abd8b1837e in_seq=0 out_seq=0 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.762+0000 7f6936ff5640 10 client.15897 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:11.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f6936ff5640 1 -- 172.21.15.78:0/2909082129 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563eed709890 con 0x563eed7b2840 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f6936ff5640 10 client.15897 handle_command_reply: tid=0 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 client.15897 shutdown 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 -- 172.21.15.78:0/2909082129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 msgr2=0x563eed79fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79fb10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6948005940 tx=0x7f6948005df0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 -- 172.21.15.78:0/2909082129 shutdown_connections 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563eed7b2840 0x563eed7b4c30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed7a2c00 0x563eed7a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed798340 0x563eed7a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 --2- 172.21.15.78:0/2909082129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79fb10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 -- 172.21.15.78:0/2909082129 >> 172.21.15.78:0/2909082129 conn(0x563eed793fb0 msgr2=0x563eed799ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f6936ff5640 0 client.15897 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.763+0000 7f695b3da740 1 -- 172.21.15.78:0/2909082129 shutdown_connections 2024-09-16T00:16:11.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.764+0000 7f695b3da740 1 -- 172.21.15.78:0/2909082129 wait complete. 2024-09-16T00:16:11.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.764+0000 7f695b3da740 20 client.15897 trim_cache size 0 max 16384 2024-09-16T00:16:11.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.786+0000 7f695b3da740 1 Processor -- start 2024-09-16T00:16:11.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.786+0000 7f695b3da740 1 -- start start 2024-09-16T00:16:11.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695b3da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695b3da740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed792f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695b3da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed7a2c00 0x563eed7b7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695b3da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69541bd890 con 0x563eed797a20 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695b3da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69541bf460 con 0x563eed7a2c00 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695b3da740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f69541bdf60 con 0x563eed798340 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43694/0 (socket says 172.21.15.78:43694) 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 -- 172.21.15.78:0/2235054564 learned_addr learned my addr 172.21.15.78:0/2235054564 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f6952d9d640 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed792f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed7a2c00 0x563eed7b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 -- 172.21.15.78:0/2235054564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 msgr2=0x563eed792f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed792f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 -- 172.21.15.78:0/2235054564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed7a2c00 msgr2=0x563eed7b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed7a2c00 0x563eed7b7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 -- 172.21.15.78:0/2235054564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eed798130 con 0x563eed797a20 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.787+0000 7f695259c640 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79a480 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f694802a4f0 tx=0x7f69480094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fccca3c3cb79f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.788+0000 7f6938ff9640 1 -- 172.21.15.78:0/2235054564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948053070 con 0x563eed797a20 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.788+0000 7f6938ff9640 1 -- 172.21.15.78:0/2235054564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6948032550 con 0x563eed797a20 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.788+0000 7f6938ff9640 1 -- 172.21.15.78:0/2235054564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948035070 con 0x563eed797a20 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 -- 172.21.15.78:0/2235054564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 msgr2=0x563eed79a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79a480 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f694802a4f0 tx=0x7f69480094e0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 -- 172.21.15.78:0/2235054564 shutdown_connections 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed7a2c00 0x563eed7b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed792f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 --2- 172.21.15.78:0/2235054564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed797a20 0x563eed79a480 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 -- 172.21.15.78:0/2235054564 >> 172.21.15.78:0/2235054564 conn(0x563eed793fb0 msgr2=0x563eed792850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 -- 172.21.15.78:0/2235054564 shutdown_connections 2024-09-16T00:16:11.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 -- 172.21.15.78:0/2235054564 wait complete. 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 -- start start 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.789+0000 7f695b3da740 1 Processor -- start 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695b3da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed7b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695b3da740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed7b8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695b3da740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed7a2c00 0x563eed7b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695b3da740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69541bed60 con 0x563eed7a2c00 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695b3da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f69541be690 con 0x563eed797a20 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695b3da740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f69541bb870 con 0x563eed798340 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f6952d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed7b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695b3da740 10 client.0 _ll_register_callbacks cb 0x7f6944009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f6952d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed7b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:46368/0 (socket says 172.21.15.78:46368) 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f6952d9d640 1 -- 172.21.15.78:0/1433017470 learned_addr learned my addr 172.21.15.78:0/1433017470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed7b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695259c640 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed7a2c00 0x563eed7b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695259c640 1 -- 172.21.15.78:0/1433017470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 msgr2=0x563eed7b8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695259c640 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed7b8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695259c640 1 -- 172.21.15.78:0/1433017470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed7a2c00 msgr2=0x563eed7b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695259c640 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed7a2c00 0x563eed7b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.790+0000 7f695259c640 1 -- 172.21.15.78:0/1433017470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eed7b59c0 con 0x563eed797a20 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f6952d9d640 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed7b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f695259c640 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed7b7bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f694804c6d0 tx=0x7f69480095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30537b41303498a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f6936ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f6936ff5640 1 -- 172.21.15.78:0/1433017470 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948053070 con 0x563eed797a20 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f6936ff5640 1 -- 172.21.15.78:0/1433017470 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f694804c980 con 0x563eed797a20 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f695b3da740 1 -- 172.21.15.78:0/1433017470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6948032550 con 0x563eed797a20 2024-09-16T00:16:11.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f6936ff5640 1 -- 172.21.15.78:0/1433017470 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6948076050 con 0x563eed797a20 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f695b3da740 10 client.25367 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f695b3da740 1 -- 172.21.15.78:0/1433017470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563eed81cd10 con 0x563eed797a20 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f6936ff5640 1 -- 172.21.15.78:0/1433017470 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f694804c460 con 0x563eed797a20 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.791+0000 7f6936ff5640 1 -- 172.21.15.78:0/1433017470 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f6948004830 con 0x563eed797a20 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f695b3da740 10 client.25367 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f695b3da740 10 client.25367 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f695b3da740 20 client.25367 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f695b3da740 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563eed81d260 0x563eed81f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f695b3da740 4 client.25367 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f695b3da740 1 -- 172.21.15.78:0/1433017470 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x7f6948032550 con 0x563eed81d260 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f6952d9d640 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563eed81d260 0x563eed81f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f6936ff5640 10 client.25367 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:11.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:11.792+0000 7f6952d9d640 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563eed81d260 0x563eed81f680 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ef915f6ea7500a server_cookie=b695957fd2c07f7a in_seq=0 out_seq=0 2024-09-16T00:16:11.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:11 smithi193 ceph-mon[35484]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 784 op/s 2024-09-16T00:16:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:11 smithi148 ceph-mon[35599]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 784 op/s 2024-09-16T00:16:12.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f6936ff5640 1 -- 172.21.15.78:0/1433017470 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f6948032550 con 0x563eed81d260 2024-09-16T00:16:12.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f6936ff5640 10 client.25367 handle_command_reply: tid=0 2024-09-16T00:16:12.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 client.25367 shutdown 2024-09-16T00:16:12.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:11 smithi078 ceph-mon[31157]: pgmap v444: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 18 MiB/s wr, 784 op/s 2024-09-16T00:16:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 -- 172.21.15.78:0/1433017470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 msgr2=0x563eed7b7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed7b7bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f694804c6d0 tx=0x7f69480095f0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 -- 172.21.15.78:0/1433017470 shutdown_connections 2024-09-16T00:16:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563eed81d260 0x563eed81f680 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563eed7a2c00 0x563eed7b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563eed798340 0x563eed7b8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f6936ff5640 0 client.25367 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 --2- 172.21.15.78:0/1433017470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563eed797a20 0x563eed7b7bf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 -- 172.21.15.78:0/1433017470 >> 172.21.15.78:0/1433017470 conn(0x563eed793fb0 msgr2=0x563eed793070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 -- 172.21.15.78:0/1433017470 shutdown_connections 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 1 -- 172.21.15.78:0/1433017470 wait complete. 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.016+0000 7f695b3da740 20 client.25367 trim_cache size 0 max 16384 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 31, 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:16:12.019 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 10, 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "362b6746": { 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 10, 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 139.6, 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:16:12.020 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:12.021 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 -- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f693c0a32c0 msgr2=0x7f693c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f693c0a32c0 0x7f693c0a5780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f694c098270 tx=0x7f694c012360 comp rx=0 tx=0).stop 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 -- 172.21.15.78:0/774578858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6954105d90 msgr2=0x7f695406cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6954105d90 0x7f695406cc50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f694c00be20 tx=0x7f694c00bf10 comp rx=0 tx=0).stop 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 -- 172.21.15.78:0/774578858 shutdown_connections 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f693c0a32c0 0x7f693c0a5780 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6954105d90 0x7f695406cc50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69541012a0 0x7f6954073ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 --2- 172.21.15.78:0/774578858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6954100980 0x7f6954073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:12.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 -- 172.21.15.78:0/774578858 >> 172.21.15.78:0/774578858 conn(0x7f69540fc610 msgr2=0x7f695410f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:12.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 -- 172.21.15.78:0/774578858 shutdown_connections 2024-09-16T00:16:12.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:12.017+0000 7f6958800640 1 -- 172.21.15.78:0/774578858 wait complete. 2024-09-16T00:16:12.367 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:16:12.367 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:16:13.244 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '362b6746', quiesce took 6.9 seconds. Will release after: 12.65 2024-09-16T00:16:13.289 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:13.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 -- 172.21.15.78:0/4265398964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc103660 msgr2=0x7fb7bc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:13.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/4265398964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc103660 0x7fb7bc10bb50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009900 tx=0x7fb7a402a480 comp rx=0 tx=0).stop 2024-09-16T00:16:13.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 -- 172.21.15.78:0/4265398964 shutdown_connections 2024-09-16T00:16:13.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/4265398964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7bc10c170 0x7fb7bc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:13.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/4265398964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc103660 0x7fb7bc10bb50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:13.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/4265398964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7bc102d40 0x7fb7bc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:13.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 -- 172.21.15.78:0/4265398964 >> 172.21.15.78:0/4265398964 conn(0x7fb7bc0fc7a0 msgr2=0x7fb7bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:13.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 -- 172.21.15.78:0/4265398964 shutdown_connections 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.794+0000 7fb7c1cca640 1 -- 172.21.15.78:0/4265398964 wait complete. 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7c1cca640 1 Processor -- start 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7c1cca640 1 -- start start 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7c1cca640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7bc102d40 0x7fb7bc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7c1cca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7bc103660 0x7fb7bc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7c1cca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc10c170 0x7fb7bc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7c1cca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc111fb0 con 0x7fb7bc10c170 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7c1cca640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc111e30 con 0x7fb7bc103660 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7c1cca640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc112130 con 0x7fb7bc102d40 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc10c170 0x7fb7bc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc10c170 0x7fb7bc101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43730/0 (socket says 172.21.15.78:43730) 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 -- 172.21.15.78:0/894034096 learned_addr learned my addr 172.21.15.78:0/894034096 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:13.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bb7fe640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7bc102d40 0x7fb7bc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7b3fff640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7bc103660 0x7fb7bc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 -- 172.21.15.78:0/894034096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7bc102d40 msgr2=0x7fb7bc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7bc102d40 0x7fb7bc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 -- 172.21.15.78:0/894034096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7bc103660 msgr2=0x7fb7bc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7bc103660 0x7fb7bc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 -- 172.21.15.78:0/894034096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc1a6960 con 0x7fb7bc10c170 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bb7fe640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7bc102d40 0x7fb7bc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7b3fff640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7bc103660 0x7fb7bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.795+0000 7fb7bbfff640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc10c170 0x7fb7bc101420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400b970 tx=0x7fb7b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.796+0000 7fb7b97fa640 1 -- 172.21.15.78:0/894034096 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4019070 con 0x7fb7bc10c170 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.796+0000 7fb7b97fa640 1 -- 172.21.15.78:0/894034096 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b400cb40 con 0x7fb7bc10c170 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.796+0000 7fb7b97fa640 1 -- 172.21.15.78:0/894034096 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4007650 con 0x7fb7bc10c170 2024-09-16T00:16:13.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.796+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc1a6c50 con 0x7fb7bc10c170 2024-09-16T00:16:13.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.796+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1a71e0 con 0x7fb7bc10c170 2024-09-16T00:16:13.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.797+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7bc103120 con 0x7fb7bc10c170 2024-09-16T00:16:13.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.798+0000 7fb7b97fa640 1 -- 172.21.15.78:0/894034096 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb7b401c070 con 0x7fb7bc10c170 2024-09-16T00:16:13.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.798+0000 7fb7b97fa640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7a80a32c0 0x7fb7a80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:13.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.798+0000 7fb7bb7fe640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7a80a32c0 0x7fb7a80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:13.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.798+0000 7fb7b97fa640 1 -- 172.21.15.78:0/894034096 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb7b40cf230 con 0x7fb7bc10c170 2024-09-16T00:16:13.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.799+0000 7fb7bb7fe640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7a80a32c0 0x7fb7a80a5780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009b40 tx=0x7fb7ac005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:13.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:13.803+0000 7fb7b97fa640 1 -- 172.21.15.78:0/894034096 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7b409bbf0 con 0x7fb7bc10c170 2024-09-16T00:16:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:13 smithi193 ceph-mon[35484]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-16T00:16:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:13 smithi148 ceph-mon[35599]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-16T00:16:14.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.003+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7bc1071e0 con 0x7fb7bc10c170 2024-09-16T00:16:14.005 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:13 smithi078 ceph-mon[31157]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 17 MiB/s wr, 866 op/s 2024-09-16T00:16:14.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.004+0000 7fb7b97fa640 1 -- 172.21.15.78:0/894034096 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7fb7b409b4f0 con 0x7fb7bc10c170 2024-09-16T00:16:14.005 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:16:14.005 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:16:14.006 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:16:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.006+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7a80a32c0 msgr2=0x7fb7a80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.006+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7a80a32c0 0x7fb7a80a5780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009b40 tx=0x7fb7ac005e50 comp rx=0 tx=0).stop 2024-09-16T00:16:14.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc10c170 msgr2=0x7fb7bc101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc10c170 0x7fb7bc101420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400b970 tx=0x7fb7b400be30 comp rx=0 tx=0).stop 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 shutdown_connections 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb7a80a32c0 0x7fb7a80a5780 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7bc10c170 0x7fb7bc101420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7bc103660 0x7fb7bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 --2- 172.21.15.78:0/894034096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7bc102d40 0x7fb7bc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 >> 172.21.15.78:0/894034096 conn(0x7fb7bc0fc7a0 msgr2=0x7fb7bc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 shutdown_connections 2024-09-16T00:16:14.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:14.007+0000 7fb7c1cca640 1 -- 172.21.15.78:0/894034096 wait complete. 2024-09-16T00:16:14.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:14 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/894034096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:14.818 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:16:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:14 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/894034096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:14 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/894034096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:15.064 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3917538238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54103660 msgr2=0x7f9e5410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3917538238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54103660 0x7f9e5410bb50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009900 tx=0x7f9e4002a480 comp rx=0 tx=0).stop 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3917538238 shutdown_connections 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3917538238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e5410c170 0x7f9e5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3917538238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54103660 0x7f9e5410bb50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3917538238 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54102d40 0x7f9e54103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3917538238 >> 172.21.15.78:0/3917538238 conn(0x7f9e540fc7a0 msgr2=0x7f9e540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3917538238 shutdown_connections 2024-09-16T00:16:15.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.573+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3917538238 wait complete. 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e5ab13640 1 Processor -- start 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e5ab13640 1 -- start start 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e5ab13640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54102d40 0x7f9e54198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e5ab13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54103660 0x7f9e541989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e5ab13640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e5410c170 0x7f9e5419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e5ab13640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54111f70 con 0x7f9e54102d40 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e5ab13640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54111df0 con 0x7f9e54103660 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e5ab13640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541120f0 con 0x7f9e5410c170 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e588af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54102d40 0x7f9e54198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e588af640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54102d40 0x7f9e54198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43760/0 (socket says 172.21.15.78:43760) 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e588af640 1 -- 172.21.15.78:0/3399406260 learned_addr learned my addr 172.21.15.78:0/3399406260 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.574+0000 7f9e53fff640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54103660 0x7f9e541989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e590b0640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e5410c170 0x7f9e5419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e588af640 1 -- 172.21.15.78:0/3399406260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e5410c170 msgr2=0x7f9e5419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e588af640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e5410c170 0x7f9e5419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e588af640 1 -- 172.21.15.78:0/3399406260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54103660 msgr2=0x7f9e541989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e588af640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54103660 0x7f9e541989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e588af640 1 -- 172.21.15.78:0/3399406260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5419d420 con 0x7f9e54102d40 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e588af640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54102d40 0x7f9e54198470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800d950 tx=0x7f9e4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e51ffb640 1 -- 172.21.15.78:0/3399406260 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48017070 con 0x7f9e54102d40 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e51ffb640 1 -- 172.21.15.78:0/3399406260 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e48009c30 con 0x7f9e54102d40 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3399406260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5419d710 con 0x7f9e54102d40 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e51ffb640 1 -- 172.21.15.78:0/3399406260 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4800f6b0 con 0x7f9e54102d40 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.575+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3399406260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e541a4ff0 con 0x7f9e54102d40 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.577+0000 7f9e51ffb640 1 -- 172.21.15.78:0/3399406260 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9e480109d0 con 0x7f9e54102d40 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e5ceec740 1 Processor -- start 2024-09-16T00:16:15.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e5ceec740 1 -- start start 2024-09-16T00:16:15.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e51ffb640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e440a32c0 0x7f9e440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced2fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fb5d0 0x555ced2fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 0x555ced308760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e590b0640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e440a32c0 0x7f9e440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555ced2612f0 con 0x555ced2fb5d0 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555ced208cf0 con 0x555ced306370 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555ced25d2b0 con 0x555ced2fcb40 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fb5d0 0x555ced2fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fb5d0 0x555ced2fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43776/0 (socket says 172.21.15.78:43776) 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e53fff640 1 -- 172.21.15.78:0/3664341145 learned_addr learned my addr 172.21.15.78:0/3664341145 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e588af640 1 --2- 172.21.15.78:0/3664341145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced2fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e51ffb640 1 -- 172.21.15.78:0/3399406260 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9e480cf970 con 0x7f9e54102d40 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e53fff640 1 -- 172.21.15.78:0/3664341145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 msgr2=0x555ced2fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:15.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e53fff640 1 --2- 172.21.15.78:0/3664341145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced2fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e53fff640 1 -- 172.21.15.78:0/3664341145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 msgr2=0x555ced308760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e53fff640 1 --2- 172.21.15.78:0/3664341145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 0x555ced308760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e53fff640 1 -- 172.21.15.78:0/3664341145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ced1ed3a0 con 0x555ced2fb5d0 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.578+0000 7f9e590b0640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e440a32c0 0x7f9e440a5780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009370 tx=0x7f9e4c00f730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.579+0000 7f9e53fff640 1 --2- 172.21.15.78:0/3664341145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fb5d0 0x555ced2fba10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9e4002a9a0 tx=0x7f9e40005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6dc1383f99cf8ea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.579+0000 7f9e3effd640 1 -- 172.21.15.78:0/3664341145 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4003f070 con 0x555ced2fb5d0 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.579+0000 7f9e3effd640 1 -- 172.21.15.78:0/3664341145 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e40004030 con 0x555ced2fb5d0 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.579+0000 7f9e3effd640 1 -- 172.21.15.78:0/3664341145 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40035070 con 0x555ced2fb5d0 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.579+0000 7f9e5ceec740 1 -- 172.21.15.78:0/3664341145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fb5d0 msgr2=0x555ced2fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.579+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/3664341145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fb5d0 0x555ced2fba10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9e4002a9a0 tx=0x7f9e40005dd0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 -- 172.21.15.78:0/3664341145 shutdown_connections 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/3664341145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 0x555ced308760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/3664341145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fb5d0 0x555ced2fba10 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/3664341145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced2fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 -- 172.21.15.78:0/3664341145 >> 172.21.15.78:0/3664341145 conn(0x555ced2c4310 msgr2=0x555ced2f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 -- 172.21.15.78:0/3664341145 shutdown_connections 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 -- 172.21.15.78:0/3664341145 wait complete. 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 -- start start 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 Processor -- start 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced305dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced303080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced306370 0x555ced303810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555ced309010 con 0x555ced2fcb40 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555ced21c060 con 0x555ced2fb5d0 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555ced309190 con 0x555ced306370 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e5ceec740 10 client.0 _ll_register_callbacks cb 0x555cecf6f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:15.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.580+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced303080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced303080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43784/0 (socket says 172.21.15.78:43784) 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e53fff640 1 -- 172.21.15.78:0/2346688091 learned_addr learned my addr 172.21.15.78:0/2346688091 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e590b0640 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced306370 0x555ced303810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e588af640 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced305dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e53fff640 1 -- 172.21.15.78:0/2346688091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced306370 msgr2=0x555ced303810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e53fff640 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced306370 0x555ced303810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e53fff640 1 -- 172.21.15.78:0/2346688091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 msgr2=0x555ced305dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e53fff640 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced305dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e53fff640 1 -- 172.21.15.78:0/2346688091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cecfd23c0 con 0x555ced2fcb40 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e588af640 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced305dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e53fff640 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced303080 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9e40006010 tx=0x7f9e40032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cb015d66645327 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:15.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e33fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e33fff640 1 -- 172.21.15.78:0/2346688091 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40035070 con 0x555ced2fcb40 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e33fff640 1 -- 172.21.15.78:0/2346688091 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9e400315b0 con 0x555ced2fcb40 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e5ceec740 1 -- 172.21.15.78:0/2346688091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e40004030 con 0x555ced2fcb40 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e33fff640 1 -- 172.21.15.78:0/2346688091 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e4003f040 con 0x555ced2fcb40 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e5ceec740 10 client.15921 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.581+0000 7f9e5ceec740 1 -- 172.21.15.78:0/2346688091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ced314b30 con 0x555ced2fcb40 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e33fff640 1 -- 172.21.15.78:0/2346688091 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e40004030 con 0x555ced2fcb40 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e33fff640 1 -- 172.21.15.78:0/2346688091 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f9e400046c0 con 0x555ced2fcb40 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e5ceec740 10 client.15921 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e5ceec740 10 client.15921 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e5ceec740 20 client.15921 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:15.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x555ced315e80 0x555ced318270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e5ceec740 4 client.15921 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e5ceec740 1 -- 172.21.15.78:0/2346688091 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ced26e860 con 0x555ced315e80 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.582+0000 7f9e588af640 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x555ced315e80 0x555ced318270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e588af640 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x555ced315e80 0x555ced318270 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98f3fbfb907af1a2 server_cookie=69a35e0aa25a4323 in_seq=0 out_seq=0 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e33fff640 10 client.15921 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e33fff640 1 -- 172.21.15.78:0/2346688091 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ced26e860 con 0x555ced315e80 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e33fff640 10 client.15921 handle_command_reply: tid=0 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 client.15921 shutdown 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 -- 172.21.15.78:0/2346688091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 msgr2=0x555ced303080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced303080 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9e40006010 tx=0x7f9e40032f60 comp rx=0 tx=0).stop 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 -- 172.21.15.78:0/2346688091 shutdown_connections 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x555ced315e80 0x555ced318270 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced306370 0x555ced303810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced303080 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/2346688091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced305dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e33fff640 0 client.15921 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.583+0000 7f9e5ceec740 1 -- 172.21.15.78:0/2346688091 >> 172.21.15.78:0/2346688091 conn(0x555ced2c4310 msgr2=0x555ced2fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.584+0000 7f9e5ceec740 1 -- 172.21.15.78:0/2346688091 shutdown_connections 2024-09-16T00:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.584+0000 7f9e5ceec740 1 -- 172.21.15.78:0/2346688091 wait complete. 2024-09-16T00:16:15.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.584+0000 7f9e5ceec740 20 client.15921 trim_cache size 0 max 16384 2024-09-16T00:16:15.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e5ceec740 1 Processor -- start 2024-09-16T00:16:15.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e5ceec740 1 -- start start 2024-09-16T00:16:15.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fb5d0 0x555ced2fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced302940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 0x555ced302080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ac9c0 con 0x555ced2fcb40 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ae590 con 0x555ced306370 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ad090 con 0x555ced2fb5d0 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced302940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced302940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43790/0 (socket says 172.21.15.78:43790) 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 -- 172.21.15.78:0/1245737687 learned_addr learned my addr 172.21.15.78:0/1245737687 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e590b0640 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 0x555ced302080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 -- 172.21.15.78:0/1245737687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fb5d0 msgr2=0x555ced2fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fb5d0 0x555ced2fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 -- 172.21.15.78:0/1245737687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 msgr2=0x555ced302080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 0x555ced302080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 -- 172.21.15.78:0/1245737687 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ced2f8cf0 con 0x555ced2fcb40 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e590b0640 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 0x555ced302080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:15.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e53fff640 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced302940 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9e40004a90 tx=0x7f9e40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db32426a148f9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.597+0000 7f9e3e7fc640 1 -- 172.21.15.78:0/1245737687 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40035030 con 0x555ced2fcb40 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e3e7fc640 1 -- 172.21.15.78:0/1245737687 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e400046c0 con 0x555ced2fcb40 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e3e7fc640 1 -- 172.21.15.78:0/1245737687 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e40031cf0 con 0x555ced2fcb40 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1245737687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 msgr2=0x555ced302940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced302940 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9e40004a90 tx=0x7f9e40000bc0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1245737687 shutdown_connections 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced306370 0x555ced302080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced2fcb40 0x555ced302940 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1245737687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fb5d0 0x555ced2fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1245737687 >> 172.21.15.78:0/1245737687 conn(0x555ced2c4310 msgr2=0x555ced301970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1245737687 shutdown_connections 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.598+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1245737687 wait complete. 2024-09-16T00:16:15.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 1 -- start start 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 1 Processor -- start 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced31ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced318de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced306370 0x555ced319360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ade90 con 0x555ced306370 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ad7c0 con 0x555ced2fb5d0 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541aa9a0 con 0x555ced2fcb40 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e5ceec740 10 client.0 _ll_register_callbacks cb 0x7f9e4800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e590b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced306370 0x555ced319360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e590b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced306370 0x555ced319360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43794/0 (socket says 172.21.15.78:43794) 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e590b0640 1 -- 172.21.15.78:0/1145169852 learned_addr learned my addr 172.21.15.78:0/1145169852 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e588af640 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced31ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e53fff640 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced318de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e590b0640 1 -- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 msgr2=0x555ced318de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e590b0640 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced318de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e590b0640 1 -- 172.21.15.78:0/1145169852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 msgr2=0x555ced31ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.599+0000 7f9e590b0640 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced31ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e590b0640 1 -- 172.21.15.78:0/1145169852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ced2fe120 con 0x555ced306370 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e53fff640 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced318de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e590b0640 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced306370 0x555ced319360 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c015000 tx=0x7f9e4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8587a52582ba41da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e33fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e33fff640 1 -- 172.21.15.78:0/1145169852 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c010b20 con 0x555ced306370 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e33fff640 1 -- 172.21.15.78:0/1145169852 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9e4c010cc0 con 0x555ced306370 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1145169852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e400046c0 con 0x555ced306370 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e33fff640 1 -- 172.21.15.78:0/1145169852 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c028470 con 0x555ced306370 2024-09-16T00:16:15.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 10 client.15927 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1145169852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555ced2f8960 con 0x555ced306370 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e33fff640 1 -- 172.21.15.78:0/1145169852 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e400046c0 con 0x555ced306370 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e33fff640 1 -- 172.21.15.78:0/1145169852 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f9e4c0114d0 con 0x555ced306370 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 10 client.15927 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 10 client.15927 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 20 client.15927 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x555ced380bb0 0x555ced382fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 4 client.15927 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.600+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1145169852 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ced268d70 con 0x555ced380bb0 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.601+0000 7f9e588af640 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x555ced380bb0 0x555ced382fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.601+0000 7f9e588af640 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x555ced380bb0 0x555ced382fd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0ab9530def3f593 server_cookie=f9c891bed692b090 in_seq=0 out_seq=0 2024-09-16T00:16:15.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.601+0000 7f9e33fff640 10 client.15927 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.601+0000 7f9e33fff640 1 -- 172.21.15.78:0/1145169852 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ced268d70 con 0x555ced380bb0 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.601+0000 7f9e33fff640 10 client.15927 handle_command_reply: tid=0 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.601+0000 7f9e5ceec740 1 client.15927 shutdown 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.601+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1145169852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced306370 msgr2=0x555ced319360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.601+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced306370 0x555ced319360 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c015000 tx=0x7f9e4c004ab0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1145169852 shutdown_connections 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555ced306370 0x555ced319360 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x555ced380bb0 0x555ced382fd0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ced2fcb40 0x555ced318de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 1 --2- 172.21.15.78:0/1145169852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555ced2fb5d0 0x555ced31ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1145169852 >> 172.21.15.78:0/1145169852 conn(0x555ced2c4310 msgr2=0x555ced2fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:15.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e33fff640 0 client.15927 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:16:15.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1145169852 shutdown_connections 2024-09-16T00:16:15.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 1 -- 172.21.15.78:0/1145169852 wait complete. 2024-09-16T00:16:15.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ceec740 20 client.15927 trim_cache size 0 max 16384 2024-09-16T00:16:15.619 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:16:15.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e440a32c0 msgr2=0x7f9e440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e440a32c0 0x7f9e440a5780 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009370 tx=0x7f9e4c00f730 comp rx=0 tx=0).stop 2024-09-16T00:16:15.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54102d40 msgr2=0x7f9e54198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:15.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54102d40 0x7f9e54198470 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800d950 tx=0x7f9e4800de10 comp rx=0 tx=0).stop 2024-09-16T00:16:15.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3399406260 shutdown_connections 2024-09-16T00:16:15.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e440a32c0 0x7f9e440a5780 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e5410c170 0x7f9e5419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54103660 0x7f9e541989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 --2- 172.21.15.78:0/3399406260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e54102d40 0x7f9e54198470 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:15.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3399406260 >> 172.21.15.78:0/3399406260 conn(0x7f9e540fc7a0 msgr2=0x7f9e540fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:15.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3399406260 shutdown_connections 2024-09-16T00:16:15.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:15.602+0000 7f9e5ab13640 1 -- 172.21.15.78:0/3399406260 wait complete. 2024-09-16T00:16:15.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:15 smithi078 ceph-mon[31157]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-16T00:16:15.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 135.0 s, 346.6 tps, lat 55.543 ms stddev 56.699 2024-09-16T00:16:15.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:15 smithi193 ceph-mon[35484]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-16T00:16:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:15 smithi148 ceph-mon[35599]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-16T00:16:16.758 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:16:17.007 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:17.879 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:17 smithi078 ceph-mon[31157]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.2 MiB/s wr, 562 op/s 2024-09-16T00:16:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:17 smithi193 ceph-mon[35484]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.2 MiB/s wr, 562 op/s 2024-09-16T00:16:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:17 smithi148 ceph-mon[35599]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.2 MiB/s wr, 562 op/s 2024-09-16T00:16:18.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 -- 172.21.15.78:0/1992259864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214106f00 msgr2=0x7f421410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 --2- 172.21.15.78:0/1992259864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214106f00 0x7f421410ea80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4204009ca0 tx=0x7f420402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 -- 172.21.15.78:0/1992259864 shutdown_connections 2024-09-16T00:16:18.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 --2- 172.21.15.78:0/1992259864 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214106f00 0x7f421410ea80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 --2- 172.21.15.78:0/1992259864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4214102780 0x7f42141067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 --2- 172.21.15.78:0/1992259864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4214101dd0 0x7f42141021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 -- 172.21.15.78:0/1992259864 >> 172.21.15.78:0/1992259864 conn(0x7f42140fd8b0 msgr2=0x7f42140ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 -- 172.21.15.78:0/1992259864 shutdown_connections 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.962+0000 7f42196c7640 1 -- 172.21.15.78:0/1992259864 wait complete. 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f42196c7640 1 Processor -- start 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f42196c7640 1 -- start start 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f42196c7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214101dd0 0x7f42140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f42196c7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4214102780 0x7f4214074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f42196c7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4214106f00 0x7f4214074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f42196c7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4214112000 con 0x7f4214101dd0 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f42196c7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4214111e80 con 0x7f4214102780 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f42196c7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4214112180 con 0x7f4214106f00 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f4212ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214101dd0 0x7f42140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f4212ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214101dd0 0x7f42140744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42650/0 (socket says 172.21.15.78:42650) 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.963+0000 7f4212ffd640 1 -- 172.21.15.78:0/3984609893 learned_addr learned my addr 172.21.15.78:0/3984609893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f42137fe640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4214106f00 0x7f4214074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f42127fc640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4214102780 0x7f4214074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f4212ffd640 1 -- 172.21.15.78:0/3984609893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4214106f00 msgr2=0x7f4214074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f4212ffd640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4214106f00 0x7f4214074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f4212ffd640 1 -- 172.21.15.78:0/3984609893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4214102780 msgr2=0x7f4214074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f4212ffd640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4214102780 0x7f4214074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f4212ffd640 1 -- 172.21.15.78:0/3984609893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42141a6a40 con 0x7f4214101dd0 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f42127fc640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4214102780 0x7f4214074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f4212ffd640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214101dd0 0x7f42140744f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f420c00b4a0 tx=0x7f420c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f41fbfff640 1 -- 172.21.15.78:0/3984609893 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c014070 con 0x7f4214101dd0 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f41fbfff640 1 -- 172.21.15.78:0/3984609893 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c010af0 con 0x7f4214101dd0 2024-09-16T00:16:18.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f42196c7640 1 -- 172.21.15.78:0/3984609893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42141a6d30 con 0x7f4214101dd0 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f41fbfff640 1 -- 172.21.15.78:0/3984609893 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c017d10 con 0x7f4214101dd0 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.964+0000 7f42196c7640 1 -- 172.21.15.78:0/3984609893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42141a7210 con 0x7f4214101dd0 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.966+0000 7f41fbfff640 1 -- 172.21.15.78:0/3984609893 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f420c017070 con 0x7f4214101dd0 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f421baa0740 1 Processor -- start 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f421baa0740 1 -- start start 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f421baa0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99fb40 0x5584fc99e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f421baa0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99e5d0 0x5584fc99ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f421baa0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc9a9370 0x5584fc9ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f421baa0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584fc9042f0 con 0x5584fc99e5d0 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f421baa0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5584fc8abcf0 con 0x5584fc99fb40 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f421baa0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5584fc9002b0 con 0x5584fc9a9370 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f4212ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99e5d0 0x5584fc99ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f4212ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99e5d0 0x5584fc99ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42652/0 (socket says 172.21.15.78:42652) 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f4212ffd640 1 -- 172.21.15.78:0/22676294 learned_addr learned my addr 172.21.15.78:0/22676294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f41fbfff640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f41fc0a3240 0x7f41fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f42137fe640 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99fb40 0x5584fc99e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f42137fe640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f41fc0a3240 0x7f41fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f42127fc640 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc9a9370 0x5584fc9ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f41fbfff640 1 -- 172.21.15.78:0/3984609893 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f420c0cefe0 con 0x7f4214101dd0 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f4212ffd640 1 -- 172.21.15.78:0/22676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc9a9370 msgr2=0x5584fc9ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f4212ffd640 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc9a9370 0x5584fc9ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f4212ffd640 1 -- 172.21.15.78:0/22676294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99fb40 msgr2=0x5584fc99e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f4212ffd640 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99fb40 0x5584fc99e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f4212ffd640 1 -- 172.21.15.78:0/22676294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584fc8903a0 con 0x5584fc99e5d0 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.967+0000 7f42127fc640 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc9a9370 0x5584fc9ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.968+0000 7f4212ffd640 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99e5d0 0x5584fc99ea10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f420c09b230 tx=0x7f420c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dfa3d69b7f8745 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:18.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.968+0000 7f42137fe640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f41fc0a3240 0x7f41fc0a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4204005fe0 tx=0x7f4204005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.968+0000 7f41f97fa640 1 -- 172.21.15.78:0/22676294 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c01e070 con 0x5584fc99e5d0 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.968+0000 7f41f97fa640 1 -- 172.21.15.78:0/22676294 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c09b2a0 con 0x5584fc99e5d0 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.968+0000 7f41f97fa640 1 -- 172.21.15.78:0/22676294 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c010ca0 con 0x5584fc99e5d0 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.968+0000 7f421baa0740 1 -- 172.21.15.78:0/22676294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99e5d0 msgr2=0x5584fc99ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.968+0000 7f421baa0740 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99e5d0 0x5584fc99ea10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f420c09b230 tx=0x7f420c0047d0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 -- 172.21.15.78:0/22676294 shutdown_connections 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc9a9370 0x5584fc9ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99e5d0 0x5584fc99ea10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 --2- 172.21.15.78:0/22676294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99fb40 0x5584fc99e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 -- 172.21.15.78:0/22676294 >> 172.21.15.78:0/22676294 conn(0x5584fc967310 msgr2=0x5584fc99ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 -- 172.21.15.78:0/22676294 shutdown_connections 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 -- 172.21.15.78:0/22676294 wait complete. 2024-09-16T00:16:18.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 -- start start 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 Processor -- start 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99e5d0 0x5584fc9a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99fb40 0x5584fc9a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc9a9370 0x5584fc9a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584fc9ac010 con 0x5584fc99fb40 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5584fc8bf060 con 0x5584fc9a9370 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5584fc9ac190 con 0x5584fc99e5d0 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f421baa0740 10 client.0 _ll_register_callbacks cb 0x5584fc6128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f42127fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99fb40 0x5584fc9a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f42127fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99fb40 0x5584fc9a5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42654/0 (socket says 172.21.15.78:42654) 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f42127fc640 1 -- 172.21.15.78:0/1312520778 learned_addr learned my addr 172.21.15.78:0/1312520778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.969+0000 7f4212ffd640 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99e5d0 0x5584fc9a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f42137fe640 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc9a9370 0x5584fc9a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f42127fc640 1 -- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99e5d0 msgr2=0x5584fc9a8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f42127fc640 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99e5d0 0x5584fc9a8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f42127fc640 1 -- 172.21.15.78:0/1312520778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc9a9370 msgr2=0x5584fc9a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f42127fc640 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc9a9370 0x5584fc9a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f42127fc640 1 -- 172.21.15.78:0/1312520778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584fc6753c0 con 0x5584fc99fb40 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f4212ffd640 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99e5d0 0x5584fc9a8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f42127fc640 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99fb40 0x5584fc9a5ff0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f420800bf70 tx=0x7f420800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a748f92ab64bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f41f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f41f6ff5640 1 -- 172.21.15.78:0/1312520778 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208018020 con 0x5584fc99fb40 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f41f6ff5640 1 -- 172.21.15.78:0/1312520778 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4208014670 con 0x5584fc99fb40 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f421baa0740 1 -- 172.21.15.78:0/1312520778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420c09b2a0 con 0x5584fc99fb40 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f41f6ff5640 1 -- 172.21.15.78:0/1312520778 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4208023050 con 0x5584fc99fb40 2024-09-16T00:16:18.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f421baa0740 10 client.15939 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f421baa0740 1 -- 172.21.15.78:0/1312520778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5584fc9b7b10 con 0x5584fc99fb40 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.970+0000 7f41f6ff5640 1 -- 172.21.15.78:0/1312520778 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c09b2a0 con 0x5584fc99fb40 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f41f6ff5640 1 -- 172.21.15.78:0/1312520778 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f4208013b50 con 0x5584fc99fb40 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f421baa0740 10 client.15939 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f421baa0740 10 client.15939 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f421baa0740 20 client.15939 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f421baa0740 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5584fc9b9070 0x5584fc9bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f421baa0740 4 client.15939 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f421baa0740 1 -- 172.21.15.78:0/1312520778 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584fc911860 con 0x5584fc9b9070 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f4212ffd640 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5584fc9b9070 0x5584fc9bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f41f6ff5640 10 client.15939 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.971+0000 7f4212ffd640 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5584fc9b9070 0x5584fc9bb4d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3926c3185b0aac81 server_cookie=dae9f45affaea458 in_seq=0 out_seq=0 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.972+0000 7f41f6ff5640 1 -- 172.21.15.78:0/1312520778 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584fc911860 con 0x5584fc9b9070 2024-09-16T00:16:18.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.972+0000 7f41f6ff5640 10 client.15939 handle_command_reply: tid=0 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.972+0000 7f421baa0740 1 client.15939 shutdown 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.972+0000 7f421baa0740 1 -- 172.21.15.78:0/1312520778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99fb40 msgr2=0x5584fc9a5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.972+0000 7f421baa0740 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99fb40 0x5584fc9a5ff0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f420800bf70 tx=0x7f420800f670 comp rx=0 tx=0).stop 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.972+0000 7f421baa0740 1 -- 172.21.15.78:0/1312520778 shutdown_connections 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.972+0000 7f421baa0740 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5584fc9b9070 0x5584fc9bb4d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.973+0000 7f421baa0740 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc9a9370 0x5584fc9a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.973+0000 7f41f6ff5640 0 client.15939 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.973+0000 7f421baa0740 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc99fb40 0x5584fc9a5ff0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.973+0000 7f421baa0740 1 --2- 172.21.15.78:0/1312520778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99e5d0 0x5584fc9a8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.973+0000 7f421baa0740 1 -- 172.21.15.78:0/1312520778 >> 172.21.15.78:0/1312520778 conn(0x5584fc967310 msgr2=0x5584fc9a0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.973+0000 7f421baa0740 1 -- 172.21.15.78:0/1312520778 shutdown_connections 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.973+0000 7f421baa0740 1 -- 172.21.15.78:0/1312520778 wait complete. 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.973+0000 7f421baa0740 20 client.15939 trim_cache size 0 max 16384 2024-09-16T00:16:18.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.985+0000 7f421baa0740 1 Processor -- start 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.985+0000 7f421baa0740 1 -- start start 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f421baa0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc99ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f421baa0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 0x5584fc9b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f421baa0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f421baa0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b0fd0 con 0x5584fc9a9370 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f421baa0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b2ba0 con 0x5584fc99e5d0 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f421baa0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b16a0 con 0x5584fc99fb40 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42668/0 (socket says 172.21.15.78:42668) 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 -- 172.21.15.78:0/2226311629 learned_addr learned my addr 172.21.15.78:0/2226311629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42127fc640 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 0x5584fc9b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f4212ffd640 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc99ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 -- 172.21.15.78:0/2226311629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 msgr2=0x5584fc9b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 0x5584fc9b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 -- 172.21.15.78:0/2226311629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 msgr2=0x5584fc99ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc99ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 -- 172.21.15.78:0/2226311629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584fc9a6950 con 0x5584fc9a9370 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f4212ffd640 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc99ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f42137fe640 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a58b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4204035000 tx=0x7f42040095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947f87b9af53cda2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f41f8ff9640 1 -- 172.21.15.78:0/2226311629 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204044040 con 0x5584fc9a9370 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f41f8ff9640 1 -- 172.21.15.78:0/2226311629 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4204036050 con 0x5584fc9a9370 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.986+0000 7f41f8ff9640 1 -- 172.21.15.78:0/2226311629 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42040047b0 con 0x5584fc9a9370 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 -- 172.21.15.78:0/2226311629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 msgr2=0x5584fc9a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a58b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4204035000 tx=0x7f42040095c0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 -- 172.21.15.78:0/2226311629 shutdown_connections 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a58b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 0x5584fc9b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 --2- 172.21.15.78:0/2226311629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc99ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 -- 172.21.15.78:0/2226311629 >> 172.21.15.78:0/2226311629 conn(0x5584fc967310 msgr2=0x5584fc99a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 -- 172.21.15.78:0/2226311629 shutdown_connections 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 -- 172.21.15.78:0/2226311629 wait complete. 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.987+0000 7f421baa0740 1 -- start start 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f421baa0740 1 Processor -- start 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f421baa0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc9b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f421baa0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 0x5584fc9a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f421baa0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f421baa0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b24a0 con 0x5584fc9a9370 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f421baa0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f42141b1dd0 con 0x5584fc99e5d0 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f421baa0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f42141aefb0 con 0x5584fc99fb40 2024-09-16T00:16:18.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f421baa0740 10 client.0 _ll_register_callbacks cb 0x7f420800b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42670/0 (socket says 172.21.15.78:42670) 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42137fe640 1 -- 172.21.15.78:0/2536266368 learned_addr learned my addr 172.21.15.78:0/2536266368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f4212ffd640 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc9b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42127fc640 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 0x5584fc9a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42137fe640 1 -- 172.21.15.78:0/2536266368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 msgr2=0x5584fc9a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42137fe640 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 0x5584fc9a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42137fe640 1 -- 172.21.15.78:0/2536266368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 msgr2=0x5584fc9b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42137fe640 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc9b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.988+0000 7f42137fe640 1 -- 172.21.15.78:0/2536266368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584fc9a3890 con 0x5584fc9a9370 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f42137fe640 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a31a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f420402ade0 tx=0x7f4204009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62e4482367211d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f41f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:18.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f41f6ff5640 1 -- 172.21.15.78:0/2536266368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204044050 con 0x5584fc9a9370 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f41f6ff5640 1 -- 172.21.15.78:0/2536266368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4204003b70 con 0x5584fc9a9370 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 1 -- 172.21.15.78:0/2536266368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4204036050 con 0x5584fc9a9370 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f41f6ff5640 1 -- 172.21.15.78:0/2536266368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f420406a050 con 0x5584fc9a9370 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 10 client.15945 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 1 -- 172.21.15.78:0/2536266368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5584fca23bc0 con 0x5584fc9a9370 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f41f6ff5640 1 -- 172.21.15.78:0/2536266368 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4204036050 con 0x5584fc9a9370 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f41f6ff5640 1 -- 172.21.15.78:0/2536266368 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f4204052070 con 0x5584fc9a9370 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 10 client.15945 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 10 client.15945 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 20 client.15945 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5584fc9b9070 0x5584fca262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 4 client.15945 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.989+0000 7f421baa0740 1 -- 172.21.15.78:0/2536266368 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584fc90bd70 con 0x5584fc9b9070 2024-09-16T00:16:18.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f4212ffd640 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5584fc9b9070 0x5584fca262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f41f6ff5640 10 client.15945 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:16:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f4212ffd640 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5584fc9b9070 0x5584fca262e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8eed47ad3c469d28 server_cookie=638009f9481efb48 in_seq=0 out_seq=0 2024-09-16T00:16:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f41f6ff5640 1 -- 172.21.15.78:0/2536266368 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584fc90bd70 con 0x5584fc9b9070 2024-09-16T00:16:18.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f41f6ff5640 10 client.15945 handle_command_reply: tid=0 2024-09-16T00:16:18.053 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 client.15945 shutdown 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 -- 172.21.15.78:0/2536266368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 msgr2=0x5584fc9a31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a31a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f420402ade0 tx=0x7f4204009930 comp rx=0 tx=0).stop 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 -- 172.21.15.78:0/2536266368 shutdown_connections 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584fc9a9370 0x5584fc9a31a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584fc99fb40 0x5584fc9a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5584fc9b9070 0x5584fca262e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 --2- 172.21.15.78:0/2536266368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584fc99e5d0 0x5584fc9b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f41f6ff5640 0 client.15945 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 -- 172.21.15.78:0/2536266368 >> 172.21.15.78:0/2536266368 conn(0x5584fc967310 msgr2=0x5584fc99a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 -- 172.21.15.78:0/2536266368 shutdown_connections 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 1 -- 172.21.15.78:0/2536266368 wait complete. 2024-09-16T00:16:18.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.990+0000 7f421baa0740 20 client.15945 trim_cache size 0 max 16384 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 -- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f41fc0a3240 msgr2=0x7f41fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f41fc0a3240 0x7f41fc0a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4204005fe0 tx=0x7f4204005ef0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 -- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214101dd0 msgr2=0x7f42140744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214101dd0 0x7f42140744f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f420c00b4a0 tx=0x7f420c00b960 comp rx=0 tx=0).stop 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 -- 172.21.15.78:0/3984609893 shutdown_connections 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f41fc0a3240 0x7f41fc0a5700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4214106f00 0x7f4214074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4214102780 0x7f4214074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 --2- 172.21.15.78:0/3984609893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4214101dd0 0x7f42140744f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 -- 172.21.15.78:0/3984609893 >> 172.21.15.78:0/3984609893 conn(0x7f42140fd8b0 msgr2=0x7f42140feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 -- 172.21.15.78:0/3984609893 shutdown_connections 2024-09-16T00:16:18.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:17.991+0000 7f42196c7640 1 -- 172.21.15.78:0/3984609893 wait complete. 2024-09-16T00:16:19.203 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:16:19.453 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:19.857 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:19 smithi078 ceph-mon[31157]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-16T00:16:19.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:19 smithi193 ceph-mon[35484]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-16T00:16:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:19 smithi148 ceph-mon[35599]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-16T00:16:19.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.936+0000 7f467d0c8640 1 -- 172.21.15.78:0/1723584398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467810c180 msgr2=0x7f467810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 --2- 172.21.15.78:0/1723584398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467810c180 0x7f467810e600 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4668009900 tx=0x7f466802a480 comp rx=0 tx=0).stop 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 -- 172.21.15.78:0/1723584398 shutdown_connections 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 --2- 172.21.15.78:0/1723584398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f467810c180 0x7f467810e600 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 --2- 172.21.15.78:0/1723584398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46781036c0 0x7f467810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 --2- 172.21.15.78:0/1723584398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4678102da0 0x7f4678103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 -- 172.21.15.78:0/1723584398 >> 172.21.15.78:0/1723584398 conn(0x7f46780fc7a0 msgr2=0x7f46780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 -- 172.21.15.78:0/1723584398 shutdown_connections 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 -- 172.21.15.78:0/1723584398 wait complete. 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.937+0000 7f467d0c8640 1 Processor -- start 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467d0c8640 1 -- start start 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467d0c8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4678102da0 0x7f4678100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467d0c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46781036c0 0x7f4678100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467d0c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f467810c180 0x7f4678101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467d0c8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111f20 con 0x7f4678102da0 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467d0c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4678111da0 con 0x7f46781036c0 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467d0c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46781120a0 con 0x7f467810c180 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4678102da0 0x7f4678100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4678102da0 0x7f4678100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42686/0 (socket says 172.21.15.78:42686) 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 -- 172.21.15.78:0/101023110 learned_addr learned my addr 172.21.15.78:0/101023110 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467659c640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46781036c0 0x7f4678100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 -- 172.21.15.78:0/101023110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f467810c180 msgr2=0x7f4678101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f467810c180 0x7f4678101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 -- 172.21.15.78:0/101023110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46781036c0 msgr2=0x7f4678100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46781036c0 0x7f4678100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 -- 172.21.15.78:0/101023110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46781a67f0 con 0x7f4678102da0 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f467659c640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46781036c0 0x7f4678100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f4676d9d640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4678102da0 0x7f4678100a00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f467000cce0 tx=0x7f467000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.938+0000 7f466ffff640 1 -- 172.21.15.78:0/101023110 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467000f650 con 0x7f4678102da0 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.939+0000 7f466ffff640 1 -- 172.21.15.78:0/101023110 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467000fcd0 con 0x7f4678102da0 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.939+0000 7f467d0c8640 1 -- 172.21.15.78:0/101023110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46781a6ae0 con 0x7f4678102da0 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.939+0000 7f466ffff640 1 -- 172.21.15.78:0/101023110 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670018c70 con 0x7f4678102da0 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.939+0000 7f467d0c8640 1 -- 172.21.15.78:0/101023110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46781a7070 con 0x7f4678102da0 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f467f4a1740 1 Processor -- start 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f466ffff640 1 -- 172.21.15.78:0/101023110 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4670004b20 con 0x7f4678102da0 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f467f4a1740 1 -- start start 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f467f4a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d82e2f0 con 0x55fd0d8c9b40 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f467f4a1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d7d5cf0 con 0x55fd0d8d3370 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.941+0000 7f467f4a1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d82a2b0 con 0x55fd0d8c85d0 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42690/0 (socket says 172.21.15.78:42690) 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 -- 172.21.15.78:0/1267101375 learned_addr learned my addr 172.21.15.78:0/1267101375 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f4676d9d640 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467659c640 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 -- 172.21.15.78:0/1267101375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 msgr2=0x55fd0d8c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 -- 172.21.15.78:0/1267101375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 msgr2=0x55fd0d8d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 -- 172.21.15.78:0/1267101375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0d7ba3a0 con 0x55fd0d8c9b40 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f467759e640 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8c8090 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4668009a20 tx=0x7f4668005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a517b53e5a9bd474 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f466d7fa640 1 -- 172.21.15.78:0/1267101375 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668040070 con 0x55fd0d8c9b40 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f466ffff640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f46640a32c0 0x7f46640a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f466d7fa640 1 -- 172.21.15.78:0/1267101375 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4668004040 con 0x55fd0d8c9b40 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f466d7fa640 1 -- 172.21.15.78:0/1267101375 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668035040 con 0x55fd0d8c9b40 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.942+0000 7f4676d9d640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f46640a32c0 0x7f46640a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f466ffff640 1 -- 172.21.15.78:0/101023110 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f46700d01a0 con 0x7f4678102da0 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 -- 172.21.15.78:0/1267101375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 msgr2=0x55fd0d8c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f4676d9d640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f46640a32c0 0x7f46640a5780 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f46700cdcb0 tx=0x7f467001a280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8c8090 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4668009a20 tx=0x7f4668005e70 comp rx=0 tx=0).stop 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 -- 172.21.15.78:0/1267101375 shutdown_connections 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1267101375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8c8090 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 -- 172.21.15.78:0/1267101375 >> 172.21.15.78:0/1267101375 conn(0x55fd0d891310 msgr2=0x55fd0d8c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 -- 172.21.15.78:0/1267101375 shutdown_connections 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.943+0000 7f467f4a1740 1 -- 172.21.15.78:0/1267101375 wait complete. 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 1 -- start start 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 1 Processor -- start 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d8d6010 con 0x55fd0d8c9b40 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d7e9060 con 0x55fd0d8d3370 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd0d8d6190 con 0x55fd0d8c85d0 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467f4a1740 10 client.0 _ll_register_callbacks cb 0x55fd0d53c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42702/0 (socket says 172.21.15.78:42702) 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467659c640 1 -- 172.21.15.78:0/331982510 learned_addr learned my addr 172.21.15.78:0/331982510 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467759e640 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467759e640 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467659c640 1 -- 172.21.15.78:0/331982510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 msgr2=0x55fd0d8d2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467659c640 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8d2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467659c640 1 -- 172.21.15.78:0/331982510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 msgr2=0x55fd0d8d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467659c640 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.944+0000 7f467659c640 1 -- 172.21.15.78:0/331982510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0d59f3c0 con 0x55fd0d8c9b40 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467659c640 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d0000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f466000f780 tx=0x7f466000fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f71b8fb68b1e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f465e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f465e7fc640 1 -- 172.21.15.78:0/331982510 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466000dcb0 con 0x55fd0d8c9b40 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 1 -- 172.21.15.78:0/331982510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4668004040 con 0x55fd0d8c9b40 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f465e7fc640 1 -- 172.21.15.78:0/331982510 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4660005020 con 0x55fd0d8c9b40 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f465e7fc640 1 -- 172.21.15.78:0/331982510 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4660023050 con 0x55fd0d8c9b40 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 10 client.15957 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 1 -- 172.21.15.78:0/331982510 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fd0d8e1b40 con 0x55fd0d8c9b40 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f465e7fc640 1 -- 172.21.15.78:0/331982510 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4668004040 con 0x55fd0d8c9b40 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f465e7fc640 1 -- 172.21.15.78:0/331982510 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f466001a430 con 0x55fd0d8c9b40 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 10 client.15957 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 10 client.15957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 20 client.15957 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55fd0d8e30a0 0x55fd0d8e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 4 client.15957 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.945+0000 7f467f4a1740 1 -- 172.21.15.78:0/331982510 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd0d83b860 con 0x55fd0d8e30a0 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.946+0000 7f467759e640 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55fd0d8e30a0 0x55fd0d8e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.946+0000 7f467759e640 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55fd0d8e30a0 0x55fd0d8e54c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8691d67d06ec4cb6 server_cookie=8a095becb0c0f0ff in_seq=0 out_seq=0 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.946+0000 7f465e7fc640 10 client.15957 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f465e7fc640 1 -- 172.21.15.78:0/331982510 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd0d83b860 con 0x55fd0d8e30a0 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f465e7fc640 10 client.15957 handle_command_reply: tid=0 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 client.15957 shutdown 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 -- 172.21.15.78:0/331982510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 msgr2=0x55fd0d8d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d0000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f466000f780 tx=0x7f466000fc40 comp rx=0 tx=0).stop 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 -- 172.21.15.78:0/331982510 shutdown_connections 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55fd0d8e30a0 0x55fd0d8e54c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d0000 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 --2- 172.21.15.78:0/331982510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8d2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f465e7fc640 0 client.15957 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:16:19.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 -- 172.21.15.78:0/331982510 >> 172.21.15.78:0/331982510 conn(0x55fd0d891310 msgr2=0x55fd0d8ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:19.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 -- 172.21.15.78:0/331982510 shutdown_connections 2024-09-16T00:16:19.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 1 -- 172.21.15.78:0/331982510 wait complete. 2024-09-16T00:16:19.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.947+0000 7f467f4a1740 20 client.15957 trim_cache size 0 max 16384 2024-09-16T00:16:19.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.960+0000 7f467f4a1740 1 Processor -- start 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.960+0000 7f467f4a1740 1 -- start start 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.960+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8df2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.960+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.960+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8d3370 0x55fd0d8eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.960+0000 7f467f4a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46781b0b80 con 0x55fd0d8d3370 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.960+0000 7f467f4a1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46781b2750 con 0x55fd0d8c85d0 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.960+0000 7f467f4a1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46781b1250 con 0x55fd0d8c9b40 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8d3370 0x55fd0d8eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8d3370 0x55fd0d8eadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42706/0 (socket says 172.21.15.78:42706) 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 -- 172.21.15.78:0/4027847438 learned_addr learned my addr 172.21.15.78:0/4027847438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467659c640 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8df2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 -- 172.21.15.78:0/4027847438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c9b40 msgr2=0x55fd0d8cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 -- 172.21.15.78:0/4027847438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8c85d0 msgr2=0x55fd0d8df2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8df2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 -- 172.21.15.78:0/4027847438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0d8cb120 con 0x55fd0d8d3370 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467659c640 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f467759e640 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8d3370 0x55fd0d8eadb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4668009900 tx=0x7f4668005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45131d26bf726e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f466cff9640 1 -- 172.21.15.78:0/4027847438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668040040 con 0x55fd0d8d3370 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f466cff9640 1 -- 172.21.15.78:0/4027847438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46680054f0 con 0x55fd0d8d3370 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.961+0000 7f466cff9640 1 -- 172.21.15.78:0/4027847438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668035040 con 0x55fd0d8d3370 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 -- 172.21.15.78:0/4027847438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8d3370 msgr2=0x55fd0d8eadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8d3370 0x55fd0d8eadb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4668009900 tx=0x7f4668005d50 comp rx=0 tx=0).stop 2024-09-16T00:16:19.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 -- 172.21.15.78:0/4027847438 shutdown_connections 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8d3370 0x55fd0d8eadb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 --2- 172.21.15.78:0/4027847438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8df2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 -- 172.21.15.78:0/4027847438 >> 172.21.15.78:0/4027847438 conn(0x55fd0d891310 msgr2=0x55fd0d8ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 -- 172.21.15.78:0/4027847438 shutdown_connections 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 -- 172.21.15.78:0/4027847438 wait complete. 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.962+0000 7f467f4a1740 1 -- start start 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467f4a1740 1 Processor -- start 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467f4a1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467f4a1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46781b2050 con 0x55fd0d8c9b40 2024-09-16T00:16:19.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467f4a1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f46781b1980 con 0x55fd0d8d3370 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467f4a1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46781aeb60 con 0x55fd0d8c85d0 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467f4a1740 10 client.0 _ll_register_callbacks cb 0x7f46600093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42722/0 (socket says 172.21.15.78:42722) 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 -- 172.21.15.78:0/1863450375 learned_addr learned my addr 172.21.15.78:0/1863450375 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467759e640 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 -- 172.21.15.78:0/1863450375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 msgr2=0x55fd0d8d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 -- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 msgr2=0x55fd0d8d7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 -- 172.21.15.78:0/1863450375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0d94cd80 con 0x55fd0d8c9b40 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467759e640 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:19.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f467659c640 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d75d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55fd0d8d1640 tx=0x7f466001a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bd3ee20de31c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f465e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f465e7fc640 1 -- 172.21.15.78:0/1863450375 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660020040 con 0x55fd0d8c9b40 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.963+0000 7f465e7fc640 1 -- 172.21.15.78:0/1863450375 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f466000dcb0 con 0x55fd0d8c9b40 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 1 -- 172.21.15.78:0/1863450375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46680054f0 con 0x55fd0d8c9b40 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f465e7fc640 1 -- 172.21.15.78:0/1863450375 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f466000fe90 con 0x55fd0d8c9b40 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 10 client.15963 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 1 -- 172.21.15.78:0/1863450375 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fd0d94e010 con 0x55fd0d8c9b40 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f465e7fc640 1 -- 172.21.15.78:0/1863450375 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46680054f0 con 0x55fd0d8c9b40 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f465e7fc640 1 -- 172.21.15.78:0/1863450375 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f4660004930 con 0x55fd0d8c9b40 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 10 client.15963 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 10 client.15963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:16:19.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 20 client.15963 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55fd0d94e820 0x55fd0d950c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 4 client.15963 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467f4a1740 1 -- 172.21.15.78:0/1863450375 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd0d835d70 con 0x55fd0d94e820 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.964+0000 7f467759e640 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55fd0d94e820 0x55fd0d950c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467759e640 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55fd0d94e820 0x55fd0d950c40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20efe1ccc5e38fb2 server_cookie=5b1321351d875dfc in_seq=0 out_seq=0 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f465e7fc640 10 client.15963 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f465e7fc640 1 -- 172.21.15.78:0/1863450375 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd0d835d70 con 0x55fd0d94e820 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f465e7fc640 10 client.15963 handle_command_reply: tid=0 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 client.15963 shutdown 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 -- 172.21.15.78:0/1863450375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 msgr2=0x55fd0d8d75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d75d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55fd0d8d1640 tx=0x7f466001a6e0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 -- 172.21.15.78:0/1863450375 shutdown_connections 2024-09-16T00:16:19.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55fd0d94e820 0x55fd0d950c40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd0d8d3370 0x55fd0d8d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f465e7fc640 0 client.15963 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:16:19.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd0d8c9b40 0x55fd0d8d75d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 --2- 172.21.15.78:0/1863450375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd0d8c85d0 0x55fd0d8d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 -- 172.21.15.78:0/1863450375 >> 172.21.15.78:0/1863450375 conn(0x55fd0d891310 msgr2=0x55fd0d8cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:19.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 -- 172.21.15.78:0/1863450375 shutdown_connections 2024-09-16T00:16:19.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 1 -- 172.21.15.78:0/1863450375 wait complete. 2024-09-16T00:16:19.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.965+0000 7f467f4a1740 20 client.15963 trim_cache size 0 max 16384 2024-09-16T00:16:19.979 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:16:19.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 -- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f46640a32c0 msgr2=0x7f46640a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f46640a32c0 0x7f46640a5780 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f46700cdcb0 tx=0x7f467001a280 comp rx=0 tx=0).stop 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 -- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4678102da0 msgr2=0x7f4678100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4678102da0 0x7f4678100a00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f467000cce0 tx=0x7f467000a620 comp rx=0 tx=0).stop 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 -- 172.21.15.78:0/101023110 shutdown_connections 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f46640a32c0 0x7f46640a5780 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f467810c180 0x7f4678101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f46781036c0 0x7f4678100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 --2- 172.21.15.78:0/101023110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4678102da0 0x7f4678100a00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 -- 172.21.15.78:0/101023110 >> 172.21.15.78:0/101023110 conn(0x7f46780fc7a0 msgr2=0x7f46780fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 -- 172.21.15.78:0/101023110 shutdown_connections 2024-09-16T00:16:19.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:19.966+0000 7f467d0c8640 1 -- 172.21.15.78:0/101023110 wait complete. 2024-09-16T00:16:20.803 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:16:20.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 140.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:16:21.050 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:21.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1462858119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 msgr2=0x7f89b410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1462858119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 0x7f89b410d950 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009cd0 tx=0x7f89ac02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1462858119 shutdown_connections 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1462858119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 0x7f89b410d950 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1462858119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89b4100a80 0x7f89b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1462858119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89b4106a80 0x7f89b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1462858119 >> 172.21.15.78:0/1462858119 conn(0x7f89b40fc7a0 msgr2=0x7f89b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1462858119 shutdown_connections 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.550+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1462858119 wait complete. 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89bc7d2640 1 Processor -- start 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89bc7d2640 1 -- start start 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89bc7d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89b4100a80 0x7f89b4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89bc7d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 0x7f89b41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89bc7d2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89b4106a80 0x7f89b419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89bc7d2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4110e80 con 0x7f89b4101420 2024-09-16T00:16:21.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89bc7d2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4110d00 con 0x7f89b4100a80 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89bc7d2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4111000 con 0x7f89b4106a80 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89b9d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 0x7f89b41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89b9d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 0x7f89b41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42744/0 (socket says 172.21.15.78:42744) 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89b9d6d640 1 -- 172.21.15.78:0/1142820708 learned_addr learned my addr 172.21.15.78:0/1142820708 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89ba56e640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89b4100a80 0x7f89b4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89b9d6d640 1 -- 172.21.15.78:0/1142820708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89b4106a80 msgr2=0x7f89b419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89b4106a80 0x7f89b419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89b9d6d640 1 -- 172.21.15.78:0/1142820708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89b4100a80 msgr2=0x7f89b4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89b4100a80 0x7f89b4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.551+0000 7f89b9d6d640 1 -- 172.21.15.78:0/1142820708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b419d320 con 0x7f89b4101420 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.552+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 0x7f89b41988b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d770 tx=0x7f89a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.552+0000 7f89ab7fe640 1 -- 172.21.15.78:0/1142820708 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4017070 con 0x7f89b4101420 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.552+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1142820708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b419d610 con 0x7f89b4101420 2024-09-16T00:16:21.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.552+0000 7f89ab7fe640 1 -- 172.21.15.78:0/1142820708 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a4005500 con 0x7f89b4101420 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.552+0000 7f89ab7fe640 1 -- 172.21.15.78:0/1142820708 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4004e00 con 0x7f89b4101420 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.552+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1142820708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b41a4ef0 con 0x7f89b4101420 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.554+0000 7f89ab7fe640 1 -- 172.21.15.78:0/1142820708 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f89a4004fa0 con 0x7f89b4101420 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.554+0000 7f89bebab740 1 Processor -- start 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.554+0000 7f89bebab740 1 -- start start 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bebab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe7827090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bebab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe7827a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bebab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 0x559fe7834760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bebab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559fe778d2f0 con 0x559fe7828b40 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bebab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fe7734cf0 con 0x559fe78275d0 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bebab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559fe77892b0 con 0x559fe7832370 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89ab7fe640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f89a00a3240 0x7f89a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bad6f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe7827090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89ba56e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe7827a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bad6f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe7827090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42750/0 (socket says 172.21.15.78:42750) 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bad6f640 1 -- 172.21.15.78:0/26305407 learned_addr learned my addr 172.21.15.78:0/26305407 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 0x559fe7834760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bad6f640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f89a00a3240 0x7f89a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89ab7fe640 1 -- 172.21.15.78:0/1142820708 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f89a40cf7a0 con 0x7f89b4101420 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89ba56e640 1 -- 172.21.15.78:0/26305407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 msgr2=0x559fe7834760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89ba56e640 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 0x559fe7834760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89ba56e640 1 -- 172.21.15.78:0/26305407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 msgr2=0x559fe7827090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89ba56e640 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe7827090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89ba56e640 1 -- 172.21.15.78:0/26305407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe77193a0 con 0x559fe78275d0 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.555+0000 7f89bad6f640 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe7827090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.556+0000 7f89ba56e640 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe7827a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89b000cb20 tx=0x7f89b000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6763baf1ef5c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.556+0000 7f89bad6f640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f89a00a3240 0x7f89a00a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f89ac031860 tx=0x7f89ac0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:21.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.556+0000 7f89a8ff9640 1 -- 172.21.15.78:0/26305407 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0014070 con 0x559fe78275d0 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.556+0000 7f89a8ff9640 1 -- 172.21.15.78:0/26305407 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b0005020 con 0x559fe78275d0 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.556+0000 7f89a8ff9640 1 -- 172.21.15.78:0/26305407 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0004c60 con 0x559fe78275d0 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.556+0000 7f89bebab740 1 -- 172.21.15.78:0/26305407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 msgr2=0x559fe7827a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.556+0000 7f89bebab740 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe7827a10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f89b000cb20 tx=0x7f89b000e350 comp rx=0 tx=0).stop 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 -- 172.21.15.78:0/26305407 shutdown_connections 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 0x559fe7834760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe7827a10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 --2- 172.21.15.78:0/26305407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe7827090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 -- 172.21.15.78:0/26305407 >> 172.21.15.78:0/26305407 conn(0x559fe77f0310 msgr2=0x559fe7823c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 -- 172.21.15.78:0/26305407 shutdown_connections 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 -- 172.21.15.78:0/26305407 wait complete. 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 -- start start 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 Processor -- start 2024-09-16T00:16:21.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 0x559fe7831d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559fe7835070 con 0x559fe7828b40 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fe7748060 con 0x559fe7832370 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559fe78351f0 con 0x559fe78275d0 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89bebab740 10 client.0 _ll_register_callbacks cb 0x559fe749b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89b9d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.557+0000 7f89b9d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42764/0 (socket says 172.21.15.78:42764) 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89b9d6d640 1 -- 172.21.15.78:0/2095199436 learned_addr learned my addr 172.21.15.78:0/2095199436 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89bad6f640 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89ba56e640 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 0x559fe7831d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89b9d6d640 1 -- 172.21.15.78:0/2095199436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 msgr2=0x559fe7831d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 0x559fe7831d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89b9d6d640 1 -- 172.21.15.78:0/2095199436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 msgr2=0x559fe782f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89b9d6d640 1 -- 172.21.15.78:0/2095199436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe74fe3c0 con 0x559fe7828b40 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782f000 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f89a4098430 tx=0x7f89a406d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3960b8d5148b2ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f899dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f899dffb640 1 -- 172.21.15.78:0/2095199436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4020040 con 0x559fe7828b40 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f899dffb640 1 -- 172.21.15.78:0/2095199436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f89a4009cc0 con 0x559fe7828b40 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89bebab740 1 -- 172.21.15.78:0/2095199436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b0005020 con 0x559fe7828b40 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f899dffb640 1 -- 172.21.15.78:0/2095199436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f89a4066900 con 0x559fe7828b40 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89bebab740 10 client.15975 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f89bebab740 1 -- 172.21.15.78:0/2095199436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559fe7840b40 con 0x559fe7828b40 2024-09-16T00:16:21.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f899dffb640 1 -- 172.21.15.78:0/2095199436 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89b0005020 con 0x559fe7828b40 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.558+0000 7f899dffb640 1 -- 172.21.15.78:0/2095199436 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f89a40980d0 con 0x559fe7828b40 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f89bebab740 10 client.15975 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f89bebab740 10 client.15975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f89bebab740 20 client.15975 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f89bebab740 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559fe7841f50 0x559fe7844360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f89bebab740 4 client.15975 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f89bebab740 1 -- 172.21.15.78:0/2095199436 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fe779a860 con 0x559fe7841f50 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f89ba56e640 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559fe7841f50 0x559fe7844360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f89ba56e640 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559fe7841f50 0x559fe7844360 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ffded592fdbcad7 server_cookie=d1a9b01bf02ea3a5 in_seq=0 out_seq=0 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.559+0000 7f899dffb640 10 client.15975 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f899dffb640 1 -- 172.21.15.78:0/2095199436 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fe779a860 con 0x559fe7841f50 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f899dffb640 10 client.15975 handle_command_reply: tid=0 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 client.15975 shutdown 2024-09-16T00:16:21.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 -- 172.21.15.78:0/2095199436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 msgr2=0x559fe782f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782f000 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f89a4098430 tx=0x7f89a406d830 comp rx=0 tx=0).stop 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 -- 172.21.15.78:0/2095199436 shutdown_connections 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559fe7841f50 0x559fe7844360 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782f000 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 --2- 172.21.15.78:0/2095199436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 0x559fe7831d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 -- 172.21.15.78:0/2095199436 >> 172.21.15.78:0/2095199436 conn(0x559fe77f0310 msgr2=0x559fe7829960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f899dffb640 0 client.15975 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 -- 172.21.15.78:0/2095199436 shutdown_connections 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 1 -- 172.21.15.78:0/2095199436 wait complete. 2024-09-16T00:16:21.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.560+0000 7f89bebab740 20 client.15975 trim_cache size 0 max 16384 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.573+0000 7f89bebab740 1 Processor -- start 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bebab740 1 -- start start 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bebab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 0x559fe7823e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bebab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe783e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bebab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bebab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41aeb40 con 0x559fe7828b40 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bebab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b0710 con 0x559fe7832370 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bebab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41af210 con 0x559fe78275d0 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe783e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe783e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42770/0 (socket says 172.21.15.78:42770) 2024-09-16T00:16:21.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 -- 172.21.15.78:0/2841458858 learned_addr learned my addr 172.21.15.78:0/2841458858 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bad6f640 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 -- 172.21.15.78:0/2841458858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 msgr2=0x559fe7823e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89ba56e640 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 0x559fe7823e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 0x559fe7823e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 -- 172.21.15.78:0/2841458858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 msgr2=0x559fe782e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 -- 172.21.15.78:0/2841458858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe782dff0 con 0x559fe7828b40 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89bad6f640 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe783e310 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f89a4012040 tx=0x7f89a4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8ade347bdbe536 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.574+0000 7f899ffff640 1 -- 172.21.15.78:0/2841458858 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4020070 con 0x559fe7828b40 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f899ffff640 1 -- 172.21.15.78:0/2841458858 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a409ba60 con 0x559fe7828b40 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f899ffff640 1 -- 172.21.15.78:0/2841458858 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4066b90 con 0x559fe7828b40 2024-09-16T00:16:21.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 -- 172.21.15.78:0/2841458858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 msgr2=0x559fe783e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe783e310 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f89a4012040 tx=0x7f89a4009c30 comp rx=0 tx=0).stop 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 -- 172.21.15.78:0/2841458858 shutdown_connections 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe7832370 0x559fe782e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe783e310 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 --2- 172.21.15.78:0/2841458858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe78275d0 0x559fe7823e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 -- 172.21.15.78:0/2841458858 >> 172.21.15.78:0/2841458858 conn(0x559fe77f0310 msgr2=0x559fe7823700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 -- 172.21.15.78:0/2841458858 shutdown_connections 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.575+0000 7f89bebab740 1 -- 172.21.15.78:0/2841458858 wait complete. 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 1 -- start start 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 1 Processor -- start 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe782bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 0x559fe782ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b0010 con 0x559fe7828b40 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41af940 con 0x559fe78275d0 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41acb20 con 0x559fe7832370 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bebab740 10 client.0 _ll_register_callbacks cb 0x7f89a400d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89b9d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89b9d6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42784/0 (socket says 172.21.15.78:42784) 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89b9d6d640 1 -- 172.21.15.78:0/856192107 learned_addr learned my addr 172.21.15.78:0/856192107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89ba56e640 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe782bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89bad6f640 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 0x559fe782ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89b9d6d640 1 -- 172.21.15.78:0/856192107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 msgr2=0x559fe782ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 0x559fe782ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89b9d6d640 1 -- 172.21.15.78:0/856192107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 msgr2=0x559fe782bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:21.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe782bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.576+0000 7f89b9d6d640 1 -- 172.21.15.78:0/856192107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe78abc00 con 0x559fe7828b40 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89b9d6d640 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782c530 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f89a40860d0 tx=0x7f89a409fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c610da0de338c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f899dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f899dffb640 1 -- 172.21.15.78:0/856192107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4020070 con 0x559fe7828b40 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f899dffb640 1 -- 172.21.15.78:0/856192107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f89a4066b90 con 0x559fe7828b40 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 1 -- 172.21.15.78:0/856192107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a409ba60 con 0x559fe7828b40 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f899dffb640 1 -- 172.21.15.78:0/856192107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f89a4066dd0 con 0x559fe7828b40 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 10 client.15981 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:21.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 1 -- 172.21.15.78:0/856192107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559fe78abc00 con 0x559fe7828b40 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f899dffb640 1 -- 172.21.15.78:0/856192107 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a409ba60 con 0x559fe7828b40 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f899dffb640 1 -- 172.21.15.78:0/856192107 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f89a40a0fa0 con 0x559fe7828b40 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 10 client.15981 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 10 client.15981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 20 client.15981 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559fe78ad120 0x559fe78af540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 4 client.15981 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89bebab740 1 -- 172.21.15.78:0/856192107 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fe7794d70 con 0x559fe78ad120 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.577+0000 7f89ba56e640 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559fe78ad120 0x559fe78af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f899dffb640 10 client.15981 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89ba56e640 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559fe78ad120 0x559fe78af540 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e83838bfe56019a server_cookie=da7e7f81e3d3d230 in_seq=0 out_seq=0 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f899dffb640 1 -- 172.21.15.78:0/856192107 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fe7794d70 con 0x559fe78ad120 2024-09-16T00:16:21.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f899dffb640 10 client.15981 handle_command_reply: tid=0 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 client.15981 shutdown 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 -- 172.21.15.78:0/856192107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 msgr2=0x559fe782c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782c530 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f89a40860d0 tx=0x7f89a409fe90 comp rx=0 tx=0).stop 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 -- 172.21.15.78:0/856192107 shutdown_connections 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x559fe78ad120 0x559fe78af540 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559fe7832370 0x559fe782ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559fe7828b40 0x559fe782c530 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 --2- 172.21.15.78:0/856192107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fe78275d0 0x559fe782bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f899dffb640 0 client.15981 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 -- 172.21.15.78:0/856192107 >> 172.21.15.78:0/856192107 conn(0x559fe77f0310 msgr2=0x559fe7831080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 -- 172.21.15.78:0/856192107 shutdown_connections 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.578+0000 7f89bebab740 1 -- 172.21.15.78:0/856192107 wait complete. 2024-09-16T00:16:21.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bebab740 20 client.15981 trim_cache size 0 max 16384 2024-09-16T00:16:21.591 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:16:21.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f89a00a3240 msgr2=0x7f89a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f89a00a3240 0x7f89a00a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f89ac031860 tx=0x7f89ac0095c0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 msgr2=0x7f89b41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 0x7f89b41988b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f89a400d770 tx=0x7f89a400dc30 comp rx=0 tx=0).stop 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1142820708 shutdown_connections 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f89a00a3240 0x7f89a00a5700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89b4106a80 0x7f89b419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89b4101420 0x7f89b41988b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 --2- 172.21.15.78:0/1142820708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f89b4100a80 0x7f89b4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1142820708 >> 172.21.15.78:0/1142820708 conn(0x7f89b40fc7a0 msgr2=0x7f89b4104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1142820708 shutdown_connections 2024-09-16T00:16:21.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:21.579+0000 7f89bc7d2640 1 -- 172.21.15.78:0/1142820708 wait complete. 2024-09-16T00:16:21.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:21 smithi078 ceph-mon[31157]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.4 MiB/s wr, 278 op/s 2024-09-16T00:16:21.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:21 smithi193 ceph-mon[35484]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.4 MiB/s wr, 278 op/s 2024-09-16T00:16:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:21 smithi148 ceph-mon[35599]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.4 MiB/s wr, 278 op/s 2024-09-16T00:16:22.429 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:16:22.677 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:23.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 -- 172.21.15.78:0/3925859178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774100e20 msgr2=0x7f2774101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 --2- 172.21.15.78:0/3925859178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774100e20 0x7f2774101200 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2770009900 tx=0x7f277002a480 comp rx=0 tx=0).stop 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 -- 172.21.15.78:0/3925859178 shutdown_connections 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 --2- 172.21.15.78:0/3925859178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2774102ef0 0x7f277410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 --2- 172.21.15.78:0/3925859178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2774102570 0x7f27741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 --2- 172.21.15.78:0/3925859178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774100e20 0x7f2774101200 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 -- 172.21.15.78:0/3925859178 >> 172.21.15.78:0/3925859178 conn(0x7f27740fc7d0 msgr2=0x7f27740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 -- 172.21.15.78:0/3925859178 shutdown_connections 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 -- 172.21.15.78:0/3925859178 wait complete. 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.194+0000 7f277b814640 1 Processor -- start 2024-09-16T00:16:23.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f277b814640 1 -- start start 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f277b814640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2774100e20 0x7f2774198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f277b814640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774102570 0x7f2774198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f277b814640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2774102ef0 0x7f277419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f277b814640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2774110f50 con 0x7f2774102570 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f277b814640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2774110dd0 con 0x7f2774100e20 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f277b814640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27741110d0 con 0x7f2774102ef0 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774102570 0x7f2774198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774102570 0x7f2774198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42814/0 (socket says 172.21.15.78:42814) 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 -- 172.21.15.78:0/1757076136 learned_addr learned my addr 172.21.15.78:0/1757076136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f27795b0640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2774100e20 0x7f2774198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2779db1640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2774102ef0 0x7f277419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 -- 172.21.15.78:0/1757076136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2774102ef0 msgr2=0x7f277419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2774102ef0 0x7f277419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 -- 172.21.15.78:0/1757076136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2774100e20 msgr2=0x7f2774198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2774100e20 0x7f2774198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 -- 172.21.15.78:0/1757076136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277419d4d0 con 0x7f2774102570 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f27795b0640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2774100e20 0x7f2774198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.195+0000 7f2778daf640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774102570 0x7f2774198a60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f276400d950 tx=0x7f276400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.196+0000 7f276a7fc640 1 -- 172.21.15.78:0/1757076136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764017070 con 0x7f2774102570 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.196+0000 7f276a7fc640 1 -- 172.21.15.78:0/1757076136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2764009c30 con 0x7f2774102570 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.196+0000 7f277b814640 1 -- 172.21.15.78:0/1757076136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277419d7c0 con 0x7f2774102570 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.196+0000 7f276a7fc640 1 -- 172.21.15.78:0/1757076136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276400f6b0 con 0x7f2774102570 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.196+0000 7f277b814640 1 -- 172.21.15.78:0/1757076136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27741a50a0 con 0x7f2774102570 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.198+0000 7f276a7fc640 1 -- 172.21.15.78:0/1757076136 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f27640109d0 con 0x7f2774102570 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.198+0000 7f277dbed740 1 Processor -- start 2024-09-16T00:16:23.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.198+0000 7f277dbed740 1 -- start start 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f276a7fc640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27600a32c0 0x7f27600a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f277dbed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 0x55f535781090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f277dbed740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f5357815d0 0x55f535781a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f277dbed740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53578e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f277dbed740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f5356e72f0 con 0x55f53578c370 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f277dbed740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f53568ecf0 con 0x55f5357815d0 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f277dbed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f5356e32b0 con 0x55f535782b40 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f276a7fc640 1 -- 172.21.15.78:0/1757076136 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f27640cf970 con 0x7f2774102570 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27600a32c0 0x7f27600a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53578e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f27795b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 0x55f535781090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53578e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42816/0 (socket says 172.21.15.78:42816) 2024-09-16T00:16:23.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 -- 172.21.15.78:0/2353536082 learned_addr learned my addr 172.21.15.78:0/2353536082 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2778daf640 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f5357815d0 0x55f535781a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 -- 172.21.15.78:0/2353536082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 msgr2=0x55f535781090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 0x55f535781090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 -- 172.21.15.78:0/2353536082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f5357815d0 msgr2=0x55f535781a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f5357815d0 0x55f535781a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 -- 172.21.15.78:0/2353536082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5356733a0 con 0x55f53578c370 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f27795b0640 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 0x55f535781090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2778daf640 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f5357815d0 0x55f535781a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27600a32c0 0x7f27600a5780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f276c009fd0 tx=0x7f276c009d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.199+0000 7f2779db1640 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53578e760 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f276c009280 tx=0x7f276c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28022e1b2e39d124 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f275f7fe640 1 -- 172.21.15.78:0/2353536082 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c004b20 con 0x55f53578c370 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f275f7fe640 1 -- 172.21.15.78:0/2353536082 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276c004cc0 con 0x55f53578c370 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f275f7fe640 1 -- 172.21.15.78:0/2353536082 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c005480 con 0x55f53578c370 2024-09-16T00:16:23.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f277dbed740 1 -- 172.21.15.78:0/2353536082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 msgr2=0x55f53578e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f277dbed740 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53578e760 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f276c009280 tx=0x7f276c01c000 comp rx=0 tx=0).stop 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f277dbed740 1 -- 172.21.15.78:0/2353536082 shutdown_connections 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f277dbed740 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53578e760 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f277dbed740 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f5357815d0 0x55f535781a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f277dbed740 1 --2- 172.21.15.78:0/2353536082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 0x55f535781090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.200+0000 7f277dbed740 1 -- 172.21.15.78:0/2353536082 >> 172.21.15.78:0/2353536082 conn(0x55f53574a310 msgr2=0x55f53577dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 -- 172.21.15.78:0/2353536082 shutdown_connections 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 -- 172.21.15.78:0/2353536082 wait complete. 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 -- start start 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 Processor -- start 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5357815d0 0x55f53578bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 0x55f535789010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f53578c370 0x55f535789600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f535793420 con 0x55f5357815d0 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f5356a2060 con 0x55f53578c370 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f5357935a0 con 0x55f535782b40 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f277dbed740 10 client.0 _ll_register_callbacks cb 0x55f5353f58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f27795b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5357815d0 0x55f53578bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f27795b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5357815d0 0x55f53578bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42826/0 (socket says 172.21.15.78:42826) 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.201+0000 7f27795b0640 1 -- 172.21.15.78:0/2631929338 learned_addr learned my addr 172.21.15.78:0/2631929338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f2779db1640 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f53578c370 0x55f535789600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f27795b0640 1 -- 172.21.15.78:0/2631929338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 msgr2=0x55f535789010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f27795b0640 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 0x55f535789010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f27795b0640 1 -- 172.21.15.78:0/2631929338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f53578c370 msgr2=0x55f535789600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f27795b0640 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f53578c370 0x55f535789600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f27795b0640 1 -- 172.21.15.78:0/2631929338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5354583c0 con 0x55f5357815d0 2024-09-16T00:16:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f27795b0640 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5357815d0 0x55f53578bd60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2770009900 tx=0x7f2770005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4b02953c040bff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f275cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f275cff9640 1 -- 172.21.15.78:0/2631929338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770035040 con 0x55f5357815d0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f275cff9640 1 -- 172.21.15.78:0/2631929338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2770004ad0 con 0x55f5357815d0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f277dbed740 1 -- 172.21.15.78:0/2631929338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f276c004cc0 con 0x55f5357815d0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f275cff9640 1 -- 172.21.15.78:0/2631929338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f277004e050 con 0x55f5357815d0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f277dbed740 10 client.15993 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f277dbed740 1 -- 172.21.15.78:0/2631929338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f53579eed0 con 0x55f5357815d0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.202+0000 7f275cff9640 1 -- 172.21.15.78:0/2631929338 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276c004cc0 con 0x55f5357815d0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f275cff9640 1 -- 172.21.15.78:0/2631929338 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f2770031540 con 0x55f5357815d0 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f277dbed740 10 client.15993 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f277dbed740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f277dbed740 20 client.15993 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f277dbed740 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f5357a0390 0x55f5357a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f277dbed740 4 client.15993 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f277dbed740 1 -- 172.21.15.78:0/2631929338 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5356f4860 con 0x55f5357a0390 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f2778daf640 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f5357a0390 0x55f5357a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f275cff9640 10 client.15993 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.203+0000 7f2778daf640 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f5357a0390 0x55f5357a27c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac21f9c324a3b91f server_cookie=b0929d6d10f30ad1 in_seq=0 out_seq=0 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f275cff9640 1 -- 172.21.15.78:0/2631929338 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5356f4860 con 0x55f5357a0390 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f275cff9640 10 client.15993 handle_command_reply: tid=0 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 client.15993 shutdown 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 -- 172.21.15.78:0/2631929338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5357815d0 msgr2=0x55f53578bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5357815d0 0x55f53578bd60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2770009900 tx=0x7f2770005300 comp rx=0 tx=0).stop 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 -- 172.21.15.78:0/2631929338 shutdown_connections 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f5357a0390 0x55f5357a27c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f53578c370 0x55f535789600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f275cff9640 0 client.15993 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f535782b40 0x55f535789010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 --2- 172.21.15.78:0/2631929338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5357815d0 0x55f53578bd60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 -- 172.21.15.78:0/2631929338 >> 172.21.15.78:0/2631929338 conn(0x55f53574a310 msgr2=0x55f535783960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.204+0000 7f277dbed740 1 -- 172.21.15.78:0/2631929338 shutdown_connections 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.205+0000 7f277dbed740 1 -- 172.21.15.78:0/2631929338 wait complete. 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.205+0000 7f277dbed740 20 client.15993 trim_cache size 0 max 16384 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.217+0000 7f277dbed740 1 Processor -- start 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.217+0000 7f277dbed740 1 -- start start 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f277dbed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f53579c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f277dbed740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f5357888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f277dbed740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53577cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f277dbed740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27741ac960 con 0x55f53578c370 2024-09-16T00:16:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f277dbed740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27741ae530 con 0x55f535782b40 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f277dbed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27741ad030 con 0x55f5357815d0 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53577cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53577cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42830/0 (socket says 172.21.15.78:42830) 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 -- 172.21.15.78:0/214685396 learned_addr learned my addr 172.21.15.78:0/214685396 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2778daf640 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f5357888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f27795b0640 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f53579c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 -- 172.21.15.78:0/214685396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 msgr2=0x55f53579c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f53579c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 -- 172.21.15.78:0/214685396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 msgr2=0x55f5357888d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f5357888d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 -- 172.21.15.78:0/214685396 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f535787fc0 con 0x55f53578c370 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f27795b0640 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f53579c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f2779db1640 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53577cbf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f276c009540 tx=0x7f276c004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdc9c8ad9e374e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f275effd640 1 -- 172.21.15.78:0/214685396 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c00c6b0 con 0x55f53578c370 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f275effd640 1 -- 172.21.15.78:0/214685396 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f276c009ac0 con 0x55f53578c370 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.218+0000 7f275effd640 1 -- 172.21.15.78:0/214685396 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c005610 con 0x55f53578c370 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 -- 172.21.15.78:0/214685396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 msgr2=0x55f53577cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53577cbf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f276c009540 tx=0x7f276c004c10 comp rx=0 tx=0).stop 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 -- 172.21.15.78:0/214685396 shutdown_connections 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f53577cbf0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f5357888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 --2- 172.21.15.78:0/214685396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f53579c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 -- 172.21.15.78:0/214685396 >> 172.21.15.78:0/214685396 conn(0x55f53574a310 msgr2=0x55f53577c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 -- 172.21.15.78:0/214685396 shutdown_connections 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 -- 172.21.15.78:0/214685396 wait complete. 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 -- start start 2024-09-16T00:16:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.219+0000 7f277dbed740 1 Processor -- start 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f277dbed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f535786100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f277dbed740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f535786640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f277dbed740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f535786b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f277dbed740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27741ade30 con 0x55f53578c370 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f277dbed740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f27741ad760 con 0x55f535782b40 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f277dbed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27741aa960 con 0x55f5357815d0 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f277dbed740 10 client.0 _ll_register_callbacks cb 0x7f2770009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f535786b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f535786b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42844/0 (socket says 172.21.15.78:42844) 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 -- 172.21.15.78:0/79718217 learned_addr learned my addr 172.21.15.78:0/79718217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2778daf640 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f535786640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f27795b0640 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f535786100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 -- 172.21.15.78:0/79718217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 msgr2=0x55f535786100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f535786100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 -- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 msgr2=0x55f535786640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f535786640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 -- 172.21.15.78:0/79718217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f53580a0b0 con 0x55f53578c370 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2778daf640 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f535786640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f2779db1640 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f535786b80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f276c010fd0 tx=0x7f276c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1490c0b98476f8de server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f275cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.220+0000 7f275cff9640 1 -- 172.21.15.78:0/79718217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c038020 con 0x55f53578c370 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f275cff9640 1 -- 172.21.15.78:0/79718217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f276c0057a0 con 0x55f53578c370 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 1 -- 172.21.15.78:0/79718217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5356eed70 con 0x55f53578c370 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f275cff9640 1 -- 172.21.15.78:0/79718217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f276c012540 con 0x55f53578c370 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 10 client.15999 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 1 -- 172.21.15.78:0/79718217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f53580b300 con 0x55f53578c370 2024-09-16T00:16:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f275cff9640 1 -- 172.21.15.78:0/79718217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f5356eed70 con 0x55f53578c370 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f275cff9640 1 -- 172.21.15.78:0/79718217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f276c01f070 con 0x55f53578c370 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 10 client.15999 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 20 client.15999 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f53580b7c0 0x55f53580dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 4 client.15999 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f277dbed740 1 -- 172.21.15.78:0/79718217 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f53566b170 con 0x55f53580b7c0 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.221+0000 7f27795b0640 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f53580b7c0 0x55f53580dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f275cff9640 10 client.15999 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f27795b0640 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f53580b7c0 0x55f53580dbb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2194c19fd4153c59 server_cookie=7db33348890cb5a3 in_seq=0 out_seq=0 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f275cff9640 1 -- 172.21.15.78:0/79718217 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f53566b170 con 0x55f53580b7c0 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f275cff9640 10 client.15999 handle_command_reply: tid=0 2024-09-16T00:16:23.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 client.15999 shutdown 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 -- 172.21.15.78:0/79718217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 msgr2=0x55f535786b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f535786b80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f276c010fd0 tx=0x7f276c005030 comp rx=0 tx=0).stop 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 -- 172.21.15.78:0/79718217 shutdown_connections 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f53580b7c0 0x55f53580dbb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f53578c370 0x55f535786b80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f535782b40 0x55f535786640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 --2- 172.21.15.78:0/79718217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f5357815d0 0x55f535786100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 -- 172.21.15.78:0/79718217 >> 172.21.15.78:0/79718217 conn(0x55f53574a310 msgr2=0x55f53577c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f275cff9640 0 client.15999 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 -- 172.21.15.78:0/79718217 shutdown_connections 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.222+0000 7f277dbed740 1 -- 172.21.15.78:0/79718217 wait complete. 2024-09-16T00:16:23.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277dbed740 20 client.15999 trim_cache size 0 max 16384 2024-09-16T00:16:23.240 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:16:23.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 -- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27600a32c0 msgr2=0x7f27600a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27600a32c0 0x7f27600a5780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f276c009fd0 tx=0x7f276c009d00 comp rx=0 tx=0).stop 2024-09-16T00:16:23.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 -- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774102570 msgr2=0x7f2774198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774102570 0x7f2774198a60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f276400d950 tx=0x7f276400de10 comp rx=0 tx=0).stop 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 -- 172.21.15.78:0/1757076136 shutdown_connections 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f27600a32c0 0x7f27600a5780 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2774102ef0 0x7f277419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2774102570 0x7f2774198a60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 --2- 172.21.15.78:0/1757076136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2774100e20 0x7f2774198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 -- 172.21.15.78:0/1757076136 >> 172.21.15.78:0/1757076136 conn(0x7f27740fc7d0 msgr2=0x7f277406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 -- 172.21.15.78:0/1757076136 shutdown_connections 2024-09-16T00:16:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:23.223+0000 7f277b814640 1 -- 172.21.15.78:0/1757076136 wait complete. 2024-09-16T00:16:23.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:23 smithi193 ceph-mon[35484]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-09-16T00:16:23.936 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:23 smithi078 ceph-mon[31157]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-09-16T00:16:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:23 smithi148 ceph-mon[35599]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-09-16T00:16:24.965 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:16:24.965 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:16:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:25.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 -- 172.21.15.78:0/2993173671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 msgr2=0x7faaa81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 --2- 172.21.15.78:0/2993173671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 0x7faaa81067f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faa98009900 tx=0x7faa9802a410 comp rx=0 tx=0).stop 2024-09-16T00:16:25.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 -- 172.21.15.78:0/2993173671 shutdown_connections 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 --2- 172.21.15.78:0/2993173671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa8106f20 0x7faaa810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 --2- 172.21.15.78:0/2993173671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 0x7faaa81067f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 --2- 172.21.15.78:0/2993173671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8101df0 0x7faaa81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 -- 172.21.15.78:0/2993173671 >> 172.21.15.78:0/2993173671 conn(0x7faaa80fd8b0 msgr2=0x7faaa80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 -- 172.21.15.78:0/2993173671 shutdown_connections 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 -- 172.21.15.78:0/2993173671 wait complete. 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 Processor -- start 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.656+0000 7faaafbd1640 1 -- start start 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaafbd1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa8101df0 0x7faaa81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaafbd1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 0x7faaa8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaafbd1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8106f20 0x7faaa819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaafbd1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8111f80 con 0x7faaa81027a0 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaafbd1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8111e00 con 0x7faaa8101df0 2024-09-16T00:16:25.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaafbd1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8112100 con 0x7faaa8106f20 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 0x7faaa8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 0x7faaa8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42864/0 (socket says 172.21.15.78:42864) 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 -- 172.21.15.78:0/3281383906 learned_addr learned my addr 172.21.15.78:0/3281383906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad96d640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa8101df0 0x7faaa81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaae16e640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8106f20 0x7faaa819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 -- 172.21.15.78:0/3281383906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8106f20 msgr2=0x7faaa819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8106f20 0x7faaa819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 -- 172.21.15.78:0/3281383906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa8101df0 msgr2=0x7faaa81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa8101df0 0x7faaa81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 -- 172.21.15.78:0/3281383906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa819d390 con 0x7faaa81027a0 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaae16e640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8106f20 0x7faaa819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaad16c640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 0x7faaa8198920 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7faa98006010 tx=0x7faa980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faa9effd640 1 -- 172.21.15.78:0/3281383906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9803f030 con 0x7faaa81027a0 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaafbd1640 1 -- 172.21.15.78:0/3281383906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa819d620 con 0x7faaa81027a0 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faa9effd640 1 -- 172.21.15.78:0/3281383906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa98003ce0 con 0x7faaa81027a0 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faa9effd640 1 -- 172.21.15.78:0/3281383906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98035070 con 0x7faaa81027a0 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.657+0000 7faaafbd1640 1 -- 172.21.15.78:0/3281383906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa81a4f60 con 0x7faaa81027a0 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faa9effd640 1 -- 172.21.15.78:0/3281383906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7faa98042030 con 0x7faaa81027a0 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faab1faa740 1 Processor -- start 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faab1faa740 1 -- start start 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faa9effd640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa940a3240 0x7faa940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faab1faa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da572f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faab1faa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faab1faa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da580130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faab1faa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637da4d8280 con 0x5637da573480 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faab1faa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5637da47fb10 con 0x5637da572b60 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.660+0000 7faab1faa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5637da4d4240 con 0x5637da57dd40 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaae16e640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa940a3240 0x7faa940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faa9effd640 1 -- 172.21.15.78:0/3281383906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faa980f0390 con 0x7faaa81027a0 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaae16e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da580130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad96d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da572f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaae16e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da580130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:40808/0 (socket says 172.21.15.78:40808) 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaae16e640 1 -- 172.21.15.78:0/2205944275 learned_addr learned my addr 172.21.15.78:0/2205944275 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad16c640 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad96d640 1 -- 172.21.15.78:0/2205944275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 msgr2=0x5637da580130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad96d640 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da580130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad96d640 1 -- 172.21.15.78:0/2205944275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 msgr2=0x5637da57d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:25.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad96d640 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad96d640 1 -- 172.21.15.78:0/2205944275 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637da48c1f0 con 0x5637da572b60 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaae16e640 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da580130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad16c640 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaae16e640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa940a3240 0x7faa940a5700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faaa0005c40 tx=0x7faaa0009e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.661+0000 7faaad96d640 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da572f40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faaa4004630 tx=0x7faaa400bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523c18a185fb1e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faa93fff640 1 -- 172.21.15.78:0/2205944275 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4003bd0 con 0x5637da572b60 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faa93fff640 1 -- 172.21.15.78:0/2205944275 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa4015070 con 0x5637da572b60 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faa93fff640 1 -- 172.21.15.78:0/2205944275 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4004ce0 con 0x5637da572b60 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 -- 172.21.15.78:0/2205944275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 msgr2=0x5637da572f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da572f40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faaa4004630 tx=0x7faaa400bb20 comp rx=0 tx=0).stop 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 -- 172.21.15.78:0/2205944275 shutdown_connections 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da580130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 --2- 172.21.15.78:0/2205944275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da572f40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 -- 172.21.15.78:0/2205944275 >> 172.21.15.78:0/2205944275 conn(0x5637da56f0f0 msgr2=0x5637da56f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 -- 172.21.15.78:0/2205944275 shutdown_connections 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.662+0000 7faab1faa740 1 -- 172.21.15.78:0/2205944275 wait complete. 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 1 -- start start 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 1 Processor -- start 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da572b60 0x5637da57ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da57dd40 0x5637da57b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637da580910 con 0x5637da573480 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5637da492f10 con 0x5637da57dd40 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5637da580a90 con 0x5637da572b60 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faab1faa740 10 client.0 _ll_register_callbacks cb 0x5637da45ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:25.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faaad16c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faaad16c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42880/0 (socket says 172.21.15.78:42880) 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faaad16c640 1 -- 172.21.15.78:0/2361396138 learned_addr learned my addr 172.21.15.78:0/2361396138 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faaae16e640 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da57dd40 0x5637da57b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.663+0000 7faaad96d640 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da572b60 0x5637da57ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faaad16c640 1 -- 172.21.15.78:0/2361396138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da572b60 msgr2=0x5637da57ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faaad16c640 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da572b60 0x5637da57ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faaad16c640 1 -- 172.21.15.78:0/2361396138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da57dd40 msgr2=0x5637da57b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faaad16c640 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da57dd40 0x5637da57b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faaad16c640 1 -- 172.21.15.78:0/2361396138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637da2496a0 con 0x5637da573480 2024-09-16T00:16:25.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faaad96d640 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da572b60 0x5637da57ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faaad16c640 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57b190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7faa980b9080 tx=0x7faa980b8a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd52c10292b429da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faa917fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faa917fa640 1 -- 172.21.15.78:0/2361396138 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98049040 con 0x5637da573480 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faa917fa640 1 -- 172.21.15.78:0/2361396138 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faa980b8d20 con 0x5637da573480 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faab1faa740 1 -- 172.21.15.78:0/2361396138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa4015070 con 0x5637da573480 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faa917fa640 1 -- 172.21.15.78:0/2361396138 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa980bc010 con 0x5637da573480 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faab1faa740 10 client.16011 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faab1faa740 1 -- 172.21.15.78:0/2361396138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5637da58c540 con 0x5637da573480 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.664+0000 7faa917fa640 1 -- 172.21.15.78:0/2361396138 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa4015070 con 0x5637da573480 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faa917fa640 1 -- 172.21.15.78:0/2361396138 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7faa980db490 con 0x5637da573480 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faab1faa740 10 client.16011 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faab1faa740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faab1faa740 20 client.16011 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:25.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faab1faa740 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5637da58d9f0 0x5637da58feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faab1faa740 4 client.16011 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:25.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faab1faa740 1 -- 172.21.15.78:0/2361396138 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637da4e57f0 con 0x5637da58d9f0 2024-09-16T00:16:25.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faaad96d640 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5637da58d9f0 0x5637da58feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faaad96d640 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5637da58d9f0 0x5637da58feb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80825005d4a04173 server_cookie=5cf4a4fa0f2545dc in_seq=0 out_seq=0 2024-09-16T00:16:25.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.665+0000 7faa917fa640 10 client.16011 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:25.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faa917fa640 1 -- 172.21.15.78:0/2361396138 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637da4e57f0 con 0x5637da58d9f0 2024-09-16T00:16:25.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faa917fa640 10 client.16011 handle_command_reply: tid=0 2024-09-16T00:16:25.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 client.16011 shutdown 2024-09-16T00:16:25.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 -- 172.21.15.78:0/2361396138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 msgr2=0x5637da57b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57b190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7faa980b9080 tx=0x7faa980b8a40 comp rx=0 tx=0).stop 2024-09-16T00:16:25.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 -- 172.21.15.78:0/2361396138 shutdown_connections 2024-09-16T00:16:25.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5637da58d9f0 0x5637da58feb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da57dd40 0x5637da57b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da57b190 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 --2- 172.21.15.78:0/2361396138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da572b60 0x5637da57ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faa917fa640 0 client.16011 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:25.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 -- 172.21.15.78:0/2361396138 >> 172.21.15.78:0/2361396138 conn(0x5637da56f0f0 msgr2=0x5637da56e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:25.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 -- 172.21.15.78:0/2361396138 shutdown_connections 2024-09-16T00:16:25.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 1 -- 172.21.15.78:0/2361396138 wait complete. 2024-09-16T00:16:25.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.666+0000 7faab1faa740 20 client.16011 trim_cache size 0 max 16384 2024-09-16T00:16:25.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.683+0000 7faab1faa740 1 Processor -- start 2024-09-16T00:16:25.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.683+0000 7faab1faa740 1 -- start start 2024-09-16T00:16:25.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faab1faa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da57cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faab1faa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da573480 0x5637da5957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faab1faa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da57dd40 0x5637da595d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faab1faa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81aeb40 con 0x5637da57dd40 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faab1faa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81b0710 con 0x5637da572b60 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faab1faa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81af210 con 0x5637da573480 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da57dd40 0x5637da595d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da57dd40 0x5637da595d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42884/0 (socket says 172.21.15.78:42884) 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 -- 172.21.15.78:0/2898066362 learned_addr learned my addr 172.21.15.78:0/2898066362 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaad96d640 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da57cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 -- 172.21.15.78:0/2898066362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da573480 msgr2=0x5637da5957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaad16c640 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da573480 0x5637da5957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da573480 0x5637da5957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 -- 172.21.15.78:0/2898066362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 msgr2=0x5637da57cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:25.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da57cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 -- 172.21.15.78:0/2898066362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637da579c00 con 0x5637da57dd40 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faaae16e640 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da57dd40 0x5637da595d00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faaa0012bf0 tx=0x7faaa001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea5f02185f3fd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faa937fe640 1 -- 172.21.15.78:0/2898066362 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0015c90 con 0x5637da57dd40 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faa937fe640 1 -- 172.21.15.78:0/2898066362 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa0015e30 con 0x5637da57dd40 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.684+0000 7faa937fe640 1 -- 172.21.15.78:0/2898066362 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0005440 con 0x5637da57dd40 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 -- 172.21.15.78:0/2898066362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da57dd40 msgr2=0x5637da595d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da57dd40 0x5637da595d00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faaa0012bf0 tx=0x7faaa001e000 comp rx=0 tx=0).stop 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 -- 172.21.15.78:0/2898066362 shutdown_connections 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da57dd40 0x5637da595d00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da573480 0x5637da5957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 --2- 172.21.15.78:0/2898066362 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da57cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 -- 172.21.15.78:0/2898066362 >> 172.21.15.78:0/2898066362 conn(0x5637da56f0f0 msgr2=0x5637da57c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:25.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 -- 172.21.15.78:0/2898066362 shutdown_connections 2024-09-16T00:16:25.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.685+0000 7faab1faa740 1 -- 172.21.15.78:0/2898066362 wait complete. 2024-09-16T00:16:25.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 1 -- start start 2024-09-16T00:16:25.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 1 Processor -- start 2024-09-16T00:16:25.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da581d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da5822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da5f7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81b0010 con 0x5637da573480 2024-09-16T00:16:25.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81af940 con 0x5637da572b60 2024-09-16T00:16:25.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faaa81acb20 con 0x5637da57dd40 2024-09-16T00:16:25.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faab1faa740 10 client.0 _ll_register_callbacks cb 0x7faa98009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:25.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faaad16c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da5822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faaad16c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da5822a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42900/0 (socket says 172.21.15.78:42900) 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faaad16c640 1 -- 172.21.15.78:0/1482595136 learned_addr learned my addr 172.21.15.78:0/1482595136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.686+0000 7faaad96d640 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da581d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faaad16c640 1 -- 172.21.15.78:0/1482595136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 msgr2=0x5637da5f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faaad16c640 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da5f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faaad16c640 1 -- 172.21.15.78:0/1482595136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 msgr2=0x5637da581d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faaad16c640 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da581d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faaad16c640 1 -- 172.21.15.78:0/1482595136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637da582b30 con 0x5637da573480 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faaad16c640 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da5822a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faa98005ab0 tx=0x7faa980b8f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2b6bb9f2b1360 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faa917fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faa917fa640 1 -- 172.21.15.78:0/1482595136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98049070 con 0x5637da573480 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faa917fa640 1 -- 172.21.15.78:0/1482595136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faa980c4e90 con 0x5637da573480 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 1 -- 172.21.15.78:0/1482595136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa0015e30 con 0x5637da573480 2024-09-16T00:16:25.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faa917fa640 1 -- 172.21.15.78:0/1482595136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faa980044e0 con 0x5637da573480 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 10 client.16017 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 1 -- 172.21.15.78:0/1482595136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5637da582b30 con 0x5637da573480 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faa917fa640 1 -- 172.21.15.78:0/1482595136 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa0015e30 con 0x5637da573480 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faa917fa640 1 -- 172.21.15.78:0/1482595136 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7faa980bc910 con 0x5637da573480 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 10 client.16017 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 20 client.16017 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5637da5f91f0 0x5637da5fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 4 client.16017 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.687+0000 7faab1faa740 1 -- 172.21.15.78:0/1482595136 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5637da4dfd00 con 0x5637da5f91f0 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.688+0000 7faaad96d640 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5637da5f91f0 0x5637da5fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.688+0000 7faa917fa640 10 client.16017 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.688+0000 7faaad96d640 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5637da5f91f0 0x5637da5fb610 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964f9447abe41a8a server_cookie=d0f5df0a3837fdb8 in_seq=0 out_seq=0 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faa917fa640 1 -- 172.21.15.78:0/1482595136 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5637da4dfd00 con 0x5637da5f91f0 2024-09-16T00:16:25.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faa917fa640 10 client.16017 handle_command_reply: tid=0 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 client.16017 shutdown 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 -- 172.21.15.78:0/1482595136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 msgr2=0x5637da5822a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da5822a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faa98005ab0 tx=0x7faa980b8f30 comp rx=0 tx=0).stop 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 -- 172.21.15.78:0/1482595136 shutdown_connections 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5637da5f91f0 0x5637da5fb610 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637da57dd40 0x5637da5f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637da573480 0x5637da5822a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 --2- 172.21.15.78:0/1482595136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637da572b60 0x5637da581d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 -- 172.21.15.78:0/1482595136 >> 172.21.15.78:0/1482595136 conn(0x5637da56f0f0 msgr2=0x5637da574e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faa917fa640 0 client.16017 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 -- 172.21.15.78:0/1482595136 shutdown_connections 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.689+0000 7faab1faa740 1 -- 172.21.15.78:0/1482595136 wait complete. 2024-09-16T00:16:25.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faab1faa740 20 client.16017 trim_cache size 0 max 16384 2024-09-16T00:16:25.696 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:16:25.696 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:16:25.696 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "e69c7e30-0798-48bc-85f4-a9066354c1bb", 2024-09-16T00:16:25.696 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:16:25.696 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 -- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa940a3240 msgr2=0x7faa940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa940a3240 0x7faa940a5700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faaa0005c40 tx=0x7faaa0009e80 comp rx=0 tx=0).stop 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 -- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 msgr2=0x7faaa8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 0x7faaa8198920 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7faa98006010 tx=0x7faa980042f0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 -- 172.21.15.78:0/3281383906 shutdown_connections 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa940a3240 0x7faa940a5700 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaa8106f20 0x7faaa819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faaa81027a0 0x7faaa8198920 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 --2- 172.21.15.78:0/3281383906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faaa8101df0 0x7faaa81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 -- 172.21.15.78:0/3281383906 >> 172.21.15.78:0/3281383906 conn(0x7faaa80fd8b0 msgr2=0x7faaa80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 -- 172.21.15.78:0/3281383906 shutdown_connections 2024-09-16T00:16:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:25.690+0000 7faaafbd1640 1 -- 172.21.15.78:0/3281383906 wait complete. 2024-09-16T00:16:25.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 145.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:16:25.894 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:16:25.917 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:25 smithi078 ceph-mon[31157]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 632 KiB/s wr, 91 op/s 2024-09-16T00:16:25.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:25 smithi193 ceph-mon[35484]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 632 KiB/s wr, 91 op/s 2024-09-16T00:16:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:25 smithi148 ceph-mon[35599]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 632 KiB/s wr, 91 op/s 2024-09-16T00:16:26.549 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:16:26.681 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:26.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:26 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:16:26.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:26 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:26 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:16:26.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:26 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:16:26.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:26 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:26 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:16:26.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:26 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:26 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:26 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:16:26.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:26 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:26 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:16:26.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:26 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:16:26.953 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:26 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.954 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:26 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:16:26.954 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:26 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.954 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:26 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:26 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:16:26.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:26 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:26 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:16:26.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:26 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:16:26.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:26 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:26 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:16:26.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:26 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:26.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:26 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:16:27.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/1213176616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 msgr2=0x7fc0a01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/1213176616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 0x7fc0a01067f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc090009900 tx=0x7fc09002a410 comp rx=0 tx=0).stop 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/1213176616 shutdown_connections 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/1213176616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0106f20 0x7fc0a010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/1213176616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 0x7fc0a01067f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/1213176616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0a0101df0 0x7fc0a01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/1213176616 >> 172.21.15.78:0/1213176616 conn(0x7fc0a00fd8b0 msgr2=0x7fc0a00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/1213176616 shutdown_connections 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.178+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/1213176616 wait complete. 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a7d9d640 1 Processor -- start 2024-09-16T00:16:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a7d9d640 1 -- start start 2024-09-16T00:16:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a7d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0a0101df0 0x7fc0a0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a7d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 0x7fc0a0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a7d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0106f20 0x7fc0a019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a7d9d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0111fa0 con 0x7fc0a01027a0 2024-09-16T00:16:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a7d9d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0111e20 con 0x7fc0a0101df0 2024-09-16T00:16:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a7d9d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a0112120 con 0x7fc0a0106f20 2024-09-16T00:16:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5338640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 0x7fc0a0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5338640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 0x7fc0a0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51248/0 (socket says 172.21.15.78:51248) 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5338640 1 -- 172.21.15.78:0/2745618348 learned_addr learned my addr 172.21.15.78:0/2745618348 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a633a640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0106f20 0x7fc0a019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5b39640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0a0101df0 0x7fc0a0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5338640 1 -- 172.21.15.78:0/2745618348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0106f20 msgr2=0x7fc0a019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5338640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0106f20 0x7fc0a019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5338640 1 -- 172.21.15.78:0/2745618348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0a0101df0 msgr2=0x7fc0a0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5338640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0a0101df0 0x7fc0a0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5338640 1 -- 172.21.15.78:0/2745618348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a019d3e0 con 0x7fc0a01027a0 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.179+0000 7fc0a5b39640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0a0101df0 0x7fc0a0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.180+0000 7fc0a5338640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 0x7fc0a0198970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc090005ab0 tx=0x7fc0900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.180+0000 7fc09effd640 1 -- 172.21.15.78:0/2745618348 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09003f030 con 0x7fc0a01027a0 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.180+0000 7fc09effd640 1 -- 172.21.15.78:0/2745618348 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc090003ce0 con 0x7fc0a01027a0 2024-09-16T00:16:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.180+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a019d670 con 0x7fc0a01027a0 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.180+0000 7fc09effd640 1 -- 172.21.15.78:0/2745618348 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090035070 con 0x7fc0a01027a0 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.180+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a01a4fb0 con 0x7fc0a01027a0 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.181+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a01032d0 con 0x7fc0a01027a0 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.182+0000 7fc09effd640 1 -- 172.21.15.78:0/2745618348 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc090042030 con 0x7fc0a01027a0 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.183+0000 7fc09effd640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc08c0a3240 0x7fc08c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.183+0000 7fc0a5b39640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc08c0a3240 0x7fc08c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.183+0000 7fc09effd640 1 -- 172.21.15.78:0/2745618348 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc0900f08c0 con 0x7fc0a01027a0 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.183+0000 7fc0a5b39640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc08c0a3240 0x7fc08c0a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc088007eb0 tx=0x7fc0880073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.186+0000 7fc09effd640 1 -- 172.21.15.78:0/2745618348 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0900bd280 con 0x7fc0a01027a0 2024-09-16T00:16:27.261 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:27.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.401+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0a010a100 con 0x7fc0a01027a0 2024-09-16T00:16:27.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.402+0000 7fc09effd640 1 -- 172.21.15.78:0/2745618348 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11990 (secure 0 0 0) 0x7fc0900bcb80 con 0x7fc0a01027a0 2024-09-16T00:16:27.403 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:16:27.404 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":31,"btime":"2024-09-16T00:13:56:527432+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:13:56.078726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:16:27.404 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 31 2024-09-16T00:16:27.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc08c0a3240 msgr2=0x7fc08c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc08c0a3240 0x7fc08c0a5700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc088007eb0 tx=0x7fc0880073c0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 msgr2=0x7fc0a0198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 0x7fc0a0198970 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc090005ab0 tx=0x7fc0900042f0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 shutdown_connections 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc08c0a3240 0x7fc08c0a5700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0a0106f20 0x7fc0a019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc0a01027a0 0x7fc0a0198970 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 --2- 172.21.15.78:0/2745618348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc0a0101df0 0x7fc0a0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 >> 172.21.15.78:0/2745618348 conn(0x7fc0a00fd8b0 msgr2=0x7fc0a00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 shutdown_connections 2024-09-16T00:16:27.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.404+0000 7fc0a7d9d640 1 -- 172.21.15.78:0/2745618348 wait complete. 2024-09-16T00:16:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 -- 172.21.15.78:0/2755122045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0101290 msgr2=0x7f07c01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 --2- 172.21.15.78:0/2755122045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0101290 0x7f07c01079e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009900 tx=0x7f07a802a410 comp rx=0 tx=0).stop 2024-09-16T00:16:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 -- 172.21.15.78:0/2755122045 shutdown_connections 2024-09-16T00:16:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 --2- 172.21.15.78:0/2755122045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c0108110 0x7f07c010ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 --2- 172.21.15.78:0/2755122045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0101290 0x7f07c01079e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 --2- 172.21.15.78:0/2755122045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07c01008e0 0x7f07c0100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 -- 172.21.15.78:0/2755122045 >> 172.21.15.78:0/2755122045 conn(0x7f07c0077210 msgr2=0x7f07c0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:27.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 -- 172.21.15.78:0/2755122045 shutdown_connections 2024-09-16T00:16:27.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 -- 172.21.15.78:0/2755122045 wait complete. 2024-09-16T00:16:27.914 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:27 smithi078 ceph-mon[31157]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 KiB/s wr, 41 op/s 2024-09-16T00:16:27.914 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:27 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2745618348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:27.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 Processor -- start 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.911+0000 7f07c577a640 1 -- start start 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07c577a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c01008e0 0x7f07c01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07c577a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07c0101290 0x7f07c01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07c577a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0108110 0x7f07c019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07c577a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0111fd0 con 0x7f07c0108110 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07c577a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0111e50 con 0x7f07c0101290 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07c577a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0112150 con 0x7f07c01008e0 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0108110 0x7f07c019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0108110 0x7f07c019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51264/0 (socket says 172.21.15.78:51264) 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 -- 172.21.15.78:0/450169663 learned_addr learned my addr 172.21.15.78:0/450169663 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07beffd640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c01008e0 0x7f07c01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07be7fc640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07c0101290 0x7f07c01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 -- 172.21.15.78:0/450169663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c01008e0 msgr2=0x7f07c01984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c01008e0 0x7f07c01984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 -- 172.21.15.78:0/450169663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07c0101290 msgr2=0x7f07c01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07c0101290 0x7f07c01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 -- 172.21.15.78:0/450169663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c019d450 con 0x7f07c0108110 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07bf7fe640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0108110 0x7f07c019cd70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009fd0 tx=0x7f07b800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.912+0000 7f07b7fff640 1 -- 172.21.15.78:0/450169663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8011830 con 0x7f07c0108110 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.913+0000 7f07c577a640 1 -- 172.21.15.78:0/450169663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c019d740 con 0x7f07c0108110 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.913+0000 7f07b7fff640 1 -- 172.21.15.78:0/450169663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8016070 con 0x7f07c0108110 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.913+0000 7f07b7fff640 1 -- 172.21.15.78:0/450169663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b800f2e0 con 0x7f07c0108110 2024-09-16T00:16:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.913+0000 7f07c577a640 1 -- 172.21.15.78:0/450169663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c01a5020 con 0x7f07c0108110 2024-09-16T00:16:27.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07b7fff640 1 -- 172.21.15.78:0/450169663 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f07b80119d0 con 0x7f07c0108110 2024-09-16T00:16:27.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07c7b53740 1 Processor -- start 2024-09-16T00:16:27.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07c7b53740 1 -- start start 2024-09-16T00:16:27.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 0x55b2b7754880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7754dc0 0x55b2b775f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b775f680 0x55b2b7761a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07c7b53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b2b76ba2e0 con 0x55b2b7754dc0 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07c7b53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b2b7661ce0 con 0x55b2b77544a0 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.915+0000 7f07c7b53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b2b76b62a0 con 0x55b2b775f680 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7754dc0 0x55b2b775f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7754dc0 0x55b2b775f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51272/0 (socket says 172.21.15.78:51272) 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 -- 172.21.15.78:0/4025677951 learned_addr learned my addr 172.21.15.78:0/4025677951 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07be7fc640 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 0x55b2b7754880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07be7fc640 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b775f680 0x55b2b7761a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07b7fff640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07ac0a3240 0x7f07ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 -- 172.21.15.78:0/4025677951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b775f680 msgr2=0x55b2b7761a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b775f680 0x55b2b7761a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 -- 172.21.15.78:0/4025677951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 msgr2=0x55b2b7754880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 0x55b2b7754880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 -- 172.21.15.78:0/4025677951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b7646440 con 0x55b2b7754dc0 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07bf7fe640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07ac0a3240 0x7f07ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07beffd640 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7754dc0 0x55b2b775f140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f07b0007ed0 tx=0x7f07b000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d19eb2f2cec6f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07b7fff640 1 -- 172.21.15.78:0/450169663 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f07b80ceeb0 con 0x7f07c0108110 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07b57fa640 1 -- 172.21.15.78:0/4025677951 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0014070 con 0x55b2b7754dc0 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07b57fa640 1 -- 172.21.15.78:0/4025677951 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b0004d20 con 0x55b2b7754dc0 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.916+0000 7f07b57fa640 1 -- 172.21.15.78:0/4025677951 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00054a0 con 0x55b2b7754dc0 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 -- 172.21.15.78:0/4025677951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7754dc0 msgr2=0x55b2b775f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07bf7fe640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07ac0a3240 0x7f07ac0a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f07b8006fd0 tx=0x7f07b80bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7754dc0 0x55b2b775f140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f07b0007ed0 tx=0x7f07b000e720 comp rx=0 tx=0).stop 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 -- 172.21.15.78:0/4025677951 shutdown_connections 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b775f680 0x55b2b7761a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7754dc0 0x55b2b775f140 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4025677951 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 0x55b2b7754880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 -- 172.21.15.78:0/4025677951 >> 172.21.15.78:0/4025677951 conn(0x55b2b7750a30 msgr2=0x55b2b7750e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 -- 172.21.15.78:0/4025677951 shutdown_connections 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.917+0000 7f07c7b53740 1 -- 172.21.15.78:0/4025677951 wait complete. 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 1 -- start start 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 1 Processor -- start 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 0x55b2b775ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b775c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b775f680 0x55b2b775c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b2b7762250 con 0x55b2b775f680 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b2b7675050 con 0x55b2b7754dc0 2024-09-16T00:16:27.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b2b77623d0 con 0x55b2b77544a0 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07c7b53740 10 client.0 _ll_register_callbacks cb 0x55b2b76b5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b775f680 0x55b2b775c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b775f680 0x55b2b775c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51274/0 (socket says 172.21.15.78:51274) 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07be7fc640 1 -- 172.21.15.78:0/4095546910 learned_addr learned my addr 172.21.15.78:0/4095546910 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07be7fc640 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 0x55b2b775ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.918+0000 7f07beffd640 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b775c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07be7fc640 1 -- 172.21.15.78:0/4095546910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 msgr2=0x55b2b775ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07be7fc640 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 0x55b2b775ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07be7fc640 1 -- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 msgr2=0x55b2b775c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07be7fc640 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b775c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07be7fc640 1 -- 172.21.15.78:0/4095546910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b742b3c0 con 0x55b2b775f680 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07beffd640 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b775c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:27.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07be7fc640 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b775f680 0x55b2b775c780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f07a8005940 tx=0x7f07a80314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc4b9fb9a0c88be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07a67fc640 1 -- 172.21.15.78:0/4095546910 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8043030 con 0x55b2b775f680 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07a67fc640 1 -- 172.21.15.78:0/4095546910 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f07a8004450 con 0x55b2b775f680 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07c7b53740 1 -- 172.21.15.78:0/4095546910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b0004d20 con 0x55b2b775f680 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07a67fc640 1 -- 172.21.15.78:0/4095546910 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f07a8052050 con 0x55b2b775f680 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07c7b53740 10 client.16035 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07c7b53740 1 -- 172.21.15.78:0/4095546910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b2b776dbd0 con 0x55b2b775f680 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07a67fc640 1 -- 172.21.15.78:0/4095546910 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b0004d20 con 0x55b2b775f680 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07a67fc640 1 -- 172.21.15.78:0/4095546910 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f07a803a040 con 0x55b2b775f680 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07c7b53740 10 client.16035 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07c7b53740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07c7b53740 20 client.16035 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.919+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b2b776f090 0x55b2b7771480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.920+0000 7f07c7b53740 4 client.16035 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:27.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.920+0000 7f07c7b53740 1 -- 172.21.15.78:0/4095546910 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b76c7850 con 0x55b2b776f090 2024-09-16T00:16:27.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.920+0000 7f07beffd640 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b2b776f090 0x55b2b7771480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.920+0000 7f07a67fc640 10 client.16035 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:27.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.920+0000 7f07beffd640 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b2b776f090 0x55b2b7771480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6bd758ca5ed9856 server_cookie=c3e568ae84e4e75f in_seq=0 out_seq=0 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07a67fc640 1 -- 172.21.15.78:0/4095546910 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2b76c7850 con 0x55b2b776f090 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07a67fc640 10 client.16035 handle_command_reply: tid=0 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 client.16035 shutdown 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 -- 172.21.15.78:0/4095546910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b775f680 msgr2=0x55b2b775c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b775f680 0x55b2b775c780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f07a8005940 tx=0x7f07a80314e0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 -- 172.21.15.78:0/4095546910 shutdown_connections 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b2b776f090 0x55b2b7771480 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b775f680 0x55b2b775c780 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b775c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07a67fc640 0 client.16035 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 --2- 172.21.15.78:0/4095546910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 0x55b2b775ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 -- 172.21.15.78:0/4095546910 >> 172.21.15.78:0/4095546910 conn(0x55b2b7750a30 msgr2=0x55b2b774fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 -- 172.21.15.78:0/4095546910 shutdown_connections 2024-09-16T00:16:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.921+0000 7f07c7b53740 1 -- 172.21.15.78:0/4095546910 wait complete. 2024-09-16T00:16:27.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.922+0000 7f07c7b53740 20 client.16035 trim_cache size 0 max 16384 2024-09-16T00:16:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:27 smithi193 ceph-mon[35484]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 KiB/s wr, 41 op/s 2024-09-16T00:16:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:27 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2745618348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.938+0000 7f07c7b53740 1 Processor -- start 2024-09-16T00:16:27.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.938+0000 7f07c7b53740 1 -- start start 2024-09-16T00:16:27.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b776b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 0x55b2b7776d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b7777240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07c7b53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01aed50 con 0x55b2b7750a30 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07c7b53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01b0920 con 0x55b2b7754dc0 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07c7b53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01af420 con 0x55b2b77544a0 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b776b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b776b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51284/0 (socket says 172.21.15.78:51284) 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 -- 172.21.15.78:0/346481283 learned_addr learned my addr 172.21.15.78:0/346481283 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07beffd640 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 0x55b2b7776d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b7777240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 -- 172.21.15.78:0/346481283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 msgr2=0x55b2b7776d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 0x55b2b7776d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 -- 172.21.15.78:0/346481283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 msgr2=0x55b2b7777240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b7777240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 -- 172.21.15.78:0/346481283 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b775f300 con 0x55b2b7750a30 2024-09-16T00:16:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07be7fc640 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b776b4b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f07a8005d00 tx=0x7f07a8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e129daeb8128216 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.939+0000 7f07b4ff9640 1 -- 172.21.15.78:0/346481283 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8043070 con 0x55b2b7750a30 2024-09-16T00:16:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07b4ff9640 1 -- 172.21.15.78:0/346481283 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a803d450 con 0x55b2b7750a30 2024-09-16T00:16:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07b4ff9640 1 -- 172.21.15.78:0/346481283 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8035040 con 0x55b2b7750a30 2024-09-16T00:16:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 -- 172.21.15.78:0/346481283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 msgr2=0x55b2b776b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b776b4b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f07a8005d00 tx=0x7f07a8005c90 comp rx=0 tx=0).stop 2024-09-16T00:16:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 -- 172.21.15.78:0/346481283 shutdown_connections 2024-09-16T00:16:27.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b7754dc0 0x55b2b7777240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b77544a0 0x55b2b7776d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 --2- 172.21.15.78:0/346481283 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b776b4b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 -- 172.21.15.78:0/346481283 >> 172.21.15.78:0/346481283 conn(0x55b2b775c8e0 msgr2=0x55b2b775ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 -- 172.21.15.78:0/346481283 shutdown_connections 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.940+0000 7f07c7b53740 1 -- 172.21.15.78:0/346481283 wait complete. 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 1 -- start start 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 1 Processor -- start 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b7763440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 0x55b2b7763980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b7754dc0 0x55b2b7763ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01b0220 con 0x55b2b7750a30 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01afb50 con 0x55b2b77544a0 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01acd30 con 0x55b2b7754dc0 2024-09-16T00:16:27.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07c7b53740 10 client.0 _ll_register_callbacks cb 0x7f07a8032d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b7763440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07be7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b7763440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51300/0 (socket says 172.21.15.78:51300) 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07be7fc640 1 -- 172.21.15.78:0/2781201358 learned_addr learned my addr 172.21.15.78:0/2781201358 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07beffd640 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 0x55b2b7763980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07be7fc640 1 -- 172.21.15.78:0/2781201358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b7754dc0 msgr2=0x55b2b7763ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07be7fc640 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b7754dc0 0x55b2b7763ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07be7fc640 1 -- 172.21.15.78:0/2781201358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 msgr2=0x55b2b7763980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07be7fc640 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 0x55b2b7763980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.941+0000 7f07be7fc640 1 -- 172.21.15.78:0/2781201358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b77d8d50 con 0x55b2b7750a30 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07be7fc640 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b7763440 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f07a803d870 tx=0x7f07a8032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38552da3df2da0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07a67fc640 1 -- 172.21.15.78:0/2781201358 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8043030 con 0x55b2b7750a30 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07a67fc640 1 -- 172.21.15.78:0/2781201358 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f07a8031aa0 con 0x55b2b7750a30 2024-09-16T00:16:27.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 1 -- 172.21.15.78:0/2781201358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a803d450 con 0x55b2b7750a30 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07a67fc640 1 -- 172.21.15.78:0/2781201358 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f07a8035040 con 0x55b2b7750a30 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 10 client.16041 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 1 -- 172.21.15.78:0/2781201358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b2b77da040 con 0x55b2b7750a30 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07a67fc640 1 -- 172.21.15.78:0/2781201358 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a803d450 con 0x55b2b7750a30 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07a67fc640 1 -- 172.21.15.78:0/2781201358 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f07a8055040 con 0x55b2b7750a30 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 10 client.16041 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 20 client.16041 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b2b775f680 0x55b2b77dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 4 client.16041 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.942+0000 7f07c7b53740 1 -- 172.21.15.78:0/2781201358 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b2b76c1d60 con 0x55b2b775f680 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07beffd640 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b2b775f680 0x55b2b77dc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:27.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07beffd640 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b2b775f680 0x55b2b77dc6b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19de99b062fe1ab5 server_cookie=d910f1eb6ab4dfb8 in_seq=0 out_seq=0 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07a67fc640 10 client.16041 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07a67fc640 1 -- 172.21.15.78:0/2781201358 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b2b76c1d60 con 0x55b2b775f680 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07a67fc640 10 client.16041 handle_command_reply: tid=0 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 client.16041 shutdown 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 -- 172.21.15.78:0/2781201358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 msgr2=0x55b2b7763440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b7763440 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f07a803d870 tx=0x7f07a8032dc0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 -- 172.21.15.78:0/2781201358 shutdown_connections 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b2b775f680 0x55b2b77dc6b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b2b7754dc0 0x55b2b7763ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b2b77544a0 0x55b2b7763980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 --2- 172.21.15.78:0/2781201358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b2b7750a30 0x55b2b7763440 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 -- 172.21.15.78:0/2781201358 >> 172.21.15.78:0/2781201358 conn(0x55b2b775c8e0 msgr2=0x55b2b775e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:27.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07a67fc640 0 client.16041 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.943+0000 7f07c7b53740 1 -- 172.21.15.78:0/2781201358 shutdown_connections 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c7b53740 1 -- 172.21.15.78:0/2781201358 wait complete. 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c7b53740 20 client.16041 trim_cache size 0 max 16384 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stdout: "e69c7e30-0798-48bc-85f4-a9066354c1bb": { 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "e69c7e30-0798-48bc-85f4-a9066354c1bb", 2024-09-16T00:16:27.951 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:16:27.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:27.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:27.952 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:16:27.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 -- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07ac0a3240 msgr2=0x7f07ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07ac0a3240 0x7f07ac0a5700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f07b8006fd0 tx=0x7f07b80bb040 comp rx=0 tx=0).stop 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 -- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0108110 msgr2=0x7f07c019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0108110 0x7f07c019cd70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009fd0 tx=0x7f07b800b8e0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 -- 172.21.15.78:0/450169663 shutdown_connections 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07ac0a3240 0x7f07ac0a5700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c0108110 0x7f07c019cd70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07c0101290 0x7f07c01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 --2- 172.21.15.78:0/450169663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c01008e0 0x7f07c01984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 -- 172.21.15.78:0/450169663 >> 172.21.15.78:0/450169663 conn(0x7f07c0077210 msgr2=0x7f07c0105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 -- 172.21.15.78:0/450169663 shutdown_connections 2024-09-16T00:16:27.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:27.944+0000 7f07c577a640 1 -- 172.21.15.78:0/450169663 wait complete. 2024-09-16T00:16:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:27 smithi148 ceph-mon[35599]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 232 KiB/s wr, 41 op/s 2024-09-16T00:16:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:27 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2745618348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:16:28.701 INFO:tasks.cephfs.filesystem:scrub status for tag:e69c7e30-0798-48bc-85f4-a9066354c1bb - {'path': '/', 'tag': 'e69c7e30-0798-48bc-85f4-a9066354c1bb', 'options': 'recursive,force'} 2024-09-16T00:16:29.172 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 362b6746 --release --await' 2024-09-16T00:16:29.172 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 362b6746 --release --await 2024-09-16T00:16:29.419 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:29.854 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:29 smithi078 ceph-mon[31157]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 67 KiB/s wr, 18 op/s 2024-09-16T00:16:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:29 smithi193 ceph-mon[35484]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 67 KiB/s wr, 18 op/s 2024-09-16T00:16:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:29 smithi148 ceph-mon[35599]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 67 KiB/s wr, 18 op/s 2024-09-16T00:16:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 -- 172.21.15.78:0/1968929511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c102ef0 msgr2=0x7f8c8c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 --2- 172.21.15.78:0/1968929511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c102ef0 0x7f8c8c10da40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009ca0 tx=0x7f8c8402a950 comp rx=0 tx=0).stop 2024-09-16T00:16:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 -- 172.21.15.78:0/1968929511 shutdown_connections 2024-09-16T00:16:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 --2- 172.21.15.78:0/1968929511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c102ef0 0x7f8c8c10da40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 --2- 172.21.15.78:0/1968929511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c102570 0x7f8c8c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 --2- 172.21.15.78:0/1968929511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c100e20 0x7f8c8c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 -- 172.21.15.78:0/1968929511 >> 172.21.15.78:0/1968929511 conn(0x7f8c8c0fc7d0 msgr2=0x7f8c8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 -- 172.21.15.78:0/1968929511 shutdown_connections 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.938+0000 7f8c91577640 1 -- 172.21.15.78:0/1968929511 wait complete. 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c91577640 1 Processor -- start 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c91577640 1 -- start start 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c91577640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c100e20 0x7f8c8c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c91577640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c102570 0x7f8c8c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c91577640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c102ef0 0x7f8c8c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c91577640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c110fa0 con 0x7f8c8c100e20 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c91577640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c110e20 con 0x7f8c8c102570 2024-09-16T00:16:29.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c91577640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c111120 con 0x7f8c8c102ef0 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c100e20 0x7f8c8c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c100e20 0x7f8c8c1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51312/0 (socket says 172.21.15.78:51312) 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c8affd640 1 -- 172.21.15.78:0/4257195208 learned_addr learned my addr 172.21.15.78:0/4257195208 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c83fff640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c102570 0x7f8c8c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c8affd640 1 -- 172.21.15.78:0/4257195208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c102ef0 msgr2=0x7f8c8c197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c8affd640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c102ef0 0x7f8c8c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c8affd640 1 -- 172.21.15.78:0/4257195208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c102570 msgr2=0x7f8c8c197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c8affd640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c102570 0x7f8c8c197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c8affd640 1 -- 172.21.15.78:0/4257195208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c1a6a50 con 0x7f8c8c100e20 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.939+0000 7f8c83fff640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c102570 0x7f8c8c197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.940+0000 7f8c8affd640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c100e20 0x7f8c8c1971e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b4a0 tx=0x7f8c7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:29.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.940+0000 7f8c88ff9640 1 -- 172.21.15.78:0/4257195208 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c014070 con 0x7f8c8c100e20 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.940+0000 7f8c88ff9640 1 -- 172.21.15.78:0/4257195208 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c010af0 con 0x7f8c8c100e20 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.940+0000 7f8c91577640 1 -- 172.21.15.78:0/4257195208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c1a6d40 con 0x7f8c8c100e20 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.940+0000 7f8c88ff9640 1 -- 172.21.15.78:0/4257195208 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c017d10 con 0x7f8c8c100e20 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.940+0000 7f8c91577640 1 -- 172.21.15.78:0/4257195208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c1a71d0 con 0x7f8c8c100e20 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.942+0000 7f8c88ff9640 1 -- 172.21.15.78:0/4257195208 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8c7c017070 con 0x7f8c8c100e20 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.942+0000 7f8c93950740 1 Processor -- start 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.942+0000 7f8c93950740 1 -- start start 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c88ff9640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8c780a3240 0x7f8c780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c93950740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376060 0x55f900376440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c93950740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376980 0x55f900380d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c93950740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f900383630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c93950740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9002db620 con 0x55f900381240 2024-09-16T00:16:29.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c93950740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f900282e50 con 0x55f900376060 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c93950740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9002d75e0 con 0x55f900376980 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8c780a3240 0x7f8c780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f900383630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c88ff9640 1 -- 172.21.15.78:0/4257195208 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8c7c0cf150 con 0x7f8c8c100e20 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f900383630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51328/0 (socket says 172.21.15.78:51328) 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 -- 172.21.15.78:0/3826771867 learned_addr learned my addr 172.21.15.78:0/3826771867 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8affd640 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376060 0x55f900376440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c83fff640 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376980 0x55f900380d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 -- 172.21.15.78:0/3826771867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376980 msgr2=0x55f900380d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376980 0x55f900380d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 -- 172.21.15.78:0/3826771867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376060 msgr2=0x55f900376440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376060 0x55f900376440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.943+0000 7f8c8b7fe640 1 -- 172.21.15.78:0/3826771867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f900273d40 con 0x55f900381240 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.944+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f900383630 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8c8402f420 tx=0x7f8c8402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54254ecaea1c0497 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.944+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8c780a3240 0x7f8c780a5700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c84030040 tx=0x7f8c84040000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.944+0000 7f8c817fa640 1 -- 172.21.15.78:0/3826771867 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8404c070 con 0x55f900381240 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.944+0000 7f8c817fa640 1 -- 172.21.15.78:0/3826771867 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8402fe40 con 0x55f900381240 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.944+0000 7f8c817fa640 1 -- 172.21.15.78:0/3826771867 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84034050 con 0x55f900381240 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- 172.21.15.78:0/3826771867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 msgr2=0x55f900383630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f900383630 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8c8402f420 tx=0x7f8c8402f450 comp rx=0 tx=0).stop 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- 172.21.15.78:0/3826771867 shutdown_connections 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f900383630 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376980 0x55f900380d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 --2- 172.21.15.78:0/3826771867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376060 0x55f900376440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- 172.21.15.78:0/3826771867 >> 172.21.15.78:0/3826771867 conn(0x55f9003725f0 msgr2=0x55f9003729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:29.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- 172.21.15.78:0/3826771867 shutdown_connections 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- 172.21.15.78:0/3826771867 wait complete. 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- start start 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 Processor -- start 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f90037e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376980 0x55f90037e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f90037ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f900383e10 con 0x55f900381240 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f900263f10 con 0x55f900376980 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f900383f90 con 0x55f900376060 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.945+0000 7f8c93950740 10 client.0 _ll_register_callbacks cb 0x55f900287db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f90037ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f90037ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51338/0 (socket says 172.21.15.78:51338) 2024-09-16T00:16:29.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 -- 172.21.15.78:0/1093074667 learned_addr learned my addr 172.21.15.78:0/1093074667 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c83fff640 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376980 0x55f90037e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8affd640 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f90037e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 -- 172.21.15.78:0/1093074667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 msgr2=0x55f90037e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f90037e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 -- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376980 msgr2=0x55f90037e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376980 0x55f90037e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 -- 172.21.15.78:0/1093074667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8fffea580 con 0x55f900381240 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c83fff640 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376980 0x55f90037e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f90037ec40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009ca0 tx=0x7f8c8402f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997accf2b4620038 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c727fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c727fc640 1 -- 172.21.15.78:0/1093074667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8404c070 con 0x55f900381240 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c727fc640 1 -- 172.21.15.78:0/1093074667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8c84003d80 con 0x55f900381240 2024-09-16T00:16:29.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c93950740 1 -- 172.21.15.78:0/1093074667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c8402fe40 con 0x55f900381240 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c727fc640 1 -- 172.21.15.78:0/1093074667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c8405e050 con 0x55f900381240 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c93950740 10 client.16053 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c93950740 1 -- 172.21.15.78:0/1093074667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f9000b4960 con 0x55f900381240 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.946+0000 7f8c727fc640 1 -- 172.21.15.78:0/1093074667 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8402fe40 con 0x55f900381240 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c727fc640 1 -- 172.21.15.78:0/1093074667 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f8c84059030 con 0x55f900381240 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c93950740 10 client.16053 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c93950740 10 client.16053 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c93950740 20 client.16053 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c93950740 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f900390cf0 0x55f9003930e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c93950740 4 client.16053 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c93950740 1 -- 172.21.15.78:0/1093074667 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9002e8b90 con 0x55f900390cf0 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c8affd640 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f900390cf0 0x55f9003930e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c8affd640 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f900390cf0 0x55f9003930e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1dd58352e2161f4 server_cookie=68da5ba78c06e8b7 in_seq=0 out_seq=0 2024-09-16T00:16:29.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.947+0000 7f8c727fc640 10 client.16053 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c727fc640 1 -- 172.21.15.78:0/1093074667 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9002e8b90 con 0x55f900390cf0 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c727fc640 10 client.16053 handle_command_reply: tid=0 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 client.16053 shutdown 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 -- 172.21.15.78:0/1093074667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 msgr2=0x55f90037ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f90037ec40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009ca0 tx=0x7f8c8402f850 comp rx=0 tx=0).stop 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 -- 172.21.15.78:0/1093074667 shutdown_connections 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f900390cf0 0x55f9003930e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900381240 0x55f90037ec40 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900376980 0x55f90037e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 --2- 172.21.15.78:0/1093074667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f90037e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 -- 172.21.15.78:0/1093074667 >> 172.21.15.78:0/1093074667 conn(0x55f9003725f0 msgr2=0x55f900378320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c727fc640 0 client.16053 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 -- 172.21.15.78:0/1093074667 shutdown_connections 2024-09-16T00:16:29.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.948+0000 7f8c93950740 1 -- 172.21.15.78:0/1093074667 wait complete. 2024-09-16T00:16:29.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.949+0000 7f8c93950740 20 client.16053 trim_cache size 0 max 16384 2024-09-16T00:16:29.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c93950740 1 Processor -- start 2024-09-16T00:16:29.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c93950740 1 -- start start 2024-09-16T00:16:29.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c93950740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f9003796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c93950740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900378210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c93950740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c93950740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b0f30 con 0x55f900376980 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c93950740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b2b00 con 0x55f900381240 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c93950740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b1600 con 0x55f900376060 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900378210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900378210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51346/0 (socket says 172.21.15.78:51346) 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.965+0000 7f8c83fff640 1 -- 172.21.15.78:0/2975790494 learned_addr learned my addr 172.21.15.78:0/2975790494 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c8affd640 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f9003796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c83fff640 1 -- 172.21.15.78:0/2975790494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 msgr2=0x55f9003796f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:29.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c83fff640 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f9003796f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c83fff640 1 -- 172.21.15.78:0/2975790494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 msgr2=0x55f9003986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c83fff640 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c83fff640 1 -- 172.21.15.78:0/2975790494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f900380100 con 0x55f900376980 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c83fff640 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900378210 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005fd0 tx=0x7f8c74013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3268e77e448bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c80ff9640 1 -- 172.21.15.78:0/2975790494 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74004030 con 0x55f900376980 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c80ff9640 1 -- 172.21.15.78:0/2975790494 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c740041d0 con 0x55f900376980 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.966+0000 7f8c80ff9640 1 -- 172.21.15.78:0/2975790494 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401c870 con 0x55f900376980 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 -- 172.21.15.78:0/2975790494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 msgr2=0x55f900378210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900378210 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005fd0 tx=0x7f8c74013830 comp rx=0 tx=0).stop 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 -- 172.21.15.78:0/2975790494 shutdown_connections 2024-09-16T00:16:29.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900378210 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 --2- 172.21.15.78:0/2975790494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f9003796f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 -- 172.21.15.78:0/2975790494 >> 172.21.15.78:0/2975790494 conn(0x55f9003725f0 msgr2=0x55f900377b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 -- 172.21.15.78:0/2975790494 shutdown_connections 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 -- 172.21.15.78:0/2975790494 wait complete. 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 -- start start 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 Processor -- start 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.967+0000 7f8c93950740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f900389600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c93950740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900389b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c93950740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c93950740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b2400 con 0x55f900376980 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c93950740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1b1d30 con 0x55f900381240 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c93950740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c1aef10 con 0x55f900376060 2024-09-16T00:16:29.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c93950740 10 client.0 _ll_register_callbacks cb 0x7f8c8402afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900389b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900389b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51352/0 (socket says 172.21.15.78:51352) 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 -- 172.21.15.78:0/3205283522 learned_addr learned my addr 172.21.15.78:0/3205283522 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c8affd640 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f900389600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c8b7fe640 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 -- 172.21.15.78:0/3205283522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 msgr2=0x55f900389600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f900389600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 -- 172.21.15.78:0/3205283522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 msgr2=0x55f9003863e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003863e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 -- 172.21.15.78:0/3205283522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f900386c10 con 0x55f900376980 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c83fff640 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900389b40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005d80 tx=0x7f8c7400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88839468cb26c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c727fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c727fc640 1 -- 172.21.15.78:0/3205283522 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7402d060 con 0x55f900376980 2024-09-16T00:16:29.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c727fc640 1 -- 172.21.15.78:0/3205283522 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8c740037f0 con 0x55f900376980 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.968+0000 7f8c93950740 1 -- 172.21.15.78:0/3205283522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c740041d0 con 0x55f900376980 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c727fc640 1 -- 172.21.15.78:0/3205283522 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c74034050 con 0x55f900376980 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c93950740 10 client.16059 fetch_fsmap learned FSMap version 31 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c93950740 1 -- 172.21.15.78:0/3205283522 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f9003fc370 con 0x55f900376980 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c727fc640 1 -- 172.21.15.78:0/3205283522 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c740041d0 con 0x55f900376980 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c727fc640 1 -- 172.21.15.78:0/3205283522 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f8c7401a020 con 0x55f900376980 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c93950740 10 client.16059 fetch_fsmap finished waiting for FSMap version 31 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c93950740 10 client.16059 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c93950740 20 client.16059 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c93950740 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f9003fcaa0 0x55f9003feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c93950740 4 client.16059 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "362b6746", "release": true, "await": true}] 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c93950740 1 -- 172.21.15.78:0/3205283522 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "362b6746", "release": true, "await": true}) -- 0x55f9002e30a0 con 0x55f9003fcaa0 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.969+0000 7f8c8affd640 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f9003fcaa0 0x55f9003feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:29.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.970+0000 7f8c727fc640 10 client.16059 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:29.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:29.970+0000 7f8c8affd640 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f9003fcaa0 0x55f9003feec0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7384b0ec4e4f58 server_cookie=e2566d3fa6e6cd28 in_seq=0 out_seq=0 2024-09-16T00:16:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c727fc640 1 -- 172.21.15.78:0/3205283522 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f9002e30a0 con 0x55f9003fcaa0 2024-09-16T00:16:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c727fc640 10 client.16059 handle_command_reply: tid=0 2024-09-16T00:16:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 client.16059 shutdown 2024-09-16T00:16:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 -- 172.21.15.78:0/3205283522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 msgr2=0x55f900389b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900389b40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005d80 tx=0x7f8c7400b900 comp rx=0 tx=0).stop 2024-09-16T00:16:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 -- 172.21.15.78:0/3205283522 shutdown_connections 2024-09-16T00:16:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f9003fcaa0 0x55f9003feec0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f900381240 0x55f9003863e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f900376980 0x55f900389b40 secure :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005d80 tx=0x7f8c7400b900 comp rx=0 tx=0).stop 2024-09-16T00:16:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 --2- 172.21.15.78:0/3205283522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f900376060 0x55f900389600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 -- 172.21.15.78:0/3205283522 >> 172.21.15.78:0/3205283522 conn(0x55f9003725f0 msgr2=0x55f900371750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:30.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c727fc640 0 client.16059 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:30.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 -- 172.21.15.78:0/3205283522 shutdown_connections 2024-09-16T00:16:30.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.045+0000 7f8c93950740 1 -- 172.21.15.78:0/3205283522 wait complete. 2024-09-16T00:16:30.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.046+0000 7f8c93950740 20 client.16059 trim_cache size 0 max 16384 2024-09-16T00:16:30.051 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:16:30.051 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 31, 2024-09-16T00:16:30.051 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:16:30.051 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 12, 2024-09-16T00:16:30.052 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:16:30.052 INFO:teuthology.orchestra.run.smithi078.stdout: "362b6746": { 2024-09-16T00:16:30.052 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 12, 2024-09-16T00:16:30.052 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:16:30.052 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:16:30.052 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:16:30.052 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:16:30.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:16:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:16:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 139.6, 2024-09-16T00:16:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:16:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:16:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:16:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:16:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:16:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:16:30.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:30.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:30.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:30.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:30.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:16:30.054 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:16:30.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.046+0000 7f8c91577640 1 -- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8c780a3240 msgr2=0x7f8c780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:30.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.046+0000 7f8c91577640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8c780a3240 0x7f8c780a5700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c84030040 tx=0x7f8c84040000 comp rx=0 tx=0).stop 2024-09-16T00:16:30.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 -- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c100e20 msgr2=0x7f8c8c1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c100e20 0x7f8c8c1971e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b4a0 tx=0x7f8c7c00b960 comp rx=0 tx=0).stop 2024-09-16T00:16:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 -- 172.21.15.78:0/4257195208 shutdown_connections 2024-09-16T00:16:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8c780a3240 0x7f8c780a5700 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c102ef0 0x7f8c8c197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c8c102570 0x7f8c8c197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:30.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 --2- 172.21.15.78:0/4257195208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c8c100e20 0x7f8c8c1971e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:30.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 -- 172.21.15.78:0/4257195208 >> 172.21.15.78:0/4257195208 conn(0x7f8c8c0fc7d0 msgr2=0x7f8c8c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:30.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 -- 172.21.15.78:0/4257195208 shutdown_connections 2024-09-16T00:16:30.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:30.047+0000 7f8c91577640 1 -- 172.21.15.78:0/4257195208 wait complete. 2024-09-16T00:16:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 150.0 s, 472.6 tps, lat 806.064 ms stddev 3685.236 2024-09-16T00:16:30.902 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '362b6746', total seconds elapsed: 24.5 2024-09-16T00:16:31.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:31 smithi193 ceph-mon[35484]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-16T00:16:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:31 smithi148 ceph-mon[35599]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-16T00:16:32.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:31 smithi078 ceph-mon[31157]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-09-16T00:16:32.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:32 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:32.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:32 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:33.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:32 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:33 smithi148 ceph-mon[35599]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 250 KiB/s wr, 43 op/s 2024-09-16T00:16:34.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:33 smithi078 ceph-mon[31157]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 250 KiB/s wr, 43 op/s 2024-09-16T00:16:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:33 smithi193 ceph-mon[35484]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 250 KiB/s wr, 43 op/s 2024-09-16T00:16:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 155.0 s, 2691.2 tps, lat 37.418 ms stddev 45.877 2024-09-16T00:16:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:35 smithi148 ceph-mon[35599]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-16T00:16:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:35 smithi078 ceph-mon[31157]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-16T00:16:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:35 smithi193 ceph-mon[35484]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-09-16T00:16:37.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:36 smithi078 ceph-mon[31157]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-16T00:16:37.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:36 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:37.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:36 smithi193 ceph-mon[35484]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-16T00:16:37.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:36 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:37.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:36 smithi148 ceph-mon[35599]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-09-16T00:16:37.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:36 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:39.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:39 smithi193 ceph-mon[35484]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.6 MiB/s wr, 483 op/s 2024-09-16T00:16:39.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:39 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:16:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:39 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:16:39.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:39 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:16:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:39 smithi148 ceph-mon[35599]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.6 MiB/s wr, 483 op/s 2024-09-16T00:16:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:39 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:16:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:39 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:16:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:39 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:16:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:39 smithi078 ceph-mon[31157]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 5.6 MiB/s wr, 483 op/s 2024-09-16T00:16:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:39 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:16:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:39 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:16:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:39 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:16:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 160.0 s, 2362.4 tps, lat 41.720 ms stddev 48.843 2024-09-16T00:16:41.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:41 smithi193 ceph-mon[35484]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 MiB/s wr, 532 op/s 2024-09-16T00:16:41.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:41 smithi148 ceph-mon[35599]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 MiB/s wr, 532 op/s 2024-09-16T00:16:42.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:41 smithi078 ceph-mon[31157]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.0 MiB/s wr, 532 op/s 2024-09-16T00:16:43.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:43 smithi193 ceph-mon[35484]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 656 op/s 2024-09-16T00:16:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:43 smithi148 ceph-mon[35599]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 656 op/s 2024-09-16T00:16:44.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:43 smithi078 ceph-mon[31157]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 MiB/s wr, 656 op/s 2024-09-16T00:16:45.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 165.0 s, 1645.6 tps, lat 61.168 ms stddev 66.439 2024-09-16T00:16:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:45 smithi193 ceph-mon[35484]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.9 MiB/s wr, 744 op/s 2024-09-16T00:16:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:45 smithi148 ceph-mon[35599]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.9 MiB/s wr, 744 op/s 2024-09-16T00:16:46.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:45 smithi078 ceph-mon[31157]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.9 MiB/s wr, 744 op/s 2024-09-16T00:16:47.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:47 smithi193 ceph-mon[35484]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 787 op/s 2024-09-16T00:16:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:47 smithi148 ceph-mon[35599]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 787 op/s 2024-09-16T00:16:48.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:47 smithi078 ceph-mon[31157]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 MiB/s wr, 787 op/s 2024-09-16T00:16:49.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:49 smithi193 ceph-mon[35484]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-16T00:16:49.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:49 smithi148 ceph-mon[35599]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-16T00:16:50.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:49 smithi078 ceph-mon[31157]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-16T00:16:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 170.0 s, 3185.8 tps, lat 31.394 ms stddev 40.954 2024-09-16T00:16:51.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:51 smithi193 ceph-mon[35484]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 MiB/s wr, 652 op/s 2024-09-16T00:16:51.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:51 smithi148 ceph-mon[35599]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 MiB/s wr, 652 op/s 2024-09-16T00:16:52.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:51 smithi078 ceph-mon[31157]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 MiB/s wr, 652 op/s 2024-09-16T00:16:53.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:53 smithi193 ceph-mon[35484]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 697 op/s 2024-09-16T00:16:53.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:53 smithi148 ceph-mon[35599]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 697 op/s 2024-09-16T00:16:54.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:53 smithi078 ceph-mon[31157]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 697 op/s 2024-09-16T00:16:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 175.0 s, 1971.4 tps, lat 50.618 ms stddev 58.497 2024-09-16T00:16:55.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:55 smithi193 ceph-mon[35484]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 715 op/s 2024-09-16T00:16:55.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:55 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:55 smithi148 ceph-mon[35599]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 715 op/s 2024-09-16T00:16:55.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:55 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:55 smithi078 ceph-mon[31157]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 715 op/s 2024-09-16T00:16:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:55 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:16:57.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:57 smithi193 ceph-mon[35484]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-09-16T00:16:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:57 smithi148 ceph-mon[35599]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-09-16T00:16:58.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:57 smithi078 ceph-mon[31157]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 684 op/s 2024-09-16T00:16:58.703 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:16:59.152 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:16:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.748+0000 7fad513d0640 1 -- 172.21.15.78:0/1365459972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad4c108da0 msgr2=0x7fad4c10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.748+0000 7fad513d0640 1 --2- 172.21.15.78:0/1365459972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad4c108da0 0x7fad4c10cdf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad40009900 tx=0x7fad4002a420 comp rx=0 tx=0).stop 2024-09-16T00:16:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.748+0000 7fad513d0640 1 -- 172.21.15.78:0/1365459972 shutdown_connections 2024-09-16T00:16:59.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.748+0000 7fad513d0640 1 --2- 172.21.15.78:0/1365459972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad4c10d520 0x7fad4c1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.748+0000 7fad513d0640 1 --2- 172.21.15.78:0/1365459972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad4c108da0 0x7fad4c10cdf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.748+0000 7fad513d0640 1 --2- 172.21.15.78:0/1365459972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1083f0 0x7fad4c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.748+0000 7fad513d0640 1 -- 172.21.15.78:0/1365459972 >> 172.21.15.78:0/1365459972 conn(0x7fad4c077af0 msgr2=0x7fad4c079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:59.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.748+0000 7fad513d0640 1 -- 172.21.15.78:0/1365459972 shutdown_connections 2024-09-16T00:16:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.750+0000 7fad513d0640 1 -- 172.21.15.78:0/1365459972 wait complete. 2024-09-16T00:16:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.751+0000 7fad513d0640 1 Processor -- start 2024-09-16T00:16:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.751+0000 7fad513d0640 1 -- start start 2024-09-16T00:16:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.751+0000 7fad513d0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1083f0 0x7fad4c073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.751+0000 7fad513d0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad4c108da0 0x7fad4c06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.751+0000 7fad513d0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad4c10d520 0x7fad4c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.751+0000 7fad513d0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c06c660 con 0x7fad4c1083f0 2024-09-16T00:16:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.751+0000 7fad513d0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c06c4e0 con 0x7fad4c108da0 2024-09-16T00:16:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.751+0000 7fad513d0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c06c7e0 con 0x7fad4c10d520 2024-09-16T00:16:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.752+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad4c10d520 0x7fad4c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.752+0000 7fad4affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1083f0 0x7fad4c073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.752+0000 7fad4affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1083f0 0x7fad4c073cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58934/0 (socket says 172.21.15.78:58934) 2024-09-16T00:16:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.752+0000 7fad4affd640 1 -- 172.21.15.78:0/616398996 learned_addr learned my addr 172.21.15.78:0/616398996 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.753+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad4c108da0 0x7fad4c06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.755+0000 7fad4affd640 1 -- 172.21.15.78:0/616398996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad4c10d520 msgr2=0x7fad4c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.755+0000 7fad4affd640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad4c10d520 0x7fad4c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.755+0000 7fad4affd640 1 -- 172.21.15.78:0/616398996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad4c108da0 msgr2=0x7fad4c06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.755+0000 7fad4affd640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad4c108da0 0x7fad4c06dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.755+0000 7fad4affd640 1 -- 172.21.15.78:0/616398996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c06eb00 con 0x7fad4c1083f0 2024-09-16T00:16:59.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.755+0000 7fad4affd640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1083f0 0x7fad4c073cd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fad4400ba50 tx=0x7fad4400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.758+0000 7fad33fff640 1 -- 172.21.15.78:0/616398996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44019070 con 0x7fad4c1083f0 2024-09-16T00:16:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.758+0000 7fad33fff640 1 -- 172.21.15.78:0/616398996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad4400cbe0 con 0x7fad4c1083f0 2024-09-16T00:16:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.758+0000 7fad33fff640 1 -- 172.21.15.78:0/616398996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad440076f0 con 0x7fad4c1083f0 2024-09-16T00:16:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.759+0000 7fad513d0640 1 -- 172.21.15.78:0/616398996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c11dce0 con 0x7fad4c1083f0 2024-09-16T00:16:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.759+0000 7fad513d0640 1 -- 172.21.15.78:0/616398996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c11e1a0 con 0x7fad4c1083f0 2024-09-16T00:16:59.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.763+0000 7fad33fff640 1 -- 172.21.15.78:0/616398996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fad4401c070 con 0x7fad4c1083f0 2024-09-16T00:16:59.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.764+0000 7fad537a9740 1 Processor -- start 2024-09-16T00:16:59.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad537a9740 1 -- start start 2024-09-16T00:16:59.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad537a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fa8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad537a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad537a9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad537a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b9a3f0e2e0 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad537a9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b9a3eb5ce0 con 0x55b9a3fb3680 2024-09-16T00:16:59.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad537a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b9a3f0a2a0 con 0x55b9a3fa84a0 2024-09-16T00:16:59.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad4a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad4a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:50548/0 (socket says 172.21.15.78:50548) 2024-09-16T00:16:59.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.765+0000 7fad4a7fc640 1 -- 172.21.15.78:0/2104177684 learned_addr learned my addr 172.21.15.78:0/2104177684 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:59.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.766+0000 7fad33fff640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fad340a3410 0x7fad340a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.766+0000 7fad33fff640 1 -- 172.21.15.78:0/616398996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fad440cf290 con 0x7fad4c1083f0 2024-09-16T00:16:59.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.767+0000 7fad4a7fc640 1 -- 172.21.15.78:0/2104177684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 msgr2=0x55b9a3fa8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.767+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/2104177684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fa8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.767+0000 7fad4a7fc640 1 -- 172.21.15.78:0/2104177684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 msgr2=0x55b9a3fb3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.767+0000 7fad4b7fe640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fad340a3410 0x7fad340a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.767+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/2104177684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.767+0000 7fad4a7fc640 1 -- 172.21.15.78:0/2104177684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a3e9a440 con 0x55b9a3fb3680 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.769+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/2104177684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb5a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad40009900 tx=0x7fad40031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c24713368788f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.770+0000 7fad317fa640 1 -- 172.21.15.78:0/2104177684 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4003f070 con 0x55b9a3fb3680 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.771+0000 7fad317fa640 1 -- 172.21.15.78:0/2104177684 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad400035f0 con 0x55b9a3fb3680 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.771+0000 7fad317fa640 1 -- 172.21.15.78:0/2104177684 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40035070 con 0x55b9a3fb3680 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 -- 172.21.15.78:0/2104177684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fb3680 msgr2=0x55b9a3fb5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad4b7fe640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fad340a3410 0x7fad340a58d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad3c006fd0 tx=0x7fad3c006f20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 --2- 172.21.15.78:0/2104177684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb5a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad40009900 tx=0x7fad40031660 comp rx=0 tx=0).stop 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 -- 172.21.15.78:0/2104177684 shutdown_connections 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 --2- 172.21.15.78:0/2104177684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb5a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 --2- 172.21.15.78:0/2104177684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 --2- 172.21.15.78:0/2104177684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fa8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 -- 172.21.15.78:0/2104177684 >> 172.21.15.78:0/2104177684 conn(0x55b9a3fa4a30 msgr2=0x55b9a3fa4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 -- 172.21.15.78:0/2104177684 shutdown_connections 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.772+0000 7fad537a9740 1 -- 172.21.15.78:0/2104177684 wait complete. 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 1 -- start start 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 1 Processor -- start 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fb3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b9a3fb6250 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b9a3ec9050 con 0x55b9a3fa84a0 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b9a3fb63d0 con 0x55b9a3fb3680 2024-09-16T00:16:59.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad537a9740 10 client.0 _ll_register_callbacks cb 0x55b9a3f09160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad4a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad4a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58966/0 (socket says 172.21.15.78:58966) 2024-09-16T00:16:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.773+0000 7fad4a7fc640 1 -- 172.21.15.78:0/3290831774 learned_addr learned my addr 172.21.15.78:0/3290831774 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.774+0000 7fad4affd640 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fb3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.774+0000 7fad4b7fe640 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.774+0000 7fad4a7fc640 1 -- 172.21.15.78:0/3290831774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 msgr2=0x55b9a3fb0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:59.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.774+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.774+0000 7fad4a7fc640 1 -- 172.21.15.78:0/3290831774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 msgr2=0x55b9a3fb3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.774+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fb3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.774+0000 7fad4a7fc640 1 -- 172.21.15.78:0/3290831774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a3c7f3c0 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.774+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb03d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fad4002a460 tx=0x7fad40004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91dd5ecc71bef2d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.775+0000 7fad2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.775+0000 7fad2eff5640 1 -- 172.21.15.78:0/3290831774 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4004a030 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.775+0000 7fad2eff5640 1 -- 172.21.15.78:0/3290831774 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fad400039b0 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.775+0000 7fad537a9740 1 -- 172.21.15.78:0/3290831774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad400035f0 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.778+0000 7fad2eff5640 1 -- 172.21.15.78:0/3290831774 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fad4003f070 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.778+0000 7fad537a9740 10 client.16065 fetch_fsmap learned FSMap version 34 2024-09-16T00:16:59.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.778+0000 7fad537a9740 1 -- 172.21.15.78:0/3290831774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b9a3fd2da0 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.779+0000 7fad2eff5640 1 -- 172.21.15.78:0/3290831774 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad400035f0 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.779+0000 7fad2eff5640 1 -- 172.21.15.78:0/3290831774 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fad40043400 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.780+0000 7fad537a9740 10 client.16065 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.780+0000 7fad537a9740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.780+0000 7fad537a9740 20 client.16065 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.780+0000 7fad537a9740 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b9a3fd40f0 0x55b9a3fd64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.780+0000 7fad537a9740 4 client.16065 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.780+0000 7fad537a9740 1 -- 172.21.15.78:0/3290831774 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9a3f1b850 con 0x55b9a3fd40f0 2024-09-16T00:16:59.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.780+0000 7fad4affd640 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b9a3fd40f0 0x55b9a3fd64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.793+0000 7fad4affd640 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b9a3fd40f0 0x55b9a3fd64e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d54e0720966baf server_cookie=d405969fa5b49293 in_seq=0 out_seq=0 2024-09-16T00:16:59.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.793+0000 7fad2eff5640 10 client.16065 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:59.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.796+0000 7fad2eff5640 1 -- 172.21.15.78:0/3290831774 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9a3f1b850 con 0x55b9a3fd40f0 2024-09-16T00:16:59.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.796+0000 7fad2eff5640 10 client.16065 handle_command_reply: tid=0 2024-09-16T00:16:59.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.797+0000 7fad537a9740 1 client.16065 shutdown 2024-09-16T00:16:59.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 -- 172.21.15.78:0/3290831774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 msgr2=0x55b9a3fb03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb03d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fad4002a460 tx=0x7fad40004230 comp rx=0 tx=0).stop 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 -- 172.21.15.78:0/3290831774 shutdown_connections 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b9a3fd40f0 0x55b9a3fd64e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad2eff5640 0 client.16065 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fb03d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 --2- 172.21.15.78:0/3290831774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 -- 172.21.15.78:0/3290831774 >> 172.21.15.78:0/3290831774 conn(0x55b9a3fa4a30 msgr2=0x55b9a3faa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.798+0000 7fad537a9740 1 -- 172.21.15.78:0/3290831774 shutdown_connections 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.799+0000 7fad537a9740 1 -- 172.21.15.78:0/3290831774 wait complete. 2024-09-16T00:16:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.799+0000 7fad537a9740 20 client.16065 trim_cache size 0 max 16384 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.828+0000 7fad537a9740 1 Processor -- start 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.828+0000 7fad537a9740 1 -- start start 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.828+0000 7fad537a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.828+0000 7fad537a9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fadb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad537a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad537a9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c11f6f0 con 0x55b9a3fb3680 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad537a9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c121230 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad537a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c11fdc0 con 0x55b9a3fa84a0 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fadb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:57398/0 (socket says 172.21.15.78:57398) 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fadb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:50560/0 (socket says 172.21.15.78:50560) 2024-09-16T00:16:59.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4affd640 1 -- 172.21.15.78:0/3470562933 learned_addr learned my addr 172.21.15.78:0/3470562933 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:59.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4a7fc640 1 -- 172.21.15.78:0/3470562933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 msgr2=0x55b9a3fad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:59.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/3470562933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4a7fc640 1 -- 172.21.15.78:0/3470562933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fb3680 msgr2=0x55b9a3fb1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:59.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/3470562933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4a7fc640 1 -- 172.21.15.78:0/3470562933 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a3fb0ed0 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4affd640 1 --2- 172.21.15.78:0/3470562933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:16:59.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/3470562933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fadb90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad40031550 tx=0x7fad40031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2a1ba8dc564600 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.829+0000 7fad30ff9640 1 -- 172.21.15.78:0/3470562933 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4004a070 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad30ff9640 1 -- 172.21.15.78:0/3470562933 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad40043400 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad30ff9640 1 -- 172.21.15.78:0/3470562933 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad40035040 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 -- 172.21.15.78:0/3470562933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa8dc0 msgr2=0x55b9a3fadb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 --2- 172.21.15.78:0/3470562933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fadb90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad40031550 tx=0x7fad40031660 comp rx=0 tx=0).stop 2024-09-16T00:16:59.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 -- 172.21.15.78:0/3470562933 shutdown_connections 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 --2- 172.21.15.78:0/3470562933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fb3680 0x55b9a3fb1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 --2- 172.21.15.78:0/3470562933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fadb90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 --2- 172.21.15.78:0/3470562933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 -- 172.21.15.78:0/3470562933 >> 172.21.15.78:0/3470562933 conn(0x55b9a3fa4a30 msgr2=0x55b9a3faf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 -- 172.21.15.78:0/3470562933 shutdown_connections 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.830+0000 7fad537a9740 1 -- 172.21.15.78:0/3470562933 wait complete. 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad537a9740 1 -- start start 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad537a9740 1 Processor -- start 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad537a9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fd71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad537a9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fd7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad537a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 0x55b9a3fd7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad4affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fd71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad4affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fd71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:50574/0 (socket says 172.21.15.78:50574) 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad4affd640 1 -- 172.21.15.78:0/1977378360 learned_addr learned my addr 172.21.15.78:0/1977378360 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c120b60 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad4b7fe640 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 0x55b9a3fd7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fd7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c120490 con 0x55b9a3fa84a0 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.831+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c082430 con 0x55b9a3fb3680 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad537a9740 10 client.0 _ll_register_callbacks cb 0x7fad400315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad4a7fc640 1 -- 172.21.15.78:0/1977378360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 msgr2=0x55b9a3fd7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 0x55b9a3fd7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad4a7fc640 1 -- 172.21.15.78:0/1977378360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 msgr2=0x55b9a3fd71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fd71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad4a7fc640 1 -- 172.21.15.78:0/1977378360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a403da30 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad4a7fc640 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fd7730 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55b9a3fb0a90 tx=0x7fad400052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7481f6182e83fce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad2eff5640 1 -- 172.21.15.78:0/1977378360 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4004a070 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad40043400 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.832+0000 7fad2eff5640 1 -- 172.21.15.78:0/1977378360 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fad400047a0 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad2eff5640 1 -- 172.21.15.78:0/1977378360 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fad40064050 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad537a9740 10 client.16071 fetch_fsmap learned FSMap version 34 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b9a403ec40 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad2eff5640 1 -- 172.21.15.78:0/1977378360 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad40043400 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad2eff5640 1 -- 172.21.15.78:0/1977378360 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fad4003b430 con 0x55b9a3fa8dc0 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad537a9740 10 client.16071 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad537a9740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad537a9740 20 client.16071 populate_metadata read hostname 'smithi078' 2024-09-16T00:16:59.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad537a9740 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b9a3fd40f0 0x55b9a40410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad537a9740 4 client.16071 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9a3f15d60 con 0x55b9a3fd40f0 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.833+0000 7fad4affd640 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b9a3fd40f0 0x55b9a40410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad2eff5640 10 client.16071 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad4affd640 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b9a3fd40f0 0x55b9a40410d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bbea51c939bfcaf server_cookie=f9bf26b2021217c4 in_seq=0 out_seq=0 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad2eff5640 1 -- 172.21.15.78:0/1977378360 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b9a3f15d60 con 0x55b9a3fd40f0 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad2eff5640 10 client.16071 handle_command_reply: tid=0 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 client.16071 shutdown 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 msgr2=0x55b9a3fd7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fd7730 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55b9a3fb0a90 tx=0x7fad400052d0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 shutdown_connections 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b9a3fd40f0 0x55b9a40410d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9a3fb3680 0x55b9a3fd7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b9a3fa8dc0 0x55b9a3fd7730 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad2eff5640 0 client.16071 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 --2- 172.21.15.78:0/1977378360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b9a3fa84a0 0x55b9a3fd71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 >> 172.21.15.78:0/1977378360 conn(0x55b9a3fa4a30 msgr2=0x55b9a3fb1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 shutdown_connections 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 1 -- 172.21.15.78:0/1977378360 wait complete. 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.834+0000 7fad537a9740 20 client.16071 trim_cache size 0 max 16384 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:16:59.840 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:16:59.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 -- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fad340a3410 msgr2=0x7fad340a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fad340a3410 0x7fad340a58d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad3c006fd0 tx=0x7fad3c006f20 comp rx=0 tx=0).stop 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 -- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1083f0 msgr2=0x7fad4c073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1083f0 0x7fad4c073cd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fad4400ba50 tx=0x7fad4400bf10 comp rx=0 tx=0).stop 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 -- 172.21.15.78:0/616398996 shutdown_connections 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fad340a3410 0x7fad340a58d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad4c10d520 0x7fad4c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad4c108da0 0x7fad4c06dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 --2- 172.21.15.78:0/616398996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad4c1083f0 0x7fad4c073cd0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 -- 172.21.15.78:0/616398996 >> 172.21.15.78:0/616398996 conn(0x7fad4c077af0 msgr2=0x7fad4c078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.835+0000 7fad513d0640 1 -- 172.21.15.78:0/616398996 shutdown_connections 2024-09-16T00:16:59.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:16:59.836+0000 7fad513d0640 1 -- 172.21.15.78:0/616398996 wait complete. 2024-09-16T00:16:59.849 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:16:59 smithi078 ceph-mon[31157]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-09-16T00:16:59.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:16:59 smithi193 ceph-mon[35484]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-09-16T00:16:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:16:59 smithi148 ceph-mon[35599]: pgmap v468: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 8.1 MiB/s wr, 717 op/s 2024-09-16T00:17:00.689 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:17:00.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:17:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 180.0 s, 2264.4 tps, lat 44.086 ms stddev 54.589 2024-09-16T00:17:01.120 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:01.913 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:01 smithi078 ceph-mon[31157]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.4 MiB/s wr, 575 op/s 2024-09-16T00:17:01.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:01 smithi193 ceph-mon[35484]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.4 MiB/s wr, 575 op/s 2024-09-16T00:17:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:01 smithi148 ceph-mon[35599]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.4 MiB/s wr, 575 op/s 2024-09-16T00:17:02.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.028+0000 7ffa7f773640 1 -- 172.21.15.78:0/2843604418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78100a90 msgr2=0x7ffa78100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:02.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.028+0000 7ffa7f773640 1 --2- 172.21.15.78:0/2843604418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78100a90 0x7ffa78100ef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009900 tx=0x7ffa6802a410 comp rx=0 tx=0).stop 2024-09-16T00:17:02.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.029+0000 7ffa7f773640 1 -- 172.21.15.78:0/2843604418 shutdown_connections 2024-09-16T00:17:02.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.029+0000 7ffa7f773640 1 --2- 172.21.15.78:0/2843604418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa78101430 0x7ffa7810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.029+0000 7ffa7f773640 1 --2- 172.21.15.78:0/2843604418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78100a90 0x7ffa78100ef0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.029+0000 7ffa7f773640 1 --2- 172.21.15.78:0/2843604418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa78106a90 0x7ffa78106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.029+0000 7ffa7f773640 1 -- 172.21.15.78:0/2843604418 >> 172.21.15.78:0/2843604418 conn(0x7ffa780fc7d0 msgr2=0x7ffa780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:02.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.029+0000 7ffa7f773640 1 -- 172.21.15.78:0/2843604418 shutdown_connections 2024-09-16T00:17:02.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.029+0000 7ffa7f773640 1 -- 172.21.15.78:0/2843604418 wait complete. 2024-09-16T00:17:02.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.029+0000 7ffa7f773640 1 Processor -- start 2024-09-16T00:17:02.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7f773640 1 -- start start 2024-09-16T00:17:02.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7f773640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa78100a90 0x7ffa7819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:02.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7f773640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa78101430 0x7ffa7819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:02.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7f773640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78106a90 0x7ffa7819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:02.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7f773640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78110e90 con 0x7ffa78100a90 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7f773640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78110d10 con 0x7ffa78106a90 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7f773640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78111010 con 0x7ffa78101430 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7d50f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa78100a90 0x7ffa7819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7d50f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa78100a90 0x7ffa7819a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59012/0 (socket says 172.21.15.78:59012) 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7d50f640 1 -- 172.21.15.78:0/3034968563 learned_addr learned my addr 172.21.15.78:0/3034968563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7dd10640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78106a90 0x7ffa7819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7cd0e640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa78101430 0x7ffa7819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7dd10640 1 -- 172.21.15.78:0/3034968563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa78101430 msgr2=0x7ffa7819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7dd10640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa78101430 0x7ffa7819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7dd10640 1 -- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa78100a90 msgr2=0x7ffa7819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7dd10640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa78100a90 0x7ffa7819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7dd10640 1 -- 172.21.15.78:0/3034968563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7819f6d0 con 0x7ffa78106a90 2024-09-16T00:17:02.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.030+0000 7ffa7cd0e640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa78101430 0x7ffa7819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:17:02.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.031+0000 7ffa7dd10640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78106a90 0x7ffa7819eff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009fd0 tx=0x7ffa7000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:02.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.031+0000 7ffa6e7fc640 1 -- 172.21.15.78:0/3034968563 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7000ce20 con 0x7ffa78106a90 2024-09-16T00:17:02.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.031+0000 7ffa6e7fc640 1 -- 172.21.15.78:0/3034968563 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa70005020 con 0x7ffa78106a90 2024-09-16T00:17:02.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.031+0000 7ffa6e7fc640 1 -- 172.21.15.78:0/3034968563 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70010720 con 0x7ffa78106a90 2024-09-16T00:17:02.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.031+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7819f9c0 con 0x7ffa78106a90 2024-09-16T00:17:02.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.031+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781a72a0 con 0x7ffa78106a90 2024-09-16T00:17:02.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.032+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78106e70 con 0x7ffa78106a90 2024-09-16T00:17:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.035+0000 7ffa6e7fc640 1 -- 172.21.15.78:0/3034968563 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ffa70005540 con 0x7ffa78106a90 2024-09-16T00:17:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.035+0000 7ffa6e7fc640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa640a3240 0x7ffa640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.036+0000 7ffa7d50f640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa640a3240 0x7ffa640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.036+0000 7ffa6e7fc640 1 -- 172.21.15.78:0/3034968563 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ffa700d0610 con 0x7ffa78106a90 2024-09-16T00:17:02.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.036+0000 7ffa7d50f640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa640a3240 0x7ffa640a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009c50 tx=0x7ffa74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.038+0000 7ffa6e7fc640 1 -- 172.21.15.78:0/3034968563 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa7009cfd0 con 0x7ffa78106a90 2024-09-16T00:17:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.399+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa7819ba80 con 0x7ffa78106a90 2024-09-16T00:17:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.400+0000 7ffa6e7fc640 1 -- 172.21.15.78:0/3034968563 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7ffa7819ba80 con 0x7ffa78106a90 2024-09-16T00:17:02.401 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:17:02.401 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:17:02.401 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:17:02.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa640a3240 msgr2=0x7ffa640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:02.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa640a3240 0x7ffa640a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009c50 tx=0x7ffa74005f90 comp rx=0 tx=0).stop 2024-09-16T00:17:02.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78106a90 msgr2=0x7ffa7819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:02.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78106a90 0x7ffa7819eff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009fd0 tx=0x7ffa7000ef10 comp rx=0 tx=0).stop 2024-09-16T00:17:02.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 shutdown_connections 2024-09-16T00:17:02.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ffa640a3240 0x7ffa640a5700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa78106a90 0x7ffa7819eff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa78101430 0x7ffa7819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 --2- 172.21.15.78:0/3034968563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa78100a90 0x7ffa7819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:02.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 >> 172.21.15.78:0/3034968563 conn(0x7ffa780fc7d0 msgr2=0x7ffa781045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:02.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 shutdown_connections 2024-09-16T00:17:02.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:02.403+0000 7ffa7f773640 1 -- 172.21.15.78:0/3034968563 wait complete. 2024-09-16T00:17:02.744 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:02 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3034968563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:02.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:02 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3034968563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:02 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3034968563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:03.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:03 smithi193 ceph-mon[35484]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.6 MiB/s wr, 688 op/s 2024-09-16T00:17:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:03 smithi148 ceph-mon[35599]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.6 MiB/s wr, 688 op/s 2024-09-16T00:17:04.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:03 smithi078 ceph-mon[31157]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.6 MiB/s wr, 688 op/s 2024-09-16T00:17:04.450 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:17:04.884 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 185.0 s, 2296.2 tps, lat 43.788 ms stddev 54.164 2024-09-16T00:17:05.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:05 smithi193 ceph-mon[35484]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-09-16T00:17:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:05 smithi148 ceph-mon[35599]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-09-16T00:17:06.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:05 smithi078 ceph-mon[31157]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.7 MiB/s wr, 703 op/s 2024-09-16T00:17:07.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:07 smithi193 ceph-mon[35484]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 635 op/s 2024-09-16T00:17:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:07 smithi148 ceph-mon[35599]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 635 op/s 2024-09-16T00:17:08.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:07 smithi078 ceph-mon[31157]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.0 MiB/s wr, 635 op/s 2024-09-16T00:17:09.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:09 smithi193 ceph-mon[35484]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-16T00:17:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:09 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:09 smithi148 ceph-mon[35599]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-16T00:17:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:09 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:10.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:09 smithi078 ceph-mon[31157]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-16T00:17:10.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:09 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:17:10.745 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:17:10.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 -- 172.21.15.78:0/3864030315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c100a90 msgr2=0x7f0e1c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/3864030315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c100a90 0x7f0e1c100ef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009890 tx=0x7f0e0402a370 comp rx=0 tx=0).stop 2024-09-16T00:17:10.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 -- 172.21.15.78:0/3864030315 shutdown_connections 2024-09-16T00:17:10.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/3864030315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e1c101430 0x7f0e1c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/3864030315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c100a90 0x7f0e1c100ef0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/3864030315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c106a90 0x7f0e1c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 -- 172.21.15.78:0/3864030315 >> 172.21.15.78:0/3864030315 conn(0x7f0e1c0fc7d0 msgr2=0x7f0e1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 -- 172.21.15.78:0/3864030315 shutdown_connections 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.665+0000 7f0e20d1a640 1 -- 172.21.15.78:0/3864030315 wait complete. 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.666+0000 7f0e20d1a640 1 Processor -- start 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.666+0000 7f0e20d1a640 1 -- start start 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.667+0000 7f0e20d1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e1c100a90 0x7f0e1c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.667+0000 7f0e20d1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c101430 0x7f0e1c197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.667+0000 7f0e20d1a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c106a90 0x7f0e1c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.667+0000 7f0e20d1a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c110e50 con 0x7f0e1c101430 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.667+0000 7f0e20d1a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c110cd0 con 0x7f0e1c100a90 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.667+0000 7f0e20d1a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c110fd0 con 0x7f0e1c106a90 2024-09-16T00:17:10.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.667+0000 7f0e1a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e1c100a90 0x7f0e1c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e1c100a90 0x7f0e1c197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48698/0 (socket says 172.21.15.78:48698) 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1a59c640 1 -- 172.21.15.78:0/2393433757 learned_addr learned my addr 172.21.15.78:0/2393433757 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1ad9d640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c106a90 0x7f0e1c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e13fff640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c101430 0x7f0e1c197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1a59c640 1 -- 172.21.15.78:0/2393433757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c106a90 msgr2=0x7f0e1c19bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c106a90 0x7f0e1c19bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1a59c640 1 -- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c101430 msgr2=0x7f0e1c197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c101430 0x7f0e1c197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1a59c640 1 -- 172.21.15.78:0/2393433757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c19c300 con 0x7f0e1c100a90 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.668+0000 7f0e1ad9d640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c106a90 0x7f0e1c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.669+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e1c100a90 0x7f0e1c197350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00e9c0 tx=0x7f0e0c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.669+0000 7f0e137fe640 1 -- 172.21.15.78:0/2393433757 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00cde0 con 0x7f0e1c100a90 2024-09-16T00:17:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.669+0000 7f0e20d1a640 1 -- 172.21.15.78:0/2393433757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c19c5f0 con 0x7f0e1c100a90 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.669+0000 7f0e137fe640 1 -- 172.21.15.78:0/2393433757 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c0035f0 con 0x7f0e1c100a90 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.669+0000 7f0e137fe640 1 -- 172.21.15.78:0/2393433757 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c010650 con 0x7f0e1c100a90 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.669+0000 7f0e20d1a640 1 -- 172.21.15.78:0/2393433757 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c06a790 con 0x7f0e1c100a90 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.673+0000 7f0e137fe640 1 -- 172.21.15.78:0/2393433757 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0e0c004020 con 0x7f0e1c100a90 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.673+0000 7f0e230f3740 1 Processor -- start 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e230f3740 1 -- start start 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x559627400090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x559627400a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627413b80 0x559627415f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e230f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5596273662f0 con 0x5596274005d0 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e230f3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55962730dcf0 con 0x559627401b40 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e230f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5596273622b0 con 0x559627413b80 2024-09-16T00:17:10.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e13fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627413b80 0x559627415f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e13fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627413b80 0x559627415f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:43838/0 (socket says 172.21.15.78:43838) 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e13fff640 1 -- 172.21.15.78:0/1048842887 learned_addr learned my addr 172.21.15.78:0/1048842887 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e137fe640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e080a32c0 0x7f0e080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.674+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x559627400a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.675+0000 7f0e1ad9d640 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x559627400090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.675+0000 7f0e137fe640 1 -- 172.21.15.78:0/2393433757 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0e0c0cfe90 con 0x7f0e1c100a90 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.675+0000 7f0e1ad9d640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e080a32c0 0x7f0e080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.675+0000 7f0e13fff640 1 -- 172.21.15.78:0/1048842887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 msgr2=0x559627400090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.675+0000 7f0e13fff640 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x559627400090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.675+0000 7f0e13fff640 1 -- 172.21.15.78:0/1048842887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 msgr2=0x559627400a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.675+0000 7f0e13fff640 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x559627400a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.675+0000 7f0e13fff640 1 -- 172.21.15.78:0/1048842887 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596272f23a0 con 0x559627413b80 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.676+0000 7f0e13fff640 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627413b80 0x559627415f70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e0402a890 tx=0x7f0e04005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebd8ce541e446456 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.676+0000 7f0e10ff9640 1 -- 172.21.15.78:0/1048842887 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04041020 con 0x559627413b80 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.676+0000 7f0e10ff9640 1 -- 172.21.15.78:0/1048842887 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e04030e40 con 0x559627413b80 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.676+0000 7f0e10ff9640 1 -- 172.21.15.78:0/1048842887 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04037040 con 0x559627413b80 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e230f3740 1 -- 172.21.15.78:0/1048842887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627413b80 msgr2=0x559627415f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e230f3740 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627413b80 0x559627415f70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e0402a890 tx=0x7f0e04005b20 comp rx=0 tx=0).stop 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e1ad9d640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e080a32c0 0x7f0e080a5780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005c40 tx=0x7f0e14009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e230f3740 1 -- 172.21.15.78:0/1048842887 shutdown_connections 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e230f3740 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627413b80 0x559627415f70 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e0402a890 tx=0x7f0e04005b20 comp rx=0 tx=0).stop 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e230f3740 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x559627400a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e230f3740 1 --2- 172.21.15.78:0/1048842887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x559627400090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e230f3740 1 -- 172.21.15.78:0/1048842887 >> 172.21.15.78:0/1048842887 conn(0x5596273c9310 msgr2=0x5596273fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.677+0000 7f0e230f3740 1 -- 172.21.15.78:0/1048842887 shutdown_connections 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.678+0000 7f0e230f3740 1 -- 172.21.15.78:0/1048842887 wait complete. 2024-09-16T00:17:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.678+0000 7f0e230f3740 1 -- start start 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.678+0000 7f0e230f3740 1 Processor -- start 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.679+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x5596274109c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.679+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x559627410f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.679+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627411440 0x55962741fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.679+0000 7f0e230f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5596274168c0 con 0x5596274005d0 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.679+0000 7f0e230f3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559627321060 con 0x559627401b40 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.679+0000 7f0e230f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559627416a40 con 0x559627411440 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.680+0000 7f0e230f3740 10 client.0 _ll_register_callbacks cb 0x5596270748c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.680+0000 7f0e1ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627411440 0x55962741fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.680+0000 7f0e1a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x5596274109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.680+0000 7f0e1a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x5596274109c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37540/0 (socket says 172.21.15.78:37540) 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.680+0000 7f0e1a59c640 1 -- 172.21.15.78:0/2211847805 learned_addr learned my addr 172.21.15.78:0/2211847805 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.681+0000 7f0e1a59c640 1 -- 172.21.15.78:0/2211847805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627411440 msgr2=0x55962741fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.681+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627411440 0x55962741fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.681+0000 7f0e1a59c640 1 -- 172.21.15.78:0/2211847805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 msgr2=0x559627410f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.681+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x559627410f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.681+0000 7f0e1a59c640 1 -- 172.21.15.78:0/2211847805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596270d73c0 con 0x5596274005d0 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.681+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x5596274109c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c6a0 tx=0x7f0e0c09c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417fe2e76015158e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.682+0000 7f0e01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.682+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2211847805 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0a4dc0 con 0x5596274005d0 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.682+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2211847805 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0e0c003b10 con 0x5596274005d0 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.682+0000 7f0e230f3740 1 -- 172.21.15.78:0/2211847805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e04030e40 con 0x5596274005d0 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.683+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2211847805 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0e0c0037a0 con 0x5596274005d0 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.683+0000 7f0e230f3740 10 client.16089 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.683+0000 7f0e230f3740 1 -- 172.21.15.78:0/2211847805 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5596274228d0 con 0x5596274005d0 2024-09-16T00:17:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.684+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2211847805 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e04030e40 con 0x5596274005d0 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.684+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2211847805 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0e0c0a0120 con 0x5596274005d0 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.684+0000 7f0e230f3740 10 client.16089 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.684+0000 7f0e230f3740 10 client.16089 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.684+0000 7f0e230f3740 20 client.16089 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.684+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559627424220 0x5596274266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.684+0000 7f0e230f3740 4 client.16089 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.684+0000 7f0e230f3740 1 -- 172.21.15.78:0/2211847805 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559627373860 con 0x559627424220 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.687+0000 7f0e13fff640 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559627424220 0x5596274266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.706+0000 7f0e13fff640 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559627424220 0x5596274266b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=999a950f80f86e15 server_cookie=9d27f0edddd7db8b in_seq=0 out_seq=0 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.706+0000 7f0e01ffb640 10 client.16089 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.707+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2211847805 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559627373860 con 0x559627424220 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.707+0000 7f0e01ffb640 10 client.16089 handle_command_reply: tid=0 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.707+0000 7f0e230f3740 1 client.16089 shutdown 2024-09-16T00:17:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.710+0000 7f0e230f3740 1 -- 172.21.15.78:0/2211847805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 msgr2=0x5596274109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.710+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x5596274109c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c6a0 tx=0x7f0e0c09c850 comp rx=0 tx=0).stop 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.711+0000 7f0e230f3740 1 -- 172.21.15.78:0/2211847805 shutdown_connections 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.711+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559627424220 0x5596274266b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.711+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559627411440 0x55962741fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.711+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x559627410f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.711+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2211847805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5596274005d0 0x5596274109c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.711+0000 7f0e230f3740 1 -- 172.21.15.78:0/2211847805 >> 172.21.15.78:0/2211847805 conn(0x5596273c9310 msgr2=0x559627402960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.713+0000 7f0e01ffb640 0 client.16089 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.719+0000 7f0e230f3740 1 -- 172.21.15.78:0/2211847805 shutdown_connections 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.719+0000 7f0e230f3740 1 -- 172.21.15.78:0/2211847805 wait complete. 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.719+0000 7f0e230f3740 20 client.16089 trim_cache size 0 max 16384 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.746+0000 7f0e230f3740 1 Processor -- start 2024-09-16T00:17:10.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.747+0000 7f0e230f3740 1 -- start start 2024-09-16T00:17:10.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.747+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 0x55962741fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.747+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x5596274200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.747+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962740f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.747+0000 7f0e230f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1aca60 con 0x559627411440 2024-09-16T00:17:10.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.747+0000 7f0e230f3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1ae5a0 con 0x559627401b40 2024-09-16T00:17:10.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.747+0000 7f0e230f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1ad130 con 0x5596274005d0 2024-09-16T00:17:10.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.747+0000 7f0e1ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962740f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.748+0000 7f0e1ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962740f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37542/0 (socket says 172.21.15.78:37542) 2024-09-16T00:17:10.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.748+0000 7f0e1ad9d640 1 -- 172.21.15.78:0/1738968033 learned_addr learned my addr 172.21.15.78:0/1738968033 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:10.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.748+0000 7f0e13fff640 1 --2- 172.21.15.78:0/1738968033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x5596274200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.748+0000 7f0e1ad9d640 1 -- 172.21.15.78:0/1738968033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 msgr2=0x55962741fb60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:17:10.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.748+0000 7f0e1ad9d640 1 --2- 172.21.15.78:0/1738968033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 0x55962741fb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.748+0000 7f0e1ad9d640 1 -- 172.21.15.78:0/1738968033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 msgr2=0x5596274200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:10.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.748+0000 7f0e1ad9d640 1 --2- 172.21.15.78:0/1738968033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x5596274200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.748+0000 7f0e1ad9d640 1 -- 172.21.15.78:0/1738968033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596273fd960 con 0x559627411440 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.749+0000 7f0e1ad9d640 1 --2- 172.21.15.78:0/1738968033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962740f290 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400fbd0 tx=0x7f0e14005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c30c92f1d33c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.750+0000 7f0e03fff640 1 -- 172.21.15.78:0/1738968033 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1402d070 con 0x559627411440 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.750+0000 7f0e03fff640 1 -- 172.21.15.78:0/1738968033 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e14028410 con 0x559627411440 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.750+0000 7f0e03fff640 1 -- 172.21.15.78:0/1738968033 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1401d640 con 0x559627411440 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.751+0000 7f0e230f3740 1 -- 172.21.15.78:0/1738968033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 msgr2=0x55962740f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.752+0000 7f0e230f3740 1 --2- 172.21.15.78:0/1738968033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962740f290 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400fbd0 tx=0x7f0e14005bd0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.752+0000 7f0e230f3740 1 -- 172.21.15.78:0/1738968033 shutdown_connections 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.752+0000 7f0e230f3740 1 --2- 172.21.15.78:0/1738968033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962740f290 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.752+0000 7f0e230f3740 1 --2- 172.21.15.78:0/1738968033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x5596274200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.752+0000 7f0e230f3740 1 --2- 172.21.15.78:0/1738968033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 0x55962741fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.752+0000 7f0e230f3740 1 -- 172.21.15.78:0/1738968033 >> 172.21.15.78:0/1738968033 conn(0x5596273c9310 msgr2=0x55962740bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.753+0000 7f0e230f3740 1 -- 172.21.15.78:0/1738968033 shutdown_connections 2024-09-16T00:17:10.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.753+0000 7f0e230f3740 1 -- 172.21.15.78:0/1738968033 wait complete. 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.753+0000 7f0e230f3740 1 -- start start 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.753+0000 7f0e230f3740 1 Processor -- start 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.754+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 0x55962740e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.754+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x55962741bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.754+0000 7f0e230f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962741c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.754+0000 7f0e13fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x55962741bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.754+0000 7f0e13fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x55962741bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48742/0 (socket says 172.21.15.78:48742) 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.754+0000 7f0e13fff640 1 -- 172.21.15.78:0/2900429063 learned_addr learned my addr 172.21.15.78:0/2900429063 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 0x55962740e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1aded0 con 0x559627411440 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1ad800 con 0x559627401b40 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c1aaa60 con 0x5596274005d0 2024-09-16T00:17:10.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e230f3740 10 client.0 _ll_register_callbacks cb 0x7f0e0c09bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e13fff640 1 -- 172.21.15.78:0/2900429063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 msgr2=0x55962740e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e13fff640 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 0x55962740e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e13fff640 1 -- 172.21.15.78:0/2900429063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 msgr2=0x55962741c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e13fff640 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962741c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e13fff640 1 -- 172.21.15.78:0/2900429063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962740cce0 con 0x559627401b40 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.755+0000 7f0e13fff640 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x55962741bcd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e04004650 tx=0x7f0e0402a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7caaf381843bc06 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.756+0000 7f0e01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.757+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2900429063 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04053060 con 0x559627401b40 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.757+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2900429063 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0e04004040 con 0x559627401b40 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.757+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e14028410 con 0x559627401b40 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.758+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2900429063 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0e04037040 con 0x559627401b40 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.759+0000 7f0e230f3740 10 client.25523 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.759+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55962748e310 con 0x559627401b40 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2900429063 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e14028410 con 0x559627401b40 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2900429063 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0e04005140 con 0x559627401b40 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e230f3740 10 client.25523 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e230f3740 10 client.25523 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e230f3740 20 client.25523 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55962748e5f0 0x5596274909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e230f3740 4 client.25523 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55962736dd70 con 0x55962748e5f0 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.761+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55962748e5f0 0x5596274909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.762+0000 7f0e1a59c640 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55962748e5f0 0x5596274909e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7e8caf1c9285062 server_cookie=501f76bb42db9d7c in_seq=0 out_seq=0 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.762+0000 7f0e01ffb640 10 client.25523 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.763+0000 7f0e01ffb640 1 -- 172.21.15.78:0/2900429063 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55962736dd70 con 0x55962748e5f0 2024-09-16T00:17:10.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.763+0000 7f0e01ffb640 10 client.25523 handle_command_reply: tid=0 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.763+0000 7f0e230f3740 1 client.25523 shutdown 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.765+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 msgr2=0x55962741bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.765+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x55962741bcd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e04004650 tx=0x7f0e0402a370 comp rx=0 tx=0).stop 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.765+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 shutdown_connections 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.765+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559627411440 0x55962741c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.765+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559627401b40 0x55962741bcd0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e04004650 tx=0x7f0e0402a370 comp rx=0 tx=0).stop 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.765+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55962748e5f0 0x5596274909e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.765+0000 7f0e230f3740 1 --2- 172.21.15.78:0/2900429063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596274005d0 0x55962740e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.765+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 >> 172.21.15.78:0/2900429063 conn(0x5596273c9310 msgr2=0x55962740f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.766+0000 7f0e01ffb640 0 client.25523 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.766+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 shutdown_connections 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.766+0000 7f0e230f3740 1 -- 172.21.15.78:0/2900429063 wait complete. 2024-09-16T00:17:10.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.768+0000 7f0e230f3740 20 client.25523 trim_cache size 0 max 16384 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.769+0000 7f0e20d1a640 1 -- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e080a32c0 msgr2=0x7f0e080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.769+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e080a32c0 0x7f0e080a5780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005c40 tx=0x7f0e14009500 comp rx=0 tx=0).stop 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.769+0000 7f0e20d1a640 1 -- 172.21.15.78:0/2393433757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e1c100a90 msgr2=0x7f0e1c197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.769+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e1c100a90 0x7f0e1c197350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00e9c0 tx=0x7f0e0c00ee80 comp rx=0 tx=0).stop 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.769+0000 7f0e20d1a640 1 -- 172.21.15.78:0/2393433757 shutdown_connections 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.769+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e080a32c0 0x7f0e080a5780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.769+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e1c106a90 0x7f0e1c19bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.769+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e1c101430 0x7f0e1c197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.770+0000 7f0e20d1a640 1 --2- 172.21.15.78:0/2393433757 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e1c100a90 0x7f0e1c197350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.770+0000 7f0e20d1a640 1 -- 172.21.15.78:0/2393433757 >> 172.21.15.78:0/2393433757 conn(0x7f0e1c0fc7d0 msgr2=0x7f0e1c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.770+0000 7f0e20d1a640 1 -- 172.21.15.78:0/2393433757 shutdown_connections 2024-09-16T00:17:10.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:09.770+0000 7f0e20d1a640 1 -- 172.21.15.78:0/2393433757 wait complete. 2024-09-16T00:17:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 190.0 s, 1391.0 tps, lat 70.634 ms stddev 129.223 2024-09-16T00:17:11.874 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:11 smithi078 ceph-mon[31157]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 822 op/s 2024-09-16T00:17:11.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:11 smithi193 ceph-mon[35484]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 822 op/s 2024-09-16T00:17:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:11 smithi148 ceph-mon[35599]: pgmap v474: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 822 op/s 2024-09-16T00:17:13.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:13 smithi193 ceph-mon[35484]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-09-16T00:17:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:13 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:17:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:13 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:17:13.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:13 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:13 smithi148 ceph-mon[35599]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-09-16T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:13 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:13 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:17:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:13 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:17:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:13 smithi078 ceph-mon[31157]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-09-16T00:17:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:13 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:17:14.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:13 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:17:14.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:13 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:17:14.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:17:15.157 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:15.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.789+0000 7f09c237a640 1 -- 172.21.15.78:0/1933747998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 msgr2=0x7f09bc100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.789+0000 7f09c237a640 1 --2- 172.21.15.78:0/1933747998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 0x7f09bc100ca0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f09b001a910 tx=0x7f09b003b430 comp rx=0 tx=0).stop 2024-09-16T00:17:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.789+0000 7f09c237a640 1 -- 172.21.15.78:0/1933747998 shutdown_connections 2024-09-16T00:17:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.789+0000 7f09c237a640 1 --2- 172.21.15.78:0/1933747998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09bc105ec0 0x7f09bc106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.789+0000 7f09c237a640 1 --2- 172.21.15.78:0/1933747998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09bc1011e0 0x7f09bc105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.789+0000 7f09c237a640 1 --2- 172.21.15.78:0/1933747998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 0x7f09bc100ca0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.789+0000 7f09c237a640 1 -- 172.21.15.78:0/1933747998 >> 172.21.15.78:0/1933747998 conn(0x7f09bc0fc7d0 msgr2=0x7f09bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.796+0000 7f09c237a640 1 -- 172.21.15.78:0/1933747998 shutdown_connections 2024-09-16T00:17:15.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.797+0000 7f09c237a640 1 -- 172.21.15.78:0/1933747998 wait complete. 2024-09-16T00:17:15.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.798+0000 7f09c237a640 1 Processor -- start 2024-09-16T00:17:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.798+0000 7f09c237a640 1 -- start start 2024-09-16T00:17:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09c237a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 0x7f09bc114e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09c237a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09bc1011e0 0x7f09bc110ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09c237a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09bc105ec0 0x7f09bc111410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09c237a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc1098c0 con 0x7f09bc1008c0 2024-09-16T00:17:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09c237a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc109740 con 0x7f09bc105ec0 2024-09-16T00:17:15.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09c237a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc109a40 con 0x7f09bc1011e0 2024-09-16T00:17:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 0x7f09bc114e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 0x7f09bc114e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37576/0 (socket says 172.21.15.78:37576) 2024-09-16T00:17:15.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09bbfff640 1 -- 172.21.15.78:0/2464175688 learned_addr learned my addr 172.21.15.78:0/2464175688 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09bb7fe640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09bc1011e0 0x7f09bc110ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.799+0000 7f09c0917640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09bc105ec0 0x7f09bc111410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09bbfff640 1 -- 172.21.15.78:0/2464175688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09bc1011e0 msgr2=0x7f09bc110ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09bbfff640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09bc1011e0 0x7f09bc110ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09bbfff640 1 -- 172.21.15.78:0/2464175688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09bc105ec0 msgr2=0x7f09bc111410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09bbfff640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09bc105ec0 0x7f09bc111410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09bbfff640 1 -- 172.21.15.78:0/2464175688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09bc111ac0 con 0x7f09bc1008c0 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09bbfff640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 0x7f09bc114e30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f09b003ba20 tx=0x7f09b0004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09b97fa640 1 -- 172.21.15.78:0/2464175688 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0012020 con 0x7f09bc1008c0 2024-09-16T00:17:15.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09c237a640 1 -- 172.21.15.78:0/2464175688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09bc06bc10 con 0x7f09bc1008c0 2024-09-16T00:17:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.800+0000 7f09c237a640 1 -- 172.21.15.78:0/2464175688 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09bc06c140 con 0x7f09bc1008c0 2024-09-16T00:17:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.802+0000 7f09b97fa640 1 -- 172.21.15.78:0/2464175688 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b0005360 con 0x7f09bc1008c0 2024-09-16T00:17:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.802+0000 7f09b97fa640 1 -- 172.21.15.78:0/2464175688 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0046030 con 0x7f09bc1008c0 2024-09-16T00:17:15.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.804+0000 7f09c4753740 1 Processor -- start 2024-09-16T00:17:15.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.807+0000 7f09c4753740 1 -- start start 2024-09-16T00:17:15.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.808+0000 7f09c4753740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019d5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.808+0000 7f09c4753740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.808+0000 7f09c4753740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.808+0000 7f09c4753740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fd0193b2f0 con 0x55fd019d6b40 2024-09-16T00:17:15.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.808+0000 7f09c4753740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fd018e2cf0 con 0x55fd019e0370 2024-09-16T00:17:15.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.808+0000 7f09c4753740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd019372b0 con 0x55fd019d55d0 2024-09-16T00:17:15.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.809+0000 7f09c0917640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019d5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.809+0000 7f09c0917640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019d5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37578/0 (socket says 172.21.15.78:37578) 2024-09-16T00:17:15.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.810+0000 7f09c0917640 1 -- 172.21.15.78:0/2420608964 learned_addr learned my addr 172.21.15.78:0/2420608964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:15.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.810+0000 7f09bb7fe640 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.811+0000 7f09b97fa640 1 -- 172.21.15.78:0/2464175688 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f09b0015020 con 0x7f09bc1008c0 2024-09-16T00:17:15.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.812+0000 7f09b97fa640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09a40a3300 0x7f09a40a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.814+0000 7f09b97fa640 1 -- 172.21.15.78:0/2464175688 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f09b00ba6b0 con 0x7f09bc1008c0 2024-09-16T00:17:15.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.814+0000 7f09bbfff640 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.814+0000 7f09c0917640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09a40a3300 0x7f09a40a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.814+0000 7f09c0917640 1 -- 172.21.15.78:0/2420608964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 msgr2=0x55fd019d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:15.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.814+0000 7f09c0917640 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.814+0000 7f09c0917640 1 -- 172.21.15.78:0/2420608964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 msgr2=0x55fd019e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:15.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.814+0000 7f09c0917640 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.814+0000 7f09c0917640 1 -- 172.21.15.78:0/2420608964 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd018c73a0 con 0x55fd019d6b40 2024-09-16T00:17:15.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.816+0000 7f09c0917640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09a40a3300 0x7f09a40a57c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f09b400cfd0 tx=0x7f09b4019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:15.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.816+0000 7f09c0917640 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019d5090 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f09b400ca80 tx=0x7f09b400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e362b6ef3e2bbba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.818+0000 7f09a27fc640 1 -- 172.21.15.78:0/2420608964 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b401e070 con 0x55fd019d6b40 2024-09-16T00:17:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.819+0000 7f09a27fc640 1 -- 172.21.15.78:0/2420608964 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4016040 con 0x55fd019d6b40 2024-09-16T00:17:15.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.819+0000 7f09a27fc640 1 -- 172.21.15.78:0/2420608964 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4015070 con 0x55fd019d6b40 2024-09-16T00:17:15.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.823+0000 7f09c4753740 1 -- 172.21.15.78:0/2420608964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 msgr2=0x55fd019d5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.823+0000 7f09c4753740 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019d5090 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f09b400ca80 tx=0x7f09b400cf40 comp rx=0 tx=0).stop 2024-09-16T00:17:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.824+0000 7f09c4753740 1 -- 172.21.15.78:0/2420608964 shutdown_connections 2024-09-16T00:17:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.824+0000 7f09c4753740 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.824+0000 7f09c4753740 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.824+0000 7f09c4753740 1 --2- 172.21.15.78:0/2420608964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019d5090 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.825+0000 7f09c4753740 1 -- 172.21.15.78:0/2420608964 >> 172.21.15.78:0/2420608964 conn(0x55fd0199e310 msgr2=0x55fd019d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:15.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.828+0000 7f09c4753740 1 -- 172.21.15.78:0/2420608964 shutdown_connections 2024-09-16T00:17:15.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.832+0000 7f09c4753740 1 -- 172.21.15.78:0/2420608964 wait complete. 2024-09-16T00:17:15.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.832+0000 7f09c4753740 1 -- start start 2024-09-16T00:17:15.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.833+0000 7f09c4753740 1 Processor -- start 2024-09-16T00:17:15.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.833+0000 7f09c4753740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.833+0000 7f09c4753740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019d6b40 0x55fd019dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.833+0000 7f09c4753740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019e0370 0x55fd019dd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.833+0000 7f09c4753740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fd019e3010 con 0x55fd019d55d0 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.833+0000 7f09c4753740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fd018f6060 con 0x55fd019d6b40 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.833+0000 7f09c4753740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd019e3190 con 0x55fd019e0370 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.833+0000 7f09c4753740 10 client.0 _ll_register_callbacks cb 0x55fd016498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09c0917640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019e0370 0x55fd019dd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09c0917640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019e0370 0x55fd019dd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:43894/0 (socket says 172.21.15.78:43894) 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09c0917640 1 -- 172.21.15.78:0/1755193691 learned_addr learned my addr 172.21.15.78:0/1755193691 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09bb7fe640 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019d6b40 0x55fd019dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09c0917640 1 -- 172.21.15.78:0/1755193691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019d6b40 msgr2=0x55fd019dd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09c0917640 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019d6b40 0x55fd019dd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09c0917640 1 -- 172.21.15.78:0/1755193691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 msgr2=0x55fd019dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09c0917640 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.834+0000 7f09c0917640 1 -- 172.21.15.78:0/1755193691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd016ac3c0 con 0x55fd019e0370 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.835+0000 7f09c0917640 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019e0370 0x55fd019dd640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c080 tx=0x7f09b400aa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=788959c620a64ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.835+0000 7f099b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.835+0000 7f099b7fe640 1 -- 172.21.15.78:0/1755193691 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4015050 con 0x55fd019e0370 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.836+0000 7f09c4753740 1 -- 172.21.15.78:0/1755193691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09b4016040 con 0x55fd019e0370 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.836+0000 7f099b7fe640 1 -- 172.21.15.78:0/1755193691 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f09b4023650 con 0x55fd019e0370 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.836+0000 7f099b7fe640 1 -- 172.21.15.78:0/1755193691 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f09b4038050 con 0x55fd019e0370 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.836+0000 7f09c4753740 10 client.25417 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:15.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.836+0000 7f09c4753740 1 -- 172.21.15.78:0/1755193691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fd019eeb70 con 0x55fd019e0370 2024-09-16T00:17:15.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.838+0000 7f099b7fe640 1 -- 172.21.15.78:0/1755193691 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4016040 con 0x55fd019e0370 2024-09-16T00:17:15.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.838+0000 7f099b7fe640 1 -- 172.21.15.78:0/1755193691 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f09b4033030 con 0x55fd019e0370 2024-09-16T00:17:15.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.838+0000 7f09c4753740 10 client.25417 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:15.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.838+0000 7f09c4753740 10 client.25417 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:17:15.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.838+0000 7f09c4753740 20 client.25417 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:15.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.838+0000 7f09c4753740 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55fd019f0030 0x55fd019f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.838+0000 7f09c4753740 4 client.25417 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:17:15.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.838+0000 7f09c4753740 1 -- 172.21.15.78:0/1755193691 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd01948860 con 0x55fd019f0030 2024-09-16T00:17:15.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.841+0000 7f09bbfff640 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55fd019f0030 0x55fd019f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.848+0000 7f09bbfff640 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55fd019f0030 0x55fd019f2420 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3a942f2d42faab0a server_cookie=1c34bf16a508603c in_seq=0 out_seq=0 2024-09-16T00:17:15.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.848+0000 7f099b7fe640 10 client.25417 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:17:15.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.850+0000 7f099b7fe640 1 -- 172.21.15.78:0/1755193691 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd01948860 con 0x55fd019f0030 2024-09-16T00:17:15.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.850+0000 7f099b7fe640 10 client.25417 handle_command_reply: tid=0 2024-09-16T00:17:15.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.850+0000 7f09c4753740 1 client.25417 shutdown 2024-09-16T00:17:15.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 -- 172.21.15.78:0/1755193691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019e0370 msgr2=0x55fd019dd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019e0370 0x55fd019dd640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c080 tx=0x7f09b400aa00 comp rx=0 tx=0).stop 2024-09-16T00:17:15.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 -- 172.21.15.78:0/1755193691 shutdown_connections 2024-09-16T00:17:15.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019e0370 0x55fd019dd640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55fd019f0030 0x55fd019f2420 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019d6b40 0x55fd019dd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 --2- 172.21.15.78:0/1755193691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f099b7fe640 0 client.25417 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:17:15.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 -- 172.21.15.78:0/1755193691 >> 172.21.15.78:0/1755193691 conn(0x55fd0199e310 msgr2=0x55fd019d7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:15.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 -- 172.21.15.78:0/1755193691 shutdown_connections 2024-09-16T00:17:15.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.853+0000 7f09c4753740 1 -- 172.21.15.78:0/1755193691 wait complete. 2024-09-16T00:17:15.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.854+0000 7f09c4753740 20 client.25417 trim_cache size 0 max 16384 2024-09-16T00:17:15.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 195.0 s, 2282.2 tps, lat 44.760 ms stddev 58.338 2024-09-16T00:17:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.874+0000 7f09c4753740 1 Processor -- start 2024-09-16T00:17:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.874+0000 7f09c4753740 1 -- start start 2024-09-16T00:17:15.875 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:15 smithi078 ceph-mon[31157]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-09-16T00:17:15.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.874+0000 7f09c4753740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019ec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.874+0000 7f09c4753740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c4753740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c4753740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09a800fbb0 con 0x55fd019d6b40 2024-09-16T00:17:15.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c4753740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09a80116f0 con 0x55fd019e0370 2024-09-16T00:17:15.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c4753740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8010280 con 0x55fd019d55d0 2024-09-16T00:17:15.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:43898/0 (socket says 172.21.15.78:43898) 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09bbfff640 1 -- 172.21.15.78:0/4062038227 learned_addr learned my addr 172.21.15.78:0/4062038227 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c0917640 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09bb7fe640 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c0917640 1 -- 172.21.15.78:0/4062038227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 msgr2=0x55fd019ec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c0917640 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019ec310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c0917640 1 -- 172.21.15.78:0/4062038227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 msgr2=0x55fd019f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c0917640 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c0917640 1 -- 172.21.15.78:0/4062038227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd019ddfd0 con 0x55fd019e0370 2024-09-16T00:17:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.875+0000 7f09c0917640 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019f80d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f09b4002230 tx=0x7f09b400c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95101de055f4e8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:15.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.876+0000 7f09a1ffb640 1 -- 172.21.15.78:0/4062038227 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4015070 con 0x55fd019e0370 2024-09-16T00:17:15.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.876+0000 7f09a1ffb640 1 -- 172.21.15.78:0/4062038227 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4016040 con 0x55fd019e0370 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.876+0000 7f09a1ffb640 1 -- 172.21.15.78:0/4062038227 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b401e040 con 0x55fd019e0370 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 -- 172.21.15.78:0/4062038227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 msgr2=0x55fd019f80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019f80d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f09b4002230 tx=0x7f09b400c770 comp rx=0 tx=0).stop 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 -- 172.21.15.78:0/4062038227 shutdown_connections 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019f80d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d6b40 0x55fd019f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 --2- 172.21.15.78:0/4062038227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d55d0 0x55fd019ec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 -- 172.21.15.78:0/4062038227 >> 172.21.15.78:0/4062038227 conn(0x55fd0199e310 msgr2=0x55fd019db900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 -- 172.21.15.78:0/4062038227 shutdown_connections 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.877+0000 7f09c4753740 1 -- 172.21.15.78:0/4062038227 wait complete. 2024-09-16T00:17:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 1 -- start start 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 1 Processor -- start 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d6b40 0x55fd019e4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019e4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8011020 con 0x55fd019d55d0 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8010950 con 0x55fd019e0370 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09a800db90 con 0x55fd019d6b40 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09c4753740 10 client.0 _ll_register_callbacks cb 0x7f09b0041770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.878+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019e4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37602/0 (socket says 172.21.15.78:37602) 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bbfff640 1 -- 172.21.15.78:0/2129529115 learned_addr learned my addr 172.21.15.78:0/2129529115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bb7fe640 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d6b40 0x55fd019e4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09c0917640 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019e4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bb7fe640 1 -- 172.21.15.78:0/2129529115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 msgr2=0x55fd019e4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bb7fe640 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019e4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bb7fe640 1 -- 172.21.15.78:0/2129529115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 msgr2=0x55fd019e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bb7fe640 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019e4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bb7fe640 1 -- 172.21.15.78:0/2129529115 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd01a59be0 con 0x55fd019d6b40 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bbfff640 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f09bb7fe640 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d6b40 0x55fd019e4780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55fd019de670 tx=0x7f09ac011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=466dbba802c08567 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f099b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.879+0000 7f099b7fe640 1 -- 172.21.15.78:0/2129529115 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac010b70 con 0x55fd019d6b40 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.880+0000 7f09c4753740 1 -- 172.21.15.78:0/2129529115 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09b4016040 con 0x55fd019d6b40 2024-09-16T00:17:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.880+0000 7f099b7fe640 1 -- 172.21.15.78:0/2129529115 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f09ac010d10 con 0x55fd019d6b40 2024-09-16T00:17:15.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.880+0000 7f099b7fe640 1 -- 172.21.15.78:0/2129529115 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f09ac0204f0 con 0x55fd019d6b40 2024-09-16T00:17:15.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.880+0000 7f09c4753740 10 client.25423 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:15.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.880+0000 7f09c4753740 1 -- 172.21.15.78:0/2129529115 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fd01a5af30 con 0x55fd019d6b40 2024-09-16T00:17:15.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.881+0000 7f099b7fe640 1 -- 172.21.15.78:0/2129529115 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4016040 con 0x55fd019d6b40 2024-09-16T00:17:15.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.881+0000 7f099b7fe640 1 -- 172.21.15.78:0/2129529115 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f09ac01b420 con 0x55fd019d6b40 2024-09-16T00:17:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f09c4753740 10 client.25423 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f09c4753740 10 client.25423 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:17:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f09c4753740 20 client.25423 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f09c4753740 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55fd01a5b690 0x55fd01a5da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f09c4753740 4 client.25423 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f09c4753740 1 -- 172.21.15.78:0/2129529115 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd01942d70 con 0x55fd01a5b690 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f09bbfff640 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55fd01a5b690 0x55fd01a5da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f09bbfff640 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55fd01a5b690 0x55fd01a5da80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=59603d4f1532881c server_cookie=67e950916dad7164 in_seq=0 out_seq=0 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.882+0000 7f099b7fe640 10 client.25423 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f099b7fe640 1 -- 172.21.15.78:0/2129529115 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd01942d70 con 0x55fd01a5b690 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f099b7fe640 10 client.25423 handle_command_reply: tid=0 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 client.25423 shutdown 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 -- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d6b40 msgr2=0x55fd019e4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d6b40 0x55fd019e4780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55fd019de670 tx=0x7f09ac011b50 comp rx=0 tx=0).stop 2024-09-16T00:17:15.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 -- 172.21.15.78:0/2129529115 shutdown_connections 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55fd01a5b690 0x55fd01a5da80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd019e0370 0x55fd019e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd019d6b40 0x55fd019e4780 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 --2- 172.21.15.78:0/2129529115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fd019d55d0 0x55fd019e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f09c4753740 1 -- 172.21.15.78:0/2129529115 >> 172.21.15.78:0/2129529115 conn(0x55fd0199e310 msgr2=0x55fd019de890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.883+0000 7f099b7fe640 0 client.25423 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c4753740 1 -- 172.21.15.78:0/2129529115 shutdown_connections 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c4753740 1 -- 172.21.15.78:0/2129529115 wait complete. 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c4753740 20 client.25423 trim_cache size 0 max 16384 2024-09-16T00:17:15.888 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:17:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 -- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09a40a3300 msgr2=0x7f09a40a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09a40a3300 0x7f09a40a57c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f09b400cfd0 tx=0x7f09b4019000 comp rx=0 tx=0).stop 2024-09-16T00:17:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 -- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 msgr2=0x7f09bc114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 0x7f09bc114e30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f09b003ba20 tx=0x7f09b0004c60 comp rx=0 tx=0).stop 2024-09-16T00:17:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 -- 172.21.15.78:0/2464175688 shutdown_connections 2024-09-16T00:17:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09a40a3300 0x7f09a40a57c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09bc105ec0 0x7f09bc111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09bc1011e0 0x7f09bc110ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 --2- 172.21.15.78:0/2464175688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc1008c0 0x7f09bc114e30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 -- 172.21.15.78:0/2464175688 >> 172.21.15.78:0/2464175688 conn(0x7f09bc0fc7d0 msgr2=0x7f09bc103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.884+0000 7f09c237a640 1 -- 172.21.15.78:0/2464175688 shutdown_connections 2024-09-16T00:17:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:15.885+0000 7f09c237a640 1 -- 172.21.15.78:0/2464175688 wait complete. 2024-09-16T00:17:15.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:15 smithi193 ceph-mon[35484]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-09-16T00:17:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:15 smithi148 ceph-mon[35599]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 MiB/s wr, 1.02k op/s 2024-09-16T00:17:17.337 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:17:17.746 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:17.916 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:17 smithi078 ceph-mon[31157]: pgmap v477: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-09-16T00:17:17.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:17 smithi193 ceph-mon[35484]: pgmap v477: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-09-16T00:17:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:17 smithi148 ceph-mon[35599]: pgmap v477: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.05k op/s 2024-09-16T00:17:18.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 -- 172.21.15.78:0/1523595728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd46811b080 msgr2=0x7fd46811d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:18.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 --2- 172.21.15.78:0/1523595728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd46811b080 0x7fd46811d470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd460009ca0 tx=0x7fd46002a980 comp rx=0 tx=0).stop 2024-09-16T00:17:18.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd465ffb640 1 -- 172.21.15.78:0/1523595728 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7fd46002e330 con 0x7fd46811b080 2024-09-16T00:17:18.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 -- 172.21.15.78:0/1523595728 shutdown_connections 2024-09-16T00:17:18.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 --2- 172.21.15.78:0/1523595728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd46811b080 0x7fd46811d470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 --2- 172.21.15.78:0/1523595728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd46807b0b0 0x7fd46807b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 --2- 172.21.15.78:0/1523595728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd468082d80 0x7fd46807ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 -- 172.21.15.78:0/1523595728 >> 172.21.15.78:0/1523595728 conn(0x7fd468077af0 msgr2=0x7fd468079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:18.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 -- 172.21.15.78:0/1523595728 shutdown_connections 2024-09-16T00:17:18.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.380+0000 7fd46d317640 1 -- 172.21.15.78:0/1523595728 wait complete. 2024-09-16T00:17:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.381+0000 7fd46d317640 1 Processor -- start 2024-09-16T00:17:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.381+0000 7fd46d317640 1 -- start start 2024-09-16T00:17:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd46d317640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd46807b0b0 0x7fd468073930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd46d317640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd468082d80 0x7fd468073e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd46d317640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd46811b080 0x7fd46806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd46d317640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd468120e00 con 0x7fd46811b080 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd46d317640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd468120c80 con 0x7fd468082d80 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd46d317640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd468120f80 con 0x7fd46807b0b0 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd46807b0b0 0x7fd468073930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd46807b0b0 0x7fd468073930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48722/0 (socket says 172.21.15.78:48722) 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd466ffd640 1 -- 172.21.15.78:0/3179996957 learned_addr learned my addr 172.21.15.78:0/3179996957 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd466ffd640 1 -- 172.21.15.78:0/3179996957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd468082d80 msgr2=0x7fd468073e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd466ffd640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd468082d80 0x7fd468073e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd466ffd640 1 -- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd46811b080 msgr2=0x7fd46806cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd466ffd640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd46811b080 0x7fd46806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.382+0000 7fd466ffd640 1 -- 172.21.15.78:0/3179996957 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46806d460 con 0x7fd46807b0b0 2024-09-16T00:17:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.383+0000 7fd466ffd640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd46807b0b0 0x7fd468073930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00d950 tx=0x7fd45c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.383+0000 7fd44ffff640 1 -- 172.21.15.78:0/3179996957 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c017070 con 0x7fd46807b0b0 2024-09-16T00:17:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.383+0000 7fd44ffff640 1 -- 172.21.15.78:0/3179996957 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45c009bb0 con 0x7fd46807b0b0 2024-09-16T00:17:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.384+0000 7fd44ffff640 1 -- 172.21.15.78:0/3179996957 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00f600 con 0x7fd46807b0b0 2024-09-16T00:17:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.384+0000 7fd46d317640 1 -- 172.21.15.78:0/3179996957 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46806d750 con 0x7fd46807b0b0 2024-09-16T00:17:18.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.384+0000 7fd46d317640 1 -- 172.21.15.78:0/3179996957 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd468071740 con 0x7fd46807b0b0 2024-09-16T00:17:18.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.387+0000 7fd44ffff640 1 -- 172.21.15.78:0/3179996957 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd45c010460 con 0x7fd46807b0b0 2024-09-16T00:17:18.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.387+0000 7fd46f6f0740 1 Processor -- start 2024-09-16T00:17:18.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd46f6f0740 1 -- start start 2024-09-16T00:17:18.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd44ffff640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4500a3380 0x7fd4500a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de36370 0x55704de38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55704dd912f0 con 0x55704de2cb40 2024-09-16T00:17:18.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55704dd38cf0 con 0x55704de36370 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55704dd8d2b0 con 0x55704de2b5d0 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de2b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49208/0 (socket says 172.21.15.78:49208) 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd466ffd640 1 -- 172.21.15.78:0/2977171802 learned_addr learned my addr 172.21.15.78:0/2977171802 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.388+0000 7fd4667fc640 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.389+0000 7fd4677fe640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4500a3380 0x7fd4500a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.389+0000 7fd4677fe640 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de36370 0x55704de38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.389+0000 7fd466ffd640 1 -- 172.21.15.78:0/2977171802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 msgr2=0x55704de2ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.389+0000 7fd466ffd640 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de2ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.389+0000 7fd466ffd640 1 -- 172.21.15.78:0/2977171802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de36370 msgr2=0x55704de38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.389+0000 7fd466ffd640 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de36370 0x55704de38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.389+0000 7fd466ffd640 1 -- 172.21.15.78:0/2977171802 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704dd1d3a0 con 0x55704de2cb40 2024-09-16T00:17:18.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.389+0000 7fd4677fe640 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de36370 0x55704de38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:17:18.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.392+0000 7fd44ffff640 1 -- 172.21.15.78:0/3179996957 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd45c0cf8d0 con 0x7fd46807b0b0 2024-09-16T00:17:18.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.392+0000 7fd4677fe640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4500a3380 0x7fd4500a5840 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd460009bf0 tx=0x7fd4600097e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:18.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.392+0000 7fd466ffd640 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de2b090 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0b1ef0 tx=0x7fd45c0afc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1f1735f3df0fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:18.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.392+0000 7fd44d7fa640 1 -- 172.21.15.78:0/2977171802 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c09ce60 con 0x55704de2cb40 2024-09-16T00:17:18.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.393+0000 7fd44d7fa640 1 -- 172.21.15.78:0/2977171802 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45c09c230 con 0x55704de2cb40 2024-09-16T00:17:18.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.393+0000 7fd44d7fa640 1 -- 172.21.15.78:0/2977171802 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c097a20 con 0x55704de2cb40 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.393+0000 7fd46f6f0740 1 -- 172.21.15.78:0/2977171802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 msgr2=0x55704de2b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.393+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de2b090 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0b1ef0 tx=0x7fd45c0afc90 comp rx=0 tx=0).stop 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 -- 172.21.15.78:0/2977171802 shutdown_connections 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de36370 0x55704de38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de2ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/2977171802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de2b090 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 -- 172.21.15.78:0/2977171802 >> 172.21.15.78:0/2977171802 conn(0x55704ddf4310 msgr2=0x55704de27c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 -- 172.21.15.78:0/2977171802 shutdown_connections 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 -- 172.21.15.78:0/2977171802 wait complete. 2024-09-16T00:17:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 -- start start 2024-09-16T00:17:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.394+0000 7fd46f6f0740 1 Processor -- start 2024-09-16T00:17:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de36370 0x55704de335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55704de39070 con 0x55704de2cb40 2024-09-16T00:17:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55704dd4c060 con 0x55704de2b5d0 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55704de391f0 con 0x55704de36370 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd46f6f0740 10 client.0 _ll_register_callbacks cb 0x55704da9f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de35d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48178/0 (socket says 172.21.15.78:48178) 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd466ffd640 1 -- 172.21.15.78:0/3560004719 learned_addr learned my addr 172.21.15.78:0/3560004719 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd4667fc640 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd466ffd640 1 -- 172.21.15.78:0/3560004719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de36370 msgr2=0x55704de335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd466ffd640 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de36370 0x55704de335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd466ffd640 1 -- 172.21.15.78:0/3560004719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 msgr2=0x55704de32fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd466ffd640 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de32fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.395+0000 7fd466ffd640 1 -- 172.21.15.78:0/3560004719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704db023c0 con 0x55704de2b5d0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.396+0000 7fd466ffd640 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de35d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd45c09bac0 tx=0x7fd45c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b47e62e8d93a8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.396+0000 7fd44aff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.396+0000 7fd44aff5640 1 -- 172.21.15.78:0/3560004719 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c09ce60 con 0x55704de2b5d0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.396+0000 7fd44aff5640 1 -- 172.21.15.78:0/3560004719 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd45c0981a0 con 0x55704de2b5d0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.396+0000 7fd46f6f0740 1 -- 172.21.15.78:0/3560004719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45c09c230 con 0x55704de2b5d0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.396+0000 7fd44aff5640 1 -- 172.21.15.78:0/3560004719 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd45c020030 con 0x55704de2b5d0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.396+0000 7fd46f6f0740 10 client.25541 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.396+0000 7fd46f6f0740 1 -- 172.21.15.78:0/3560004719 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55704de39070 con 0x55704de2b5d0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.397+0000 7fd44aff5640 1 -- 172.21.15.78:0/3560004719 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45c09c230 con 0x55704de2b5d0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.397+0000 7fd44aff5640 1 -- 172.21.15.78:0/3560004719 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd45c0b6cd0 con 0x55704de2b5d0 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.397+0000 7fd46f6f0740 10 client.25541 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.397+0000 7fd46f6f0740 10 client.25541 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:17:18.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.397+0000 7fd46f6f0740 20 client.25541 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:18.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.397+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55704de45bc0 0x55704de47fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.397+0000 7fd46f6f0740 4 client.25541 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:17:18.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.397+0000 7fd46f6f0740 1 -- 172.21.15.78:0/3560004719 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55704dd9e860 con 0x55704de45bc0 2024-09-16T00:17:18.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.398+0000 7fd4667fc640 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55704de45bc0 0x55704de47fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.398+0000 7fd44aff5640 10 client.25541 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.398+0000 7fd4667fc640 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55704de45bc0 0x55704de47fb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfd746879d54cd49 server_cookie=a105fb461b522698 in_seq=0 out_seq=0 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd44aff5640 1 -- 172.21.15.78:0/3560004719 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55704dd9e860 con 0x55704de45bc0 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd44aff5640 10 client.25541 handle_command_reply: tid=0 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 client.25541 shutdown 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 -- 172.21.15.78:0/3560004719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 msgr2=0x55704de35d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de35d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd45c09bac0 tx=0x7fd45c009f60 comp rx=0 tx=0).stop 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 -- 172.21.15.78:0/3560004719 shutdown_connections 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de36370 0x55704de335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de2cb40 0x55704de32fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55704de45bc0 0x55704de47fb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd44aff5640 0 client.25541 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:17:18.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/3560004719 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de35d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.399+0000 7fd46f6f0740 1 -- 172.21.15.78:0/3560004719 >> 172.21.15.78:0/3560004719 conn(0x55704ddf4310 msgr2=0x55704de2d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:18.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.400+0000 7fd46f6f0740 1 -- 172.21.15.78:0/3560004719 shutdown_connections 2024-09-16T00:17:18.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.400+0000 7fd46f6f0740 1 -- 172.21.15.78:0/3560004719 wait complete. 2024-09-16T00:17:18.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.400+0000 7fd46f6f0740 20 client.25541 trim_cache size 0 max 16384 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.420+0000 7fd46f6f0740 1 Processor -- start 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.420+0000 7fd46f6f0740 1 -- start start 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2cb40 0x55704de2e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 0x55704de2e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681c6310 con 0x55704de36370 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681c7e50 con 0x55704de2cb40 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681c69e0 con 0x55704de2b5d0 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de26c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48746/0 (socket says 172.21.15.78:48746) 2024-09-16T00:17:18.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd466ffd640 1 -- 172.21.15.78:0/964258304 learned_addr learned my addr 172.21.15.78:0/964258304 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd4677fe640 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 0x55704de2e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd4667fc640 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2cb40 0x55704de2e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd466ffd640 1 -- 172.21.15.78:0/964258304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2cb40 msgr2=0x55704de2e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd466ffd640 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2cb40 0x55704de2e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd466ffd640 1 -- 172.21.15.78:0/964258304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 msgr2=0x55704de2e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd466ffd640 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 0x55704de2e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.421+0000 7fd466ffd640 1 -- 172.21.15.78:0/964258304 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704de33940 con 0x55704de2b5d0 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.422+0000 7fd466ffd640 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de26c10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0b82d0 tx=0x7fd45c09b5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1feb9bc94ad38724 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.422+0000 7fd44cff9640 1 -- 172.21.15.78:0/964258304 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c09c230 con 0x55704de2b5d0 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.422+0000 7fd44cff9640 1 -- 172.21.15.78:0/964258304 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45c097a20 con 0x55704de2b5d0 2024-09-16T00:17:18.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.422+0000 7fd44cff9640 1 -- 172.21.15.78:0/964258304 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0a4310 con 0x55704de2b5d0 2024-09-16T00:17:18.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.423+0000 7fd46f6f0740 1 -- 172.21.15.78:0/964258304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 msgr2=0x55704de26c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:18.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.423+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de26c10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0b82d0 tx=0x7fd45c09b5c0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.423+0000 7fd46f6f0740 1 -- 172.21.15.78:0/964258304 shutdown_connections 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.423+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 0x55704de2e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.423+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2cb40 0x55704de2e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.423+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/964258304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2b5d0 0x55704de26c10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.423+0000 7fd46f6f0740 1 -- 172.21.15.78:0/964258304 >> 172.21.15.78:0/964258304 conn(0x55704ddf4310 msgr2=0x55704de26500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.423+0000 7fd46f6f0740 1 -- 172.21.15.78:0/964258304 shutdown_connections 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.424+0000 7fd46f6f0740 1 -- 172.21.15.78:0/964258304 wait complete. 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.424+0000 7fd46f6f0740 1 -- start start 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.424+0000 7fd46f6f0740 1 Processor -- start 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2cb40 0x55704de30620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd46f6f0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 0x55704de30b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681c7780 con 0x55704de36370 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681c70b0 con 0x55704de2b5d0 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd46f6f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd4681c42f0 con 0x55704de2cb40 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd46f6f0740 10 client.0 _ll_register_callbacks cb 0x7fd458006ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd466ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de300e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48196/0 (socket says 172.21.15.78:48196) 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.425+0000 7fd466ffd640 1 -- 172.21.15.78:0/4165096866 learned_addr learned my addr 172.21.15.78:0/4165096866 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd4667fc640 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2cb40 0x55704de30620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd466ffd640 1 -- 172.21.15.78:0/4165096866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2cb40 msgr2=0x55704de30620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd4677fe640 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 0x55704de30b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd466ffd640 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2cb40 0x55704de30620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd466ffd640 1 -- 172.21.15.78:0/4165096866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 msgr2=0x55704de30b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd466ffd640 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 0x55704de30b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd466ffd640 1 -- 172.21.15.78:0/4165096866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704deaf720 con 0x55704de2b5d0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd466ffd640 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de300e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd45c09b4d0 tx=0x7fd45c09c0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d874f977ed7bf335 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.426+0000 7fd44aff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd44aff5640 1 -- 172.21.15.78:0/4165096866 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c09ce60 con 0x55704de2b5d0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd44aff5640 1 -- 172.21.15.78:0/4165096866 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd45c09b180 con 0x55704de2b5d0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd46f6f0740 1 -- 172.21.15.78:0/4165096866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45c097a20 con 0x55704de2b5d0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd44aff5640 1 -- 172.21.15.78:0/4165096866 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd45c09fee0 con 0x55704de2b5d0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd46f6f0740 10 client.25547 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd46f6f0740 1 -- 172.21.15.78:0/4165096866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55704deaf720 con 0x55704de2b5d0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd44aff5640 1 -- 172.21.15.78:0/4165096866 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd45c09c230 con 0x55704de2b5d0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd44aff5640 1 -- 172.21.15.78:0/4165096866 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd45c010750 con 0x55704de2b5d0 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd46f6f0740 10 client.25547 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.427+0000 7fd46f6f0740 10 client.25547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:17:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.428+0000 7fd46f6f0740 20 client.25547 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:18.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.428+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55704deb0be0 0x55704deb3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:18.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.428+0000 7fd46f6f0740 4 client.25547 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:17:18.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.428+0000 7fd46f6f0740 1 -- 172.21.15.78:0/4165096866 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd45c097a20 con 0x55704deb0be0 2024-09-16T00:17:18.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.428+0000 7fd4667fc640 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55704deb0be0 0x55704deb3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:18.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.429+0000 7fd4667fc640 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55704deb0be0 0x55704deb3000 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=938bb2c822e068b3 server_cookie=e29f3a30fcd19fc2 in_seq=0 out_seq=0 2024-09-16T00:17:18.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.429+0000 7fd44aff5640 10 client.25547 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:17:18.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.429+0000 7fd44aff5640 1 -- 172.21.15.78:0/4165096866 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd45c097a20 con 0x55704deb0be0 2024-09-16T00:17:18.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.429+0000 7fd44aff5640 10 client.25547 handle_command_reply: tid=0 2024-09-16T00:17:18.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.429+0000 7fd46f6f0740 1 client.25547 shutdown 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 -- 172.21.15.78:0/4165096866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 msgr2=0x55704de300e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de300e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd45c09b4d0 tx=0x7fd45c09c0c0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 -- 172.21.15.78:0/4165096866 shutdown_connections 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55704de36370 0x55704de30b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55704de2cb40 0x55704de30620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55704deb0be0 0x55704deb3000 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd44aff5640 0 client.25547 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 --2- 172.21.15.78:0/4165096866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55704de2b5d0 0x55704de300e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 -- 172.21.15.78:0/4165096866 >> 172.21.15.78:0/4165096866 conn(0x55704ddf4310 msgr2=0x55704de26d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 -- 172.21.15.78:0/4165096866 shutdown_connections 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 1 -- 172.21.15.78:0/4165096866 wait complete. 2024-09-16T00:17:18.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.430+0000 7fd46f6f0740 20 client.25547 trim_cache size 0 max 16384 2024-09-16T00:17:18.435 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:17:18.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 -- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4500a3380 msgr2=0x7fd4500a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4500a3380 0x7fd4500a5840 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd460009bf0 tx=0x7fd4600097e0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 -- 172.21.15.78:0/3179996957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd46807b0b0 msgr2=0x7fd468073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd46807b0b0 0x7fd468073930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00d950 tx=0x7fd45c00de10 comp rx=0 tx=0).stop 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 -- 172.21.15.78:0/3179996957 shutdown_connections 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4500a3380 0x7fd4500a5840 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd46811b080 0x7fd46806cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd468082d80 0x7fd468073e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 --2- 172.21.15.78:0/3179996957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd46807b0b0 0x7fd468073930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 -- 172.21.15.78:0/3179996957 >> 172.21.15.78:0/3179996957 conn(0x7fd468077af0 msgr2=0x7fd468080d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 -- 172.21.15.78:0/3179996957 shutdown_connections 2024-09-16T00:17:18.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:18.432+0000 7fd46d317640 1 -- 172.21.15.78:0/3179996957 wait complete. 2024-09-16T00:17:19.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:17:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:19.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:19 smithi193 ceph-mon[35484]: pgmap v478: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2024-09-16T00:17:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:19 smithi148 ceph-mon[35599]: pgmap v478: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2024-09-16T00:17:19.961 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:19 smithi078 ceph-mon[31157]: pgmap v478: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 MiB/s wr, 1.10k op/s 2024-09-16T00:17:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.671+0000 7f1354802640 1 -- 172.21.15.78:0/3169626192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 msgr2=0x7f13501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.671+0000 7f1354802640 1 --2- 172.21.15.78:0/3169626192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 0x7f13501029b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1344009900 tx=0x7f134402a480 comp rx=0 tx=0).stop 2024-09-16T00:17:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.672+0000 7f1354802640 1 -- 172.21.15.78:0/3169626192 shutdown_connections 2024-09-16T00:17:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.672+0000 7f1354802640 1 --2- 172.21.15.78:0/3169626192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350102ef0 0x7f135010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.672+0000 7f1354802640 1 --2- 172.21.15.78:0/3169626192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 0x7f13501029b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.672+0000 7f1354802640 1 --2- 172.21.15.78:0/3169626192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350100e20 0x7f1350101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.672+0000 7f1354802640 1 -- 172.21.15.78:0/3169626192 >> 172.21.15.78:0/3169626192 conn(0x7f13500fc7d0 msgr2=0x7f13500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:20.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.672+0000 7f1354802640 1 -- 172.21.15.78:0/3169626192 shutdown_connections 2024-09-16T00:17:20.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.672+0000 7f1354802640 1 -- 172.21.15.78:0/3169626192 wait complete. 2024-09-16T00:17:20.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.673+0000 7f1354802640 1 Processor -- start 2024-09-16T00:17:20.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.673+0000 7f1354802640 1 -- start start 2024-09-16T00:17:20.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.674+0000 7f1354802640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350100e20 0x7f13501970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.674+0000 7f1354802640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 0x7f1350197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.674+0000 7f1354802640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350102ef0 0x7f135019c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.674+0000 7f1354802640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1350110fd0 con 0x7f1350102570 2024-09-16T00:17:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.674+0000 7f1354802640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1350110e50 con 0x7f1350102ef0 2024-09-16T00:17:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.674+0000 7f1354802640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1350111150 con 0x7f1350100e20 2024-09-16T00:17:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 0x7f1350197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134dd9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 0x7f1350197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49270/0 (socket says 172.21.15.78:49270) 2024-09-16T00:17:20.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134dd9b640 1 -- 172.21.15.78:0/2145623228 learned_addr learned my addr 172.21.15.78:0/2145623228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:20.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134e59c640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350100e20 0x7f13501970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134dd9b640 1 -- 172.21.15.78:0/2145623228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350100e20 msgr2=0x7f13501970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134dd9b640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350100e20 0x7f13501970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134dd9b640 1 -- 172.21.15.78:0/2145623228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350102ef0 msgr2=0x7f135019c140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134dd9b640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350102ef0 0x7f135019c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.675+0000 7f134dd9b640 1 -- 172.21.15.78:0/2145623228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135019c680 con 0x7f1350102570 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.676+0000 7f134dd9b640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 0x7f1350197630 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f134402aa70 tx=0x7f1344003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.676+0000 7f133f7fe640 1 -- 172.21.15.78:0/2145623228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134403f070 con 0x7f1350102570 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.676+0000 7f1354802640 1 -- 172.21.15.78:0/2145623228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135019c910 con 0x7f1350102570 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.676+0000 7f1354802640 1 -- 172.21.15.78:0/2145623228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135019cdf0 con 0x7f1350102570 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.676+0000 7f133f7fe640 1 -- 172.21.15.78:0/2145623228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1344031c30 con 0x7f1350102570 2024-09-16T00:17:20.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.677+0000 7f133f7fe640 1 -- 172.21.15.78:0/2145623228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344035030 con 0x7f1350102570 2024-09-16T00:17:20.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.679+0000 7f1356bdb740 1 Processor -- start 2024-09-16T00:17:20.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.680+0000 7f1356bdb740 1 -- start start 2024-09-16T00:17:20.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.680+0000 7f133f7fe640 1 -- 172.21.15.78:0/2145623228 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1344042070 con 0x7f1350102570 2024-09-16T00:17:20.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.680+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a0879c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.680+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a0879c5d0 0x556a0879ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.680+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.680+0000 7f1356bdb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a087022f0 con 0x556a0879db40 2024-09-16T00:17:20.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.680+0000 7f1356bdb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556a086a9cf0 con 0x556a0879c5d0 2024-09-16T00:17:20.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.680+0000 7f1356bdb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a086fe2b0 con 0x556a087a7370 2024-09-16T00:17:20.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.681+0000 7f134e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a0879c5d0 0x556a0879ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.681+0000 7f134e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a0879c5d0 0x556a0879ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48216/0 (socket says 172.21.15.78:48216) 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.681+0000 7f134e59c640 1 -- 172.21.15.78:0/1494932398 learned_addr learned my addr 172.21.15.78:0/1494932398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.681+0000 7f134e59c640 1 -- 172.21.15.78:0/1494932398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 msgr2=0x556a087a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.681+0000 7f134dd9b640 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.682+0000 7f134e59c640 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.682+0000 7f134e59c640 1 -- 172.21.15.78:0/1494932398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 msgr2=0x556a0879c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.682+0000 7f134e59c640 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a0879c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.682+0000 7f134e59c640 1 -- 172.21.15.78:0/1494932398 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0868e3a0 con 0x556a0879c5d0 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.682+0000 7f134dd9b640 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.682+0000 7f134e59c640 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a0879c5d0 0x556a0879ca10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1340009bc0 tx=0x7f134000e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f938d9cc257a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.683+0000 7f133cff9640 1 -- 172.21.15.78:0/1494932398 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340016070 con 0x556a0879c5d0 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.683+0000 7f133cff9640 1 -- 172.21.15.78:0/1494932398 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134000ee70 con 0x556a0879c5d0 2024-09-16T00:17:20.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.683+0000 7f133cff9640 1 -- 172.21.15.78:0/1494932398 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340011470 con 0x556a0879c5d0 2024-09-16T00:17:20.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.683+0000 7f133f7fe640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13380a3410 0x7f13380a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f134ed9d640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13380a3410 0x7f13380a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f133f7fe640 1 -- 172.21.15.78:0/2145623228 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f13440f01c0 con 0x7f1350102570 2024-09-16T00:17:20.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 -- 172.21.15.78:0/1494932398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a0879c5d0 msgr2=0x556a0879ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:20.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a0879c5d0 0x556a0879ca10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1340009bc0 tx=0x7f134000e6c0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 -- 172.21.15.78:0/1494932398 shutdown_connections 2024-09-16T00:17:20.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a0879c5d0 0x556a0879ca10 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1340009bc0 tx=0x7f134000e6c0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 --2- 172.21.15.78:0/1494932398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a0879c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 -- 172.21.15.78:0/1494932398 >> 172.21.15.78:0/1494932398 conn(0x556a08765310 msgr2=0x556a08798c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f134ed9d640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13380a3410 0x7f13380a58d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1348005fd0 tx=0x7f1348009940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 -- 172.21.15.78:0/1494932398 shutdown_connections 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.684+0000 7f1356bdb740 1 -- 172.21.15.78:0/1494932398 wait complete. 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.685+0000 7f1356bdb740 1 -- start start 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.685+0000 7f1356bdb740 1 Processor -- start 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.685+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.685+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a7370 0x556a087a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.685+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a4c90 0x556a087b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.685+0000 7f1356bdb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a087aa070 con 0x556a0879db40 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f1356bdb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556a086bd060 con 0x556a087a7370 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a4c90 0x556a087b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a4c90 0x556a087b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48796/0 (socket says 172.21.15.78:48796) 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 -- 172.21.15.78:0/3638976387 learned_addr learned my addr 172.21.15.78:0/3638976387 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134dd9b640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a7370 0x556a087a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a087aa1f0 con 0x556a087a4c90 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f1356bdb740 10 client.0 _ll_register_callbacks cb 0x556a084108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134e59c640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 -- 172.21.15.78:0/3638976387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a7370 msgr2=0x556a087a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a7370 0x556a087a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 -- 172.21.15.78:0/3638976387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 msgr2=0x556a087a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 -- 172.21.15.78:0/3638976387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a084733c0 con 0x556a087a4c90 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134dd9b640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a7370 0x556a087a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134e59c640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f134ed9d640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a4c90 0x556a087b35b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f13480106b0 tx=0x7f1348004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503bc23cc5acb588 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.686+0000 7f1335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.687+0000 7f1335ffb640 1 -- 172.21.15.78:0/3638976387 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348021020 con 0x556a087a4c90 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.687+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134000ee70 con 0x556a087a4c90 2024-09-16T00:17:20.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.687+0000 7f1335ffb640 1 -- 172.21.15.78:0/3638976387 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1348004cc0 con 0x556a087a4c90 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.688+0000 7f1335ffb640 1 -- 172.21.15.78:0/3638976387 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f134802c050 con 0x556a087a4c90 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.688+0000 7f1356bdb740 10 client.25444 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.688+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a087b6190 con 0x556a087a4c90 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.689+0000 7f1335ffb640 1 -- 172.21.15.78:0/3638976387 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1348005510 con 0x556a087a4c90 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.690+0000 7f1335ffb640 1 -- 172.21.15.78:0/3638976387 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f134801c8d0 con 0x556a087a4c90 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.690+0000 7f1356bdb740 10 client.25444 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.690+0000 7f1356bdb740 10 client.25444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.690+0000 7f1356bdb740 20 client.25444 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.690+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x556a087b75f0 0x556a087b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.690+0000 7f1356bdb740 4 client.25444 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.690+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f134000ee70 con 0x556a087b75f0 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.691+0000 7f134e59c640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x556a087b75f0 0x556a087b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.692+0000 7f134e59c640 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x556a087b75f0 0x556a087b9a70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=316663e42cb741e0 server_cookie=218234fb50c06739 in_seq=0 out_seq=0 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.692+0000 7f1335ffb640 10 client.25444 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1335ffb640 1 -- 172.21.15.78:0/3638976387 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f134000ee70 con 0x556a087b75f0 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1335ffb640 10 client.25444 handle_command_reply: tid=0 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 client.25444 shutdown 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a4c90 msgr2=0x556a087b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a4c90 0x556a087b35b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f13480106b0 tx=0x7f1348004a40 comp rx=0 tx=0).stop 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 shutdown_connections 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x556a087b75f0 0x556a087b9a70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a4c90 0x556a087b35b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a7370 0x556a087a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3638976387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 >> 172.21.15.78:0/3638976387 conn(0x556a08765310 msgr2=0x556a0879e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:20.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1335ffb640 0 client.25444 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:17:20.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 shutdown_connections 2024-09-16T00:17:20.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 1 -- 172.21.15.78:0/3638976387 wait complete. 2024-09-16T00:17:20.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.693+0000 7f1356bdb740 20 client.25444 trim_cache size 0 max 16384 2024-09-16T00:17:20.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.714+0000 7f1356bdb740 1 Processor -- start 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.714+0000 7f1356bdb740 1 -- start start 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.714+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.714+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.714+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f1356bdb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f133000f910 con 0x556a0879db40 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f1356bdb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1330011450 con 0x556a087a4c90 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f1356bdb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f133000ffe0 con 0x556a087a7370 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134dd9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49294/0 (socket says 172.21.15.78:49294) 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134e59c640 1 -- 172.21.15.78:0/680537422 learned_addr learned my addr 172.21.15.78:0/680537422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:20.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134ed9d640 1 --2- 172.21.15.78:0/680537422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134ed9d640 1 -- 172.21.15.78:0/680537422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 msgr2=0x556a087b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134ed9d640 1 --2- 172.21.15.78:0/680537422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134ed9d640 1 -- 172.21.15.78:0/680537422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 msgr2=0x556a087a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134ed9d640 1 --2- 172.21.15.78:0/680537422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134ed9d640 1 -- 172.21.15.78:0/680537422 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0879d3a0 con 0x556a087a7370 2024-09-16T00:17:20.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.715+0000 7f134ed9d640 1 --2- 172.21.15.78:0/680537422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087b38e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f134801cd70 tx=0x7f13480095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1524c5a56dc529ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.716+0000 7f1337fff640 1 -- 172.21.15.78:0/680537422 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348021070 con 0x556a087a7370 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.716+0000 7f1337fff640 1 -- 172.21.15.78:0/680537422 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134800ee70 con 0x556a087a7370 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.716+0000 7f1337fff640 1 -- 172.21.15.78:0/680537422 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134801ddb0 con 0x556a087a7370 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 -- 172.21.15.78:0/680537422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 msgr2=0x556a087b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 --2- 172.21.15.78:0/680537422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087b38e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f134801cd70 tx=0x7f13480095c0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 -- 172.21.15.78:0/680537422 shutdown_connections 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 --2- 172.21.15.78:0/680537422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087b38e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 --2- 172.21.15.78:0/680537422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 --2- 172.21.15.78:0/680537422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 -- 172.21.15.78:0/680537422 >> 172.21.15.78:0/680537422 conn(0x556a08765310 msgr2=0x556a087a5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:20.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 -- 172.21.15.78:0/680537422 shutdown_connections 2024-09-16T00:17:20.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 -- 172.21.15.78:0/680537422 wait complete. 2024-09-16T00:17:20.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.717+0000 7f1356bdb740 1 -- start start 2024-09-16T00:17:20.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f1356bdb740 1 Processor -- start 2024-09-16T00:17:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f1356bdb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f1356bdb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1330010d80 con 0x556a0879db40 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f1356bdb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13300106b0 con 0x556a087a4c90 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f1356bdb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f133000d9a0 con 0x556a087a7370 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f1356bdb740 10 client.0 _ll_register_callbacks cb 0x7f1348014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f134e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f134e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49310/0 (socket says 172.21.15.78:49310) 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f134e59c640 1 -- 172.21.15.78:0/3231698009 learned_addr learned my addr 172.21.15.78:0/3231698009 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f134ed9d640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.718+0000 7f134dd9b640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.719+0000 7f134e59c640 1 -- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 msgr2=0x556a087a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.720+0000 7f134e59c640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.720+0000 7f134e59c640 1 -- 172.21.15.78:0/3231698009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 msgr2=0x556a087a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:20.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.720+0000 7f134e59c640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.720+0000 7f134e59c640 1 -- 172.21.15.78:0/3231698009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a087ae780 con 0x556a0879db40 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.720+0000 7f134ed9d640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.720+0000 7f134dd9b640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f134e59c640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a0560 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x556a08799ac0 tx=0x7f13400097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c10d5ab3a34e484 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f1335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f1335ffb640 1 -- 172.21.15.78:0/3231698009 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134002c070 con 0x556a0879db40 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f1356bdb740 1 -- 172.21.15.78:0/3231698009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134800ee70 con 0x556a0879db40 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f1335ffb640 1 -- 172.21.15.78:0/3231698009 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1340004820 con 0x556a0879db40 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f1335ffb640 1 -- 172.21.15.78:0/3231698009 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1340016040 con 0x556a0879db40 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f1356bdb740 10 client.16128 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f1356bdb740 1 -- 172.21.15.78:0/3231698009 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a087afad0 con 0x556a0879db40 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.721+0000 7f1335ffb640 1 -- 172.21.15.78:0/3231698009 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134800ee70 con 0x556a0879db40 2024-09-16T00:17:20.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f1335ffb640 1 -- 172.21.15.78:0/3231698009 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1340025040 con 0x556a0879db40 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f1356bdb740 10 client.16128 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f1356bdb740 10 client.16128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f1356bdb740 20 client.16128 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x556a087afe10 0x556a08824320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f1356bdb740 4 client.16128 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f1356bdb740 1 -- 172.21.15.78:0/3231698009 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a0870f860 con 0x556a087afe10 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f134dd9b640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x556a087afe10 0x556a08824320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f1335ffb640 10 client.16128 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.722+0000 7f134dd9b640 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x556a087afe10 0x556a08824320 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36964b91b01e497f server_cookie=9c6df6bbfd0df258 in_seq=0 out_seq=0 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1335ffb640 1 -- 172.21.15.78:0/3231698009 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a0870f860 con 0x556a087afe10 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1335ffb640 10 client.16128 handle_command_reply: tid=0 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 client.16128 shutdown 2024-09-16T00:17:20.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 -- 172.21.15.78:0/3231698009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 msgr2=0x556a087a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a0560 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x556a08799ac0 tx=0x7f13400097c0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 -- 172.21.15.78:0/3231698009 shutdown_connections 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x556a087afe10 0x556a08824320 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a087a7370 0x556a087a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1335ffb640 0 client.16128 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a087a4c90 0x556a087a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 --2- 172.21.15.78:0/3231698009 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a0879db40 0x556a087a0560 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 -- 172.21.15.78:0/3231698009 >> 172.21.15.78:0/3231698009 conn(0x556a08765310 msgr2=0x556a0879e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 -- 172.21.15.78:0/3231698009 shutdown_connections 2024-09-16T00:17:20.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 1 -- 172.21.15.78:0/3231698009 wait complete. 2024-09-16T00:17:20.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.723+0000 7f1356bdb740 20 client.16128 trim_cache size 0 max 16384 2024-09-16T00:17:20.729 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:17:20.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 -- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13380a3410 msgr2=0x7f13380a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:20.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13380a3410 0x7f13380a58d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1348005fd0 tx=0x7f1348009940 comp rx=0 tx=0).stop 2024-09-16T00:17:20.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 -- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 msgr2=0x7f1350197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:20.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 0x7f1350197630 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f134402aa70 tx=0x7f1344003c10 comp rx=0 tx=0).stop 2024-09-16T00:17:20.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 -- 172.21.15.78:0/2145623228 shutdown_connections 2024-09-16T00:17:20.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13380a3410 0x7f13380a58d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1350102ef0 0x7f135019c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1350102570 0x7f1350197630 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 --2- 172.21.15.78:0/2145623228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1350100e20 0x7f13501970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:20.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.724+0000 7f1354802640 1 -- 172.21.15.78:0/2145623228 >> 172.21.15.78:0/2145623228 conn(0x7f13500fc7d0 msgr2=0x7f1350106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:20.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.725+0000 7f1354802640 1 -- 172.21.15.78:0/2145623228 shutdown_connections 2024-09-16T00:17:20.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:20.725+0000 7f1354802640 1 -- 172.21.15.78:0/2145623228 wait complete. 2024-09-16T00:17:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 200.0 s, 1789.6 tps, lat 54.897 ms stddev 65.860 2024-09-16T00:17:21.599 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:17:21.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:21 smithi193 ceph-mon[35484]: pgmap v479: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 860 op/s 2024-09-16T00:17:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:21 smithi148 ceph-mon[35599]: pgmap v479: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 860 op/s 2024-09-16T00:17:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:22.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:21 smithi078 ceph-mon[31157]: pgmap v479: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 MiB/s wr, 860 op/s 2024-09-16T00:17:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.648+0000 7ff53e5f2640 1 -- 172.21.15.78:0/1950200634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538101420 msgr2=0x7ff53810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.648+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/1950200634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538101420 0x7ff53810d950 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff530009ca0 tx=0x7ff53002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.653+0000 7ff53e5f2640 1 -- 172.21.15.78:0/1950200634 shutdown_connections 2024-09-16T00:17:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.653+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/1950200634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538101420 0x7ff53810d950 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.653+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/1950200634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff538100a80 0x7ff538100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.653+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/1950200634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff538106a80 0x7ff538106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.653+0000 7ff53e5f2640 1 -- 172.21.15.78:0/1950200634 >> 172.21.15.78:0/1950200634 conn(0x7ff5380fc7a0 msgr2=0x7ff5380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.658+0000 7ff53e5f2640 1 -- 172.21.15.78:0/1950200634 shutdown_connections 2024-09-16T00:17:22.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.661+0000 7ff53e5f2640 1 -- 172.21.15.78:0/1950200634 wait complete. 2024-09-16T00:17:22.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.662+0000 7ff53e5f2640 1 Processor -- start 2024-09-16T00:17:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.663+0000 7ff53e5f2640 1 -- start start 2024-09-16T00:17:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.663+0000 7ff53e5f2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff538100a80 0x7ff538116470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.663+0000 7ff53e5f2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff538101420 0x7ff5381104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.663+0000 7ff53e5f2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538106a80 0x7ff538110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.663+0000 7ff53e5f2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff53810ef50 con 0x7ff538100a80 2024-09-16T00:17:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.663+0000 7ff53e5f2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff53810edd0 con 0x7ff538106a80 2024-09-16T00:17:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.663+0000 7ff53e5f2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff53810f0d0 con 0x7ff538101420 2024-09-16T00:17:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.665+0000 7ff537fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff538100a80 0x7ff538116470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.665+0000 7ff537fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff538100a80 0x7ff538116470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49324/0 (socket says 172.21.15.78:49324) 2024-09-16T00:17:22.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.665+0000 7ff537fff640 1 -- 172.21.15.78:0/2285885199 learned_addr learned my addr 172.21.15.78:0/2285885199 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.668+0000 7ff5377fe640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff538101420 0x7ff5381104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.668+0000 7ff53cb8f640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538106a80 0x7ff538110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.670+0000 7ff53cb8f640 1 -- 172.21.15.78:0/2285885199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff538101420 msgr2=0x7ff5381104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.670+0000 7ff53cb8f640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff538101420 0x7ff5381104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.670+0000 7ff53cb8f640 1 -- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff538100a80 msgr2=0x7ff538116470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.670+0000 7ff53cb8f640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff538100a80 0x7ff538116470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.670+0000 7ff53cb8f640 1 -- 172.21.15.78:0/2285885199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff538111100 con 0x7ff538106a80 2024-09-16T00:17:22.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.674+0000 7ff53cb8f640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538106a80 0x7ff538110a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff530009ca0 tx=0x7ff530003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.675+0000 7ff5357fa640 1 -- 172.21.15.78:0/2285885199 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530032e80 con 0x7ff538106a80 2024-09-16T00:17:22.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.677+0000 7ff5357fa640 1 -- 172.21.15.78:0/2285885199 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff530004520 con 0x7ff538106a80 2024-09-16T00:17:22.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.677+0000 7ff5357fa640 1 -- 172.21.15.78:0/2285885199 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530030d70 con 0x7ff538106a80 2024-09-16T00:17:22.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.677+0000 7ff53e5f2640 1 -- 172.21.15.78:0/2285885199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53806bc10 con 0x7ff538106a80 2024-09-16T00:17:22.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.677+0000 7ff53e5f2640 1 -- 172.21.15.78:0/2285885199 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53806c100 con 0x7ff538106a80 2024-09-16T00:17:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.681+0000 7ff5409cb740 1 Processor -- start 2024-09-16T00:17:22.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.684+0000 7ff5409cb740 1 -- start start 2024-09-16T00:17:22.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.684+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1ddf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.684+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1ddfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.684+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1dea370 0x558ba1dec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.684+0000 7ff5409cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558ba1d452f0 con 0x558ba1ddf5d0 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.684+0000 7ff5409cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558ba1ceccf0 con 0x558ba1de0b40 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.684+0000 7ff5409cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558ba1d412b0 con 0x558ba1dea370 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.684+0000 7ff5357fa640 1 -- 172.21.15.78:0/2285885199 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff530004040 con 0x7ff538106a80 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.685+0000 7ff537fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1ddfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.685+0000 7ff537fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1ddfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49340/0 (socket says 172.21.15.78:49340) 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.685+0000 7ff537fff640 1 -- 172.21.15.78:0/2498881834 learned_addr learned my addr 172.21.15.78:0/2498881834 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.685+0000 7ff5357fa640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5200a3380 0x7ff5200a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.686+0000 7ff5357fa640 1 -- 172.21.15.78:0/2285885199 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff5300f0fa0 con 0x7ff538106a80 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.686+0000 7ff537fff640 1 -- 172.21.15.78:0/2498881834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1dea370 msgr2=0x558ba1dec760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.686+0000 7ff537fff640 1 --2- 172.21.15.78:0/2498881834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1dea370 0x558ba1dec760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.686+0000 7ff537fff640 1 -- 172.21.15.78:0/2498881834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 msgr2=0x558ba1ddf090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.686+0000 7ff537fff640 1 --2- 172.21.15.78:0/2498881834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1ddf090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.686+0000 7ff537fff640 1 -- 172.21.15.78:0/2498881834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba1cd13a0 con 0x558ba1ddf5d0 2024-09-16T00:17:22.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.686+0000 7ff537fff640 1 --2- 172.21.15.78:0/2498881834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1ddfa10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff52800bd40 tx=0x7ff5280107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633f7ebf2e380e42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.686+0000 7ff53cb8f640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5200a3380 0x7ff5200a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.687+0000 7ff51e7fc640 1 -- 172.21.15.78:0/2498881834 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52801c680 con 0x558ba1ddf5d0 2024-09-16T00:17:22.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.687+0000 7ff51e7fc640 1 -- 172.21.15.78:0/2498881834 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528011920 con 0x558ba1ddf5d0 2024-09-16T00:17:22.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.687+0000 7ff51e7fc640 1 -- 172.21.15.78:0/2498881834 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52801a800 con 0x558ba1ddf5d0 2024-09-16T00:17:22.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 -- 172.21.15.78:0/2498881834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 msgr2=0x558ba1ddfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2498881834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1ddfa10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff52800bd40 tx=0x7ff5280107a0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff53cb8f640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5200a3380 0x7ff5200a5840 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff5300b9660 tx=0x7ff53002a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 -- 172.21.15.78:0/2498881834 shutdown_connections 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2498881834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1dea370 0x558ba1dec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2498881834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1ddfa10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2498881834 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1ddf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 -- 172.21.15.78:0/2498881834 >> 172.21.15.78:0/2498881834 conn(0x558ba1da8310 msgr2=0x558ba1ddbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 -- 172.21.15.78:0/2498881834 shutdown_connections 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.688+0000 7ff5409cb740 1 -- 172.21.15.78:0/2498881834 wait complete. 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.689+0000 7ff5409cb740 1 -- start start 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.689+0000 7ff5409cb740 1 Processor -- start 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.689+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1de9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.689+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1de6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.689+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 0x558ba1de7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.689+0000 7ff5409cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558ba1ded110 con 0x558ba1dea370 2024-09-16T00:17:22.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.689+0000 7ff5409cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558ba1d00060 con 0x558ba1de0b40 2024-09-16T00:17:22.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.689+0000 7ff5409cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558ba1ded290 con 0x558ba1ddf5d0 2024-09-16T00:17:22.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.690+0000 7ff5409cb740 10 client.0 _ll_register_callbacks cb 0x558ba1a538c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:22.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.690+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1de9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.690+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1de9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48846/0 (socket says 172.21.15.78:48846) 2024-09-16T00:17:22.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.690+0000 7ff5377fe640 1 -- 172.21.15.78:0/4236567998 learned_addr learned my addr 172.21.15.78:0/4236567998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:22.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.690+0000 7ff5377fe640 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 0x558ba1de7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.690+0000 7ff537fff640 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1de6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.692+0000 7ff537fff640 1 -- 172.21.15.78:0/4236567998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 msgr2=0x558ba1de9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:22.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.692+0000 7ff537fff640 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1de9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.692+0000 7ff537fff640 1 -- 172.21.15.78:0/4236567998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 msgr2=0x558ba1de7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:22.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.692+0000 7ff537fff640 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 0x558ba1de7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.692+0000 7ff537fff640 1 -- 172.21.15.78:0/4236567998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba1ab63c0 con 0x558ba1de0b40 2024-09-16T00:17:22.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.695+0000 7ff537fff640 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1de6df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff528006010 tx=0x7ff528011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d0027d2b6a18e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.696+0000 7ff5177fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:17:22.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.696+0000 7ff5177fe640 1 -- 172.21.15.78:0/4236567998 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52802a060 con 0x558ba1de0b40 2024-09-16T00:17:22.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.696+0000 7ff5409cb740 1 -- 172.21.15.78:0/4236567998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff528011920 con 0x558ba1de0b40 2024-09-16T00:17:22.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.696+0000 7ff5177fe640 1 -- 172.21.15.78:0/4236567998 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff528011650 con 0x558ba1de0b40 2024-09-16T00:17:22.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.696+0000 7ff5177fe640 1 -- 172.21.15.78:0/4236567998 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff528031050 con 0x558ba1de0b40 2024-09-16T00:17:22.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.696+0000 7ff5409cb740 10 client.25571 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:22.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.697+0000 7ff5409cb740 1 -- 172.21.15.78:0/4236567998 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558ba1df8aa0 con 0x558ba1de0b40 2024-09-16T00:17:22.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.697+0000 7ff5177fe640 1 -- 172.21.15.78:0/4236567998 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff528011920 con 0x558ba1de0b40 2024-09-16T00:17:22.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.697+0000 7ff5177fe640 1 -- 172.21.15.78:0/4236567998 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff528004c60 con 0x558ba1de0b40 2024-09-16T00:17:22.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.699+0000 7ff5409cb740 10 client.25571 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:22.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.699+0000 7ff5409cb740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:17:22.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.699+0000 7ff5409cb740 20 client.25571 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:22.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.699+0000 7ff5409cb740 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558ba1df9df0 0x558ba1dfc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.699+0000 7ff5409cb740 4 client.25571 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:17:22.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.699+0000 7ff5409cb740 1 -- 172.21.15.78:0/4236567998 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ba1d52860 con 0x558ba1df9df0 2024-09-16T00:17:22.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.699+0000 7ff5377fe640 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558ba1df9df0 0x558ba1dfc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.700+0000 7ff5377fe640 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558ba1df9df0 0x558ba1dfc1e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3dea8558e9804e server_cookie=9aed591b3d373ee9 in_seq=0 out_seq=0 2024-09-16T00:17:22.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.701+0000 7ff5177fe640 10 client.25571 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:22.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.702+0000 7ff5177fe640 1 -- 172.21.15.78:0/4236567998 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ba1d52860 con 0x558ba1df9df0 2024-09-16T00:17:22.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.702+0000 7ff5177fe640 10 client.25571 handle_command_reply: tid=0 2024-09-16T00:17:22.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.702+0000 7ff5409cb740 1 client.25571 shutdown 2024-09-16T00:17:22.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.703+0000 7ff5409cb740 1 -- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 msgr2=0x558ba1de6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.703+0000 7ff5409cb740 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1de6df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff528006010 tx=0x7ff528011d70 comp rx=0 tx=0).stop 2024-09-16T00:17:22.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.704+0000 7ff5409cb740 1 -- 172.21.15.78:0/4236567998 shutdown_connections 2024-09-16T00:17:22.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.704+0000 7ff5409cb740 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558ba1df9df0 0x558ba1dfc1e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.704+0000 7ff5409cb740 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 0x558ba1de7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.704+0000 7ff5409cb740 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1de6df0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.704+0000 7ff5409cb740 1 --2- 172.21.15.78:0/4236567998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1de9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.704+0000 7ff5409cb740 1 -- 172.21.15.78:0/4236567998 >> 172.21.15.78:0/4236567998 conn(0x558ba1da8310 msgr2=0x558ba1de1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:22.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.706+0000 7ff5177fe640 0 client.25571 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:22.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.707+0000 7ff5409cb740 1 -- 172.21.15.78:0/4236567998 shutdown_connections 2024-09-16T00:17:22.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.709+0000 7ff5409cb740 1 -- 172.21.15.78:0/4236567998 wait complete. 2024-09-16T00:17:22.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.710+0000 7ff5409cb740 20 client.25571 trim_cache size 0 max 16384 2024-09-16T00:17:22.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5409cb740 1 Processor -- start 2024-09-16T00:17:22.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5409cb740 1 -- start start 2024-09-16T00:17:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1de0b40 0x558ba1debca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1dea370 0x558ba1de56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5409cb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff52400fb70 con 0x558ba1ddf5d0 2024-09-16T00:17:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5409cb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff5240116b0 con 0x558ba1dea370 2024-09-16T00:17:22.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5409cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff524010240 con 0x558ba1de0b40 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49354/0 (socket says 172.21.15.78:49354) 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 -- 172.21.15.78:0/3904765968 learned_addr learned my addr 172.21.15.78:0/3904765968 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 --2- 172.21.15.78:0/3904765968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1dea370 0x558ba1de56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 -- 172.21.15.78:0/3904765968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1de0b40 msgr2=0x558ba1debca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 --2- 172.21.15.78:0/3904765968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1de0b40 0x558ba1debca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 -- 172.21.15.78:0/3904765968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1dea370 msgr2=0x558ba1de56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 --2- 172.21.15.78:0/3904765968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1dea370 0x558ba1de56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 -- 172.21.15.78:0/3904765968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba1ddc1a0 con 0x558ba1ddf5d0 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.736+0000 7ff5377fe640 1 --2- 172.21.15.78:0/3904765968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb760 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff52c001eb0 tx=0x7ff52c005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248a9b22fd7469e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.737+0000 7ff51dffb640 1 -- 172.21.15.78:0/3904765968 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c010970 con 0x558ba1ddf5d0 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.737+0000 7ff51dffb640 1 -- 172.21.15.78:0/3904765968 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff52c005020 con 0x558ba1ddf5d0 2024-09-16T00:17:22.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.737+0000 7ff51dffb640 1 -- 172.21.15.78:0/3904765968 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c004b20 con 0x558ba1ddf5d0 2024-09-16T00:17:22.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.737+0000 7ff5409cb740 1 -- 172.21.15.78:0/3904765968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 msgr2=0x558ba1deb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.737+0000 7ff5409cb740 1 --2- 172.21.15.78:0/3904765968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb760 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff52c001eb0 tx=0x7ff52c005d20 comp rx=0 tx=0).stop 2024-09-16T00:17:22.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.738+0000 7ff5409cb740 1 -- 172.21.15.78:0/3904765968 shutdown_connections 2024-09-16T00:17:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.738+0000 7ff5409cb740 1 --2- 172.21.15.78:0/3904765968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1dea370 0x558ba1de56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.738+0000 7ff5409cb740 1 --2- 172.21.15.78:0/3904765968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1de0b40 0x558ba1debca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.738+0000 7ff5409cb740 1 --2- 172.21.15.78:0/3904765968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb760 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.738+0000 7ff5409cb740 1 -- 172.21.15.78:0/3904765968 >> 172.21.15.78:0/3904765968 conn(0x558ba1da8310 msgr2=0x558ba1de6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.738+0000 7ff5409cb740 1 -- 172.21.15.78:0/3904765968 shutdown_connections 2024-09-16T00:17:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.738+0000 7ff5409cb740 1 -- 172.21.15.78:0/3904765968 wait complete. 2024-09-16T00:17:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.738+0000 7ff5409cb740 1 -- start start 2024-09-16T00:17:22.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.739+0000 7ff5409cb740 1 Processor -- start 2024-09-16T00:17:22.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.739+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.740+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1deb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.740+0000 7ff5409cb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 0x558ba1df1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.740+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.740+0000 7ff5377fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48868/0 (socket says 172.21.15.78:48868) 2024-09-16T00:17:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.740+0000 7ff5377fe640 1 -- 172.21.15.78:0/2699721202 learned_addr learned my addr 172.21.15.78:0/2699721202 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.740+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff524010fe0 con 0x558ba1dea370 2024-09-16T00:17:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.740+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff524010910 con 0x558ba1de0b40 2024-09-16T00:17:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.740+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff52400db50 con 0x558ba1ddf5d0 2024-09-16T00:17:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.741+0000 7ff5409cb740 10 client.0 _ll_register_callbacks cb 0x7ff52800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:22.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.742+0000 7ff537fff640 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1deb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.742+0000 7ff5377fe640 1 -- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 msgr2=0x558ba1deb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.742+0000 7ff5377fe640 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1deb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.742+0000 7ff5377fe640 1 -- 172.21.15.78:0/2699721202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 msgr2=0x558ba1df1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.742+0000 7ff5377fe640 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 0x558ba1df1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.742+0000 7ff5377fe640 1 -- 172.21.15.78:0/2699721202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba1df1b80 con 0x558ba1ddf5d0 2024-09-16T00:17:22.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.742+0000 7ff537fff640 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1deb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:17:22.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.743+0000 7ff5377fe640 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff52c01f040 tx=0x7ff52c005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58fcb4c5898eb377 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:22.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.743+0000 7ff5177fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:17:22.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.743+0000 7ff5177fe640 1 -- 172.21.15.78:0/2699721202 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c010e40 con 0x558ba1ddf5d0 2024-09-16T00:17:22.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.743+0000 7ff5177fe640 1 -- 172.21.15.78:0/2699721202 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff52c004730 con 0x558ba1ddf5d0 2024-09-16T00:17:22.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.743+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff52c005020 con 0x558ba1ddf5d0 2024-09-16T00:17:22.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5177fe640 1 -- 172.21.15.78:0/2699721202 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff52c004930 con 0x558ba1ddf5d0 2024-09-16T00:17:22.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5409cb740 10 client.25459 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:22.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558ba1df2e00 con 0x558ba1ddf5d0 2024-09-16T00:17:22.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5177fe640 1 -- 172.21.15.78:0/2699721202 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff52c005020 con 0x558ba1ddf5d0 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5177fe640 1 -- 172.21.15.78:0/2699721202 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff52c032030 con 0x558ba1ddf5d0 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5409cb740 10 client.25459 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5409cb740 10 client.25459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5409cb740 20 client.25459 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558ba1e64810 0x558ba1e66c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5409cb740 4 client.25459 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ba1d4cd70 con 0x558ba1e64810 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.744+0000 7ff537fff640 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558ba1e64810 0x558ba1e66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5177fe640 10 client.25459 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff537fff640 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558ba1e64810 0x558ba1e66c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7fb6b7956af0788 server_cookie=954ab2a56104b905 in_seq=0 out_seq=0 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5177fe640 1 -- 172.21.15.78:0/2699721202 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ba1d4cd70 con 0x558ba1e64810 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5177fe640 10 client.25459 handle_command_reply: tid=0 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 client.25459 shutdown 2024-09-16T00:17:22.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 msgr2=0x558ba1deb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff52c01f040 tx=0x7ff52c005770 comp rx=0 tx=0).stop 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 shutdown_connections 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558ba1e64810 0x558ba1e66c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558ba1dea370 0x558ba1df1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558ba1de0b40 0x558ba1deb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 --2- 172.21.15.78:0/2699721202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ba1ddf5d0 0x558ba1deb370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 >> 172.21.15.78:0/2699721202 conn(0x558ba1da8310 msgr2=0x558ba1de1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.745+0000 7ff5177fe640 0 client.25459 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.746+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 shutdown_connections 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.746+0000 7ff5409cb740 1 -- 172.21.15.78:0/2699721202 wait complete. 2024-09-16T00:17:22.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.746+0000 7ff5409cb740 20 client.25459 trim_cache size 0 max 16384 2024-09-16T00:17:22.749 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:17:22.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.746+0000 7ff53e5f2640 1 -- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5200a3380 msgr2=0x7ff5200a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.746+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5200a3380 0x7ff5200a5840 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff5300b9660 tx=0x7ff53002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.746+0000 7ff53e5f2640 1 -- 172.21.15.78:0/2285885199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538106a80 msgr2=0x7ff538110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.746+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538106a80 0x7ff538110a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff530009ca0 tx=0x7ff530003ba0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.747+0000 7ff53e5f2640 1 -- 172.21.15.78:0/2285885199 shutdown_connections 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.747+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff5200a3380 0x7ff5200a5840 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.747+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff538106a80 0x7ff538110a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.747+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff538101420 0x7ff5381104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.747+0000 7ff53e5f2640 1 --2- 172.21.15.78:0/2285885199 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff538100a80 0x7ff538116470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.747+0000 7ff53e5f2640 1 -- 172.21.15.78:0/2285885199 >> 172.21.15.78:0/2285885199 conn(0x7ff5380fc7a0 msgr2=0x7ff5380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.747+0000 7ff53e5f2640 1 -- 172.21.15.78:0/2285885199 shutdown_connections 2024-09-16T00:17:22.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:22.747+0000 7ff53e5f2640 1 -- 172.21.15.78:0/2285885199 wait complete. 2024-09-16T00:17:23.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:23 smithi193 ceph-mon[35484]: pgmap v480: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 856 op/s 2024-09-16T00:17:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:23 smithi148 ceph-mon[35599]: pgmap v480: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 856 op/s 2024-09-16T00:17:24.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:23 smithi078 ceph-mon[31157]: pgmap v480: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 856 op/s 2024-09-16T00:17:24.713 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:17:24.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:17:25.152 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 205.0 s, 2059.2 tps, lat 49.375 ms stddev 57.715 2024-09-16T00:17:25.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:25 smithi193 ceph-mon[35484]: pgmap v481: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 606 op/s 2024-09-16T00:17:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:25 smithi148 ceph-mon[35599]: pgmap v481: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 606 op/s 2024-09-16T00:17:26.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:25 smithi078 ceph-mon[31157]: pgmap v481: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 MiB/s wr, 606 op/s 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 -- 172.21.15.78:0/219743157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 msgr2=0x7f5344100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 --2- 172.21.15.78:0/219743157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 0x7f5344100d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f532c009860 tx=0x7f532c02a3b0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 -- 172.21.15.78:0/219743157 shutdown_connections 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 --2- 172.21.15.78:0/219743157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5344105d90 0x7f5344106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 --2- 172.21.15.78:0/219743157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53441012a0 0x7f5344105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 --2- 172.21.15.78:0/219743157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 0x7f5344100d60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 -- 172.21.15.78:0/219743157 >> 172.21.15.78:0/219743157 conn(0x7f53440fc610 msgr2=0x7f53440fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 -- 172.21.15.78:0/219743157 shutdown_connections 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.263+0000 7f5348f0e640 1 -- 172.21.15.78:0/219743157 wait complete. 2024-09-16T00:17:26.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f5348f0e640 1 Processor -- start 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f5348f0e640 1 -- start start 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f5348f0e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 0x7f5344073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f5348f0e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53441012a0 0x7f534406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f53437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 0x7f5344073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f53437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 0x7f5344073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54190/0 (socket says 172.21.15.78:54190) 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f53437fe640 1 -- 172.21.15.78:0/2506354280 learned_addr learned my addr 172.21.15.78:0/2506354280 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f5348f0e640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5344105d90 0x7f534406c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53441183a0 con 0x7f5344100980 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5344118220 con 0x7f5344105d90 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.264+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5344118520 con 0x7f53441012a0 2024-09-16T00:17:26.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.265+0000 7f5342ffd640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53441012a0 0x7f534406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.265+0000 7f5343fff640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5344105d90 0x7f534406c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.265+0000 7f5343fff640 1 -- 172.21.15.78:0/2506354280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53441012a0 msgr2=0x7f534406bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.265+0000 7f5343fff640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53441012a0 0x7f534406bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.265+0000 7f5343fff640 1 -- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 msgr2=0x7f5344073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.265+0000 7f5343fff640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 0x7f5344073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.265+0000 7f5343fff640 1 -- 172.21.15.78:0/2506354280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534406ca40 con 0x7f5344105d90 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.265+0000 7f53437fe640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 0x7f5344073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.266+0000 7f5343fff640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5344105d90 0x7f534406c220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f533c00ef00 tx=0x7f533c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.266+0000 7f5340ff9640 1 -- 172.21.15.78:0/2506354280 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c004b20 con 0x7f5344105d90 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.266+0000 7f5340ff9640 1 -- 172.21.15.78:0/2506354280 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c003e50 con 0x7f5344105d90 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.266+0000 7f5340ff9640 1 -- 172.21.15.78:0/2506354280 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c00b570 con 0x7f5344105d90 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.266+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53441b3650 con 0x7f5344105d90 2024-09-16T00:17:26.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.266+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53441b3a80 con 0x7f5344105d90 2024-09-16T00:17:26.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.269+0000 7f5340ff9640 1 -- 172.21.15.78:0/2506354280 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f533c01d020 con 0x7f5344105d90 2024-09-16T00:17:26.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.269+0000 7f534c549740 1 Processor -- start 2024-09-16T00:17:26.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f534c549740 1 -- start start 2024-09-16T00:17:26.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f534c549740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd29150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f534c549740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f534c549740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd33f50 0x55570dd36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f534c549740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55570dc8e280 con 0x55570dd28d70 2024-09-16T00:17:26.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f534c549740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55570dc35b10 con 0x55570dd33f50 2024-09-16T00:17:26.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f534c549740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55570dc8a240 con 0x55570dd29690 2024-09-16T00:17:26.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f53437fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f53437fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd33a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53096/0 (socket says 172.21.15.78:53096) 2024-09-16T00:17:26.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f53437fe640 1 -- 172.21.15.78:0/504960477 learned_addr learned my addr 172.21.15.78:0/504960477 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:26.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.270+0000 7f5342ffd640 1 --2- 172.21.15.78:0/504960477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd33f50 0x55570dd36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.271+0000 7f5342ffd640 1 -- 172.21.15.78:0/504960477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 msgr2=0x55570dd33a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.271+0000 7f5342ffd640 1 --2- 172.21.15.78:0/504960477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd33a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.271+0000 7f5342ffd640 1 -- 172.21.15.78:0/504960477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 msgr2=0x55570dd29150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.271+0000 7f5342ffd640 1 --2- 172.21.15.78:0/504960477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd29150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.271+0000 7f5342ffd640 1 -- 172.21.15.78:0/504960477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570dc421f0 con 0x55570dd33f50 2024-09-16T00:17:26.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.271+0000 7f5342ffd640 1 --2- 172.21.15.78:0/504960477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd33f50 0x55570dd36340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f534406d530 tx=0x7f5334011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd2cc6f438c3c17a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.271+0000 7f5340ff9640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53300a3410 0x7f53300a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.272+0000 7f5340ff9640 1 -- 172.21.15.78:0/2506354280 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f533c0cf990 con 0x7f5344105d90 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.272+0000 7f53437fe640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53300a3410 0x7f53300a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.272+0000 7f5339ffb640 1 -- 172.21.15.78:0/504960477 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334004220 con 0x55570dd33f50 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.272+0000 7f5339ffb640 1 -- 172.21.15.78:0/504960477 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53340043c0 con 0x55570dd33f50 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.272+0000 7f5339ffb640 1 -- 172.21.15.78:0/504960477 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53340035f0 con 0x55570dd33f50 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 -- 172.21.15.78:0/504960477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd33f50 msgr2=0x55570dd36340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 --2- 172.21.15.78:0/504960477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd33f50 0x55570dd36340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f534406d530 tx=0x7f5334011810 comp rx=0 tx=0).stop 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f53437fe640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53300a3410 0x7f53300a58d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f532c0095f0 tx=0x7f532c0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 -- 172.21.15.78:0/504960477 shutdown_connections 2024-09-16T00:17:26.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 --2- 172.21.15.78:0/504960477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd33f50 0x55570dd36340 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 --2- 172.21.15.78:0/504960477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 --2- 172.21.15.78:0/504960477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd29150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 -- 172.21.15.78:0/504960477 >> 172.21.15.78:0/504960477 conn(0x55570dd25300 msgr2=0x55570dd256e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 -- 172.21.15.78:0/504960477 shutdown_connections 2024-09-16T00:17:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.273+0000 7f534c549740 1 -- 172.21.15.78:0/504960477 wait complete. 2024-09-16T00:17:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 1 -- start start 2024-09-16T00:17:26.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 1 Processor -- start 2024-09-16T00:17:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd30f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd31470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd33f50 0x55570dd319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55570dd36b20 con 0x55570dd33f50 2024-09-16T00:17:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55570dc48f10 con 0x55570dd28d70 2024-09-16T00:17:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55570dd36ca0 con 0x55570dd29690 2024-09-16T00:17:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.274+0000 7f534c549740 10 client.0 _ll_register_callbacks cb 0x55570dc11a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:26.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f53437fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd30f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.408 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:17:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:17:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "3d3d8c98-5483-4a47-8507-25080cbce279", 2024-09-16T00:17:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:17:26.408 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:17:26.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5343fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd33f50 0x55570dd319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5343fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd33f50 0x55570dd319b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54204/0 (socket says 172.21.15.78:54204) 2024-09-16T00:17:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5343fff640 1 -- 172.21.15.78:0/2927641801 learned_addr learned my addr 172.21.15.78:0/2927641801 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5342ffd640 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd31470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5343fff640 1 -- 172.21.15.78:0/2927641801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 msgr2=0x55570dd31470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5343fff640 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd31470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5343fff640 1 -- 172.21.15.78:0/2927641801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 msgr2=0x55570dd30f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5343fff640 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd30f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.275+0000 7f5343fff640 1 -- 172.21.15.78:0/2927641801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570d9ff6a0 con 0x55570dd33f50 2024-09-16T00:17:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.276+0000 7f5343fff640 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd33f50 0x55570dd319b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f533c0a4330 tx=0x7f533c09c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b3d8e423fd1aa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.276+0000 7f5322ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:17:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.276+0000 7f5322ffd640 1 -- 172.21.15.78:0/2927641801 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c01a040 con 0x55570dd33f50 2024-09-16T00:17:26.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.276+0000 7f5322ffd640 1 -- 172.21.15.78:0/2927641801 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f533c0cd1e0 con 0x55570dd33f50 2024-09-16T00:17:26.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.276+0000 7f534c549740 1 -- 172.21.15.78:0/2927641801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53340043c0 con 0x55570dd33f50 2024-09-16T00:17:26.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.276+0000 7f5322ffd640 1 -- 172.21.15.78:0/2927641801 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f533c0a4860 con 0x55570dd33f50 2024-09-16T00:17:26.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.276+0000 7f534c549740 10 client.16152 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:26.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.276+0000 7f534c549740 1 -- 172.21.15.78:0/2927641801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55570dd42880 con 0x55570dd33f50 2024-09-16T00:17:26.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f5322ffd640 1 -- 172.21.15.78:0/2927641801 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53340043c0 con 0x55570dd33f50 2024-09-16T00:17:26.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f5322ffd640 1 -- 172.21.15.78:0/2927641801 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f533c00b040 con 0x55570dd33f50 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f534c549740 10 client.16152 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f534c549740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f534c549740 20 client.16152 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f534c549740 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55570dd441c0 0x55570dd465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f534c549740 4 client.16152 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f534c549740 1 -- 172.21.15.78:0/2927641801 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570dc9b7f0 con 0x55570dd441c0 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.277+0000 7f53437fe640 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55570dd441c0 0x55570dd465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.278+0000 7f5322ffd640 10 client.16152 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:26.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.278+0000 7f53437fe640 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55570dd441c0 0x55570dd465f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef9d3b30b8dd6e1 server_cookie=9b883458282ad62a in_seq=0 out_seq=0 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.280+0000 7f5322ffd640 1 -- 172.21.15.78:0/2927641801 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55570dc9b7f0 con 0x55570dd441c0 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.280+0000 7f5322ffd640 10 client.16152 handle_command_reply: tid=0 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.280+0000 7f534c549740 1 client.16152 shutdown 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.280+0000 7f534c549740 1 -- 172.21.15.78:0/2927641801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd33f50 msgr2=0x55570dd319b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.280+0000 7f534c549740 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd33f50 0x55570dd319b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f533c0a4330 tx=0x7f533c09c3b0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 1 -- 172.21.15.78:0/2927641801 shutdown_connections 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55570dd441c0 0x55570dd465f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd33f50 0x55570dd319b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f5322ffd640 0 client.16152 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:26.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd29690 0x55570dd31470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 1 --2- 172.21.15.78:0/2927641801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd30f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 1 -- 172.21.15.78:0/2927641801 >> 172.21.15.78:0/2927641801 conn(0x55570dd25300 msgr2=0x55570dd2b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 1 -- 172.21.15.78:0/2927641801 shutdown_connections 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 1 -- 172.21.15.78:0/2927641801 wait complete. 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.281+0000 7f534c549740 20 client.16152 trim_cache size 0 max 16384 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f534c549740 1 Processor -- start 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f534c549740 1 -- start start 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f534c549740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd31800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f534c549740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd29690 0x55570dd4bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f534c549740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd4c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f534c549740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53441bd900 con 0x55570dd28d70 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f53437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd31800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f534c549740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53441bf440 con 0x55570dd29690 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f534c549740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53441bdfd0 con 0x55570dd33f50 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f53437fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd31800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54206/0 (socket says 172.21.15.78:54206) 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f53437fe640 1 -- 172.21.15.78:0/506511346 learned_addr learned my addr 172.21.15.78:0/506511346 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f5343fff640 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd4c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.309+0000 7f5342ffd640 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd29690 0x55570dd4bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53437fe640 1 -- 172.21.15.78:0/506511346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 msgr2=0x55570dd4c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53437fe640 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd4c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53437fe640 1 -- 172.21.15.78:0/506511346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd29690 msgr2=0x55570dd4bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53437fe640 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd29690 0x55570dd4bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53437fe640 1 -- 172.21.15.78:0/506511346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55570dd33800 con 0x55570dd28d70 2024-09-16T00:17:26.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f5342ffd640 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd29690 0x55570dd4bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53437fe640 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd31800 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55570dd26160 tx=0x7f532c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb87d536976248fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53397fa640 1 -- 172.21.15.78:0/506511346 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c041070 con 0x55570dd28d70 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53397fa640 1 -- 172.21.15.78:0/506511346 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532c042040 con 0x55570dd28d70 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.310+0000 7f53397fa640 1 -- 172.21.15.78:0/506511346 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c037040 con 0x55570dd28d70 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.311+0000 7f534c549740 1 -- 172.21.15.78:0/506511346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 msgr2=0x55570dd31800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.311+0000 7f534c549740 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd31800 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55570dd26160 tx=0x7f532c02c000 comp rx=0 tx=0).stop 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.311+0000 7f534c549740 1 -- 172.21.15.78:0/506511346 shutdown_connections 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.311+0000 7f534c549740 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd4c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.311+0000 7f534c549740 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd29690 0x55570dd4bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.312+0000 7f534c549740 1 --2- 172.21.15.78:0/506511346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd28d70 0x55570dd31800 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.312+0000 7f534c549740 1 -- 172.21.15.78:0/506511346 >> 172.21.15.78:0/506511346 conn(0x55570dd25300 msgr2=0x55570dd32ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.312+0000 7f534c549740 1 -- 172.21.15.78:0/506511346 shutdown_connections 2024-09-16T00:17:26.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.312+0000 7f534c549740 1 -- 172.21.15.78:0/506511346 wait complete. 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.312+0000 7f534c549740 1 -- start start 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f534c549740 1 Processor -- start 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f534c549740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd37e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f534c549740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd29690 0x55570dd38340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f534c549740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd38880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f534c549740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f53441bed70 con 0x55570dd29690 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f534c549740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53441be6a0 con 0x55570dd28d70 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f534c549740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53441bb8e0 con 0x55570dd33f50 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f5343fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f5343fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd38880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53110/0 (socket says 172.21.15.78:53110) 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f5343fff640 1 -- 172.21.15.78:0/501902872 learned_addr learned my addr 172.21.15.78:0/501902872 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:26.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f534c549740 10 client.0 _ll_register_callbacks cb 0x7f533c09b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:26.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.313+0000 7f53437fe640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd37e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5343fff640 1 -- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 msgr2=0x55570dd37e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:26.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5342ffd640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd29690 0x55570dd38340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5343fff640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd37e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5343fff640 1 -- 172.21.15.78:0/501902872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd29690 msgr2=0x55570dd38340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:26.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5343fff640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd29690 0x55570dd38340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5343fff640 1 -- 172.21.15.78:0/501902872 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570ddae0a0 con 0x55570dd33f50 2024-09-16T00:17:26.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f53437fe640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd37e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:17:26.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5342ffd640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd29690 0x55570dd38340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:17:26.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5343fff640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd38880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f533c099ea0 tx=0x7f533c09b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7179c6209d97a389 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:26.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5322ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:17:26.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5322ffd640 1 -- 172.21.15.78:0/501902872 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c0b9020 con 0x55570dd33f50 2024-09-16T00:17:26.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.314+0000 7f5322ffd640 1 -- 172.21.15.78:0/501902872 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f533c00e0b0 con 0x55570dd33f50 2024-09-16T00:17:26.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.315+0000 7f534c549740 1 -- 172.21.15.78:0/501902872 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f532c042040 con 0x55570dd33f50 2024-09-16T00:17:26.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.315+0000 7f5322ffd640 1 -- 172.21.15.78:0/501902872 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f533c01a040 con 0x55570dd33f50 2024-09-16T00:17:26.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.315+0000 7f534c549740 10 client.25477 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:26.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.315+0000 7f534c549740 1 -- 172.21.15.78:0/501902872 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55570ddaf340 con 0x55570dd33f50 2024-09-16T00:17:26.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.315+0000 7f5322ffd640 1 -- 172.21.15.78:0/501902872 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c0a43d0 con 0x55570dd33f50 2024-09-16T00:17:26.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.316+0000 7f5322ffd640 1 -- 172.21.15.78:0/501902872 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f533c09ab80 con 0x55570dd33f50 2024-09-16T00:17:26.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.316+0000 7f534c549740 10 client.25477 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:26.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.316+0000 7f534c549740 10 client.25477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:17:26.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.316+0000 7f534c549740 20 client.25477 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:26.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.316+0000 7f534c549740 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55570dd441c0 0x55570ddb1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:26.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.316+0000 7f534c549740 4 client.25477 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:17:26.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.316+0000 7f534c549740 1 -- 172.21.15.78:0/501902872 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f532c042040 con 0x55570dd441c0 2024-09-16T00:17:26.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.317+0000 7f53437fe640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55570dd441c0 0x55570ddb1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:26.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.318+0000 7f5322ffd640 10 client.25477 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:26.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.318+0000 7f53437fe640 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55570dd441c0 0x55570ddb1830 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8943ea066e6551b9 server_cookie=bceb87fda2af1866 in_seq=0 out_seq=0 2024-09-16T00:17:26.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f5322ffd640 1 -- 172.21.15.78:0/501902872 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f532c042040 con 0x55570dd441c0 2024-09-16T00:17:26.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f5322ffd640 10 client.25477 handle_command_reply: tid=0 2024-09-16T00:17:26.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 client.25477 shutdown 2024-09-16T00:17:26.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 -- 172.21.15.78:0/501902872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 msgr2=0x55570dd38880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd38880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f533c099ea0 tx=0x7f533c09b460 comp rx=0 tx=0).stop 2024-09-16T00:17:26.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 -- 172.21.15.78:0/501902872 shutdown_connections 2024-09-16T00:17:26.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55570dd441c0 0x55570ddb1830 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55570dd33f50 0x55570dd38880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55570dd29690 0x55570dd38340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 --2- 172.21.15.78:0/501902872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55570dd28d70 0x55570dd37e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.319+0000 7f534c549740 1 -- 172.21.15.78:0/501902872 >> 172.21.15.78:0/501902872 conn(0x55570dd25300 msgr2=0x55570dd23bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:26.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.320+0000 7f5322ffd640 0 client.25477 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:26.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.320+0000 7f534c549740 1 -- 172.21.15.78:0/501902872 shutdown_connections 2024-09-16T00:17:26.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.320+0000 7f534c549740 1 -- 172.21.15.78:0/501902872 wait complete. 2024-09-16T00:17:26.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.320+0000 7f534c549740 20 client.25477 trim_cache size 0 max 16384 2024-09-16T00:17:26.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53300a3410 msgr2=0x7f53300a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53300a3410 0x7f53300a58d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f532c0095f0 tx=0x7f532c0056d0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5344105d90 msgr2=0x7f534406c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:26.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5344105d90 0x7f534406c220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f533c00ef00 tx=0x7f533c00c740 comp rx=0 tx=0).stop 2024-09-16T00:17:26.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 shutdown_connections 2024-09-16T00:17:26.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f53300a3410 0x7f53300a58d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5344105d90 0x7f534406c220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53441012a0 0x7f534406bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 --2- 172.21.15.78:0/2506354280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5344100980 0x7f5344073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:26.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 >> 172.21.15.78:0/2506354280 conn(0x7f53440fc610 msgr2=0x7f534410f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:26.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 shutdown_connections 2024-09-16T00:17:26.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:26.321+0000 7f5348f0e640 1 -- 172.21.15.78:0/2506354280 wait complete. 2024-09-16T00:17:26.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:26 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:17:26.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:26 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:26.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:26 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:17:26.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:26 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:26.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:26 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:17:26.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:26 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.545 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:17:27.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:27 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:17:27.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:27 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:27 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:17:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:27 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:17:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:27 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:27 smithi078 ceph-mon[31157]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.8 MiB/s wr, 641 op/s 2024-09-16T00:17:27.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:27 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:27 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:17:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:27 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:27 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:17:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:27 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:17:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:27 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:27 smithi193 ceph-mon[35484]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.8 MiB/s wr, 641 op/s 2024-09-16T00:17:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:27 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.932 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:27 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:17:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:27 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:27 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:17:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:27 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:17:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:27 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:27 smithi148 ceph-mon[35599]: pgmap v482: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.8 MiB/s wr, 641 op/s 2024-09-16T00:17:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:27 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:17:28.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.512+0000 7fab20b48640 1 -- 172.21.15.78:0/228580640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c102790 msgr2=0x7fab1c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.512+0000 7fab20b48640 1 --2- 172.21.15.78:0/228580640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c102790 0x7fab1c1067e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fab10009900 tx=0x7fab1002a410 comp rx=0 tx=0).stop 2024-09-16T00:17:28.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.513+0000 7fab20b48640 1 -- 172.21.15.78:0/228580640 shutdown_connections 2024-09-16T00:17:28.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.513+0000 7fab20b48640 1 --2- 172.21.15.78:0/228580640 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c106f10 0x7fab1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.513+0000 7fab20b48640 1 --2- 172.21.15.78:0/228580640 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c102790 0x7fab1c1067e0 secure :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fab10009900 tx=0x7fab1002a410 comp rx=0 tx=0).stop 2024-09-16T00:17:28.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.513+0000 7fab20b48640 1 --2- 172.21.15.78:0/228580640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c101de0 0x7fab1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.513+0000 7fab20b48640 1 -- 172.21.15.78:0/228580640 >> 172.21.15.78:0/228580640 conn(0x7fab1c0fd8c0 msgr2=0x7fab1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:28.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.513+0000 7fab20b48640 1 -- 172.21.15.78:0/228580640 shutdown_connections 2024-09-16T00:17:28.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.513+0000 7fab20b48640 1 -- 172.21.15.78:0/228580640 wait complete. 2024-09-16T00:17:28.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.514+0000 7fab20b48640 1 Processor -- start 2024-09-16T00:17:28.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.514+0000 7fab20b48640 1 -- start start 2024-09-16T00:17:28.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.514+0000 7fab20b48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c101de0 0x7fab1c1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.514+0000 7fab20b48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c106f10 0x7fab1c1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab20b48640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c19a690 0x7fab1c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab20b48640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c111f70 con 0x7fab1c106f10 2024-09-16T00:17:28.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab20b48640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c111df0 con 0x7fab1c101de0 2024-09-16T00:17:28.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab20b48640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1120f0 con 0x7fab1c19a690 2024-09-16T00:17:28.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c101de0 0x7fab1c1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c101de0 0x7fab1c1a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37456/0 (socket says 172.21.15.78:37456) 2024-09-16T00:17:28.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1ad9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c19a690 0x7fab1c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1a59c640 1 -- 172.21.15.78:0/2961683992 learned_addr learned my addr 172.21.15.78:0/2961683992 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:28.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab19d9b640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c106f10 0x7fab1c1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1ad9d640 1 -- 172.21.15.78:0/2961683992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c101de0 msgr2=0x7fab1c1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1ad9d640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c101de0 0x7fab1c1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1ad9d640 1 -- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c106f10 msgr2=0x7fab1c1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1ad9d640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c106f10 0x7fab1c1a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1ad9d640 1 -- 172.21.15.78:0/2961683992 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1c19b240 con 0x7fab1c19a690 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab19d9b640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c106f10 0x7fab1c1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab1ad9d640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c19a690 0x7fab1c19ab10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab14009fd0 tx=0x7fab1400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.515+0000 7fab0b7fe640 1 -- 172.21.15.78:0/2961683992 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1400ce20 con 0x7fab1c19a690 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.516+0000 7fab0b7fe640 1 -- 172.21.15.78:0/2961683992 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab14005020 con 0x7fab1c19a690 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.516+0000 7fab0b7fe640 1 -- 172.21.15.78:0/2961683992 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14010720 con 0x7fab1c19a690 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.516+0000 7fab20b48640 1 -- 172.21.15.78:0/2961683992 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1c1a7370 con 0x7fab1c19a690 2024-09-16T00:17:28.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.516+0000 7fab20b48640 1 -- 172.21.15.78:0/2961683992 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1c1a7710 con 0x7fab1c19a690 2024-09-16T00:17:28.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.519+0000 7fab22f21740 1 Processor -- start 2024-09-16T00:17:28.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.519+0000 7fab0b7fe640 1 -- 172.21.15.78:0/2961683992 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fab14005540 con 0x7fab1c19a690 2024-09-16T00:17:28.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab22f21740 1 -- start start 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab22f21740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9528090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab22f21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d95285d0 0x5605d9532950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab22f21740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 0x5605d9535280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab22f21740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605d94852e0 con 0x5605d9532e90 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab22f21740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5605d942cce0 con 0x5605d9527cb0 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab22f21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605d94812a0 con 0x5605d95285d0 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab19d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9528090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab19d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9528090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37460/0 (socket says 172.21.15.78:37460) 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab19d9b640 1 -- 172.21.15.78:0/2614164975 learned_addr learned my addr 172.21.15.78:0/2614164975 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab1a59c640 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d95285d0 0x5605d9532950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab19d9b640 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 0x5605d9535280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab0b7fe640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab040a3240 0x7fab040a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab19d9b640 1 -- 172.21.15.78:0/2614164975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d95285d0 msgr2=0x5605d9532950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab19d9b640 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d95285d0 0x5605d9532950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab19d9b640 1 -- 172.21.15.78:0/2614164975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 msgr2=0x5605d9535280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.520+0000 7fab19d9b640 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 0x5605d9535280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.521+0000 7fab19d9b640 1 -- 172.21.15.78:0/2614164975 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605d9411440 con 0x5605d9527cb0 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.521+0000 7fab0b7fe640 1 -- 172.21.15.78:0/2961683992 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fab140d0300 con 0x7fab1c19a690 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.521+0000 7fab19d9b640 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9528090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab1c102cf0 tx=0x7fab10005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adbfd80d65061bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.521+0000 7fab1ad9d640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab040a3240 0x7fab040a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.521+0000 7fab08ff9640 1 -- 172.21.15.78:0/2614164975 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10035040 con 0x5605d9527cb0 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.521+0000 7fab08ff9640 1 -- 172.21.15.78:0/2614164975 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab100319d0 con 0x5605d9527cb0 2024-09-16T00:17:28.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.521+0000 7fab08ff9640 1 -- 172.21.15.78:0/2614164975 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10003ce0 con 0x5605d9527cb0 2024-09-16T00:17:28.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 -- 172.21.15.78:0/2614164975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 msgr2=0x5605d9528090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9528090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab1c102cf0 tx=0x7fab10005b70 comp rx=0 tx=0).stop 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab1ad9d640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab040a3240 0x7fab040a5700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fab14005ba0 tx=0x7fab14005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 -- 172.21.15.78:0/2614164975 shutdown_connections 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 0x5605d9535280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d95285d0 0x5605d9532950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 --2- 172.21.15.78:0/2614164975 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9528090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 -- 172.21.15.78:0/2614164975 >> 172.21.15.78:0/2614164975 conn(0x5605d9524240 msgr2=0x5605d9524620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 -- 172.21.15.78:0/2614164975 shutdown_connections 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 -- 172.21.15.78:0/2614164975 wait complete. 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.522+0000 7fab22f21740 1 -- start start 2024-09-16T00:17:28.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.523+0000 7fab22f21740 1 Processor -- start 2024-09-16T00:17:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab22f21740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9532700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab22f21740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d952f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab19d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9532700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab19d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9532700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37474/0 (socket says 172.21.15.78:37474) 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab19d9b640 1 -- 172.21.15.78:0/3494753623 learned_addr learned my addr 172.21.15.78:0/3494753623 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab22f21740 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 0x5605d952fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5605d9535a60 con 0x5605d95285d0 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5605d9440050 con 0x5605d9527cb0 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.524+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605d9535be0 con 0x5605d9532e90 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.525+0000 7fab1a59c640 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d952f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.525+0000 7fab19d9b640 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 0x5605d952fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.525+0000 7fab1a59c640 1 -- 172.21.15.78:0/3494753623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 msgr2=0x5605d952fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.525+0000 7fab1a59c640 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 0x5605d952fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.525+0000 7fab1a59c640 1 -- 172.21.15.78:0/3494753623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 msgr2=0x5605d9532700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:28.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.525+0000 7fab1a59c640 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9532700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.525+0000 7fab1a59c640 1 -- 172.21.15.78:0/3494753623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605d91f63c0 con 0x5605d95285d0 2024-09-16T00:17:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.526+0000 7fab22f21740 10 client.0 _ll_register_callbacks cb 0x5605d9480160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.526+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab100319d0 con 0x5605d95285d0 2024-09-16T00:17:28.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.526+0000 7fab1a59c640 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d952f9b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0126d0 tx=0x7fab0c012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bde09a64d2072df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.530+0000 7fab01ffb640 10 client.16167 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:17:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.530+0000 7fab01ffb640 1 -- 172.21.15.78:0/3494753623 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c004050 con 0x5605d95285d0 2024-09-16T00:17:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.530+0000 7fab01ffb640 1 -- 172.21.15.78:0/3494753623 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fab0c0041f0 con 0x5605d95285d0 2024-09-16T00:17:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.530+0000 7fab01ffb640 1 -- 172.21.15.78:0/3494753623 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab0c004430 con 0x5605d95285d0 2024-09-16T00:17:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.531+0000 7fab22f21740 10 client.16167 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:28.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.531+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5605d95411d0 con 0x5605d95285d0 2024-09-16T00:17:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.532+0000 7fab01ffb640 1 -- 172.21.15.78:0/3494753623 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fab0c00f640 con 0x5605d95285d0 2024-09-16T00:17:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.532+0000 7fab22f21740 10 client.16167 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.532+0000 7fab22f21740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:17:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.532+0000 7fab22f21740 20 client.16167 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.532+0000 7fab22f21740 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5605d95425d0 0x5605d9544a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.532+0000 7fab22f21740 4 client.16167 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:17:28.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.532+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605d9492850 con 0x5605d95425d0 2024-09-16T00:17:28.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.535+0000 7fab19d9b640 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5605d95425d0 0x5605d9544a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.552+0000 7fab19d9b640 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5605d95425d0 0x5605d9544a20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18c4acc3cd759fd server_cookie=ab355f88cd5333c3 in_seq=0 out_seq=0 2024-09-16T00:17:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.553+0000 7fab01ffb640 10 client.16167 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.554+0000 7fab01ffb640 1 -- 172.21.15.78:0/3494753623 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605d9492850 con 0x5605d95425d0 2024-09-16T00:17:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.554+0000 7fab01ffb640 10 client.16167 handle_command_reply: tid=0 2024-09-16T00:17:28.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.555+0000 7fab22f21740 1 client.16167 shutdown 2024-09-16T00:17:28.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.566+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 msgr2=0x5605d952f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.566+0000 7fab22f21740 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d952f9b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0126d0 tx=0x7fab0c012ad0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.571+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 shutdown_connections 2024-09-16T00:17:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.571+0000 7fab22f21740 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5605d95425d0 0x5605d9544a20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.571+0000 7fab22f21740 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 0x5605d952fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.571+0000 7fab22f21740 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d952f9b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.571+0000 7fab01ffb640 0 client.16167 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.572+0000 7fab22f21740 1 --2- 172.21.15.78:0/3494753623 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d9532700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.572+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 >> 172.21.15.78:0/3494753623 conn(0x5605d9524240 msgr2=0x5605d951ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:28.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.572+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 shutdown_connections 2024-09-16T00:17:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.574+0000 7fab22f21740 1 -- 172.21.15.78:0/3494753623 wait complete. 2024-09-16T00:17:28.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.575+0000 7fab22f21740 20 client.16167 trim_cache size 0 max 16384 2024-09-16T00:17:28.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.607+0000 7fab22f21740 1 Processor -- start 2024-09-16T00:17:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab22f21740 1 -- start start 2024-09-16T00:17:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab22f21740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d954a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab22f21740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d954a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab22f21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 0x5605d9529700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab22f21740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1acfe0 con 0x5605d95285d0 2024-09-16T00:17:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab22f21740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1aeb20 con 0x5605d9527cb0 2024-09-16T00:17:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab22f21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1ad6b0 con 0x5605d9532e90 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab19d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d954a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab19d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d954a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37476/0 (socket says 172.21.15.78:37476) 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab19d9b640 1 -- 172.21.15.78:0/206899427 learned_addr learned my addr 172.21.15.78:0/206899427 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.608+0000 7fab19d9b640 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 0x5605d9529700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.609+0000 7fab1a59c640 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d954a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.609+0000 7fab19d9b640 1 -- 172.21.15.78:0/206899427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 msgr2=0x5605d9529700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.609+0000 7fab19d9b640 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 0x5605d9529700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.609+0000 7fab19d9b640 1 -- 172.21.15.78:0/206899427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 msgr2=0x5605d954a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.609+0000 7fab19d9b640 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d954a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.609+0000 7fab19d9b640 1 -- 172.21.15.78:0/206899427 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605d9524070 con 0x5605d9527cb0 2024-09-16T00:17:28.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.611+0000 7fab19d9b640 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d954a330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab10001eb0 tx=0x7fab10032f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a750edc04710b250 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.611+0000 7fab03fff640 1 -- 172.21.15.78:0/206899427 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10035040 con 0x5605d9527cb0 2024-09-16T00:17:28.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.611+0000 7fab03fff640 1 -- 172.21.15.78:0/206899427 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1003d960 con 0x5605d9527cb0 2024-09-16T00:17:28.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.612+0000 7fab22f21740 1 -- 172.21.15.78:0/206899427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 msgr2=0x5605d954a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.612+0000 7fab22f21740 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d954a330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab10001eb0 tx=0x7fab10032f40 comp rx=0 tx=0).stop 2024-09-16T00:17:28.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.613+0000 7fab22f21740 1 -- 172.21.15.78:0/206899427 shutdown_connections 2024-09-16T00:17:28.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.613+0000 7fab22f21740 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9532e90 0x5605d9529700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.613+0000 7fab22f21740 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d95285d0 0x5605d954a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.613+0000 7fab22f21740 1 --2- 172.21.15.78:0/206899427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d9527cb0 0x5605d954a330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.613+0000 7fab22f21740 1 -- 172.21.15.78:0/206899427 >> 172.21.15.78:0/206899427 conn(0x5605d9524240 msgr2=0x5605d952b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:28.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.615+0000 7fab22f21740 1 -- 172.21.15.78:0/206899427 shutdown_connections 2024-09-16T00:17:28.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.616+0000 7fab22f21740 1 -- 172.21.15.78:0/206899427 wait complete. 2024-09-16T00:17:28.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.616+0000 7fab22f21740 1 -- start start 2024-09-16T00:17:28.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.616+0000 7fab22f21740 1 Processor -- start 2024-09-16T00:17:28.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab22f21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9527cb0 0x5605d95a8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab22f21740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d95285d0 0x5605d953ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab22f21740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 0x5605d953b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab22f21740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1ae450 con 0x5605d9532e90 2024-09-16T00:17:28.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab22f21740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1add80 con 0x5605d95285d0 2024-09-16T00:17:28.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab22f21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fab1c1ab020 con 0x5605d9527cb0 2024-09-16T00:17:28.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab22f21740 10 client.0 _ll_register_callbacks cb 0x7fab0c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:28.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab19d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9527cb0 0x5605d95a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab19d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9527cb0 0x5605d95a8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53156/0 (socket says 172.21.15.78:53156) 2024-09-16T00:17:28.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.617+0000 7fab19d9b640 1 -- 172.21.15.78:0/1314654149 learned_addr learned my addr 172.21.15.78:0/1314654149 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:28.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.619+0000 7fab1a59c640 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d95285d0 0x5605d953ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.620+0000 7fab19d9b640 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 0x5605d953b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.621+0000 7fab19d9b640 1 -- 172.21.15.78:0/1314654149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d95285d0 msgr2=0x5605d953ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:28.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.621+0000 7fab19d9b640 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d95285d0 0x5605d953ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.621+0000 7fab19d9b640 1 -- 172.21.15.78:0/1314654149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 msgr2=0x5605d953b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:28.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.621+0000 7fab19d9b640 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 0x5605d953b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.621+0000 7fab19d9b640 1 -- 172.21.15.78:0/1314654149 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605d953b960 con 0x5605d9527cb0 2024-09-16T00:17:28.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.621+0000 7fab19d9b640 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9527cb0 0x5605d95a8940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab10005ec0 tx=0x7fab1004c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5491900eeeccaeae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:28.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.622+0000 7fab01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:17:28.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.622+0000 7fab01ffb640 1 -- 172.21.15.78:0/1314654149 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10061070 con 0x5605d9527cb0 2024-09-16T00:17:28.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.622+0000 7fab01ffb640 1 -- 172.21.15.78:0/1314654149 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fab1003d700 con 0x5605d9527cb0 2024-09-16T00:17:28.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.623+0000 7fab22f21740 1 -- 172.21.15.78:0/1314654149 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1003d960 con 0x5605d9527cb0 2024-09-16T00:17:28.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.623+0000 7fab01ffb640 1 -- 172.21.15.78:0/1314654149 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab10035070 con 0x5605d9527cb0 2024-09-16T00:17:28.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.623+0000 7fab22f21740 10 client.25492 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:28.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.623+0000 7fab22f21740 1 -- 172.21.15.78:0/1314654149 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605d95be310 con 0x5605d9527cb0 2024-09-16T00:17:28.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.624+0000 7fab01ffb640 1 -- 172.21.15.78:0/1314654149 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1003d960 con 0x5605d9527cb0 2024-09-16T00:17:28.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.624+0000 7fab01ffb640 1 -- 172.21.15.78:0/1314654149 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fab10064020 con 0x5605d9527cb0 2024-09-16T00:17:28.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.624+0000 7fab22f21740 10 client.25492 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:28.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.624+0000 7fab22f21740 10 client.25492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:17:28.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.624+0000 7fab22f21740 20 client.25492 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:28.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.624+0000 7fab22f21740 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5605d95425d0 0x5605d95c0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:28.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.624+0000 7fab22f21740 4 client.25492 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:17:28.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.624+0000 7fab22f21740 1 -- 172.21.15.78:0/1314654149 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5605d948cd60 con 0x5605d95425d0 2024-09-16T00:17:28.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.625+0000 7fab1a59c640 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5605d95425d0 0x5605d95c0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:28.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.625+0000 7fab1a59c640 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5605d95425d0 0x5605d95c0880 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594261854518e33f server_cookie=3b3d9941a53bd325 in_seq=0 out_seq=0 2024-09-16T00:17:28.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.625+0000 7fab01ffb640 10 client.25492 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:28.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab01ffb640 1 -- 172.21.15.78:0/1314654149 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5605d948cd60 con 0x5605d95425d0 2024-09-16T00:17:28.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab01ffb640 10 client.25492 handle_command_reply: tid=0 2024-09-16T00:17:28.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 client.25492 shutdown 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 -- 172.21.15.78:0/1314654149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9527cb0 msgr2=0x5605d95a8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9527cb0 0x5605d95a8940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fab10005ec0 tx=0x7fab1004c4e0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 -- 172.21.15.78:0/1314654149 shutdown_connections 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5605d95425d0 0x5605d95c0880 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5605d9532e90 0x5605d953b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5605d95285d0 0x5605d953ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 --2- 172.21.15.78:0/1314654149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605d9527cb0 0x5605d95a8940 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 -- 172.21.15.78:0/1314654149 >> 172.21.15.78:0/1314654149 conn(0x5605d9524240 msgr2=0x5605d951a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab01ffb640 0 client.25492 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 -- 172.21.15.78:0/1314654149 shutdown_connections 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.626+0000 7fab22f21740 1 -- 172.21.15.78:0/1314654149 wait complete. 2024-09-16T00:17:28.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.627+0000 7fab22f21740 20 client.25492 trim_cache size 0 max 16384 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout: "3d3d8c98-5483-4a47-8507-25080cbce279": { 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "3d3d8c98-5483-4a47-8507-25080cbce279", 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:17:28.629 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:17:28.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 -- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab040a3240 msgr2=0x7fab040a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab040a3240 0x7fab040a5700 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fab14005ba0 tx=0x7fab14005940 comp rx=0 tx=0).stop 2024-09-16T00:17:28.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 -- 172.21.15.78:0/2961683992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c19a690 msgr2=0x7fab1c19ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:28.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c19a690 0x7fab1c19ab10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab14009fd0 tx=0x7fab1400ef10 comp rx=0 tx=0).stop 2024-09-16T00:17:28.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 -- 172.21.15.78:0/2961683992 shutdown_connections 2024-09-16T00:17:28.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab040a3240 0x7fab040a5700 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab1c19a690 0x7fab1c19ab10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab1c106f10 0x7fab1c1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 --2- 172.21.15.78:0/2961683992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fab1c101de0 0x7fab1c1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:28.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 -- 172.21.15.78:0/2961683992 >> 172.21.15.78:0/2961683992 conn(0x7fab1c0fd8c0 msgr2=0x7fab1c10d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:28.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 -- 172.21.15.78:0/2961683992 shutdown_connections 2024-09-16T00:17:28.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:28.629+0000 7fab20b48640 1 -- 172.21.15.78:0/2961683992 wait complete. 2024-09-16T00:17:29.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:29 smithi193 ceph-mon[35484]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 687 op/s 2024-09-16T00:17:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:29 smithi148 ceph-mon[35599]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 687 op/s 2024-09-16T00:17:30.001 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:29 smithi078 ceph-mon[31157]: pgmap v483: 129 pgs: 129 active+clean; 1.1 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 687 op/s 2024-09-16T00:17:30.054 INFO:tasks.cephfs.filesystem:scrub status for tag:3d3d8c98-5483-4a47-8507-25080cbce279 - {'path': '/', 'tag': '3d3d8c98-5483-4a47-8507-25080cbce279', 'options': 'recursive,force'} 2024-09-16T00:17:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 210.0 s, 2041.0 tps, lat 48.740 ms stddev 61.037 2024-09-16T00:17:31.922 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:31 smithi078 ceph-mon[31157]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 MiB/s wr, 602 op/s 2024-09-16T00:17:31.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:31 smithi193 ceph-mon[35484]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 MiB/s wr, 602 op/s 2024-09-16T00:17:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:31 smithi148 ceph-mon[35599]: pgmap v484: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 MiB/s wr, 602 op/s 2024-09-16T00:17:33.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:33 smithi193 ceph-mon[35484]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 639 op/s 2024-09-16T00:17:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:33 smithi148 ceph-mon[35599]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 639 op/s 2024-09-16T00:17:34.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:33 smithi078 ceph-mon[31157]: pgmap v485: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 639 op/s 2024-09-16T00:17:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 215.0 s, 1832.6 tps, lat 54.825 ms stddev 63.339 2024-09-16T00:17:35.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:35 smithi193 ceph-mon[35484]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.8 MiB/s wr, 609 op/s 2024-09-16T00:17:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:35 smithi148 ceph-mon[35599]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.8 MiB/s wr, 609 op/s 2024-09-16T00:17:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:35 smithi078 ceph-mon[31157]: pgmap v486: 129 pgs: 129 active+clean; 1.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.8 MiB/s wr, 609 op/s 2024-09-16T00:17:37.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:37 smithi193 ceph-mon[35484]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 621 op/s 2024-09-16T00:17:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:37 smithi148 ceph-mon[35599]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 621 op/s 2024-09-16T00:17:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:37 smithi078 ceph-mon[31157]: pgmap v487: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.9 MiB/s wr, 621 op/s 2024-09-16T00:17:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:39 smithi148 ceph-mon[35599]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-09-16T00:17:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:39 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:17:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:39 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:17:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:39 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:17:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:39 smithi078 ceph-mon[31157]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-09-16T00:17:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:39 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:17:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:39 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:17:40.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:39 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:17:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:39 smithi193 ceph-mon[35484]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 10 MiB/s wr, 809 op/s 2024-09-16T00:17:40.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:39 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:17:40.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:39 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:17:40.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:39 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:17:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 220.0 s, 1481.2 tps, lat 66.824 ms stddev 99.405 2024-09-16T00:17:41.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:41 smithi148 ceph-mon[35599]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 MiB/s wr, 746 op/s 2024-09-16T00:17:41.980 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:41 smithi193 ceph-mon[35484]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 MiB/s wr, 746 op/s 2024-09-16T00:17:42.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:41 smithi078 ceph-mon[31157]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 10 MiB/s wr, 746 op/s 2024-09-16T00:17:43.517 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.8 2024-09-16T00:17:43.518 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:17:43.921 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:43 smithi148 ceph-mon[35599]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-09-16T00:17:44.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:43 smithi078 ceph-mon[31157]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-09-16T00:17:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:43 smithi193 ceph-mon[35484]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.506+0000 7f6eda3a4640 1 -- 172.21.15.78:0/654884582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ecc0051d0 msgr2=0x7f6ecc097fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.506+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/654884582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ecc0051d0 0x7f6ecc097fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4050280 tx=0x7f6ed4096c40 comp rx=0 tx=0).stop 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.506+0000 7f6eda3a4640 1 -- 172.21.15.78:0/654884582 shutdown_connections 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.506+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/654884582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ecc0051d0 0x7f6ecc097fd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.506+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/654884582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ecc004740 0x7f6ecc004b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.506+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/654884582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ecc09a4c0 0x7f6ecc09a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.506+0000 7f6eda3a4640 1 -- 172.21.15.78:0/654884582 >> 172.21.15.78:0/654884582 conn(0x7f6ecc00a1b0 msgr2=0x7f6ecc00a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.507+0000 7f6eda3a4640 1 -- 172.21.15.78:0/654884582 shutdown_connections 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.507+0000 7f6eda3a4640 1 -- 172.21.15.78:0/654884582 wait complete. 2024-09-16T00:17:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.507+0000 7f6eda3a4640 1 Processor -- start 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6eda3a4640 1 -- start start 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6eda3a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ecc004740 0x7f6ecc143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6eda3a4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ecc0051d0 0x7f6ecc143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6eda3a4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ecc09a4c0 0x7f6ecc147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6eda3a4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0abf10 con 0x7f6ecc0051d0 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6ed93a2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ecc004740 0x7f6ecc143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6ed93a2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ecc004740 0x7f6ecc143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52842/0 (socket says 172.21.15.78:52842) 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6ed93a2640 1 -- 172.21.15.78:0/3923590790 learned_addr learned my addr 172.21.15.78:0/3923590790 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0abd90 con 0x7f6ecc004740 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc0ac090 con 0x7f6ecc09a4c0 2024-09-16T00:17:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.508+0000 7f6ed9ba3640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ecc09a4c0 0x7f6ecc147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6ed93a2640 1 -- 172.21.15.78:0/3923590790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ecc09a4c0 msgr2=0x7f6ecc147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6ed93a2640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ecc09a4c0 0x7f6ecc147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6ed93a2640 1 -- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ecc0051d0 msgr2=0x7f6ecc143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6ed93a2640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ecc0051d0 0x7f6ecc143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6ed93a2640 1 -- 172.21.15.78:0/3923590790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc1482c0 con 0x7f6ecc004740 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6ed93a2640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ecc004740 0x7f6ecc143310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000e9c0 tx=0x7f6ed000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6eca7fc640 1 -- 172.21.15.78:0/3923590790 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000ccc0 con 0x7f6ecc004740 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc1485b0 con 0x7f6ecc004740 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6eca7fc640 1 -- 172.21.15.78:0/3923590790 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ed0005500 con 0x7f6ecc004740 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc14fe90 con 0x7f6ecc004740 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.509+0000 7f6eca7fc640 1 -- 172.21.15.78:0/3923590790 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0010430 con 0x7f6ecc004740 2024-09-16T00:17:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.510+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc097fd0 con 0x7f6ecc004740 2024-09-16T00:17:44.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.512+0000 7f6eca7fc640 1 -- 172.21.15.78:0/3923590790 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6ed00046c0 con 0x7f6ecc004740 2024-09-16T00:17:44.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.513+0000 7f6eca7fc640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ec00a3350 0x7f6ec00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:44.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.513+0000 7f6eca7fc640 1 -- 172.21.15.78:0/3923590790 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6ed00cff80 con 0x7f6ecc004740 2024-09-16T00:17:44.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.513+0000 7f6ed8ba1640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ec00a3350 0x7f6ec00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:44.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.514+0000 7f6ed8ba1640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ec00a3350 0x7f6ec00a5810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00a940 tx=0x7f6ec4005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:44.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.517+0000 7f6eca7fc640 1 -- 172.21.15.78:0/3923590790 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ed009c8e0 con 0x7f6ecc004740 2024-09-16T00:17:44.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.841+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ecc144670 con 0x7f6ecc004740 2024-09-16T00:17:44.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.842+0000 7f6eca7fc640 1 -- 172.21.15.78:0/3923590790 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f6ed00bae20 con 0x7f6ecc004740 2024-09-16T00:17:44.844 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:17:44.844 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:17:44.844 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:17:44.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ec00a3350 msgr2=0x7f6ec00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:44.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ec00a3350 0x7f6ec00a5810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00a940 tx=0x7f6ec4005df0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ecc004740 msgr2=0x7f6ecc143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ecc004740 0x7f6ecc143310 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000e9c0 tx=0x7f6ed000ee80 comp rx=0 tx=0).stop 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 shutdown_connections 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6ec00a3350 0x7f6ec00a5810 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ecc09a4c0 0x7f6ecc147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ecc0051d0 0x7f6ecc143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 --2- 172.21.15.78:0/3923590790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ecc004740 0x7f6ecc143310 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.845+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 >> 172.21.15.78:0/3923590790 conn(0x7f6ecc00a1b0 msgr2=0x7f6ecc091e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.846+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 shutdown_connections 2024-09-16T00:17:44.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:44.846+0000 7f6eda3a4640 1 -- 172.21.15.78:0/3923590790 wait complete. 2024-09-16T00:17:45.737 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0' 2024-09-16T00:17:45.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0 2024-09-16T00:17:45.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 225.0 s, 1398.0 tps, lat 70.799 ms stddev 81.829 2024-09-16T00:17:46.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:45 smithi078 ceph-mon[31157]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-09-16T00:17:46.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:45 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3923590790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:45 smithi193 ceph-mon[35484]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-09-16T00:17:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:45 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3923590790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:45 smithi148 ceph-mon[35599]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-09-16T00:17:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:45 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3923590790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.827+0000 7f58f4930640 1 -- 172.21.15.78:0/1123926626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0101270 msgr2=0x7f58f010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.827+0000 7f58f4930640 1 --2- 172.21.15.78:0/1123926626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0101270 0x7f58f010d7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009be0 tx=0x7f58e802a4c0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.827+0000 7f58f4930640 1 -- 172.21.15.78:0/1123926626 shutdown_connections 2024-09-16T00:17:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.827+0000 7f58f4930640 1 --2- 172.21.15.78:0/1123926626 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f0101270 0x7f58f010d7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.827+0000 7f58f4930640 1 --2- 172.21.15.78:0/1123926626 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f01008d0 0x7f58f0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.827+0000 7f58f4930640 1 --2- 172.21.15.78:0/1123926626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01068d0 0x7f58f0106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.827+0000 7f58f4930640 1 -- 172.21.15.78:0/1123926626 >> 172.21.15.78:0/1123926626 conn(0x7f58f00fc610 msgr2=0x7f58f00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.829+0000 7f58f4930640 1 -- 172.21.15.78:0/1123926626 shutdown_connections 2024-09-16T00:17:46.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.830+0000 7f58f4930640 1 -- 172.21.15.78:0/1123926626 wait complete. 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.831+0000 7f58f4930640 1 Processor -- start 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.831+0000 7f58f4930640 1 -- start start 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.831+0000 7f58f4930640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01008d0 0x7f58f0073c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.831+0000 7f58f4930640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f0101270 0x7f58f0074190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.831+0000 7f58f4930640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f01068d0 0x7f58f006deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.831+0000 7f58f4930640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58f006c370 con 0x7f58f0101270 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.831+0000 7f58f4930640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58f006c1f0 con 0x7f58f01068d0 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.831+0000 7f58f4930640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58f006c4f0 con 0x7f58f01008d0 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.832+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01008d0 0x7f58f0073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.832+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01008d0 0x7f58f0073c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:45186/0 (socket says 172.21.15.78:45186) 2024-09-16T00:17:46.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.832+0000 7f58ef7fe640 1 -- 172.21.15.78:0/1158433042 learned_addr learned my addr 172.21.15.78:0/1158433042 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58effff640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f01068d0 0x7f58f006deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58ef7fe640 1 -- 172.21.15.78:0/1158433042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f01068d0 msgr2=0x7f58f006deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f01068d0 0x7f58f006deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58ef7fe640 1 -- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f0101270 msgr2=0x7f58f0074190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58eeffd640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f0101270 0x7f58f0074190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f0101270 0x7f58f0074190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58ef7fe640 1 -- 172.21.15.78:0/1158433042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f006e770 con 0x7f58f01008d0 2024-09-16T00:17:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58effff640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f01068d0 0x7f58f006deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:17:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.833+0000 7f58eeffd640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f0101270 0x7f58f0074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:17:46.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.836+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01008d0 0x7f58f0073c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f58e400b450 tx=0x7f58e400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:46.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.838+0000 7f58ecff9640 1 -- 172.21.15.78:0/1158433042 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4015020 con 0x7f58f01008d0 2024-09-16T00:17:46.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.838+0000 7f58ecff9640 1 -- 172.21.15.78:0/1158433042 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e4011d70 con 0x7f58f01008d0 2024-09-16T00:17:46.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.838+0000 7f58ecff9640 1 -- 172.21.15.78:0/1158433042 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e400f340 con 0x7f58f01008d0 2024-09-16T00:17:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.838+0000 7f58f4930640 1 -- 172.21.15.78:0/1158433042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f006ea60 con 0x7f58f01008d0 2024-09-16T00:17:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.838+0000 7f58f4930640 1 -- 172.21.15.78:0/1158433042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01ab3d0 con 0x7f58f01008d0 2024-09-16T00:17:46.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.842+0000 7f58f7f6b740 1 Processor -- start 2024-09-16T00:17:46.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.842+0000 7f58f7f6b740 1 -- start start 2024-09-16T00:17:46.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.842+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a42214900 0x563a42214ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.842+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 0x563a4221f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.842+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a4221fae0 0x563a42221ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.842+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563a4217a2a0 con 0x563a4221fae0 2024-09-16T00:17:46.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.842+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563a42121b10 con 0x563a42214900 2024-09-16T00:17:46.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.842+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a42176260 con 0x563a42215220 2024-09-16T00:17:46.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.843+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a4221fae0 0x563a42221ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.843+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a4221fae0 0x563a42221ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39218/0 (socket says 172.21.15.78:39218) 2024-09-16T00:17:46.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.843+0000 7f58eeffd640 1 -- 172.21.15.78:0/1427515882 learned_addr learned my addr 172.21.15.78:0/1427515882 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:46.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.843+0000 7f58eeffd640 1 -- 172.21.15.78:0/1427515882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 msgr2=0x563a4221f5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:17:46.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.843+0000 7f58eeffd640 1 --2- 172.21.15.78:0/1427515882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 0x563a4221f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.843+0000 7f58eeffd640 1 -- 172.21.15.78:0/1427515882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a42214900 msgr2=0x563a42214ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:46.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.843+0000 7f58eeffd640 1 --2- 172.21.15.78:0/1427515882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a42214900 0x563a42214ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.843+0000 7f58eeffd640 1 -- 172.21.15.78:0/1427515882 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a41eebb90 con 0x563a4221fae0 2024-09-16T00:17:46.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.844+0000 7f58eeffd640 1 --2- 172.21.15.78:0/1427515882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a4221fae0 0x563a42221ed0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f58e0020ae0 tx=0x7f58e001a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66377e63f1ba2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:46.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.846+0000 7f58ecff9640 1 -- 172.21.15.78:0/1158433042 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f58e4018020 con 0x7f58f01008d0 2024-09-16T00:17:46.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.848+0000 7f58ecff9640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58d80a3450 0x7f58d80a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.848+0000 7f58effff640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58d80a3450 0x7f58d80a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.848+0000 7f58ecff9640 1 -- 172.21.15.78:0/1158433042 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f58e40cf630 con 0x7f58f01008d0 2024-09-16T00:17:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.848+0000 7f58effff640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58d80a3450 0x7f58d80a5910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f58e8034040 tx=0x7f58e80098d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.848+0000 7f58d5ffb640 1 -- 172.21.15.78:0/1427515882 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e001ade0 con 0x563a4221fae0 2024-09-16T00:17:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.849+0000 7f58d5ffb640 1 -- 172.21.15.78:0/1427515882 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e0003610 con 0x563a4221fae0 2024-09-16T00:17:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.849+0000 7f58d5ffb640 1 -- 172.21.15.78:0/1427515882 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e00296e0 con 0x563a4221fae0 2024-09-16T00:17:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.850+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1427515882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a4221fae0 msgr2=0x563a42221ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.850+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1427515882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a4221fae0 0x563a42221ed0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f58e0020ae0 tx=0x7f58e001a270 comp rx=0 tx=0).stop 2024-09-16T00:17:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.851+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1427515882 shutdown_connections 2024-09-16T00:17:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.851+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1427515882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a4221fae0 0x563a42221ed0 secure :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f58e0020ae0 tx=0x7f58e001a270 comp rx=0 tx=0).stop 2024-09-16T00:17:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.851+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1427515882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 0x563a4221f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.851+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1427515882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a42214900 0x563a42214ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.851+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1427515882 >> 172.21.15.78:0/1427515882 conn(0x563a42210e90 msgr2=0x563a42211270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.851+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1427515882 shutdown_connections 2024-09-16T00:17:46.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.851+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1427515882 wait complete. 2024-09-16T00:17:46.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.852+0000 7f58f7f6b740 1 -- start start 2024-09-16T00:17:46.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.852+0000 7f58f7f6b740 1 Processor -- start 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.852+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a4221ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.852+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a4221d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a4222bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563a422226b0 con 0x563a42215220 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563a421026e0 con 0x563a4221d790 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a42222830 con 0x563a42214900 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a4221ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a4221ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:45192/0 (socket says 172.21.15.78:45192) 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58ef7fe640 1 -- 172.21.15.78:0/957619635 learned_addr learned my addr 172.21.15.78:0/957619635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58f7f6b740 10 client.0 _ll_register_callbacks cb 0x563a42126a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58effff640 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a4222bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58ef7fe640 1 -- 172.21.15.78:0/957619635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 msgr2=0x563a4222bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a4222bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58ef7fe640 1 -- 172.21.15.78:0/957619635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 msgr2=0x563a4221d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a4221d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.853+0000 7f58ef7fe640 1 -- 172.21.15.78:0/957619635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4222c2f0 con 0x563a42214900 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.854+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a4221ccc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f58e4000c00 tx=0x7f58e409c050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94722af754ff75db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.854+0000 7f58ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.854+0000 7f58ceffd640 1 -- 172.21.15.78:0/957619635 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e40a4500 con 0x563a42214900 2024-09-16T00:17:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.854+0000 7f58f7f6b740 1 -- 172.21.15.78:0/957619635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e0003610 con 0x563a42214900 2024-09-16T00:17:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.854+0000 7f58ceffd640 1 -- 172.21.15.78:0/957619635 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f58e40cf340 con 0x563a42214900 2024-09-16T00:17:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.857+0000 7f58ceffd640 1 -- 172.21.15.78:0/957619635 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f58e40ba080 con 0x563a42214900 2024-09-16T00:17:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.857+0000 7f58f7f6b740 10 client.25504 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.857+0000 7f58f7f6b740 1 -- 172.21.15.78:0/957619635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a4222ed30 con 0x563a42214900 2024-09-16T00:17:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.858+0000 7f58ceffd640 1 -- 172.21.15.78:0/957619635 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e40b4070 con 0x563a42214900 2024-09-16T00:17:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.858+0000 7f58ceffd640 1 -- 172.21.15.78:0/957619635 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f58e40a4170 con 0x563a42214900 2024-09-16T00:17:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.859+0000 7f58f7f6b740 10 client.25504 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.859+0000 7f58f7f6b740 10 client.25504 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:17:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.859+0000 7f58f7f6b740 20 client.25504 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.859+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a42230600 0x563a42232a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.859+0000 7f58f7f6b740 4 client.25504 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:17:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.859+0000 7f58f7f6b740 1 -- 172.21.15.78:0/957619635 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f58e0003610 con 0x563a42230600 2024-09-16T00:17:46.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.860+0000 7f58eeffd640 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a42230600 0x563a42232a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.860+0000 7f58ceffd640 10 client.25504 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:46.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.860+0000 7f58eeffd640 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a42230600 0x563a42232a60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e978c51e49cb164a server_cookie=75d822ca2a496f29 in_seq=0 out_seq=0 2024-09-16T00:17:46.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.861+0000 7f58ceffd640 1 -- 172.21.15.78:0/957619635 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f58e0003610 con 0x563a42230600 2024-09-16T00:17:46.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.861+0000 7f58ceffd640 10 client.25504 handle_command_reply: tid=0 2024-09-16T00:17:46.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.861+0000 7f58f7f6b740 1 client.25504 shutdown 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 -- 172.21.15.78:0/957619635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 msgr2=0x563a4221ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a4221ccc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f58e4000c00 tx=0x7f58e409c050 comp rx=0 tx=0).stop 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 -- 172.21.15.78:0/957619635 shutdown_connections 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a42230600 0x563a42232a60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a4222bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a4221d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/957619635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a4221ccc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 -- 172.21.15.78:0/957619635 >> 172.21.15.78:0/957619635 conn(0x563a42210e90 msgr2=0x563a42216bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58ceffd640 0 client.25504 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:46.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 -- 172.21.15.78:0/957619635 shutdown_connections 2024-09-16T00:17:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 1 -- 172.21.15.78:0/957619635 wait complete. 2024-09-16T00:17:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.862+0000 7f58f7f6b740 20 client.25504 trim_cache size 0 max 16384 2024-09-16T00:17:46.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.885+0000 7f58f7f6b740 1 Processor -- start 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.885+0000 7f58f7f6b740 1 -- start start 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.885+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42214900 0x563a4222b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.885+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 0x563a42216ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.885+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42238370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc00fac0 con 0x563a42214900 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc011600 con 0x563a4221d790 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc010190 con 0x563a42215220 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42214900 0x563a4222b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42214900 0x563a4222b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39250/0 (socket says 172.21.15.78:39250) 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 -- 172.21.15.78:0/1192342758 learned_addr learned my addr 172.21.15.78:0/1192342758 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58eeffd640 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 0x563a42216ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58effff640 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42238370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 -- 172.21.15.78:0/1192342758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 msgr2=0x563a42216ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:46.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 0x563a42216ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 -- 172.21.15.78:0/1192342758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 msgr2=0x563a42238370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:46.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42238370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 -- 172.21.15.78:0/1192342758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4221bc90 con 0x563a42214900 2024-09-16T00:17:46.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42214900 0x563a4222b9a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f58e409b590 tx=0x7f58e4009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7616ccbf4d5eef3b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:46.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.886+0000 7f58d57fa640 1 -- 172.21.15.78:0/1192342758 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e40ccdf0 con 0x563a42214900 2024-09-16T00:17:46.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58d57fa640 1 -- 172.21.15.78:0/1192342758 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e40b4040 con 0x563a42214900 2024-09-16T00:17:46.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58d57fa640 1 -- 172.21.15.78:0/1192342758 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e40a42d0 con 0x563a42214900 2024-09-16T00:17:46.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1192342758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42214900 msgr2=0x563a4222b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:46.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42214900 0x563a4222b9a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f58e409b590 tx=0x7f58e4009300 comp rx=0 tx=0).stop 2024-09-16T00:17:46.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1192342758 shutdown_connections 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42238370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42215220 0x563a42216ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1192342758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42214900 0x563a4222b9a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1192342758 >> 172.21.15.78:0/1192342758 conn(0x563a42210e90 msgr2=0x563a422163d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1192342758 shutdown_connections 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.887+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1192342758 wait complete. 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 1 -- start start 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 1 Processor -- start 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a42227b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a42228050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42224ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc010f30 con 0x563a42215220 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc010860 con 0x563a4221d790 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc00daa0 con 0x563a42214900 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58f7f6b740 10 client.0 _ll_register_callbacks cb 0x7f58e409b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a42228050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58eeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a42228050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39266/0 (socket says 172.21.15.78:39266) 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58eeffd640 1 -- 172.21.15.78:0/1199774561 learned_addr learned my addr 172.21.15.78:0/1199774561 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a42227b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.888+0000 7f58effff640 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42224ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58effff640 1 -- 172.21.15.78:0/1199774561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 msgr2=0x563a42227b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58effff640 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a42227b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58effff640 1 -- 172.21.15.78:0/1199774561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 msgr2=0x563a42228050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58effff640 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a42228050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:46.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58effff640 1 -- 172.21.15.78:0/1199774561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a422250a0 con 0x563a4221d790 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58eeffd640 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a42228050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58effff640 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42224ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f58e802aab0 tx=0x7f58e802a4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd504fb8d23952e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58ceffd640 1 -- 172.21.15.78:0/1199774561 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e804f020 con 0x563a4221d790 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1199774561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58e40b4040 con 0x563a4221d790 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.889+0000 7f58ceffd640 1 -- 172.21.15.78:0/1199774561 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f58e8004040 con 0x563a4221d790 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58ceffd640 1 -- 172.21.15.78:0/1199774561 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f58e805a050 con 0x563a4221d790 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58f7f6b740 10 client.25625 fetch_fsmap learned FSMap version 34 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1199774561 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a4229bc40 con 0x563a4221d790 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58ceffd640 1 -- 172.21.15.78:0/1199774561 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e40b4040 con 0x563a4221d790 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58ceffd640 1 -- 172.21.15.78:0/1199774561 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f58e804a070 con 0x563a4221d790 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58f7f6b740 10 client.25625 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58f7f6b740 10 client.25625 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:17:46.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58f7f6b740 20 client.25625 populate_metadata read hostname 'smithi078' 2024-09-16T00:17:46.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a4229c400 0x563a4229e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:46.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58f7f6b740 4 client.25625 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}] 2024-09-16T00:17:46.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.890+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1199774561 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}) -- 0x563a421879d0 con 0x563a4229c400 2024-09-16T00:17:46.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.892+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a4229c400 0x563a4229e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:46.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.892+0000 7f58ef7fe640 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a4229c400 0x563a4229e820 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d869a0c69bf5a3d server_cookie=b52d6ffe1f742074 in_seq=0 out_seq=0 2024-09-16T00:17:46.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:46.892+0000 7f58ceffd640 10 client.25625 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.143+0000 7f58ceffd640 1 -- 172.21.15.78:0/1199774561 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563a421879d0 con 0x563a4229c400 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.143+0000 7f58ceffd640 10 client.25625 handle_command_reply: tid=0 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.143+0000 7f58f7f6b740 1 client.25625 shutdown 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.143+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 msgr2=0x563a42224ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.143+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42224ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f58e802aab0 tx=0x7f58e802a4c0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.143+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1199774561 shutdown_connections 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.143+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a4229c400 0x563a4229e820 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.143+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a4221d790 0x563a42224ae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58ceffd640 0 client.25625 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a42215220 0x563a42228050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f7f6b740 1 --2- 172.21.15.78:0/1199774561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a42214900 0x563a42227b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1199774561 >> 172.21.15.78:0/1199774561 conn(0x563a42210e90 msgr2=0x563a42210000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1199774561 shutdown_connections 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f7f6b740 1 -- 172.21.15.78:0/1199774561 wait complete. 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f7f6b740 20 client.25625 trim_cache size 0 max 16384 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 14, 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:17:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "d80a4fc7": { 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 14, 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 133.8, 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:17:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:17:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:17:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:17:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:17:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:17:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:17:47.148 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 -- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58d80a3450 msgr2=0x7f58d80a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58d80a3450 0x7f58d80a5910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f58e8034040 tx=0x7f58e80098d0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 -- 172.21.15.78:0/1158433042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01008d0 msgr2=0x7f58f0073c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01008d0 0x7f58f0073c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f58e400b450 tx=0x7f58e400b910 comp rx=0 tx=0).stop 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 -- 172.21.15.78:0/1158433042 shutdown_connections 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58d80a3450 0x7f58d80a5910 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58f01068d0 0x7f58f006deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58f0101270 0x7f58f0074190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 --2- 172.21.15.78:0/1158433042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58f01008d0 0x7f58f0073c50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.144+0000 7f58f4930640 1 -- 172.21.15.78:0/1158433042 >> 172.21.15.78:0/1158433042 conn(0x7f58f00fc610 msgr2=0x7f58f00fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.145+0000 7f58f4930640 1 -- 172.21.15.78:0/1158433042 shutdown_connections 2024-09-16T00:17:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:47.145+0000 7f58f4930640 1 -- 172.21.15.78:0/1158433042 wait complete. 2024-09-16T00:17:48.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:47 smithi078 ceph-mon[31157]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-09-16T00:17:48.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:47 smithi193 ceph-mon[35484]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-09-16T00:17:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:47 smithi148 ceph-mon[35599]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 18 MiB/s wr, 1.06k op/s 2024-09-16T00:17:49.099 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd80a4fc7', quiesce took 5.6 seconds. Will release after: 8.200000000000001 2024-09-16T00:17:50.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:49 smithi078 ceph-mon[31157]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-16T00:17:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:49 smithi193 ceph-mon[35484]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-16T00:17:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:49 smithi148 ceph-mon[35599]: pgmap v493: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 1.12k op/s 2024-09-16T00:17:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 230.0 s, 249.5 tps, lat 89.538 ms stddev 96.181 2024-09-16T00:17:52.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:51 smithi078 ceph-mon[31157]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 847 op/s 2024-09-16T00:17:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:51 smithi193 ceph-mon[35484]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 847 op/s 2024-09-16T00:17:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:51 smithi148 ceph-mon[35599]: pgmap v494: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 847 op/s 2024-09-16T00:17:54.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:53 smithi078 ceph-mon[31157]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 729 op/s 2024-09-16T00:17:54.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:53 smithi193 ceph-mon[35484]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 729 op/s 2024-09-16T00:17:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:53 smithi148 ceph-mon[35599]: pgmap v495: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 729 op/s 2024-09-16T00:17:55.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:54 smithi078 ceph-mon[31157]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-09-16T00:17:55.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:54 smithi193 ceph-mon[35484]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-09-16T00:17:55.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:54 smithi148 ceph-mon[35599]: pgmap v496: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 470 op/s 2024-09-16T00:17:55.860 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 235.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:17:57.299 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:17:57.558 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:57.838 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:57 smithi078 ceph-mon[31157]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-16T00:17:57.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:57 smithi193 ceph-mon[35484]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-16T00:17:57.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:57 smithi148 ceph-mon[35599]: pgmap v497: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-09-16T00:17:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 -- 172.21.15.78:0/2681032862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70101420 msgr2=0x7fce7010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:58.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 --2- 172.21.15.78:0/2681032862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70101420 0x7fce7010d950 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fce68009cd0 tx=0x7fce6802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 -- 172.21.15.78:0/2681032862 shutdown_connections 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 --2- 172.21.15.78:0/2681032862 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70101420 0x7fce7010d950 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 --2- 172.21.15.78:0/2681032862 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70100a80 0x7fce70100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 --2- 172.21.15.78:0/2681032862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce70106a80 0x7fce70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 -- 172.21.15.78:0/2681032862 >> 172.21.15.78:0/2681032862 conn(0x7fce700fc7a0 msgr2=0x7fce700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 -- 172.21.15.78:0/2681032862 shutdown_connections 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 -- 172.21.15.78:0/2681032862 wait complete. 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 Processor -- start 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 -- start start 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70100a80 0x7fce7010e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:58.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.058+0000 7fce76e48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101420 0x7fce7010ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce76e48640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce70106a80 0x7fce7010f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce76e48640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce70111f20 con 0x7fce70100a80 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce76e48640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce70111da0 con 0x7fce70101420 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce76e48640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce701120a0 con 0x7fce70106a80 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70100a80 0x7fce7010e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70100a80 0x7fce7010e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54042/0 (socket says 172.21.15.78:54042) 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 -- 172.21.15.78:0/1040586324 learned_addr learned my addr 172.21.15.78:0/1040586324 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce6ffff640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101420 0x7fce7010ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce753e5640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce70106a80 0x7fce7010f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 -- 172.21.15.78:0/1040586324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce70106a80 msgr2=0x7fce7010f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce70106a80 0x7fce7010f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 -- 172.21.15.78:0/1040586324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101420 msgr2=0x7fce7010ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:58.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101420 0x7fce7010ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 -- 172.21.15.78:0/1040586324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce701a6820 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce74be4640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70100a80 0x7fce7010e670 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c890 tx=0x7fce6400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce6dffb640 1 -- 172.21.15.78:0/1040586324 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64004220 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce6dffb640 1 -- 172.21.15.78:0/1040586324 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce640043c0 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce701a6b10 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.059+0000 7fce6dffb640 1 -- 172.21.15.78:0/1040586324 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64004a00 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.060+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701a7050 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.060+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce70100ee0 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.062+0000 7fce6dffb640 1 -- 172.21.15.78:0/1040586324 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fce640079b0 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.062+0000 7fce6dffb640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce600a3240 0x7fce600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.062+0000 7fce6ffff640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce600a3240 0x7fce600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.062+0000 7fce6dffb640 1 -- 172.21.15.78:0/1040586324 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fce640cfce0 con 0x7fce70100a80 2024-09-16T00:17:58.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.063+0000 7fce6ffff640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce600a3240 0x7fce600a5700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fce70110100 tx=0x7fce5c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:17:58.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.065+0000 7fce6dffb640 1 -- 172.21.15.78:0/1040586324 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce64014030 con 0x7fce70100a80 2024-09-16T00:17:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.265+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce7010fba0 con 0x7fce70100a80 2024-09-16T00:17:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.266+0000 7fce6dffb640 1 -- 172.21.15.78:0/1040586324 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fce6409c6a0 con 0x7fce70100a80 2024-09-16T00:17:58.267 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:17:58.267 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:17:58.268 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:17:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce600a3240 msgr2=0x7fce600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce600a3240 0x7fce600a5700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fce70110100 tx=0x7fce5c00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70100a80 msgr2=0x7fce7010e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:17:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70100a80 0x7fce7010e670 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c890 tx=0x7fce6400cc90 comp rx=0 tx=0).stop 2024-09-16T00:17:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 shutdown_connections 2024-09-16T00:17:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce600a3240 0x7fce600a5700 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce70106a80 0x7fce7010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101420 0x7fce7010ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 --2- 172.21.15.78:0/1040586324 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70100a80 0x7fce7010e670 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:17:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 >> 172.21.15.78:0/1040586324 conn(0x7fce700fc7a0 msgr2=0x7fce700fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:17:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 shutdown_connections 2024-09-16T00:17:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:17:58.269+0000 7fce76e48640 1 -- 172.21.15.78:0/1040586324 wait complete. 2024-09-16T00:17:58.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:58 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1040586324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:58.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:58 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1040586324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:59.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:58 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1040586324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:17:59.684 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id d80a4fc7 --release --await' 2024-09-16T00:17:59.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id d80a4fc7 --release --await 2024-09-16T00:17:59.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:17:59 smithi193 ceph-mon[35484]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 297 op/s 2024-09-16T00:17:59.936 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:17:59.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:17:59 smithi148 ceph-mon[35599]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 297 op/s 2024-09-16T00:18:00.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:17:59 smithi078 ceph-mon[31157]: pgmap v498: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.5 MiB/s wr, 297 op/s 2024-09-16T00:18:00.055 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:18:00.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 -- 172.21.15.78:0/2844705976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 msgr2=0x7ff4041067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 --2- 172.21.15.78:0/2844705976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 0x7ff4041067d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009900 tx=0x7ff3f402a410 comp rx=0 tx=0).stop 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 -- 172.21.15.78:0/2844705976 shutdown_connections 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 --2- 172.21.15.78:0/2844705976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff404106f00 0x7ff40410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 --2- 172.21.15.78:0/2844705976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 0x7ff4041067d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 --2- 172.21.15.78:0/2844705976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff404101dd0 0x7ff4041021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 -- 172.21.15.78:0/2844705976 >> 172.21.15.78:0/2844705976 conn(0x7ff4040fd8b0 msgr2=0x7ff4040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 -- 172.21.15.78:0/2844705976 shutdown_connections 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.427+0000 7ff40be79640 1 -- 172.21.15.78:0/2844705976 wait complete. 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40be79640 1 Processor -- start 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40be79640 1 -- start start 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40be79640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff404101dd0 0x7ff4040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40be79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 0x7ff404074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40be79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff404106f00 0x7ff404074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40be79640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff404111f20 con 0x7ff404102780 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40be79640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff404111da0 con 0x7ff404101dd0 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40be79640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041120a0 con 0x7ff404106f00 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff409414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 0x7ff404074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff409414640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 0x7ff404074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54052/0 (socket says 172.21.15.78:54052) 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff409414640 1 -- 172.21.15.78:0/888046810 learned_addr learned my addr 172.21.15.78:0/888046810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff40a416640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff404106f00 0x7ff404074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.428+0000 7ff409c15640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff404101dd0 0x7ff4040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff409414640 1 -- 172.21.15.78:0/888046810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff404106f00 msgr2=0x7ff404074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff409414640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff404106f00 0x7ff404074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff409414640 1 -- 172.21.15.78:0/888046810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff404101dd0 msgr2=0x7ff4040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff409414640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff404101dd0 0x7ff4040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff409414640 1 -- 172.21.15.78:0/888046810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4041a68e0 con 0x7ff404102780 2024-09-16T00:18:00.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff409c15640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff404101dd0 0x7ff4040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:00.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff40a416640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff404106f00 0x7ff404074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:00.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff409414640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 0x7ff404074a30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4006010 tx=0x7ff3f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:00.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff3faffd640 1 -- 172.21.15.78:0/888046810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f403f030 con 0x7ff404102780 2024-09-16T00:18:00.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff40be79640 1 -- 172.21.15.78:0/888046810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041a6b70 con 0x7ff404102780 2024-09-16T00:18:00.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff3faffd640 1 -- 172.21.15.78:0/888046810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f4003ce0 con 0x7ff404102780 2024-09-16T00:18:00.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff40be79640 1 -- 172.21.15.78:0/888046810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041a70a0 con 0x7ff404102780 2024-09-16T00:18:00.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.429+0000 7ff3faffd640 1 -- 172.21.15.78:0/888046810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4049060 con 0x7ff404102780 2024-09-16T00:18:00.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40e252740 1 Processor -- start 2024-09-16T00:18:00.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40e252740 1 -- start start 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40e252740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348060 0x55b36f348440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40e252740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348980 0x55b36f352d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40e252740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 0x55b36f355630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40e252740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b36f2ad620 con 0x55b36f348980 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40e252740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b36f254e50 con 0x55b36f353240 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40e252740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b36f2a95e0 con 0x55b36f348060 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff409c15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348980 0x55b36f352d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff409c15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348980 0x55b36f352d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54058/0 (socket says 172.21.15.78:54058) 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff409c15640 1 -- 172.21.15.78:0/3765022607 learned_addr learned my addr 172.21.15.78:0/3765022607 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff40a416640 1 --2- 172.21.15.78:0/3765022607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348060 0x55b36f348440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.432+0000 7ff3faffd640 1 -- 172.21.15.78:0/888046810 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff3f4042030 con 0x7ff404102780 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff409c15640 1 -- 172.21.15.78:0/3765022607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348060 msgr2=0x55b36f348440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff409c15640 1 --2- 172.21.15.78:0/3765022607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348060 0x55b36f348440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff409c15640 1 -- 172.21.15.78:0/3765022607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 msgr2=0x55b36f355630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff409c15640 1 --2- 172.21.15.78:0/3765022607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 0x55b36f355630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff409c15640 1 -- 172.21.15.78:0/3765022607 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36f245d40 con 0x55b36f348980 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff409c15640 1 --2- 172.21.15.78:0/3765022607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348980 0x55b36f352d00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff400009c50 tx=0x7ff40000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c9b565f824fee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff3effff640 1 -- 172.21.15.78:0/3765022607 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400016070 con 0x55b36f348980 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff3effff640 1 -- 172.21.15.78:0/3765022607 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff400005500 con 0x55b36f348980 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff3effff640 1 -- 172.21.15.78:0/3765022607 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400011670 con 0x55b36f348980 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff3faffd640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3f00a3240 0x7ff3f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff40a416640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3f00a3240 0x7ff3f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.433+0000 7ff3faffd640 1 -- 172.21.15.78:0/888046810 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff3f40f0600 con 0x7ff404102780 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 -- 172.21.15.78:0/3765022607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348980 msgr2=0x55b36f352d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40a416640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3f00a3240 0x7ff3f00a5700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc005fd0 tx=0x7ff3fc009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 --2- 172.21.15.78:0/3765022607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348980 0x55b36f352d00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff400009c50 tx=0x7ff40000e6c0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 -- 172.21.15.78:0/3765022607 shutdown_connections 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 --2- 172.21.15.78:0/3765022607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 0x55b36f355630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 --2- 172.21.15.78:0/3765022607 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348980 0x55b36f352d00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 --2- 172.21.15.78:0/3765022607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348060 0x55b36f348440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 -- 172.21.15.78:0/3765022607 >> 172.21.15.78:0/3765022607 conn(0x55b36f3445f0 msgr2=0x55b36f3449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 -- 172.21.15.78:0/3765022607 shutdown_connections 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 -- 172.21.15.78:0/3765022607 wait complete. 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 -- start start 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.434+0000 7ff40e252740 1 Processor -- start 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff40e252740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f350110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff40e252740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f350650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff40e252740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 0x55b36f350b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff40e252740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b36f355e10 con 0x55b36f348060 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff40e252740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b36f235f10 con 0x55b36f353240 2024-09-16T00:18:00.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff40e252740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b36f355f90 con 0x55b36f348980 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff40e252740 10 client.0 _ll_register_callbacks cb 0x55b36f259db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409c15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f350110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409c15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f350110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54074/0 (socket says 172.21.15.78:54074) 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409c15640 1 -- 172.21.15.78:0/3269776477 learned_addr learned my addr 172.21.15.78:0/3269776477 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff40a416640 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 0x55b36f350b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409414640 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f350650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409c15640 1 -- 172.21.15.78:0/3269776477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 msgr2=0x55b36f350650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409c15640 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f350650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409c15640 1 -- 172.21.15.78:0/3269776477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 msgr2=0x55b36f350b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409c15640 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 0x55b36f350b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409c15640 1 -- 172.21.15.78:0/3269776477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36efbc580 con 0x55b36f348060 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.435+0000 7ff409414640 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f350650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:00.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff409c15640 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f350110 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff400005940 tx=0x7ff400009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dac7853157531c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff3ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/3269776477 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400016040 con 0x55b36f348060 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/3269776477 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff400004020 con 0x55b36f348060 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff40e252740 1 -- 172.21.15.78:0/3269776477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff400005500 con 0x55b36f348060 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/3269776477 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff400004240 con 0x55b36f348060 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff40e252740 10 client.16206 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff40e252740 1 -- 172.21.15.78:0/3269776477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b36f086960 con 0x55b36f348060 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/3269776477 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff400005500 con 0x55b36f348060 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/3269776477 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff40000f040 con 0x55b36f348060 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.436+0000 7ff40e252740 10 client.16206 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.437+0000 7ff40e252740 10 client.16206 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.437+0000 7ff40e252740 20 client.16206 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.437+0000 7ff40e252740 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b36f362dc0 0x55b36f3651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.437+0000 7ff40e252740 4 client.16206 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:00.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.437+0000 7ff40e252740 1 -- 172.21.15.78:0/3269776477 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b36f2bab90 con 0x55b36f362dc0 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.437+0000 7ff409414640 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b36f362dc0 0x55b36f3651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.437+0000 7ff3ed7fa640 10 client.16206 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.437+0000 7ff409414640 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b36f362dc0 0x55b36f3651b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93799fc25deb188a server_cookie=550cd1a724fe6d1c in_seq=0 out_seq=0 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/3269776477 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b36f2bab90 con 0x55b36f362dc0 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff3ed7fa640 10 client.16206 handle_command_reply: tid=0 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 client.16206 shutdown 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 -- 172.21.15.78:0/3269776477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 msgr2=0x55b36f350110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f350110 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff400005940 tx=0x7ff400009990 comp rx=0 tx=0).stop 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 -- 172.21.15.78:0/3269776477 shutdown_connections 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b36f362dc0 0x55b36f3651b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f353240 0x55b36f350b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f350650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 --2- 172.21.15.78:0/3269776477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f350110 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff3ed7fa640 0 client.16206 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:00.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.438+0000 7ff40e252740 1 -- 172.21.15.78:0/3269776477 >> 172.21.15.78:0/3269776477 conn(0x55b36f3445f0 msgr2=0x55b36f34a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:00.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.439+0000 7ff40e252740 1 -- 172.21.15.78:0/3269776477 shutdown_connections 2024-09-16T00:18:00.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.439+0000 7ff40e252740 1 -- 172.21.15.78:0/3269776477 wait complete. 2024-09-16T00:18:00.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.439+0000 7ff40e252740 20 client.16206 trim_cache size 0 max 16384 2024-09-16T00:18:00.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.455+0000 7ff40e252740 1 Processor -- start 2024-09-16T00:18:00.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff40e252740 1 -- start start 2024-09-16T00:18:00.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff40e252740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f3449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff40e252740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348980 0x55b36f34a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff40e252740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f353240 0x55b36f36a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff40e252740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041b0de0 con 0x55b36f348060 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff40e252740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041b2920 con 0x55b36f348980 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff40e252740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041b14b0 con 0x55b36f353240 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f3449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f3449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54086/0 (socket says 172.21.15.78:54086) 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 -- 172.21.15.78:0/3749819265 learned_addr learned my addr 172.21.15.78:0/3749819265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409414640 1 --2- 172.21.15.78:0/3749819265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348980 0x55b36f34a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 -- 172.21.15.78:0/3749819265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f353240 msgr2=0x55b36f36a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 --2- 172.21.15.78:0/3749819265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f353240 0x55b36f36a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 -- 172.21.15.78:0/3749819265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348980 msgr2=0x55b36f34a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 --2- 172.21.15.78:0/3749819265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348980 0x55b36f34a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 -- 172.21.15.78:0/3749819265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36f34f0c0 con 0x55b36f348060 2024-09-16T00:18:00.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.456+0000 7ff409c15640 1 --2- 172.21.15.78:0/3749819265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f3449d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff400003a90 tx=0x7ff400004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a50e883225ff42a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:00.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff3ef7fe640 1 -- 172.21.15.78:0/3749819265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4000035f0 con 0x55b36f348060 2024-09-16T00:18:00.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff3ef7fe640 1 -- 172.21.15.78:0/3749819265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff400003790 con 0x55b36f348060 2024-09-16T00:18:00.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff3ef7fe640 1 -- 172.21.15.78:0/3749819265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400011d20 con 0x55b36f348060 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff40e252740 1 -- 172.21.15.78:0/3749819265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 msgr2=0x55b36f3449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff40e252740 1 --2- 172.21.15.78:0/3749819265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f3449d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff400003a90 tx=0x7ff400004570 comp rx=0 tx=0).stop 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff40e252740 1 -- 172.21.15.78:0/3749819265 shutdown_connections 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff40e252740 1 --2- 172.21.15.78:0/3749819265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f353240 0x55b36f36a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff40e252740 1 --2- 172.21.15.78:0/3749819265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348980 0x55b36f34a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff40e252740 1 --2- 172.21.15.78:0/3749819265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f348060 0x55b36f3449d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.457+0000 7ff40e252740 1 -- 172.21.15.78:0/3749819265 >> 172.21.15.78:0/3749819265 conn(0x55b36f3445f0 msgr2=0x55b36f349ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 -- 172.21.15.78:0/3749819265 shutdown_connections 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 -- 172.21.15.78:0/3749819265 wait complete. 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 -- start start 2024-09-16T00:18:00.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 Processor -- start 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348060 0x55b36f35b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f356c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f353240 0x55b36f3571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041b2250 con 0x55b36f353240 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041b1b80 con 0x55b36f348060 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041aedc0 con 0x55b36f348980 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40e252740 10 client.0 _ll_register_callbacks cb 0x7ff400005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.458+0000 7ff40a416640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f353240 0x55b36f3571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40a416640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f353240 0x55b36f3571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54090/0 (socket says 172.21.15.78:54090) 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40a416640 1 -- 172.21.15.78:0/2329386450 learned_addr learned my addr 172.21.15.78:0/2329386450 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff409c15640 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348060 0x55b36f35b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff409414640 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f356c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40a416640 1 -- 172.21.15.78:0/2329386450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 msgr2=0x55b36f356c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40a416640 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f356c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40a416640 1 -- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348060 msgr2=0x55b36f35b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40a416640 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348060 0x55b36f35b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40a416640 1 -- 172.21.15.78:0/2329386450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36f3579b0 con 0x55b36f353240 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff409c15640 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348060 0x55b36f35b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40a416640 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f353240 0x55b36f3571d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc011c90 tx=0x7ff3fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb81594e041cac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff3ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/2329386450 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc011cc0 con 0x55b36f353240 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/2329386450 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff3fc021400 con 0x55b36f353240 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40e252740 1 -- 172.21.15.78:0/2329386450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff400003790 con 0x55b36f353240 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/2329386450 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff3fc011e60 con 0x55b36f353240 2024-09-16T00:18:00.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40e252740 10 client.16212 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.459+0000 7ff40e252740 1 -- 172.21.15.78:0/2329386450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b36f3cd9c0 con 0x55b36f353240 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/2329386450 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3fc021ae0 con 0x55b36f353240 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/2329386450 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff3fc00f040 con 0x55b36f353240 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff40e252740 10 client.16212 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff40e252740 10 client.16212 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff40e252740 20 client.16212 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff40e252740 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b36f3ce190 0x55b36f3d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff40e252740 4 client.16212 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d80a4fc7", "release": true, "await": true}] 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff40e252740 1 -- 172.21.15.78:0/2329386450 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d80a4fc7", "release": true, "await": true}) -- 0x7ff400003790 con 0x55b36f3ce190 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.460+0000 7ff409c15640 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b36f3ce190 0x55b36f3d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.461+0000 7ff409c15640 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b36f3ce190 0x55b36f3d05b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d29288c4da5bc06e server_cookie=74cdecaa23a32732 in_seq=0 out_seq=0 2024-09-16T00:18:00.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.461+0000 7ff3ed7fa640 10 client.16212 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:00.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.508+0000 7ff3ed7fa640 1 -- 172.21.15.78:0/2329386450 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7ff400003790 con 0x55b36f3ce190 2024-09-16T00:18:00.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.508+0000 7ff3ed7fa640 10 client.16212 handle_command_reply: tid=0 2024-09-16T00:18:00.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.508+0000 7ff40e252740 1 client.16212 shutdown 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 -- 172.21.15.78:0/2329386450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f353240 msgr2=0x55b36f3571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f353240 0x55b36f3571d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc011c90 tx=0x7ff3fc004ab0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 -- 172.21.15.78:0/2329386450 shutdown_connections 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55b36f3ce190 0x55b36f3d05b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b36f353240 0x55b36f3571d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff3ed7fa640 0 client.16212 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36f348980 0x55b36f356c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 --2- 172.21.15.78:0/2329386450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b36f348060 0x55b36f35b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 -- 172.21.15.78:0/2329386450 >> 172.21.15.78:0/2329386450 conn(0x55b36f3445f0 msgr2=0x55b36f34a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:00.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 -- 172.21.15.78:0/2329386450 shutdown_connections 2024-09-16T00:18:00.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 1 -- 172.21.15.78:0/2329386450 wait complete. 2024-09-16T00:18:00.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.510+0000 7ff40e252740 20 client.16212 trim_cache size 0 max 16384 2024-09-16T00:18:00.512 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:18:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:18:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:18:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 16, 2024-09-16T00:18:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "d80a4fc7": { 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 16, 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 133.8, 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:18:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:18:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:00.514 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:18:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 -- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3f00a3240 msgr2=0x7ff3f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3f00a3240 0x7ff3f00a5700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc005fd0 tx=0x7ff3fc009780 comp rx=0 tx=0).stop 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 -- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 msgr2=0x7ff404074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 0x7ff404074a30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4006010 tx=0x7ff3f40042f0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 -- 172.21.15.78:0/888046810 shutdown_connections 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3f00a3240 0x7ff3f00a5700 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff404106f00 0x7ff404074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff404102780 0x7ff404074a30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 --2- 172.21.15.78:0/888046810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff404101dd0 0x7ff4040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 -- 172.21.15.78:0/888046810 >> 172.21.15.78:0/888046810 conn(0x7ff4040fd8b0 msgr2=0x7ff4040fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 -- 172.21.15.78:0/888046810 shutdown_connections 2024-09-16T00:18:00.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:00.511+0000 7ff40be79640 1 -- 172.21.15.78:0/888046810 wait complete. 2024-09-16T00:18:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 240.0 s, 80.8 tps, lat 3315.082 ms stddev 5836.591 2024-09-16T00:18:01.387 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd80a4fc7', total seconds elapsed: 17.9 2024-09-16T00:18:01.419 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:01.843 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:01 smithi078 ceph-mon[31157]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 MiB/s wr, 155 op/s 2024-09-16T00:18:01.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:01 smithi193 ceph-mon[35484]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 MiB/s wr, 155 op/s 2024-09-16T00:18:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:01 smithi148 ceph-mon[35599]: pgmap v499: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 MiB/s wr, 155 op/s 2024-09-16T00:18:01.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 -- 172.21.15.78:0/316757856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06c200 msgr2=0x7fbebc11b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:01.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 --2- 172.21.15.78:0/316757856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06c200 0x7fbebc11b010 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009860 tx=0x7fbeac02a3d0 comp rx=0 tx=0).stop 2024-09-16T00:18:01.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 -- 172.21.15.78:0/316757856 shutdown_connections 2024-09-16T00:18:01.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 --2- 172.21.15.78:0/316757856 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc11b550 0x7fbebc11d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:01.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 --2- 172.21.15.78:0/316757856 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06c200 0x7fbebc11b010 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:01.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 --2- 172.21.15.78:0/316757856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbebc06b8e0 0x7fbebc06bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:01.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 -- 172.21.15.78:0/316757856 >> 172.21.15.78:0/316757856 conn(0x7fbebc071630 msgr2=0x7fbebc073a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:02.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 -- 172.21.15.78:0/316757856 shutdown_connections 2024-09-16T00:18:02.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.998+0000 7fbec261a640 1 -- 172.21.15.78:0/316757856 wait complete. 2024-09-16T00:18:02.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.999+0000 7fbec261a640 1 Processor -- start 2024-09-16T00:18:02.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.999+0000 7fbec261a640 1 -- start start 2024-09-16T00:18:02.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.999+0000 7fbec261a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06b8e0 0x7fbebc1168e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.999+0000 7fbec261a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc06c200 0x7fbebc10e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.999+0000 7fbec261a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbebc11b550 0x7fbebc10edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.999+0000 7fbec261a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc120e00 con 0x7fbebc06c200 2024-09-16T00:18:02.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.999+0000 7fbec261a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc120c80 con 0x7fbebc06b8e0 2024-09-16T00:18:02.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:01.999+0000 7fbec261a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc120f80 con 0x7fbebc11b550 2024-09-16T00:18:02.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.000+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06b8e0 0x7fbebc1168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.000+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc06c200 0x7fbebc10e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.000+0000 7fbec0bb7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbebc11b550 0x7fbebc10edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.000+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc06c200 0x7fbebc10e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54106/0 (socket says 172.21.15.78:54106) 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.000+0000 7fbebb7fe640 1 -- 172.21.15.78:0/2855030752 learned_addr learned my addr 172.21.15.78:0/2855030752 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.001+0000 7fbebbfff640 1 -- 172.21.15.78:0/2855030752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbebc11b550 msgr2=0x7fbebc10edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.001+0000 7fbebbfff640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbebc11b550 0x7fbebc10edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.001+0000 7fbebbfff640 1 -- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc06c200 msgr2=0x7fbebc10e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.001+0000 7fbebbfff640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc06c200 0x7fbebc10e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.001+0000 7fbebbfff640 1 -- 172.21.15.78:0/2855030752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc10f610 con 0x7fbebc06b8e0 2024-09-16T00:18:02.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.002+0000 7fbebbfff640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06b8e0 0x7fbebc1168e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400ef60 tx=0x7fbeb400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:02.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.004+0000 7fbeb97fa640 1 -- 172.21.15.78:0/2855030752 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb401b070 con 0x7fbebc06b8e0 2024-09-16T00:18:02.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.004+0000 7fbeb97fa640 1 -- 172.21.15.78:0/2855030752 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb400ba50 con 0x7fbebc06b8e0 2024-09-16T00:18:02.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.004+0000 7fbeb97fa640 1 -- 172.21.15.78:0/2855030752 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4016c90 con 0x7fbebc06b8e0 2024-09-16T00:18:02.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.005+0000 7fbec261a640 1 -- 172.21.15.78:0/2855030752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc1bc0f0 con 0x7fbebc06b8e0 2024-09-16T00:18:02.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.005+0000 7fbec261a640 1 -- 172.21.15.78:0/2855030752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc1bc5e0 con 0x7fbebc06b8e0 2024-09-16T00:18:02.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.011+0000 7fbeb97fa640 1 -- 172.21.15.78:0/2855030752 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fbeb400bc00 con 0x7fbebc06b8e0 2024-09-16T00:18:02.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec49f3740 1 Processor -- start 2024-09-16T00:18:02.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec49f3740 1 -- start start 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f20432880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec49f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f203982e0 con 0x564f204324a0 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec49f3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f2033fce0 con 0x564f2043d680 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec49f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564f203942a0 con 0x564f20432dc0 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec0bb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f20432880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec0bb7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f20432880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54116/0 (socket says 172.21.15.78:54116) 2024-09-16T00:18:02.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbec0bb7640 1 -- 172.21.15.78:0/2871319495 learned_addr learned my addr 172.21.15.78:0/2871319495 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbeb97fa640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbea80a33d0 0x7fbea80a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.012+0000 7fbebbfff640 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbec0bb7640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbea80a33d0 0x7fbea80a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbebb7fe640 1 -- 172.21.15.78:0/2871319495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 msgr2=0x564f2043d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbebb7fe640 1 -- 172.21.15.78:0/2871319495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 msgr2=0x564f20432880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f20432880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbebb7fe640 1 -- 172.21.15.78:0/2871319495 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f20324440 con 0x564f2043d680 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbec0bb7640 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f20432880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbeb97fa640 1 -- 172.21.15.78:0/2855030752 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbeb40d0010 con 0x7fbebc06b8e0 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043fa70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbeac02a9c0 tx=0x7fbeac030bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcdf18b8022950c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:02.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbec0bb7640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbea80a33d0 0x7fbea80a5890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbea4006aa0 tx=0x7fbea40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:02.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbeb27fc640 1 -- 172.21.15.78:0/2871319495 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac004230 con 0x564f2043d680 2024-09-16T00:18:02.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.013+0000 7fbeb27fc640 1 -- 172.21.15.78:0/2871319495 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeac041070 con 0x564f2043d680 2024-09-16T00:18:02.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbeb27fc640 1 -- 172.21.15.78:0/2871319495 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac005540 con 0x564f2043d680 2024-09-16T00:18:02.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbec49f3740 1 -- 172.21.15.78:0/2871319495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 msgr2=0x564f2043fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:02.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbec49f3740 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043fa70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbeac02a9c0 tx=0x7fbeac030bf0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbec49f3740 1 -- 172.21.15.78:0/2871319495 shutdown_connections 2024-09-16T00:18:02.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbec49f3740 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043fa70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbec49f3740 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbec49f3740 1 --2- 172.21.15.78:0/2871319495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f20432880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbec49f3740 1 -- 172.21.15.78:0/2871319495 >> 172.21.15.78:0/2871319495 conn(0x564f2042ea30 msgr2=0x564f2042ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.014+0000 7fbec49f3740 1 -- 172.21.15.78:0/2871319495 shutdown_connections 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.015+0000 7fbec49f3740 1 -- 172.21.15.78:0/2871319495 wait complete. 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.015+0000 7fbec49f3740 1 -- start start 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.015+0000 7fbec49f3740 1 Processor -- start 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f2043cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbec49f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x564f204402f0 con 0x564f204324a0 2024-09-16T00:18:02.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbec49f3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564f20353050 con 0x564f2043d680 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbec49f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564f20440470 con 0x564f20432dc0 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbec49f3740 10 client.0 _ll_register_callbacks cb 0x564f20393160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f2043cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f2043cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54128/0 (socket says 172.21.15.78:54128) 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 -- 172.21.15.78:0/1263368664 learned_addr learned my addr 172.21.15.78:0/1263368664 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbec0bb7640 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 -- 172.21.15.78:0/1263368664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 msgr2=0x564f2043a1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043a1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 -- 172.21.15.78:0/1263368664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 msgr2=0x564f2043a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 -- 172.21.15.78:0/1263368664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f201093c0 con 0x564f204324a0 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbebbfff640 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f2043cf20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbeb409c2c0 tx=0x7fbeb4005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce82907081abfea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:02.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.016+0000 7fbe9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:02.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.017+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/1263368664 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40d22f0 con 0x564f204324a0 2024-09-16T00:18:02.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.017+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/1263368664 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbeb40a0160 con 0x564f204324a0 2024-09-16T00:18:02.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.017+0000 7fbec49f3740 1 -- 172.21.15.78:0/1263368664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeac041070 con 0x564f204324a0 2024-09-16T00:18:02.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.017+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/1263368664 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbeb409c380 con 0x564f204324a0 2024-09-16T00:18:02.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.017+0000 7fbec49f3740 10 client.16221 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:02.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.017+0000 7fbec49f3740 1 -- 172.21.15.78:0/1263368664 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564f2044bc60 con 0x564f204324a0 2024-09-16T00:18:02.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/1263368664 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeac041070 con 0x564f204324a0 2024-09-16T00:18:02.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/1263368664 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbeb400b040 con 0x564f204324a0 2024-09-16T00:18:02.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbec49f3740 10 client.16221 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:02.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbec49f3740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:02.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbec49f3740 20 client.16221 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:02.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbec49f3740 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x564f2044d110 0x564f2044f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbec49f3740 4 client.16221 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:02.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbec49f3740 1 -- 172.21.15.78:0/1263368664 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f203a5850 con 0x564f2044d110 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.019+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x564f2044d110 0x564f2044f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.020+0000 7fbe9b7fe640 10 client.16221 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.020+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x564f2044d110 0x564f2044f560 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e375097ca3b32ef0 server_cookie=552311db0e47983b in_seq=0 out_seq=0 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/1263368664 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f203a5850 con 0x564f2044d110 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbe9b7fe640 10 client.16221 handle_command_reply: tid=0 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 client.16221 shutdown 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 -- 172.21.15.78:0/1263368664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 msgr2=0x564f2043cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f2043cf20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbeb409c2c0 tx=0x7fbeb4005b70 comp rx=0 tx=0).stop 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 -- 172.21.15.78:0/1263368664 shutdown_connections 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x564f2044d110 0x564f2044f560 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f2043a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 --2- 172.21.15.78:0/1263368664 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f2043cf20 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbec49f3740 1 -- 172.21.15.78:0/1263368664 >> 172.21.15.78:0/1263368664 conn(0x564f2042ea30 msgr2=0x564f20434760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.021+0000 7fbe9b7fe640 0 client.16221 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.022+0000 7fbec49f3740 1 -- 172.21.15.78:0/1263368664 shutdown_connections 2024-09-16T00:18:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.022+0000 7fbec49f3740 1 -- 172.21.15.78:0/1263368664 wait complete. 2024-09-16T00:18:02.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.022+0000 7fbec49f3740 20 client.16221 trim_cache size 0 max 16384 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.045+0000 7fbec49f3740 1 Processor -- start 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec49f3740 1 -- start start 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f204495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f20454e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec49f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1c6240 con 0x564f204324a0 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec49f3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1c7d80 con 0x564f2043d680 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec49f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1c6910 con 0x564f20432dc0 2024-09-16T00:18:02.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f204495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f204495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54138/0 (socket says 172.21.15.78:54138) 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebbfff640 1 -- 172.21.15.78:0/2626399104 learned_addr learned my addr 172.21.15.78:0/2626399104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec0bb7640 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f20454e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebbfff640 1 -- 172.21.15.78:0/2626399104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 msgr2=0x564f2043c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebbfff640 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebbfff640 1 -- 172.21.15.78:0/2626399104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 msgr2=0x564f20454e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebbfff640 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f20454e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbebbfff640 1 -- 172.21.15.78:0/2626399104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f2042fbc0 con 0x564f204324a0 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.046+0000 7fbec0bb7640 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f20454e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.047+0000 7fbebbfff640 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f204495a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbeb409c080 tx=0x7fbeb400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9350eac27d87c660 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.047+0000 7fbeb1ffb640 1 -- 172.21.15.78:0/2626399104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb409bec0 con 0x564f204324a0 2024-09-16T00:18:02.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.047+0000 7fbeb1ffb640 1 -- 172.21.15.78:0/2626399104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb4016410 con 0x564f204324a0 2024-09-16T00:18:02.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.047+0000 7fbeb1ffb640 1 -- 172.21.15.78:0/2626399104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40d20f0 con 0x564f204324a0 2024-09-16T00:18:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 -- 172.21.15.78:0/2626399104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 msgr2=0x564f204495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f204495a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbeb409c080 tx=0x7fbeb400bf90 comp rx=0 tx=0).stop 2024-09-16T00:18:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 -- 172.21.15.78:0/2626399104 shutdown_connections 2024-09-16T00:18:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f2043d680 0x564f20454e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f20432dc0 0x564f2043c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 --2- 172.21.15.78:0/2626399104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f204324a0 0x564f204495a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 -- 172.21.15.78:0/2626399104 >> 172.21.15.78:0/2626399104 conn(0x564f2042ea30 msgr2=0x564f2043ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 -- 172.21.15.78:0/2626399104 shutdown_connections 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 -- 172.21.15.78:0/2626399104 wait complete. 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.048+0000 7fbec49f3740 1 -- start start 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbec49f3740 1 Processor -- start 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f204324a0 0x564f204414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f20432dc0 0x564f204419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbec49f3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f2043d680 0x564f20441f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbec49f3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1c76b0 con 0x564f2043d680 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbec49f3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1c6fe0 con 0x564f20432dc0 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbec49f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc1c4220 con 0x564f204324a0 2024-09-16T00:18:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbec49f3740 10 client.0 _ll_register_callbacks cb 0x7fbeb4016630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f204324a0 0x564f204414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbebbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f204324a0 0x564f204414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:38788/0 (socket says 172.21.15.78:38788) 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbebbfff640 1 -- 172.21.15.78:0/3510305470 learned_addr learned my addr 172.21.15.78:0/3510305470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.049+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f20432dc0 0x564f204419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.050+0000 7fbec0bb7640 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f2043d680 0x564f20441f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.050+0000 7fbebb7fe640 1 -- 172.21.15.78:0/3510305470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f204324a0 msgr2=0x564f204414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.050+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f204324a0 0x564f204414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.050+0000 7fbebb7fe640 1 -- 172.21.15.78:0/3510305470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f2043d680 msgr2=0x564f20441f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.050+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f2043d680 0x564f20441f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.050+0000 7fbebb7fe640 1 -- 172.21.15.78:0/3510305470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f204b6e50 con 0x564f20432dc0 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.050+0000 7fbebbfff640 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f204324a0 0x564f204414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.050+0000 7fbebb7fe640 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f20432dc0 0x564f204419e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x564f2043b810 tx=0x7fbeac0096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5589ca183c4052ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.051+0000 7fbe9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:18:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.051+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/3510305470 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac056060 con 0x564f20432dc0 2024-09-16T00:18:02.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.051+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/3510305470 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbeac0307b0 con 0x564f20432dc0 2024-09-16T00:18:02.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.051+0000 7fbec49f3740 1 -- 172.21.15.78:0/3510305470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb4016410 con 0x564f20432dc0 2024-09-16T00:18:02.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.052+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/3510305470 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbeac009450 con 0x564f20432dc0 2024-09-16T00:18:02.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.052+0000 7fbec49f3740 10 client.25658 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.052+0000 7fbec49f3740 1 -- 172.21.15.78:0/3510305470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564f204b8180 con 0x564f20432dc0 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.052+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/3510305470 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeb4016410 con 0x564f20432dc0 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.052+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/3510305470 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbeac004aa0 con 0x564f20432dc0 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.053+0000 7fbec49f3740 10 client.25658 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.053+0000 7fbec49f3740 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.053+0000 7fbec49f3740 20 client.25658 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.053+0000 7fbec49f3740 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x564f204b84c0 0x564f204ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.053+0000 7fbec49f3740 4 client.25658 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.053+0000 7fbec49f3740 1 -- 172.21.15.78:0/3510305470 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f2039fd60 con 0x564f204b84c0 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.053+0000 7fbebbfff640 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x564f204b84c0 0x564f204ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbebbfff640 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x564f204b84c0 0x564f204ba8e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eda9de6e9ff09d5c server_cookie=a00a18d16da0d0c3 in_seq=0 out_seq=0 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbe9b7fe640 10 client.25658 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:02.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbe9b7fe640 1 -- 172.21.15.78:0/3510305470 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f2039fd60 con 0x564f204b84c0 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbe9b7fe640 10 client.25658 handle_command_reply: tid=0 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 client.25658 shutdown 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 -- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f20432dc0 msgr2=0x564f204419e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f20432dc0 0x564f204419e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x564f2043b810 tx=0x7fbeac0096d0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 -- 172.21.15.78:0/3510305470 shutdown_connections 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x564f204b84c0 0x564f204ba8e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbe9b7fe640 0 client.25658 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x564f2043d680 0x564f20441f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564f20432dc0 0x564f204419e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 --2- 172.21.15.78:0/3510305470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f204324a0 0x564f204414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 -- 172.21.15.78:0/3510305470 >> 172.21.15.78:0/3510305470 conn(0x564f2042ea30 msgr2=0x564f20433ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 -- 172.21.15.78:0/3510305470 shutdown_connections 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.054+0000 7fbec49f3740 1 -- 172.21.15.78:0/3510305470 wait complete. 2024-09-16T00:18:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec49f3740 20 client.25658 trim_cache size 0 max 16384 2024-09-16T00:18:02.060 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:18:02.060 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:18:02.060 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:18:02.060 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 -- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbea80a33d0 msgr2=0x7fbea80a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbea80a33d0 0x7fbea80a5890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbea4006aa0 tx=0x7fbea40046c0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 -- 172.21.15.78:0/2855030752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06b8e0 msgr2=0x7fbebc1168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06b8e0 0x7fbebc1168e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400ef60 tx=0x7fbeb400ef90 comp rx=0 tx=0).stop 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 -- 172.21.15.78:0/2855030752 shutdown_connections 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbea80a33d0 0x7fbea80a5890 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbebc11b550 0x7fbebc10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbebc06c200 0x7fbebc10e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 --2- 172.21.15.78:0/2855030752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbebc06b8e0 0x7fbebc1168e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.055+0000 7fbec261a640 1 -- 172.21.15.78:0/2855030752 >> 172.21.15.78:0/2855030752 conn(0x7fbebc071630 msgr2=0x7fbebc072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.056+0000 7fbec261a640 1 -- 172.21.15.78:0/2855030752 shutdown_connections 2024-09-16T00:18:02.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:02.056+0000 7fbec261a640 1 -- 172.21.15.78:0/2855030752 wait complete. 2024-09-16T00:18:02.880 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:18:02.881 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:18:03.377 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:03.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:03 smithi078 ceph-mon[31157]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-09-16T00:18:03.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:03 smithi193 ceph-mon[35484]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-09-16T00:18:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:03 smithi148 ceph-mon[35599]: pgmap v500: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.3 MiB/s wr, 170 op/s 2024-09-16T00:18:04.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.990+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/2827483987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 msgr2=0x7f4d941067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.990+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/2827483987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 0x7f4d941067d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009900 tx=0x7f4d8002a410 comp rx=0 tx=0).stop 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.992+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/2827483987 shutdown_connections 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.992+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/2827483987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d94106f00 0x7f4d9410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.992+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/2827483987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 0x7f4d941067d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.992+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/2827483987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d94101dd0 0x7f4d941021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.992+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/2827483987 >> 172.21.15.78:0/2827483987 conn(0x7f4d940fd8b0 msgr2=0x7f4d940ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.992+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/2827483987 shutdown_connections 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.992+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/2827483987 wait complete. 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.993+0000 7f4d9ac3f640 1 Processor -- start 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.993+0000 7f4d9ac3f640 1 -- start start 2024-09-16T00:18:04.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d9ac3f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d94101dd0 0x7f4d94075e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d9ac3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 0x7f4d94071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d9ac3f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d94106f00 0x7f4d94072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d9ac3f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9406c4d0 con 0x7f4d94102780 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d9ac3f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9406c350 con 0x7f4d94106f00 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d9ac3f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9406c650 con 0x7f4d94101dd0 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 0x7f4d94071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d93fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 0x7f4d94071ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54170/0 (socket says 172.21.15.78:54170) 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d93fff640 1 -- 172.21.15.78:0/3622318726 learned_addr learned my addr 172.21.15.78:0/3622318726 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d989db640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d94101dd0 0x7f4d94075e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d991dc640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d94106f00 0x7f4d94072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d93fff640 1 -- 172.21.15.78:0/3622318726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d94101dd0 msgr2=0x7f4d94075e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:04.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d94101dd0 0x7f4d94075e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d93fff640 1 -- 172.21.15.78:0/3622318726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d94106f00 msgr2=0x7f4d94072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d94106f00 0x7f4d94072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d93fff640 1 -- 172.21.15.78:0/3622318726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d94072ce0 con 0x7f4d94102780 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.994+0000 7f4d991dc640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d94106f00 0x7f4d94072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.995+0000 7f4d93fff640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 0x7f4d94071ee0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4d80004770 tx=0x7f4d80003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.995+0000 7f4d91ffb640 1 -- 172.21.15.78:0/3622318726 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8003f070 con 0x7f4d94102780 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.995+0000 7f4d91ffb640 1 -- 172.21.15.78:0/3622318726 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d80031ca0 con 0x7f4d94102780 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.995+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d941af420 con 0x7f4d94102780 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.995+0000 7f4d91ffb640 1 -- 172.21.15.78:0/3622318726 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80035030 con 0x7f4d94102780 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.995+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d941af980 con 0x7f4d94102780 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.996+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d941032b0 con 0x7f4d94102780 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.998+0000 7f4d91ffb640 1 -- 172.21.15.78:0/3622318726 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4d80042070 con 0x7f4d94102780 2024-09-16T00:18:04.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.999+0000 7f4d91ffb640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4d840a32c0 0x7f4d840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:04.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.999+0000 7f4d91ffb640 1 -- 172.21.15.78:0/3622318726 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4d800f07e0 con 0x7f4d94102780 2024-09-16T00:18:04.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:03.999+0000 7f4d989db640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4d840a32c0 0x7f4d840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:04.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.003+0000 7f4d989db640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4d840a32c0 0x7f4d840a5780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4d88005fd0 tx=0x7f4d88009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:04.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.004+0000 7f4d91ffb640 1 -- 172.21.15.78:0/3622318726 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d800bd260 con 0x7f4d94102780 2024-09-16T00:18:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.371+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d9410a0e0 con 0x7f4d94102780 2024-09-16T00:18:04.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.372+0000 7f4d91ffb640 1 -- 172.21.15.78:0/3622318726 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f4d800bcb60 con 0x7f4d94102780 2024-09-16T00:18:04.373 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:18:04.373 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:18:04.373 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:18:04.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4d840a32c0 msgr2=0x7f4d840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:04.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4d840a32c0 0x7f4d840a5780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4d88005fd0 tx=0x7f4d88009500 comp rx=0 tx=0).stop 2024-09-16T00:18:04.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 msgr2=0x7f4d94071ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:04.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 0x7f4d94071ee0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4d80004770 tx=0x7f4d80003c10 comp rx=0 tx=0).stop 2024-09-16T00:18:04.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 shutdown_connections 2024-09-16T00:18:04.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4d840a32c0 0x7f4d840a5780 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4d94106f00 0x7f4d94072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d94102780 0x7f4d94071ee0 secure :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4d80004770 tx=0x7f4d80003c10 comp rx=0 tx=0).stop 2024-09-16T00:18:04.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 --2- 172.21.15.78:0/3622318726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d94101dd0 0x7f4d94075e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:04.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.374+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 >> 172.21.15.78:0/3622318726 conn(0x7f4d940fd8b0 msgr2=0x7f4d940fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:04.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.375+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 shutdown_connections 2024-09-16T00:18:04.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:04.375+0000 7f4d9ac3f640 1 -- 172.21.15.78:0/3622318726 wait complete. 2024-09-16T00:18:04.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:04 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3622318726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:04.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:04 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3622318726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:04.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:04 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3622318726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:05.209 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:18:05.649 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 245.0 s, 1542.0 tps, lat 72.565 ms stddev 328.478 2024-09-16T00:18:05.926 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:05 smithi078 ceph-mon[31157]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.6 MiB/s wr, 201 op/s 2024-09-16T00:18:05.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:05 smithi193 ceph-mon[35484]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.6 MiB/s wr, 201 op/s 2024-09-16T00:18:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:05 smithi148 ceph-mon[35599]: pgmap v501: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.6 MiB/s wr, 201 op/s 2024-09-16T00:18:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.242+0000 7f0315242640 1 -- 172.21.15.78:0/646987249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031010c180 msgr2=0x7f031010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.242+0000 7f0315242640 1 --2- 172.21.15.78:0/646987249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031010c180 0x7f031010e610 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0308009900 tx=0x7f030802a480 comp rx=0 tx=0).stop 2024-09-16T00:18:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.243+0000 7f0315242640 1 -- 172.21.15.78:0/646987249 shutdown_connections 2024-09-16T00:18:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.243+0000 7f0315242640 1 --2- 172.21.15.78:0/646987249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f031010c180 0x7f031010e610 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.243+0000 7f0315242640 1 --2- 172.21.15.78:0/646987249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0310103700 0x7f031010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.243+0000 7f0315242640 1 --2- 172.21.15.78:0/646987249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0310102de0 0x7f03101031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.243+0000 7f0315242640 1 -- 172.21.15.78:0/646987249 >> 172.21.15.78:0/646987249 conn(0x7f03100fc7a0 msgr2=0x7f03100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.243+0000 7f0315242640 1 -- 172.21.15.78:0/646987249 shutdown_connections 2024-09-16T00:18:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.243+0000 7f0315242640 1 -- 172.21.15.78:0/646987249 wait complete. 2024-09-16T00:18:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.244+0000 7f0315242640 1 Processor -- start 2024-09-16T00:18:06.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.244+0000 7f0315242640 1 -- start start 2024-09-16T00:18:06.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.245+0000 7f0315242640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0310102de0 0x7f0310100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.245+0000 7f0315242640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0310103700 0x7f0310100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.245+0000 7f0315242640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f031010c180 0x7f03101014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f0315242640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0310111f70 con 0x7f0310103700 2024-09-16T00:18:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030f59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f031010c180 0x7f03101014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030f59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f031010c180 0x7f03101014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48202/0 (socket says 172.21.15.78:48202) 2024-09-16T00:18:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030f59e640 1 -- 172.21.15.78:0/4158119531 learned_addr learned my addr 172.21.15.78:0/4158119531 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0310111df0 con 0x7f0310102de0 2024-09-16T00:18:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03101120f0 con 0x7f031010c180 2024-09-16T00:18:06.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030ed9d640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0310102de0 0x7f0310100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030e59c640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0310103700 0x7f0310100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030f59e640 1 -- 172.21.15.78:0/4158119531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0310102de0 msgr2=0x7f0310100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030f59e640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0310102de0 0x7f0310100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030f59e640 1 -- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0310103700 msgr2=0x7f0310100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030f59e640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0310103700 0x7f0310100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.246+0000 7f030f59e640 1 -- 172.21.15.78:0/4158119531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03101a6690 con 0x7f031010c180 2024-09-16T00:18:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.247+0000 7f030f59e640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f031010c180 0x7f03101014c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0308005940 tx=0x7f03080305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.247+0000 7f02f7fff640 1 -- 172.21.15.78:0/4158119531 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030803f070 con 0x7f031010c180 2024-09-16T00:18:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.247+0000 7f02f7fff640 1 -- 172.21.15.78:0/4158119531 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0308004460 con 0x7f031010c180 2024-09-16T00:18:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.247+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03101a6920 con 0x7f031010c180 2024-09-16T00:18:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.247+0000 7f02f7fff640 1 -- 172.21.15.78:0/4158119531 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308035030 con 0x7f031010c180 2024-09-16T00:18:06.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.247+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03101a6d90 con 0x7f031010c180 2024-09-16T00:18:06.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f02f7fff640 1 -- 172.21.15.78:0/4158119531 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0308042070 con 0x7f031010c180 2024-09-16T00:18:06.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f031761b740 1 Processor -- start 2024-09-16T00:18:06.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f031761b740 1 -- start start 2024-09-16T00:18:06.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f031761b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f031761b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bc5d0 0x55e36c7bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f031761b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7cfb80 0x55e36c7d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f031761b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e36c7222f0 con 0x55e36c7bc5d0 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f031761b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e36c6c9cf0 con 0x55e36c7cfb80 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.251+0000 7f031761b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e36c71e2b0 con 0x55e36c7bdb40 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7bc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48206/0 (socket says 172.21.15.78:48206) 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 -- 172.21.15.78:0/3019791999 learned_addr learned my addr 172.21.15.78:0/3019791999 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f02f7fff640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02f80a32c0 0x7f02f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7cfb80 0x55e36c7d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030ed9d640 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bc5d0 0x55e36c7bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030f59e640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02f80a32c0 0x7f02f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 -- 172.21.15.78:0/3019791999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7cfb80 msgr2=0x55e36c7d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:06.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7cfb80 0x55e36c7d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 -- 172.21.15.78:0/3019791999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bc5d0 msgr2=0x55e36c7bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bc5d0 0x55e36c7bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 -- 172.21.15.78:0/3019791999 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36c6ae3a0 con 0x55e36c7bdb40 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f02f7fff640 1 -- 172.21.15.78:0/4158119531 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f03080f05e0 con 0x7f031010c180 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030f59e640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02f80a32c0 0x7f02f80a5780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f03080b8c50 tx=0x7f03080b8b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.252+0000 7f030e59c640 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7bc090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f03040138f0 tx=0x7f030400e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb634e469bed52 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.253+0000 7f02f57fa640 1 -- 172.21.15.78:0/3019791999 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304004020 con 0x55e36c7bdb40 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.253+0000 7f02f57fa640 1 -- 172.21.15.78:0/3019791999 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030400be10 con 0x55e36c7bdb40 2024-09-16T00:18:06.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.253+0000 7f02f57fa640 1 -- 172.21.15.78:0/3019791999 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040038e0 con 0x55e36c7bdb40 2024-09-16T00:18:06.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 -- 172.21.15.78:0/3019791999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 msgr2=0x55e36c7bc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7bc090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f03040138f0 tx=0x7f030400e430 comp rx=0 tx=0).stop 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 -- 172.21.15.78:0/3019791999 shutdown_connections 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7cfb80 0x55e36c7d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bc5d0 0x55e36c7bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 --2- 172.21.15.78:0/3019791999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7bc090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 -- 172.21.15.78:0/3019791999 >> 172.21.15.78:0/3019791999 conn(0x55e36c785310 msgr2=0x55e36c7b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 -- 172.21.15.78:0/3019791999 shutdown_connections 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 -- 172.21.15.78:0/3019791999 wait complete. 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.254+0000 7f031761b740 1 -- start start 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.255+0000 7f031761b740 1 Processor -- start 2024-09-16T00:18:06.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.255+0000 7f031761b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.255+0000 7f031761b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.255+0000 7f031761b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 0x55e36c7cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.255+0000 7f030e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f030e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59322/0 (socket says 172.21.15.78:59322) 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f031761b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e36c7bf550 con 0x55e36c7cfb80 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f031761b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e36c6dd060 con 0x55e36c7bc5d0 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f031761b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e36c7bf6d0 con 0x55e36c7bdb40 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f031761b740 10 client.0 _ll_register_callbacks cb 0x55e36c4308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f030e59c640 1 -- 172.21.15.78:0/213808830 learned_addr learned my addr 172.21.15.78:0/213808830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f030ed9d640 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f030e59c640 1 -- 172.21.15.78:0/213808830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 msgr2=0x55e36c7c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f030e59c640 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f030e59c640 1 -- 172.21.15.78:0/213808830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 msgr2=0x55e36c7cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f030e59c640 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 0x55e36c7cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.256+0000 7f030e59c640 1 -- 172.21.15.78:0/213808830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36c4933c0 con 0x55e36c7bc5d0 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.257+0000 7f030e59c640 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7c93c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f030400ee40 tx=0x7f03040045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e1a9b73513d6a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.257+0000 7f02f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.257+0000 7f02f2ff5640 1 -- 172.21.15.78:0/213808830 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304024030 con 0x55e36c7bc5d0 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.258+0000 7f031761b740 1 -- 172.21.15.78:0/213808830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030400be10 con 0x55e36c7bc5d0 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.257+0000 7f02f2ff5640 1 -- 172.21.15.78:0/213808830 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f030400b930 con 0x55e36c7bc5d0 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.258+0000 7f02f2ff5640 1 -- 172.21.15.78:0/213808830 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0304031050 con 0x55e36c7bc5d0 2024-09-16T00:18:06.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.258+0000 7f031761b740 10 client.25673 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.258+0000 7f031761b740 1 -- 172.21.15.78:0/213808830 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e36c7d5ca0 con 0x55e36c7bc5d0 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.258+0000 7f02f2ff5640 1 -- 172.21.15.78:0/213808830 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030400be10 con 0x55e36c7bc5d0 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.258+0000 7f02f2ff5640 1 -- 172.21.15.78:0/213808830 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0304004a50 con 0x55e36c7bc5d0 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f031761b740 10 client.25673 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f031761b740 10 client.25673 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f031761b740 20 client.25673 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f031761b740 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e36c7d6ff0 0x55e36c7d93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f031761b740 4 client.25673 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f031761b740 1 -- 172.21.15.78:0/213808830 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e36c72f860 con 0x55e36c7d6ff0 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f030ed9d640 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e36c7d6ff0 0x55e36c7d93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f030ed9d640 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e36c7d6ff0 0x55e36c7d93e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=910c3e20b156f738 server_cookie=e8d118b2cc183c0b in_seq=0 out_seq=0 2024-09-16T00:18:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.259+0000 7f02f2ff5640 10 client.25673 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:18:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f02f2ff5640 1 -- 172.21.15.78:0/213808830 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e36c72f860 con 0x55e36c7d6ff0 2024-09-16T00:18:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f02f2ff5640 10 client.25673 handle_command_reply: tid=0 2024-09-16T00:18:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 client.25673 shutdown 2024-09-16T00:18:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 -- 172.21.15.78:0/213808830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 msgr2=0x55e36c7c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7c93c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f030400ee40 tx=0x7f03040045f0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 -- 172.21.15.78:0/213808830 shutdown_connections 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 0x55e36c7cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e36c7d6ff0 0x55e36c7d93e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bdb40 0x55e36c7c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 --2- 172.21.15.78:0/213808830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7c93c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.260+0000 7f031761b740 1 -- 172.21.15.78:0/213808830 >> 172.21.15.78:0/213808830 conn(0x55e36c785310 msgr2=0x55e36c7be960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.261+0000 7f02f2ff5640 0 client.25673 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.261+0000 7f031761b740 1 -- 172.21.15.78:0/213808830 shutdown_connections 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.261+0000 7f031761b740 1 -- 172.21.15.78:0/213808830 wait complete. 2024-09-16T00:18:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.261+0000 7f031761b740 20 client.25673 trim_cache size 0 max 16384 2024-09-16T00:18:06.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.281+0000 7f031761b740 1 Processor -- start 2024-09-16T00:18:06.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.281+0000 7f031761b740 1 -- start start 2024-09-16T00:18:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.282+0000 7f031761b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7b8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59326/0 (socket says 172.21.15.78:59326) 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030e59c640 1 -- 172.21.15.78:0/2843783456 learned_addr learned my addr 172.21.15.78:0/2843783456 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f031761b740 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bdb40 0x55e36c7cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f031761b740 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7cfb80 0x55e36c7dc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f031761b740 1 -- 172.21.15.78:0/2843783456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ac930 con 0x55e36c7bdb40 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f031761b740 1 -- 172.21.15.78:0/2843783456 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ae470 con 0x55e36c7bc5d0 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f031761b740 1 -- 172.21.15.78:0/2843783456 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ad000 con 0x55e36c7cfb80 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030ed9d640 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bdb40 0x55e36c7cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030ed9d640 1 -- 172.21.15.78:0/2843783456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7cfb80 msgr2=0x55e36c7dc020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030ed9d640 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7cfb80 0x55e36c7dc020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030ed9d640 1 -- 172.21.15.78:0/2843783456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 msgr2=0x55e36c7b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030ed9d640 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7b8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030ed9d640 1 -- 172.21.15.78:0/2843783456 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e36c7c2c50 con 0x55e36c7bdb40 2024-09-16T00:18:06.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.285+0000 7f030ed9d640 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bdb40 0x55e36c7cf010 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55e36c7c5d20 tx=0x7f0300009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c315a62ff84ff279 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:06.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.286+0000 7f02f4ff9640 1 -- 172.21.15.78:0/2843783456 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300003b70 con 0x55e36c7bdb40 2024-09-16T00:18:06.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.286+0000 7f02f4ff9640 1 -- 172.21.15.78:0/2843783456 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0300003d10 con 0x55e36c7bdb40 2024-09-16T00:18:06.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.286+0000 7f02f4ff9640 1 -- 172.21.15.78:0/2843783456 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300021970 con 0x55e36c7bdb40 2024-09-16T00:18:06.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 -- 172.21.15.78:0/2843783456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bdb40 msgr2=0x55e36c7cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bdb40 0x55e36c7cf010 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55e36c7c5d20 tx=0x7f0300009500 comp rx=0 tx=0).stop 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 -- 172.21.15.78:0/2843783456 shutdown_connections 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7cfb80 0x55e36c7dc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7bdb40 0x55e36c7cf010 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 --2- 172.21.15.78:0/2843783456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bc5d0 0x55e36c7b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 -- 172.21.15.78:0/2843783456 >> 172.21.15.78:0/2843783456 conn(0x55e36c785310 msgr2=0x55e36c7b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 -- 172.21.15.78:0/2843783456 shutdown_connections 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 -- 172.21.15.78:0/2843783456 wait complete. 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 -- start start 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.287+0000 7f031761b740 1 Processor -- start 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f031761b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bc5d0 0x55e36c7dc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f031761b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bdb40 0x55e36c7d9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f031761b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 0x55e36c7d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f031761b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03101adda0 con 0x55e36c7cfb80 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f031761b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03101ad6d0 con 0x55e36c7bdb40 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f031761b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03101aa950 con 0x55e36c7bc5d0 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f030e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bc5d0 0x55e36c7dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f030e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bc5d0 0x55e36c7dc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48238/0 (socket says 172.21.15.78:48238) 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f030e59c640 1 -- 172.21.15.78:0/551837010 learned_addr learned my addr 172.21.15.78:0/551837010 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f030ed9d640 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bdb40 0x55e36c7d9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.288+0000 7f031761b740 10 client.0 _ll_register_callbacks cb 0x7f030400ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f030e59c640 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 0x55e36c7d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f030ed9d640 1 -- 172.21.15.78:0/551837010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bc5d0 msgr2=0x55e36c7dc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f030ed9d640 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bc5d0 0x55e36c7dc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f030ed9d640 1 -- 172.21.15.78:0/551837010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 msgr2=0x55e36c7d9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:06.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f030ed9d640 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 0x55e36c7d9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f030ed9d640 1 -- 172.21.15.78:0/551837010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36c7da7e0 con 0x55e36c7bdb40 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f030e59c640 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bc5d0 0x55e36c7dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f030ed9d640 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bdb40 0x55e36c7d9aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55e36c7c5520 tx=0x7f0300003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74682a35f3b79542 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f02f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f02f2ff5640 1 -- 172.21.15.78:0/551837010 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300004460 con 0x55e36c7bdb40 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.289+0000 7f02f2ff5640 1 -- 172.21.15.78:0/551837010 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0300004600 con 0x55e36c7bdb40 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.290+0000 7f031761b740 1 -- 172.21.15.78:0/551837010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0300003d10 con 0x55e36c7bdb40 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.290+0000 7f02f2ff5640 1 -- 172.21.15.78:0/551837010 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f030004c050 con 0x55e36c7bdb40 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.290+0000 7f031761b740 10 client.25679 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:06.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.290+0000 7f031761b740 1 -- 172.21.15.78:0/551837010 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e36c8418e0 con 0x55e36c7bdb40 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.290+0000 7f02f2ff5640 1 -- 172.21.15.78:0/551837010 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0300021a30 con 0x55e36c7bdb40 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.291+0000 7f02f2ff5640 1 -- 172.21.15.78:0/551837010 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0300039070 con 0x55e36c7bdb40 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.291+0000 7f031761b740 10 client.25679 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.291+0000 7f031761b740 10 client.25679 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.291+0000 7f031761b740 20 client.25679 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.291+0000 7f031761b740 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e36c841e80 0x55e36c8442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.291+0000 7f031761b740 4 client.25679 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.291+0000 7f031761b740 1 -- 172.21.15.78:0/551837010 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0300003d10 con 0x55e36c841e80 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.291+0000 7f030e59c640 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e36c841e80 0x55e36c8442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:06.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.292+0000 7f030e59c640 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e36c841e80 0x55e36c8442a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c879a94eb4d17a51 server_cookie=714c7bd4e37b8708 in_seq=0 out_seq=0 2024-09-16T00:18:06.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.292+0000 7f02f2ff5640 10 client.25679 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:18:06.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.293+0000 7f02f2ff5640 1 -- 172.21.15.78:0/551837010 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0300003d10 con 0x55e36c841e80 2024-09-16T00:18:06.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.293+0000 7f02f2ff5640 10 client.25679 handle_command_reply: tid=0 2024-09-16T00:18:06.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.293+0000 7f031761b740 1 client.25679 shutdown 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 -- 172.21.15.78:0/551837010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bdb40 msgr2=0x55e36c7d9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bdb40 0x55e36c7d9aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55e36c7c5520 tx=0x7f0300003ba0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 -- 172.21.15.78:0/551837010 shutdown_connections 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e36c7cfb80 0x55e36c7d9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e36c7bdb40 0x55e36c7d9aa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55e36c841e80 0x55e36c8442a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 --2- 172.21.15.78:0/551837010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e36c7bc5d0 0x55e36c7dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 -- 172.21.15.78:0/551837010 >> 172.21.15.78:0/551837010 conn(0x55e36c785310 msgr2=0x55e36c7b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:06.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f02f2ff5640 0 client.25679 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:18:06.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 -- 172.21.15.78:0/551837010 shutdown_connections 2024-09-16T00:18:06.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 1 -- 172.21.15.78:0/551837010 wait complete. 2024-09-16T00:18:06.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.295+0000 7f031761b740 20 client.25679 trim_cache size 0 max 16384 2024-09-16T00:18:06.300 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02f80a32c0 msgr2=0x7f02f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02f80a32c0 0x7f02f80a5780 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f03080b8c50 tx=0x7f03080b8b60 comp rx=0 tx=0).stop 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f031010c180 msgr2=0x7f03101014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f031010c180 0x7f03101014c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0308005940 tx=0x7f03080305d0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 shutdown_connections 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02f80a32c0 0x7f02f80a5780 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f031010c180 0x7f03101014c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0310103700 0x7f0310100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 --2- 172.21.15.78:0/4158119531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0310102de0 0x7f0310100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 >> 172.21.15.78:0/4158119531 conn(0x7f03100fc7a0 msgr2=0x7f031010dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 shutdown_connections 2024-09-16T00:18:06.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:06.296+0000 7f0315242640 1 -- 172.21.15.78:0/4158119531 wait complete. 2024-09-16T00:18:07.166 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:18:07.585 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:07.911 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:07 smithi078 ceph-mon[31157]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.0 MiB/s wr, 222 op/s 2024-09-16T00:18:07.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:07 smithi193 ceph-mon[35484]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.0 MiB/s wr, 222 op/s 2024-09-16T00:18:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:07 smithi148 ceph-mon[35599]: pgmap v502: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.0 MiB/s wr, 222 op/s 2024-09-16T00:18:08.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.230+0000 7f7ab30a6640 1 -- 172.21.15.78:0/636699348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 msgr2=0x7f7aa40a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.230+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/636699348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 0x7f7aa40a2470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7aac051500 tx=0x7f7aac09f7e0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.237+0000 7f7ab30a6640 1 -- 172.21.15.78:0/636699348 shutdown_connections 2024-09-16T00:18:08.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.237+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/636699348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 0x7f7aa40a2470 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.237+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/636699348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa409a1a0 0x7f7aa409a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.237+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/636699348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa4095cb0 0x7f7aa4096090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.237+0000 7f7ab30a6640 1 -- 172.21.15.78:0/636699348 >> 172.21.15.78:0/636699348 conn(0x7f7aa400c690 msgr2=0x7f7aa400caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:08.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.240+0000 7f7ab30a6640 1 -- 172.21.15.78:0/636699348 shutdown_connections 2024-09-16T00:18:08.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.240+0000 7f7ab30a6640 1 -- 172.21.15.78:0/636699348 wait complete. 2024-09-16T00:18:08.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.240+0000 7f7ab30a6640 1 Processor -- start 2024-09-16T00:18:08.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.245+0000 7f7ab30a6640 1 -- start start 2024-09-16T00:18:08.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.245+0000 7f7ab30a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa4095cb0 0x7f7aa41308d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.245+0000 7f7ab30a6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa409a1a0 0x7f7aa4130e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.245+0000 7f7ab30a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 0x7f7aa4129b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.245+0000 7f7ab30a6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40a36d0 con 0x7f7aa4095cb0 2024-09-16T00:18:08.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.245+0000 7f7ab30a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40a3550 con 0x7f7aa409ab60 2024-09-16T00:18:08.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.245+0000 7f7ab30a6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40a3850 con 0x7f7aa409a1a0 2024-09-16T00:18:08.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.248+0000 7f7ab28a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 0x7f7aa4129b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.248+0000 7f7ab28a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 0x7f7aa4129b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59352/0 (socket says 172.21.15.78:59352) 2024-09-16T00:18:08.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.248+0000 7f7ab28a5640 1 -- 172.21.15.78:0/352269787 learned_addr learned my addr 172.21.15.78:0/352269787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:08.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.248+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa409a1a0 0x7f7aa4130e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.249+0000 7f7ab18a3640 1 -- 172.21.15.78:0/352269787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 msgr2=0x7f7aa4129b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.249+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 0x7f7aa4129b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.249+0000 7f7ab18a3640 1 -- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa4095cb0 msgr2=0x7f7aa41308d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.249+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa4095cb0 0x7f7aa41308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.249+0000 7f7ab18a3640 1 -- 172.21.15.78:0/352269787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa412a400 con 0x7f7aa409a1a0 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.250+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa409a1a0 0x7f7aa4130e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009fd0 tx=0x7f7a9c00b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.250+0000 7f7aa37fe640 1 -- 172.21.15.78:0/352269787 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c015020 con 0x7f7aa409a1a0 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.251+0000 7f7aa37fe640 1 -- 172.21.15.78:0/352269787 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c011d10 con 0x7f7aa409a1a0 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.251+0000 7f7ab30a6640 1 -- 172.21.15.78:0/352269787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa412a690 con 0x7f7aa409a1a0 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.251+0000 7f7aa37fe640 1 -- 172.21.15.78:0/352269787 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00f2e0 con 0x7f7aa409a1a0 2024-09-16T00:18:08.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.251+0000 7f7ab30a6640 1 -- 172.21.15.78:0/352269787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa412e680 con 0x7f7aa409a1a0 2024-09-16T00:18:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.258+0000 7f7aa37fe640 1 -- 172.21.15.78:0/352269787 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7a9c018020 con 0x7f7aa409a1a0 2024-09-16T00:18:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.259+0000 7f7aa37fe640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a980a3460 0x7f7a980a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.260+0000 7f7aa37fe640 1 -- 172.21.15.78:0/352269787 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7a9c0d0490 con 0x7f7aa409a1a0 2024-09-16T00:18:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.260+0000 7f7ab66e1740 1 Processor -- start 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.260+0000 7f7ab66e1740 1 -- start start 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.260+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff15b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.261+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.261+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff168940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.261+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560aff0c12f0 con 0x560aff15cd20 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.261+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560aff068cf0 con 0x560aff166550 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.261+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560aff0bd2b0 con 0x560aff15b7b0 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.261+0000 7f7ab18a3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.261+0000 7f7ab18a3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48262/0 (socket says 172.21.15.78:48262) 2024-09-16T00:18:08.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.261+0000 7f7ab18a3640 1 -- 172.21.15.78:0/3194042448 learned_addr learned my addr 172.21.15.78:0/3194042448 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:08.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.263+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a980a3460 0x7f7a980a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.263+0000 7f7ab28a5640 1 --2- 172.21.15.78:0/3194042448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff168940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.263+0000 7f7ab18a3640 1 -- 172.21.15.78:0/3194042448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 msgr2=0x560aff168940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:08.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.263+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/3194042448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff168940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.263+0000 7f7ab18a3640 1 -- 172.21.15.78:0/3194042448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 msgr2=0x560aff15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:08.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.263+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/3194042448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.263+0000 7f7ab18a3640 1 -- 172.21.15.78:0/3194042448 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aff04d3a0 con 0x560aff15b7b0 2024-09-16T00:18:08.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.264+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/3194042448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff15bbf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009240 tx=0x7f7a9c09c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb6891594e9defdc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:08.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.264+0000 7f7aa0dea640 1 -- 172.21.15.78:0/3194042448 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0a5360 con 0x560aff15b7b0 2024-09-16T00:18:08.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.264+0000 7f7aa0dea640 1 -- 172.21.15.78:0/3194042448 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0b5070 con 0x560aff15b7b0 2024-09-16T00:18:08.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.265+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3194042448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 msgr2=0x560aff15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.265+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3194042448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff15bbf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009240 tx=0x7f7a9c09c150 comp rx=0 tx=0).stop 2024-09-16T00:18:08.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.269+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3194042448 shutdown_connections 2024-09-16T00:18:08.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.269+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3194042448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff168940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.269+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3194042448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff15bbf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.269+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3194042448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.269+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3194042448 >> 172.21.15.78:0/3194042448 conn(0x560aff124310 msgr2=0x560aff157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:08.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.269+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3194042448 shutdown_connections 2024-09-16T00:18:08.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.273+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a980a3460 0x7f7a980a5920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800ab40 tx=0x7f7aa800a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:08.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.274+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3194042448 wait complete. 2024-09-16T00:18:08.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.274+0000 7f7ab66e1740 1 -- start start 2024-09-16T00:18:08.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.274+0000 7f7ab66e1740 1 Processor -- start 2024-09-16T00:18:08.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.275+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff165e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.275+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15cd20 0x560aff163110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.275+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff166550 0x560aff163650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.275+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560aff169240 con 0x560aff166550 2024-09-16T00:18:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.275+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560aff07c060 con 0x560aff15b7b0 2024-09-16T00:18:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.275+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560aff1693c0 con 0x560aff15cd20 2024-09-16T00:18:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.276+0000 7f7ab66e1740 10 client.0 _ll_register_callbacks cb 0x560afedcf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.276+0000 7f7ab18a3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15cd20 0x560aff163110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.276+0000 7f7ab18a3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15cd20 0x560aff163110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48268/0 (socket says 172.21.15.78:48268) 2024-09-16T00:18:08.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.276+0000 7f7ab18a3640 1 -- 172.21.15.78:0/3632624991 learned_addr learned my addr 172.21.15.78:0/3632624991 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.278+0000 7f7ab28a5640 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff166550 0x560aff163650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.278+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.278+0000 7f7ab18a3640 1 -- 172.21.15.78:0/3632624991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 msgr2=0x560aff165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:08.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.278+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff165e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.278+0000 7f7ab18a3640 1 -- 172.21.15.78:0/3632624991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff166550 msgr2=0x560aff163650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.278+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff166550 0x560aff163650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.278+0000 7f7ab18a3640 1 -- 172.21.15.78:0/3632624991 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560afee323c0 con 0x560aff15cd20 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.280+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15cd20 0x560aff163110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c098b50 tx=0x7f7a9c0a54b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=123961cf21a3872e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.281+0000 7f7a95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.281+0000 7f7a95ffb640 1 -- 172.21.15.78:0/3632624991 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0aa060 con 0x560aff15cd20 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.281+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3632624991 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a9c0b5070 con 0x560aff15cd20 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.282+0000 7f7a95ffb640 1 -- 172.21.15.78:0/3632624991 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7a9c09bb40 con 0x560aff15cd20 2024-09-16T00:18:08.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.282+0000 7f7a95ffb640 1 -- 172.21.15.78:0/3632624991 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c0bb070 con 0x560aff15cd20 2024-09-16T00:18:08.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.283+0000 7f7ab66e1740 10 client.25570 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:08.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.283+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3632624991 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560aff169240 con 0x560aff15cd20 2024-09-16T00:18:08.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.288+0000 7f7a95ffb640 1 -- 172.21.15.78:0/3632624991 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c09bd80 con 0x560aff15cd20 2024-09-16T00:18:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.289+0000 7f7a95ffb640 1 -- 172.21.15.78:0/3632624991 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7a9c09b6e0 con 0x560aff15cd20 2024-09-16T00:18:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.290+0000 7f7ab66e1740 10 client.25570 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.290+0000 7f7ab66e1740 10 client.25570 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:18:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.290+0000 7f7ab66e1740 20 client.25570 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.290+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x560aff175e00 0x560aff1781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.290+0000 7f7ab66e1740 4 client.25570 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:08.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.291+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3632624991 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7a9c0b5070 con 0x560aff175e00 2024-09-16T00:18:08.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.291+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x560aff175e00 0x560aff1781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.297+0000 7f7a95ffb640 10 client.25570 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:18:08.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.297+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x560aff175e00 0x560aff1781f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6127ed0a03d84ecb server_cookie=7f38d5c2dbd25e1c in_seq=0 out_seq=0 2024-09-16T00:18:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.299+0000 7f7a95ffb640 1 -- 172.21.15.78:0/3632624991 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7a9c0b5070 con 0x560aff175e00 2024-09-16T00:18:08.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.299+0000 7f7a95ffb640 10 client.25570 handle_command_reply: tid=0 2024-09-16T00:18:08.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.299+0000 7f7ab66e1740 1 client.25570 shutdown 2024-09-16T00:18:08.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.302+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3632624991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15cd20 msgr2=0x560aff163110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.302+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15cd20 0x560aff163110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c098b50 tx=0x7f7a9c0a54b0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.302+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3632624991 shutdown_connections 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.302+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff166550 0x560aff163650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.302+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15cd20 0x560aff163110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.302+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x560aff175e00 0x560aff1781f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.302+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/3632624991 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff165e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.302+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3632624991 >> 172.21.15.78:0/3632624991 conn(0x560aff124310 msgr2=0x560aff15db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.303+0000 7f7a95ffb640 0 client.25570 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.303+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3632624991 shutdown_connections 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.303+0000 7f7ab66e1740 1 -- 172.21.15.78:0/3632624991 wait complete. 2024-09-16T00:18:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.303+0000 7f7ab66e1740 20 client.25570 trim_cache size 0 max 16384 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.328+0000 7f7ab66e1740 1 Processor -- start 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.328+0000 7f7ab66e1740 1 -- start start 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff172560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff157790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9000fa20 con 0x560aff15cd20 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90011560 con 0x560aff166550 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab18a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab28a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab28a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59378/0 (socket says 172.21.15.78:59378) 2024-09-16T00:18:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab18a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55502/0 (socket says 172.21.15.78:55502) 2024-09-16T00:18:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab28a5640 1 -- 172.21.15.78:0/487637786 learned_addr learned my addr 172.21.15.78:0/487637786 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff172560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.329+0000 7f7ab66e1740 1 -- 172.21.15.78:0/487637786 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a900100f0 con 0x560aff15b7b0 2024-09-16T00:18:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.330+0000 7f7ab20a4640 1 -- 172.21.15.78:0/487637786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 msgr2=0x560aff157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.330+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.330+0000 7f7ab20a4640 1 -- 172.21.15.78:0/487637786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 msgr2=0x560aff157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.330+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.330+0000 7f7ab20a4640 1 -- 172.21.15.78:0/487637786 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aff15c940 con 0x560aff15b7b0 2024-09-16T00:18:08.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.330+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:08.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.330+0000 7f7ab28a5640 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:08.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.330+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff172560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8005fd0 tx=0x7f7aa8010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e15cf2d3b5340b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:08.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.331+0000 7f7a97fff640 1 -- 172.21.15.78:0/487637786 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa801edb0 con 0x560aff15b7b0 2024-09-16T00:18:08.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.331+0000 7f7a97fff640 1 -- 172.21.15.78:0/487637786 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa8016040 con 0x560aff15b7b0 2024-09-16T00:18:08.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.331+0000 7f7a97fff640 1 -- 172.21.15.78:0/487637786 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8003de0 con 0x560aff15b7b0 2024-09-16T00:18:08.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.332+0000 7f7ab66e1740 1 -- 172.21.15.78:0/487637786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 msgr2=0x560aff172560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.332+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff172560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8005fd0 tx=0x7f7aa8010000 comp rx=0 tx=0).stop 2024-09-16T00:18:08.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.332+0000 7f7ab66e1740 1 -- 172.21.15.78:0/487637786 shutdown_connections 2024-09-16T00:18:08.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.332+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff166550 0x560aff157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.332+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.332+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/487637786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff15b7b0 0x560aff172560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.332+0000 7f7ab66e1740 1 -- 172.21.15.78:0/487637786 >> 172.21.15.78:0/487637786 conn(0x560aff124310 msgr2=0x560aff165190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:08.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.333+0000 7f7ab66e1740 1 -- 172.21.15.78:0/487637786 shutdown_connections 2024-09-16T00:18:08.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.333+0000 7f7ab66e1740 1 -- 172.21.15.78:0/487637786 wait complete. 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.334+0000 7f7ab66e1740 1 -- start start 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.334+0000 7f7ab66e1740 1 Processor -- start 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab66e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff166550 0x560aff160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90010e90 con 0x560aff15cd20 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a900107c0 con 0x560aff15b7b0 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab66e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9000dab0 con 0x560aff166550 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab66e1740 10 client.0 _ll_register_callbacks cb 0x7f7a9c09c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:08.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab18a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab18a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55508/0 (socket says 172.21.15.78:55508) 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.335+0000 7f7ab18a3640 1 -- 172.21.15.78:0/1774955693 learned_addr learned my addr 172.21.15.78:0/1774955693 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab28a5640 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff166550 0x560aff160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab18a3640 1 -- 172.21.15.78:0/1774955693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff166550 msgr2=0x560aff160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff166550 0x560aff160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab18a3640 1 -- 172.21.15.78:0/1774955693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 msgr2=0x560aff160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab18a3640 1 -- 172.21.15.78:0/1774955693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aff1df790 con 0x560aff15cd20 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7ab18a3640 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff160680 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0a5520 tx=0x7f7a9c09bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf9a4518b42a529 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:08.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.336+0000 7f7a95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.337+0000 7f7a95ffb640 1 -- 172.21.15.78:0/1774955693 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0aa040 con 0x560aff15cd20 2024-09-16T00:18:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.337+0000 7f7a95ffb640 1 -- 172.21.15.78:0/1774955693 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7a9c09b5b0 con 0x560aff15cd20 2024-09-16T00:18:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.337+0000 7f7ab66e1740 1 -- 172.21.15.78:0/1774955693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa8016040 con 0x560aff15cd20 2024-09-16T00:18:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.337+0000 7f7a95ffb640 1 -- 172.21.15.78:0/1774955693 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7a9c09c6f0 con 0x560aff15cd20 2024-09-16T00:18:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.337+0000 7f7ab66e1740 10 client.16257 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:08.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.337+0000 7f7ab66e1740 1 -- 172.21.15.78:0/1774955693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560aff1e0a50 con 0x560aff15cd20 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7a95ffb640 1 -- 172.21.15.78:0/1774955693 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0b5040 con 0x560aff15cd20 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7a95ffb640 1 -- 172.21.15.78:0/1774955693 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7a9c098440 con 0x560aff15cd20 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7ab66e1740 10 client.16257 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7ab66e1740 10 client.16257 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7ab66e1740 20 client.16257 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x560aff1e0fe0 0x560aff1e3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7ab66e1740 4 client.16257 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7ab66e1740 1 -- 172.21.15.78:0/1774955693 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7aa8016040 con 0x560aff1e0fe0 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.338+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x560aff1e0fe0 0x560aff1e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.339+0000 7f7a95ffb640 10 client.16257 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.339+0000 7f7ab20a4640 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x560aff1e0fe0 0x560aff1e3400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8d499ad60184a054 server_cookie=71d95cbb234b846 in_seq=0 out_seq=0 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.339+0000 7f7a95ffb640 1 -- 172.21.15.78:0/1774955693 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7aa8016040 con 0x560aff1e0fe0 2024-09-16T00:18:08.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.339+0000 7f7a95ffb640 10 client.16257 handle_command_reply: tid=0 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.339+0000 7f7ab66e1740 1 client.16257 shutdown 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 -- 172.21.15.78:0/1774955693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 msgr2=0x560aff160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff160680 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0a5520 tx=0x7f7a9c09bd70 comp rx=0 tx=0).stop 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 -- 172.21.15.78:0/1774955693 shutdown_connections 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aff166550 0x560aff160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aff15cd20 0x560aff160680 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x560aff1e0fe0 0x560aff1e3400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 --2- 172.21.15.78:0/1774955693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aff15b7b0 0x560aff160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 -- 172.21.15.78:0/1774955693 >> 172.21.15.78:0/1774955693 conn(0x560aff124310 msgr2=0x560aff15d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7a95ffb640 0 client.16257 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 -- 172.21.15.78:0/1774955693 shutdown_connections 2024-09-16T00:18:08.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 1 -- 172.21.15.78:0/1774955693 wait complete. 2024-09-16T00:18:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.340+0000 7f7ab66e1740 20 client.16257 trim_cache size 0 max 16384 2024-09-16T00:18:08.345 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:18:08.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 -- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a980a3460 msgr2=0x7f7a980a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a980a3460 0x7f7a980a5920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800ab40 tx=0x7f7aa800a430 comp rx=0 tx=0).stop 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 -- 172.21.15.78:0/352269787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa409a1a0 msgr2=0x7f7aa4130e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa409a1a0 0x7f7aa4130e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009fd0 tx=0x7f7a9c00b870 comp rx=0 tx=0).stop 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 -- 172.21.15.78:0/352269787 shutdown_connections 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a980a3460 0x7f7a980a5920 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7aa409ab60 0x7f7aa4129b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aa409a1a0 0x7f7aa4130e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 --2- 172.21.15.78:0/352269787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7aa4095cb0 0x7f7aa41308d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 -- 172.21.15.78:0/352269787 >> 172.21.15.78:0/352269787 conn(0x7f7aa400c690 msgr2=0x7f7aa40934c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 -- 172.21.15.78:0/352269787 shutdown_connections 2024-09-16T00:18:08.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:08.341+0000 7f7ab30a6640 1 -- 172.21.15.78:0/352269787 wait complete. 2024-09-16T00:18:09.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:18:09.773 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:09 smithi078 ceph-mon[31157]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.4 MiB/s wr, 393 op/s 2024-09-16T00:18:09.906 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:09.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:09 smithi193 ceph-mon[35484]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.4 MiB/s wr, 393 op/s 2024-09-16T00:18:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:09 smithi148 ceph-mon[35599]: pgmap v503: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.4 MiB/s wr, 393 op/s 2024-09-16T00:18:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 250.0 s, 1747.0 tps, lat 56.360 ms stddev 74.405 2024-09-16T00:18:10.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.863+0000 7fba7beb4640 1 -- 172.21.15.78:0/1495407412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 msgr2=0x7fba74116970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.863+0000 7fba7beb4640 1 --2- 172.21.15.78:0/1495407412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 0x7fba74116970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fba64009900 tx=0x7fba6402a410 comp rx=0 tx=0).stop 2024-09-16T00:18:10.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.864+0000 7fba7beb4640 1 -- 172.21.15.78:0/1495407412 shutdown_connections 2024-09-16T00:18:10.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.864+0000 7fba7beb4640 1 --2- 172.21.15.78:0/1495407412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba74116eb0 0x7fba741192a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.864+0000 7fba7beb4640 1 --2- 172.21.15.78:0/1495407412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 0x7fba74116970 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.864+0000 7fba7beb4640 1 --2- 172.21.15.78:0/1495407412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba741091f0 0x7fba741095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.864+0000 7fba7beb4640 1 -- 172.21.15.78:0/1495407412 >> 172.21.15.78:0/1495407412 conn(0x7fba7406e140 msgr2=0x7fba7406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:10.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.865+0000 7fba7beb4640 1 -- 172.21.15.78:0/1495407412 shutdown_connections 2024-09-16T00:18:10.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.866+0000 7fba7beb4640 1 -- 172.21.15.78:0/1495407412 wait complete. 2024-09-16T00:18:10.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.867+0000 7fba7beb4640 1 Processor -- start 2024-09-16T00:18:10.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.868+0000 7fba7beb4640 1 -- start start 2024-09-16T00:18:10.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7beb4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba741091f0 0x7fba741a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7beb4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 0x7fba741a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7beb4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba74116eb0 0x7fba741abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7beb4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba74113790 con 0x7fba741091f0 2024-09-16T00:18:10.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7beb4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba74113610 con 0x7fba74116eb0 2024-09-16T00:18:10.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7beb4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba74113910 con 0x7fba74109b10 2024-09-16T00:18:10.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7a451640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba74116eb0 0x7fba741abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7a451640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba74116eb0 0x7fba741abbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59408/0 (socket says 172.21.15.78:59408) 2024-09-16T00:18:10.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7a451640 1 -- 172.21.15.78:0/3915869574 learned_addr learned my addr 172.21.15.78:0/3915869574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:10.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.869+0000 7fba7944f640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 0x7fba741a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.870+0000 7fba79c50640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba741091f0 0x7fba741a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.871+0000 7fba7a451640 1 -- 172.21.15.78:0/3915869574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 msgr2=0x7fba741a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.871+0000 7fba7a451640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 0x7fba741a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.871+0000 7fba7a451640 1 -- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba741091f0 msgr2=0x7fba741a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.871+0000 7fba7a451640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba741091f0 0x7fba741a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.871+0000 7fba7a451640 1 -- 172.21.15.78:0/3915869574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba741ac2a0 con 0x7fba74116eb0 2024-09-16T00:18:10.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.873+0000 7fba7944f640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 0x7fba741a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:10.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.873+0000 7fba7a451640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba74116eb0 0x7fba741abbc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00cab0 tx=0x7fba6c00cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:10.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.873+0000 7fba6affd640 1 -- 172.21.15.78:0/3915869574 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00ae20 con 0x7fba74116eb0 2024-09-16T00:18:10.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.874+0000 7fba7beb4640 1 -- 172.21.15.78:0/3915869574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba741ac590 con 0x7fba74116eb0 2024-09-16T00:18:10.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.874+0000 7fba7beb4640 1 -- 172.21.15.78:0/3915869574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba741b3e70 con 0x7fba74116eb0 2024-09-16T00:18:10.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.874+0000 7fba6affd640 1 -- 172.21.15.78:0/3915869574 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6c00f930 con 0x7fba74116eb0 2024-09-16T00:18:10.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.874+0000 7fba6affd640 1 -- 172.21.15.78:0/3915869574 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c018b80 con 0x7fba74116eb0 2024-09-16T00:18:10.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.879+0000 7fba7e28d740 1 Processor -- start 2024-09-16T00:18:10.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7e28d740 1 -- start start 2024-09-16T00:18:10.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb28972090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb289725d0 0x55eb28972a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb2897d370 0x55eb2897f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7e28d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb288d82f0 con 0x55eb28973b40 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7e28d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb2887fcf0 con 0x55eb2897d370 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7e28d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb288d42b0 con 0x55eb289725d0 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7944f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb2897d370 0x55eb2897f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba79c50640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb289725d0 0x55eb28972a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7944f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb2897d370 0x55eb2897f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59414/0 (socket says 172.21.15.78:59414) 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.880+0000 7fba7944f640 1 -- 172.21.15.78:0/969566020 learned_addr learned my addr 172.21.15.78:0/969566020 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.881+0000 7fba6affd640 1 -- 172.21.15.78:0/3915869574 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fba6c00fae0 con 0x7fba74116eb0 2024-09-16T00:18:10.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.881+0000 7fba7944f640 1 --2- 172.21.15.78:0/969566020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb28972090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.881+0000 7fba7944f640 1 -- 172.21.15.78:0/969566020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb289725d0 msgr2=0x55eb28972a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:10.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.881+0000 7fba7944f640 1 --2- 172.21.15.78:0/969566020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb289725d0 0x55eb28972a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.881+0000 7fba7944f640 1 -- 172.21.15.78:0/969566020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 msgr2=0x55eb28972090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:10.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.881+0000 7fba7944f640 1 --2- 172.21.15.78:0/969566020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb28972090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.881+0000 7fba7944f640 1 -- 172.21.15.78:0/969566020 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb288643a0 con 0x55eb2897d370 2024-09-16T00:18:10.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.881+0000 7fba7944f640 1 --2- 172.21.15.78:0/969566020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb2897d370 0x55eb2897f760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fba64004500 tx=0x7fba64000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9843c378b2079d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:10.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.882+0000 7fba6affd640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fba600a3450 0x7fba600a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.882+0000 7fba6affd640 1 -- 172.21.15.78:0/3915869574 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fba6c0d01a0 con 0x7fba74116eb0 2024-09-16T00:18:10.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.883+0000 7fba5ffff640 1 -- 172.21.15.78:0/969566020 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64042020 con 0x55eb2897d370 2024-09-16T00:18:10.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.883+0000 7fba5ffff640 1 -- 172.21.15.78:0/969566020 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba64003cd0 con 0x55eb2897d370 2024-09-16T00:18:10.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.884+0000 7fba79c50640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fba600a3450 0x7fba600a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.884+0000 7fba7e28d740 1 -- 172.21.15.78:0/969566020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb2897d370 msgr2=0x55eb2897f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.884+0000 7fba7e28d740 1 --2- 172.21.15.78:0/969566020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb2897d370 0x55eb2897f760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fba64004500 tx=0x7fba64000bc0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.884+0000 7fba7e28d740 1 -- 172.21.15.78:0/969566020 shutdown_connections 2024-09-16T00:18:10.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.884+0000 7fba7e28d740 1 --2- 172.21.15.78:0/969566020 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb2897d370 0x55eb2897f760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.884+0000 7fba7e28d740 1 --2- 172.21.15.78:0/969566020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb289725d0 0x55eb28972a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.884+0000 7fba7e28d740 1 --2- 172.21.15.78:0/969566020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb28972090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.884+0000 7fba7e28d740 1 -- 172.21.15.78:0/969566020 >> 172.21.15.78:0/969566020 conn(0x55eb2893b310 msgr2=0x55eb2896ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:10.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.885+0000 7fba7e28d740 1 -- 172.21.15.78:0/969566020 shutdown_connections 2024-09-16T00:18:10.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.885+0000 7fba7e28d740 1 -- 172.21.15.78:0/969566020 wait complete. 2024-09-16T00:18:10.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.886+0000 7fba7e28d740 1 -- start start 2024-09-16T00:18:10.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.886+0000 7fba7e28d740 1 Processor -- start 2024-09-16T00:18:10.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.886+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb2897f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb2897c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7e28d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb28980060 con 0x55eb289725d0 2024-09-16T00:18:10.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7e28d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb28893060 con 0x55eb28973b40 2024-09-16T00:18:10.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7e28d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb289801e0 con 0x55eb2897cc70 2024-09-16T00:18:10.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7e28d740 10 client.0 _ll_register_callbacks cb 0x55eb285e68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7944f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb2897c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7a451640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7a451640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48324/0 (socket says 172.21.15.78:48324) 2024-09-16T00:18:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7a451640 1 -- 172.21.15.78:0/3049953746 learned_addr learned my addr 172.21.15.78:0/3049953746 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:10.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba79c50640 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb2897f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7a451640 1 -- 172.21.15.78:0/3049953746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 msgr2=0x55eb2897c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.887+0000 7fba7a451640 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb2897c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba7a451640 1 -- 172.21.15.78:0/3049953746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 msgr2=0x55eb2897f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba7a451640 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb2897f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba79c50640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fba600a3450 0x7fba600a5910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55eb2896fac0 tx=0x7fba70008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba7a451640 1 -- 172.21.15.78:0/3049953746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb286493c0 con 0x55eb2897cc70 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba7944f640 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb2897c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba7a451640 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897d0f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0a4b40 tx=0x7fba6c09be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9726992df18d37e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba5d7fa640 1 -- 172.21.15.78:0/3049953746 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c09c400 con 0x55eb2897cc70 2024-09-16T00:18:10.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba7e28d740 1 -- 172.21.15.78:0/3049953746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb288e5860 con 0x55eb2897cc70 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba5d7fa640 1 -- 172.21.15.78:0/3049953746 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fba6c0cd9f0 con 0x55eb2897cc70 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba5d7fa640 1 -- 172.21.15.78:0/3049953746 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fba6c0a07b0 con 0x55eb2897cc70 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba7e28d740 10 client.25585 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.888+0000 7fba7e28d740 1 -- 172.21.15.78:0/3049953746 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eb2898be90 con 0x55eb2897cc70 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba5d7fa640 1 -- 172.21.15.78:0/3049953746 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55eb288e5860 con 0x55eb2897cc70 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba5d7fa640 1 -- 172.21.15.78:0/3049953746 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fba6c0a4be0 con 0x55eb2897cc70 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba7e28d740 10 client.25585 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba7e28d740 10 client.25585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba7e28d740 20 client.25585 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba7e28d740 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55eb2898d710 0x55eb2898fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba7e28d740 4 client.25585 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:10.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba7e28d740 1 -- 172.21.15.78:0/3049953746 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb288dfd70 con 0x55eb2898d710 2024-09-16T00:18:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.889+0000 7fba79c50640 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55eb2898d710 0x55eb2898fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.890+0000 7fba5d7fa640 10 client.25585 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:18:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.890+0000 7fba79c50640 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55eb2898d710 0x55eb2898fb90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff562b9b673d057f server_cookie=8c4713836b8a49f2 in_seq=0 out_seq=0 2024-09-16T00:18:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.892+0000 7fba5d7fa640 1 -- 172.21.15.78:0/3049953746 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb288dfd70 con 0x55eb2898d710 2024-09-16T00:18:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.892+0000 7fba5d7fa640 10 client.25585 handle_command_reply: tid=0 2024-09-16T00:18:10.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.892+0000 7fba7e28d740 1 client.25585 shutdown 2024-09-16T00:18:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.893+0000 7fba7e28d740 1 -- 172.21.15.78:0/3049953746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 msgr2=0x55eb2897d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.893+0000 7fba7e28d740 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897d0f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0a4b40 tx=0x7fba6c09be50 comp rx=0 tx=0).stop 2024-09-16T00:18:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.894+0000 7fba7e28d740 1 -- 172.21.15.78:0/3049953746 shutdown_connections 2024-09-16T00:18:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.894+0000 7fba7e28d740 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897d0f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.894+0000 7fba7e28d740 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55eb2898d710 0x55eb2898fb90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.894+0000 7fba7e28d740 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb2897c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.894+0000 7fba7e28d740 1 --2- 172.21.15.78:0/3049953746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb2897f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.894+0000 7fba5d7fa640 0 client.25585 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:18:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.894+0000 7fba7e28d740 1 -- 172.21.15.78:0/3049953746 >> 172.21.15.78:0/3049953746 conn(0x55eb2893b310 msgr2=0x55eb28974990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.896+0000 7fba7e28d740 1 -- 172.21.15.78:0/3049953746 shutdown_connections 2024-09-16T00:18:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.896+0000 7fba7e28d740 1 -- 172.21.15.78:0/3049953746 wait complete. 2024-09-16T00:18:10.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.896+0000 7fba7e28d740 20 client.25585 trim_cache size 0 max 16384 2024-09-16T00:18:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.917+0000 7fba7e28d740 1 Processor -- start 2024-09-16T00:18:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.918+0000 7fba7e28d740 1 -- start start 2024-09-16T00:18:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb289725d0 0x55eb2896ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb289954a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba79c50640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb289725d0 0x55eb2896ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba79c50640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb289725d0 0x55eb2896ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59428/0 (socket says 172.21.15.78:59428) 2024-09-16T00:18:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba7944f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb289954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba79c50640 1 -- 172.21.15.78:0/2990685815 learned_addr learned my addr 172.21.15.78:0/2990685815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:10.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba7e28d740 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb289959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba7e28d740 1 -- 172.21.15.78:0/2990685815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba5800fb50 con 0x55eb28973b40 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.919+0000 7fba7e28d740 1 -- 172.21.15.78:0/2990685815 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba58011690 con 0x55eb289725d0 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.920+0000 7fba7e28d740 1 -- 172.21.15.78:0/2990685815 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba58010220 con 0x55eb2897cc70 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.920+0000 7fba7a451640 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb289959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.920+0000 7fba7944f640 1 -- 172.21.15.78:0/2990685815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 msgr2=0x55eb289959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.920+0000 7fba7944f640 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb289959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.920+0000 7fba7944f640 1 -- 172.21.15.78:0/2990685815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb289725d0 msgr2=0x55eb2896ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.920+0000 7fba7944f640 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb289725d0 0x55eb2896ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.920+0000 7fba7944f640 1 -- 172.21.15.78:0/2990685815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb28990210 con 0x55eb28973b40 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.920+0000 7fba7944f640 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb289954a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fba64009850 tx=0x7fba64009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0579e522f5b71f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.921+0000 7fba5f7fe640 1 -- 172.21.15.78:0/2990685815 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6404e070 con 0x55eb28973b40 2024-09-16T00:18:10.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.921+0000 7fba5f7fe640 1 -- 172.21.15.78:0/2990685815 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba64031a50 con 0x55eb28973b40 2024-09-16T00:18:10.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.921+0000 7fba5f7fe640 1 -- 172.21.15.78:0/2990685815 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64042030 con 0x55eb28973b40 2024-09-16T00:18:10.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.921+0000 7fba7e28d740 1 -- 172.21.15.78:0/2990685815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 msgr2=0x55eb289954a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.921+0000 7fba7e28d740 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb289954a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fba64009850 tx=0x7fba64009260 comp rx=0 tx=0).stop 2024-09-16T00:18:10.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.921+0000 7fba7e28d740 1 -- 172.21.15.78:0/2990685815 shutdown_connections 2024-09-16T00:18:10.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.921+0000 7fba7e28d740 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb289959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.922+0000 7fba7e28d740 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb28973b40 0x55eb289954a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.922+0000 7fba7e28d740 1 --2- 172.21.15.78:0/2990685815 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb289725d0 0x55eb2896ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.922+0000 7fba7e28d740 1 -- 172.21.15.78:0/2990685815 >> 172.21.15.78:0/2990685815 conn(0x55eb2893b310 msgr2=0x55eb28976940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.922+0000 7fba7e28d740 1 -- 172.21.15.78:0/2990685815 shutdown_connections 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.922+0000 7fba7e28d740 1 -- 172.21.15.78:0/2990685815 wait complete. 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.922+0000 7fba7e28d740 1 -- start start 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.922+0000 7fba7e28d740 1 Processor -- start 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.923+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb28979700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.923+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb28979c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.923+0000 7fba7e28d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.923+0000 7fba7e28d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba58010fc0 con 0x55eb289725d0 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.923+0000 7fba7e28d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba580108f0 con 0x55eb28973b40 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.923+0000 7fba7e28d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba5800db30 con 0x55eb2897cc70 2024-09-16T00:18:10.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.923+0000 7fba7e28d740 10 client.0 _ll_register_callbacks cb 0x7fba6c0076e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7944f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb28979c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48330/0 (socket says 172.21.15.78:48330) 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 -- 172.21.15.78:0/534113606 learned_addr learned my addr 172.21.15.78:0/534113606 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba79c50640 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb28979700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 -- 172.21.15.78:0/534113606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 msgr2=0x55eb28979c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb28979c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 -- 172.21.15.78:0/534113606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 msgr2=0x55eb28979700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb28979700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 -- 172.21.15.78:0/534113606 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb289f7470 con 0x55eb2897cc70 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba79c50640 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb28979700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:10.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba7a451640 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897a180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fba6c09c100 tx=0x7fba6c004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e3c873ad6db3da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.924+0000 7fba5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.925+0000 7fba5d7fa640 1 -- 172.21.15.78:0/534113606 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0b9060 con 0x55eb2897cc70 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.925+0000 7fba5d7fa640 1 -- 172.21.15.78:0/534113606 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fba6c09bad0 con 0x55eb2897cc70 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.925+0000 7fba7e28d740 1 -- 172.21.15.78:0/534113606 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb2885c170 con 0x55eb2897cc70 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.925+0000 7fba5d7fa640 1 -- 172.21.15.78:0/534113606 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fba6c0986a0 con 0x55eb2897cc70 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.925+0000 7fba7e28d740 10 client.25591 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.925+0000 7fba7e28d740 1 -- 172.21.15.78:0/534113606 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eb289f86d0 con 0x55eb2897cc70 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.925+0000 7fba5d7fa640 1 -- 172.21.15.78:0/534113606 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55eb2885c170 con 0x55eb2897cc70 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.926+0000 7fba5d7fa640 1 -- 172.21.15.78:0/534113606 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fba6c0b5060 con 0x55eb2897cc70 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.926+0000 7fba7e28d740 10 client.25591 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.926+0000 7fba7e28d740 10 client.25591 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.926+0000 7fba7e28d740 20 client.25591 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:10.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.926+0000 7fba7e28d740 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55eb289f8df0 0x55eb289fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.926+0000 7fba7e28d740 4 client.25591 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.926+0000 7fba7e28d740 1 -- 172.21.15.78:0/534113606 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb288a99b0 con 0x55eb289f8df0 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.926+0000 7fba79c50640 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55eb289f8df0 0x55eb289fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba5d7fa640 10 client.25591 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba79c50640 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55eb289f8df0 0x55eb289fb210 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f627b5c3f981a549 server_cookie=859f292a9214fa64 in_seq=0 out_seq=0 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba5d7fa640 1 -- 172.21.15.78:0/534113606 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb288a99b0 con 0x55eb289f8df0 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba5d7fa640 10 client.25591 handle_command_reply: tid=0 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba7e28d740 1 client.25591 shutdown 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba7e28d740 1 -- 172.21.15.78:0/534113606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 msgr2=0x55eb2897a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba7e28d740 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897a180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fba6c09c100 tx=0x7fba6c004c60 comp rx=0 tx=0).stop 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba7e28d740 1 -- 172.21.15.78:0/534113606 shutdown_connections 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba7e28d740 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb2897cc70 0x55eb2897a180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba7e28d740 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55eb289f8df0 0x55eb289fb210 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba7e28d740 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb28973b40 0x55eb28979c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.927+0000 7fba7e28d740 1 --2- 172.21.15.78:0/534113606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb289725d0 0x55eb28979700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba7e28d740 1 -- 172.21.15.78:0/534113606 >> 172.21.15.78:0/534113606 conn(0x55eb2893b310 msgr2=0x55eb2897df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba5d7fa640 0 client.25591 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba7e28d740 1 -- 172.21.15.78:0/534113606 shutdown_connections 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba7e28d740 1 -- 172.21.15.78:0/534113606 wait complete. 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba7e28d740 20 client.25591 trim_cache size 0 max 16384 2024-09-16T00:18:10.934 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:18:10.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba7beb4640 1 -- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fba600a3450 msgr2=0x7fba600a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba7beb4640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fba600a3450 0x7fba600a5910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55eb2896fac0 tx=0x7fba70008040 comp rx=0 tx=0).stop 2024-09-16T00:18:10.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba7beb4640 1 -- 172.21.15.78:0/3915869574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba74116eb0 msgr2=0x7fba741abbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:10.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.928+0000 7fba7beb4640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba74116eb0 0x7fba741abbc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00cab0 tx=0x7fba6c00cf70 comp rx=0 tx=0).stop 2024-09-16T00:18:10.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.929+0000 7fba7beb4640 1 -- 172.21.15.78:0/3915869574 shutdown_connections 2024-09-16T00:18:10.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.929+0000 7fba7beb4640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fba600a3450 0x7fba600a5910 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.929+0000 7fba7beb4640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba74116eb0 0x7fba741abbc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.929+0000 7fba7beb4640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba74109b10 0x7fba741a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.929+0000 7fba7beb4640 1 --2- 172.21.15.78:0/3915869574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba741091f0 0x7fba741a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:10.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.929+0000 7fba7beb4640 1 -- 172.21.15.78:0/3915869574 >> 172.21.15.78:0/3915869574 conn(0x7fba7406e140 msgr2=0x7fba74114230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:10.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.929+0000 7fba7beb4640 1 -- 172.21.15.78:0/3915869574 shutdown_connections 2024-09-16T00:18:10.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:10.929+0000 7fba7beb4640 1 -- 172.21.15.78:0/3915869574 wait complete. 2024-09-16T00:18:11.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:11 smithi078 ceph-mon[31157]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 402 op/s 2024-09-16T00:18:11.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:11 smithi193 ceph-mon[35484]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 402 op/s 2024-09-16T00:18:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:11 smithi148 ceph-mon[35599]: pgmap v504: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 4.9 MiB/s wr, 402 op/s 2024-09-16T00:18:12.088 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:18:12.528 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:13.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.122+0000 7fa8fc768640 1 -- 172.21.15.78:0/770949050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8f4102ef0 msgr2=0x7fa8f4069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.122+0000 7fa8fc768640 1 --2- 172.21.15.78:0/770949050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8f4102ef0 0x7fa8f4069f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0013f10 tx=0x7fa8f0032a30 comp rx=0 tx=0).stop 2024-09-16T00:18:13.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.123+0000 7fa8fc768640 1 -- 172.21.15.78:0/770949050 shutdown_connections 2024-09-16T00:18:13.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.123+0000 7fa8fc768640 1 --2- 172.21.15.78:0/770949050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8f4102ef0 0x7fa8f4069f70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.123+0000 7fa8fc768640 1 --2- 172.21.15.78:0/770949050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8f4102570 0x7fa8f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.123+0000 7fa8fc768640 1 --2- 172.21.15.78:0/770949050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4100e20 0x7fa8f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.123+0000 7fa8fc768640 1 -- 172.21.15.78:0/770949050 >> 172.21.15.78:0/770949050 conn(0x7fa8f40fc7d0 msgr2=0x7fa8f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:13.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.124+0000 7fa8fc768640 1 -- 172.21.15.78:0/770949050 shutdown_connections 2024-09-16T00:18:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.127+0000 7fa8fc768640 1 -- 172.21.15.78:0/770949050 wait complete. 2024-09-16T00:18:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.128+0000 7fa8fc768640 1 Processor -- start 2024-09-16T00:18:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.128+0000 7fa8fc768640 1 -- start start 2024-09-16T00:18:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.128+0000 7fa8fc768640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8f4100e20 0x7fa8f41a7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.128+0000 7fa8fc768640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4102570 0x7fa8f41a7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.128+0000 7fa8fc768640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8f4102ef0 0x7fa8f41abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.128+0000 7fa8fc768640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f410ff20 con 0x7fa8f4100e20 2024-09-16T00:18:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.128+0000 7fa8fc768640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f410fda0 con 0x7fa8f4102570 2024-09-16T00:18:13.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.128+0000 7fa8fc768640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41100a0 con 0x7fa8f4102ef0 2024-09-16T00:18:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.129+0000 7fa8f9d03640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4102570 0x7fa8f41a7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.129+0000 7fa8f9d03640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4102570 0x7fa8f41a7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59466/0 (socket says 172.21.15.78:59466) 2024-09-16T00:18:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.130+0000 7fa8f9d03640 1 -- 172.21.15.78:0/767581301 learned_addr learned my addr 172.21.15.78:0/767581301 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.131+0000 7fa8f9d03640 1 -- 172.21.15.78:0/767581301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8f4102ef0 msgr2=0x7fa8f41abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.131+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8f4102ef0 0x7fa8f41abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.131+0000 7fa8f9d03640 1 -- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8f4100e20 msgr2=0x7fa8f41a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.131+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8f4100e20 0x7fa8f41a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.131+0000 7fa8f9d03640 1 -- 172.21.15.78:0/767581301 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f41ac2d0 con 0x7fa8f4102570 2024-09-16T00:18:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.137+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4102570 0x7fa8f41a7860 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00ba50 tx=0x7fa8ec00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.137+0000 7fa8eb7fe640 1 -- 172.21.15.78:0/767581301 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec019070 con 0x7fa8f4102570 2024-09-16T00:18:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.137+0000 7fa8fc768640 1 -- 172.21.15.78:0/767581301 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f41ac5c0 con 0x7fa8f4102570 2024-09-16T00:18:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.138+0000 7fa8fc768640 1 -- 172.21.15.78:0/767581301 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f419b3f0 con 0x7fa8f4102570 2024-09-16T00:18:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.141+0000 7fa8feb41740 1 Processor -- start 2024-09-16T00:18:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.141+0000 7fa8eb7fe640 1 -- 172.21.15.78:0/767581301 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ec00cbe0 con 0x7fa8f4102570 2024-09-16T00:18:13.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.141+0000 7fa8eb7fe640 1 -- 172.21.15.78:0/767581301 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec007630 con 0x7fa8f4102570 2024-09-16T00:18:13.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.144+0000 7fa8feb41740 1 -- start start 2024-09-16T00:18:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e8b40 0x5601455e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 0x5601455f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8feb41740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56014554d2f0 con 0x5601455e75d0 2024-09-16T00:18:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8feb41740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5601454f4cf0 con 0x5601455e8b40 2024-09-16T00:18:13.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8feb41740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5601455492b0 con 0x5601455f2370 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8fad05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e8b40 0x5601455e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8fad05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e8b40 0x5601455e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59478/0 (socket says 172.21.15.78:59478) 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8fad05640 1 -- 172.21.15.78:0/463458031 learned_addr learned my addr 172.21.15.78:0/463458031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.145+0000 7fa8fa504640 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.146+0000 7fa8fad05640 1 -- 172.21.15.78:0/463458031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 msgr2=0x5601455f4760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.146+0000 7fa8fad05640 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 0x5601455f4760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.146+0000 7fa8fad05640 1 -- 172.21.15.78:0/463458031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 msgr2=0x5601455e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.146+0000 7fa8fad05640 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.146+0000 7fa8fad05640 1 -- 172.21.15.78:0/463458031 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601454d93a0 con 0x5601455e8b40 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.146+0000 7fa8fa504640 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.146+0000 7fa8fad05640 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e8b40 0x5601455e7090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009490 tx=0x7fa8f0013320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4074fcd67f33e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.147+0000 7fa8eb7fe640 1 -- 172.21.15.78:0/767581301 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa8ec01c070 con 0x7fa8f4102570 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.148+0000 7fa8eb7fe640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8e00a3450 0x7fa8e00a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.149+0000 7fa8eb7fe640 1 -- 172.21.15.78:0/767581301 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa8ec0cf1d0 con 0x7fa8f4102570 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.150+0000 7fa8fa504640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8e00a3450 0x7fa8e00a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.151+0000 7fa8e8ff9640 1 -- 172.21.15.78:0/463458031 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f003a030 con 0x5601455e8b40 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.151+0000 7fa8e8ff9640 1 -- 172.21.15.78:0/463458031 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f0012ba0 con 0x5601455e8b40 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.152+0000 7fa8feb41740 1 -- 172.21.15.78:0/463458031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e8b40 msgr2=0x5601455e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.152+0000 7fa8feb41740 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e8b40 0x5601455e7090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009490 tx=0x7fa8f0013320 comp rx=0 tx=0).stop 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.154+0000 7fa8feb41740 1 -- 172.21.15.78:0/463458031 shutdown_connections 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.154+0000 7fa8feb41740 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 0x5601455f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.154+0000 7fa8feb41740 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.154+0000 7fa8feb41740 1 --2- 172.21.15.78:0/463458031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e8b40 0x5601455e7090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.154+0000 7fa8feb41740 1 -- 172.21.15.78:0/463458031 >> 172.21.15.78:0/463458031 conn(0x5601455b0310 msgr2=0x5601455e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.154+0000 7fa8feb41740 1 -- 172.21.15.78:0/463458031 shutdown_connections 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.154+0000 7fa8feb41740 1 -- 172.21.15.78:0/463458031 wait complete. 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 1 -- start start 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 1 Processor -- start 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e75d0 0x5601455f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x5601455eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5601455f4ed0 con 0x5601455e8b40 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560145508060 con 0x5601455f2370 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5601455f5050 con 0x5601455e75d0 2024-09-16T00:18:13.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.155+0000 7fa8feb41740 10 client.0 _ll_register_callbacks cb 0x56014525b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.156+0000 7fa8fad05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.156+0000 7fa8fad05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ef5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59480/0 (socket says 172.21.15.78:59480) 2024-09-16T00:18:13.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.156+0000 7fa8fad05640 1 -- 172.21.15.78:0/604683033 learned_addr learned my addr 172.21.15.78:0/604683033 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.157+0000 7fa8fa504640 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e75d0 0x5601455f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.157+0000 7fa8fad05640 1 -- 172.21.15.78:0/604683033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e75d0 msgr2=0x5601455f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:13.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.157+0000 7fa8fad05640 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e75d0 0x5601455f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.157+0000 7fa8fad05640 1 -- 172.21.15.78:0/604683033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 msgr2=0x5601455eefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.157+0000 7fa8fad05640 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x5601455eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.157+0000 7fa8fad05640 1 -- 172.21.15.78:0/604683033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601452be3c0 con 0x5601455f2370 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.157+0000 7fa8fad05640 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ef5d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0005c40 tx=0x7fa8f0013050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b95e2835c3ce0cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.158+0000 7fa8ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.159+0000 7fa8fa504640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8e00a3450 0x7fa8e00a5910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4005d00 tx=0x7fa8e40357e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.159+0000 7fa8ddffb640 1 -- 172.21.15.78:0/604683033 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f004b020 con 0x5601455f2370 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.159+0000 7fa8feb41740 1 -- 172.21.15.78:0/604683033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f0012ba0 con 0x5601455f2370 2024-09-16T00:18:13.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.159+0000 7fa8ddffb640 1 -- 172.21.15.78:0/604683033 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa8f00048d0 con 0x5601455f2370 2024-09-16T00:18:13.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.163+0000 7fa8ddffb640 1 -- 172.21.15.78:0/604683033 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa8f003a070 con 0x5601455f2370 2024-09-16T00:18:13.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.163+0000 7fa8feb41740 10 client.25715 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:13.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.163+0000 7fa8feb41740 1 -- 172.21.15.78:0/604683033 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560145600a10 con 0x5601455f2370 2024-09-16T00:18:13.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.164+0000 7fa8ddffb640 1 -- 172.21.15.78:0/604683033 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f0012ba0 con 0x5601455f2370 2024-09-16T00:18:13.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.164+0000 7fa8ddffb640 1 -- 172.21.15.78:0/604683033 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa8f00035f0 con 0x5601455f2370 2024-09-16T00:18:13.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.164+0000 7fa8feb41740 10 client.25715 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:13.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.164+0000 7fa8feb41740 10 client.25715 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:18:13.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.164+0000 7fa8feb41740 20 client.25715 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:13.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.164+0000 7fa8feb41740 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560145601fd0 0x560145604430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.164+0000 7fa8feb41740 4 client.25715 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:13.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.164+0000 7fa8feb41740 1 -- 172.21.15.78:0/604683033 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56014555a860 con 0x560145601fd0 2024-09-16T00:18:13.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.165+0000 7fa8fa504640 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560145601fd0 0x560145604430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.166+0000 7fa8ddffb640 10 client.25715 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:18:13.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.166+0000 7fa8fa504640 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560145601fd0 0x560145604430 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f635b7e528a56787 server_cookie=f738df2a90a71182 in_seq=0 out_seq=0 2024-09-16T00:18:13.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.167+0000 7fa8ddffb640 1 -- 172.21.15.78:0/604683033 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56014555a860 con 0x560145601fd0 2024-09-16T00:18:13.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.167+0000 7fa8ddffb640 10 client.25715 handle_command_reply: tid=0 2024-09-16T00:18:13.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.169+0000 7fa8feb41740 1 client.25715 shutdown 2024-09-16T00:18:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.169+0000 7fa8feb41740 1 -- 172.21.15.78:0/604683033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 msgr2=0x5601455ef5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.169+0000 7fa8feb41740 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ef5d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0005c40 tx=0x7fa8f0013050 comp rx=0 tx=0).stop 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.171+0000 7fa8feb41740 1 -- 172.21.15.78:0/604683033 shutdown_connections 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.171+0000 7fa8feb41740 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560145601fd0 0x560145604430 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.171+0000 7fa8feb41740 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ef5d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.171+0000 7fa8feb41740 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x5601455eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.171+0000 7fa8feb41740 1 --2- 172.21.15.78:0/604683033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e75d0 0x5601455f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.171+0000 7fa8feb41740 1 -- 172.21.15.78:0/604683033 >> 172.21.15.78:0/604683033 conn(0x5601455b0310 msgr2=0x5601455e9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.171+0000 7fa8ddffb640 0 client.25715 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.172+0000 7fa8feb41740 1 -- 172.21.15.78:0/604683033 shutdown_connections 2024-09-16T00:18:13.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.172+0000 7fa8feb41740 1 -- 172.21.15.78:0/604683033 wait complete. 2024-09-16T00:18:13.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.172+0000 7fa8feb41740 20 client.25715 trim_cache size 0 max 16384 2024-09-16T00:18:13.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.196+0000 7fa8feb41740 1 Processor -- start 2024-09-16T00:18:13.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8feb41740 1 -- start start 2024-09-16T00:18:13.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455ed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e8b40 0x5601455eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8feb41740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d800fc30 con 0x5601455e75d0 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8feb41740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8011770 con 0x5601455f2370 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8feb41740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8010300 con 0x5601455e8b40 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8f9d03640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e8b40 0x5601455eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8f9d03640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e8b40 0x5601455eddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48360/0 (socket says 172.21.15.78:48360) 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8f9d03640 1 -- 172.21.15.78:0/376640749 learned_addr learned my addr 172.21.15.78:0/376640749 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8fad05640 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.197+0000 7fa8fa504640 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455ed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8f9d03640 1 -- 172.21.15.78:0/376640749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 msgr2=0x5601455ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:13.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8f9d03640 1 -- 172.21.15.78:0/376640749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 msgr2=0x5601455ed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:13.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455ed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8f9d03640 1 -- 172.21.15.78:0/376640749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601455f20a0 con 0x5601455e8b40 2024-09-16T00:18:13.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8fa504640 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:13.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e8b40 0x5601455eddf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00ba20 tx=0x7fa8ec014a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cbf21999d72ea40 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:13.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8dffff640 1 -- 172.21.15.78:0/376640749 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0b4070 con 0x5601455e8b40 2024-09-16T00:18:13.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8dffff640 1 -- 172.21.15.78:0/376640749 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ec09bb30 con 0x5601455e8b40 2024-09-16T00:18:13.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.198+0000 7fa8dffff640 1 -- 172.21.15.78:0/376640749 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec014b60 con 0x5601455e8b40 2024-09-16T00:18:13.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.199+0000 7fa8feb41740 1 -- 172.21.15.78:0/376640749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e8b40 msgr2=0x5601455eddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.199+0000 7fa8feb41740 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e8b40 0x5601455eddf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00ba20 tx=0x7fa8ec014a70 comp rx=0 tx=0).stop 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.201+0000 7fa8feb41740 1 -- 172.21.15.78:0/376640749 shutdown_connections 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.201+0000 7fa8feb41740 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455f2370 0x5601455ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.201+0000 7fa8feb41740 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455e8b40 0x5601455eddf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.201+0000 7fa8feb41740 1 --2- 172.21.15.78:0/376640749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e75d0 0x5601455ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.201+0000 7fa8feb41740 1 -- 172.21.15.78:0/376640749 >> 172.21.15.78:0/376640749 conn(0x5601455b0310 msgr2=0x5601455ea2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.202+0000 7fa8feb41740 1 -- 172.21.15.78:0/376640749 shutdown_connections 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.202+0000 7fa8feb41740 1 -- 172.21.15.78:0/376640749 wait complete. 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.202+0000 7fa8feb41740 1 -- start start 2024-09-16T00:18:13.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.202+0000 7fa8feb41740 1 Processor -- start 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e75d0 0x560145606f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x560145607470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8feb41740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 0x56014566ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8feb41740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d80110a0 con 0x5601455e8b40 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8f9d03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x560145607470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8fa504640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e75d0 0x560145606f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8f9d03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x560145607470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55610/0 (socket says 172.21.15.78:55610) 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8f9d03640 1 -- 172.21.15.78:0/4126280589 learned_addr learned my addr 172.21.15.78:0/4126280589 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:13.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8fad05640 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 0x56014566ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8feb41740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d80109d0 con 0x5601455e75d0 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d800dc10 con 0x5601455f2370 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8feb41740 10 client.0 _ll_register_callbacks cb 0x7fa8e400b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8f9d03640 1 -- 172.21.15.78:0/4126280589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 msgr2=0x56014566ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 0x56014566ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8f9d03640 1 -- 172.21.15.78:0/4126280589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e75d0 msgr2=0x560145606f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e75d0 0x560145606f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.203+0000 7fa8f9d03640 1 -- 172.21.15.78:0/4126280589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560145607d50 con 0x5601455e8b40 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.204+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ec09bb30 con 0x5601455e8b40 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.204+0000 7fa8f9d03640 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x560145607470 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec014a40 tx=0x7fa8ec0a40a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e7e0b51d043d80 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.204+0000 7fa8ddffb640 10 client.16278 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.204+0000 7fa8ddffb640 1 -- 172.21.15.78:0/4126280589 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0b4040 con 0x5601455e8b40 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.204+0000 7fa8ddffb640 1 -- 172.21.15.78:0/4126280589 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa8ec0b41e0 con 0x5601455e8b40 2024-09-16T00:18:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.204+0000 7fa8ddffb640 1 -- 172.21.15.78:0/4126280589 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa8ec0b43e0 con 0x5601455e8b40 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.204+0000 7fa8feb41740 10 client.16278 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.204+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({fsmap=34}) -- 0x56014566c380 con 0x5601455e8b40 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.205+0000 7fa8ddffb640 1 -- 172.21.15.78:0/4126280589 <== mon.0 v2:172.21.15.78:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa8ec0a5020 con 0x5601455e8b40 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.205+0000 7fa8feb41740 10 client.16278 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.205+0000 7fa8feb41740 10 client.16278 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.205+0000 7fa8feb41740 20 client.16278 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.205+0000 7fa8feb41740 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56014566c970 0x56014566ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.205+0000 7fa8feb41740 4 client.16278 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.205+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560145554d70 con 0x56014566c970 2024-09-16T00:18:13.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.205+0000 7fa8fa504640 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56014566c970 0x56014566ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.206+0000 7fa8fa504640 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56014566c970 0x56014566ed90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5200bd2a150486da server_cookie=4b46d839832140ef in_seq=0 out_seq=0 2024-09-16T00:18:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.206+0000 7fa8ddffb640 10 client.16278 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:18:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.206+0000 7fa8ddffb640 1 -- 172.21.15.78:0/4126280589 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560145554d70 con 0x56014566c970 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.206+0000 7fa8ddffb640 10 client.16278 handle_command_reply: tid=0 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.206+0000 7fa8feb41740 1 client.16278 shutdown 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 msgr2=0x560145607470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x560145607470 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec014a40 tx=0x7fa8ec0a40a0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 shutdown_connections 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56014566c970 0x56014566ed90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601455f2370 0x56014566ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5601455e8b40 0x560145607470 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 --2- 172.21.15.78:0/4126280589 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5601455e75d0 0x560145606f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 >> 172.21.15.78:0/4126280589 conn(0x5601455b0310 msgr2=0x560145604d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8ddffb640 0 client.16278 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 shutdown_connections 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 1 -- 172.21.15.78:0/4126280589 wait complete. 2024-09-16T00:18:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.207+0000 7fa8feb41740 20 client.16278 trim_cache size 0 max 16384 2024-09-16T00:18:13.212 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:18:13.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 -- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8e00a3450 msgr2=0x7fa8e00a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8e00a3450 0x7fa8e00a5910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4005d00 tx=0x7fa8e40357e0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 -- 172.21.15.78:0/767581301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4102570 msgr2=0x7fa8f41a7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:13.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4102570 0x7fa8f41a7860 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00ba50 tx=0x7fa8ec00bf10 comp rx=0 tx=0).stop 2024-09-16T00:18:13.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 -- 172.21.15.78:0/767581301 shutdown_connections 2024-09-16T00:18:13.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8e00a3450 0x7fa8e00a5910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8f4102ef0 0x7fa8f41abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8f4102570 0x7fa8f41a7860 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.208+0000 7fa8fc768640 1 --2- 172.21.15.78:0/767581301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8f4100e20 0x7fa8f41a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:13.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.209+0000 7fa8fc768640 1 -- 172.21.15.78:0/767581301 >> 172.21.15.78:0/767581301 conn(0x7fa8f40fc7d0 msgr2=0x7fa8f410e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:13.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.209+0000 7fa8fc768640 1 -- 172.21.15.78:0/767581301 shutdown_connections 2024-09-16T00:18:13.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:13.209+0000 7fa8fc768640 1 -- 172.21.15.78:0/767581301 wait complete. 2024-09-16T00:18:13.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:13 smithi193 ceph-mon[35484]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.9 MiB/s wr, 505 op/s 2024-09-16T00:18:13.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:13 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:18:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:13 smithi148 ceph-mon[35599]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.9 MiB/s wr, 505 op/s 2024-09-16T00:18:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:13 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:18:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:13 smithi078 ceph-mon[31157]: pgmap v505: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.9 MiB/s wr, 505 op/s 2024-09-16T00:18:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:13 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:18:14.161 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:18:14.637 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:14.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:18:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:18:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:18:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:18:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:18:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:18:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:18:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:18:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:18:15.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 255.0 s, 1461.8 tps, lat 68.863 ms stddev 82.059 2024-09-16T00:18:15.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:15 smithi193 ceph-mon[35484]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.2 MiB/s wr, 615 op/s 2024-09-16T00:18:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:15 smithi148 ceph-mon[35599]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.2 MiB/s wr, 615 op/s 2024-09-16T00:18:16.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:15 smithi078 ceph-mon[31157]: pgmap v506: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 7.2 MiB/s wr, 615 op/s 2024-09-16T00:18:16.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 -- 172.21.15.78:0/1978674258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594108e00 msgr2=0x7f559410ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:16.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 --2- 172.21.15.78:0/1978674258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594108e00 0x7f559410ce00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5588009900 tx=0x7f558802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 -- 172.21.15.78:0/1978674258 shutdown_connections 2024-09-16T00:18:16.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 --2- 172.21.15.78:0/1978674258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f559410d530 0x7f55941150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 --2- 172.21.15.78:0/1978674258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594108e00 0x7f559410ce00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 --2- 172.21.15.78:0/1978674258 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5594108450 0x7f5594108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 -- 172.21.15.78:0/1978674258 >> 172.21.15.78:0/1978674258 conn(0x7f5594077ba0 msgr2=0x7f5594079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:16.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 -- 172.21.15.78:0/1978674258 shutdown_connections 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.970+0000 7f559a4e6640 1 -- 172.21.15.78:0/1978674258 wait complete. 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f559a4e6640 1 Processor -- start 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f559a4e6640 1 -- start start 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f559a4e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5594108450 0x7f55941a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f559a4e6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594108e00 0x7f55941a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f559a4e6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f559410d530 0x7f559419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f559a4e6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5594118530 con 0x7f5594108e00 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f559a4e6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55941183b0 con 0x7f5594108450 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f559a4e6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55941186b0 con 0x7f559410d530 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.971+0000 7f5598a83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f559410d530 0x7f559419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5598a83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f559410d530 0x7f559419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:54028/0 (socket says 172.21.15.78:54028) 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5598a83640 1 -- 172.21.15.78:0/3605743342 learned_addr learned my addr 172.21.15.78:0/3605743342 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5593fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5594108450 0x7f55941a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f55937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594108e00 0x7f55941a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5593fff640 1 -- 172.21.15.78:0/3605743342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f559410d530 msgr2=0x7f559419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5593fff640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f559410d530 0x7f559419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5593fff640 1 -- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594108e00 msgr2=0x7f55941a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5593fff640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594108e00 0x7f55941a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5593fff640 1 -- 172.21.15.78:0/3605743342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559419ef90 con 0x7f5594108450 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f5593fff640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5594108450 0x7f55941a46a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f558c00eb00 tx=0x7f558c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f55917fa640 1 -- 172.21.15.78:0/3605743342 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c010430 con 0x7f5594108450 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.972+0000 7f559a4e6640 1 -- 172.21.15.78:0/3605743342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559419f280 con 0x7f5594108450 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.973+0000 7f55917fa640 1 -- 172.21.15.78:0/3605743342 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c010ab0 con 0x7f5594108450 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.973+0000 7f55917fa640 1 -- 172.21.15.78:0/3605743342 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c017bb0 con 0x7f5594108450 2024-09-16T00:18:16.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.973+0000 7f559a4e6640 1 -- 172.21.15.78:0/3605743342 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55941ab570 con 0x7f5594108450 2024-09-16T00:18:16.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f55917fa640 1 -- 172.21.15.78:0/3605743342 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f558c021430 con 0x7f5594108450 2024-09-16T00:18:16.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f559c8bf740 1 Processor -- start 2024-09-16T00:18:16.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f559c8bf740 1 -- start start 2024-09-16T00:18:16.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c69fc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fc5d0 0x5587c69fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f55917fa640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f557c0a32c0 0x7f557c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a09760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f559c8bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5587c69622f0 con 0x5587c69fc5d0 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f559c8bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5587c6909cf0 con 0x5587c6a07370 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.976+0000 7f559c8bf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587c695e2b0 con 0x5587c69fdb40 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c69fc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c69fc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:54030/0 (socket says 172.21.15.78:54030) 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 -- 172.21.15.78:0/828829734 learned_addr learned my addr 172.21.15.78:0/828829734 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f55917fa640 1 -- 172.21.15.78:0/3605743342 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f558c0d0570 con 0x7f5594108450 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f55937fe640 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a09760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5593fff640 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fc5d0 0x5587c69fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 -- 172.21.15.78:0/828829734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 msgr2=0x5587c6a09760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:16.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a09760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 -- 172.21.15.78:0/828829734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fc5d0 msgr2=0x5587c69fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fc5d0 0x5587c69fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 -- 172.21.15.78:0/828829734 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c68ee3a0 con 0x5587c69fdb40 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5593fff640 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fc5d0 0x5587c69fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c69fc090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5584005f60 tx=0x7f5584013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d204c0b3a3fef64d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.977+0000 7f5598a83640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f557c0a32c0 0x7f557c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.978+0000 7f557a7fc640 1 -- 172.21.15.78:0/828829734 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584020070 con 0x5587c69fdb40 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.978+0000 7f557a7fc640 1 -- 172.21.15.78:0/828829734 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558401b9f0 con 0x5587c69fdb40 2024-09-16T00:18:16.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.978+0000 7f557a7fc640 1 -- 172.21.15.78:0/828829734 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558400fcf0 con 0x5587c69fdb40 2024-09-16T00:18:16.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.979+0000 7f559c8bf740 1 -- 172.21.15.78:0/828829734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 msgr2=0x5587c69fc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.979+0000 7f559c8bf740 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c69fc090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5584005f60 tx=0x7f5584013830 comp rx=0 tx=0).stop 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f5598a83640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f557c0a32c0 0x7f557c0a5780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f558401d000 tx=0x7f558400ff70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f559c8bf740 1 -- 172.21.15.78:0/828829734 shutdown_connections 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f559c8bf740 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a09760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f559c8bf740 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fc5d0 0x5587c69fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f559c8bf740 1 --2- 172.21.15.78:0/828829734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c69fc090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f559c8bf740 1 -- 172.21.15.78:0/828829734 >> 172.21.15.78:0/828829734 conn(0x5587c69c5310 msgr2=0x5587c69f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f559c8bf740 1 -- 172.21.15.78:0/828829734 shutdown_connections 2024-09-16T00:18:16.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f559c8bf740 1 -- 172.21.15.78:0/828829734 wait complete. 2024-09-16T00:18:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.981+0000 7f559c8bf740 1 -- start start 2024-09-16T00:18:16.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.982+0000 7f559c8bf740 1 Processor -- start 2024-09-16T00:18:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.982+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c69fc5d0 0x5587c6a06da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.982+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c6a04050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.982+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c6a07370 0x5587c6a047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.982+0000 7f559c8bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5587c6a0a070 con 0x5587c6a07370 2024-09-16T00:18:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.982+0000 7f559c8bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5587c691d060 con 0x5587c69fc5d0 2024-09-16T00:18:16.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.982+0000 7f559c8bf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587c6a0a1f0 con 0x5587c69fdb40 2024-09-16T00:18:16.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.982+0000 7f559c8bf740 10 client.0 _ll_register_callbacks cb 0x5587c66708c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:16.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.985+0000 7f5598a83640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c6a07370 0x5587c6a047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.985+0000 7f55937fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c6a04050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.985+0000 7f5598a83640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c6a07370 0x5587c6a047e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37948/0 (socket says 172.21.15.78:37948) 2024-09-16T00:18:16.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.985+0000 7f5598a83640 1 -- 172.21.15.78:0/1439376167 learned_addr learned my addr 172.21.15.78:0/1439376167 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:16.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.985+0000 7f5593fff640 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c69fc5d0 0x5587c6a06da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.990+0000 7f5593fff640 1 -- 172.21.15.78:0/1439376167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 msgr2=0x5587c6a04050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.990+0000 7f5593fff640 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c6a04050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.990+0000 7f5593fff640 1 -- 172.21.15.78:0/1439376167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c6a07370 msgr2=0x5587c6a047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.990+0000 7f5593fff640 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c6a07370 0x5587c6a047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.990+0000 7f5593fff640 1 -- 172.21.15.78:0/1439376167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c66d33c0 con 0x5587c69fc5d0 2024-09-16T00:18:16.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.993+0000 7f5593fff640 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c69fc5d0 0x5587c6a06da0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f558c09c7c0 tx=0x7f558c09c250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec25b62e0f23f7a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:16.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.994+0000 7f55737fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:18:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.998+0000 7f55737fe640 1 -- 172.21.15.78:0/1439376167 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0a0be0 con 0x5587c69fc5d0 2024-09-16T00:18:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.998+0000 7f55737fe640 1 -- 172.21.15.78:0/1439376167 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f558c09ab30 con 0x5587c69fc5d0 2024-09-16T00:18:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:16.998+0000 7f559c8bf740 1 -- 172.21.15.78:0/1439376167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558401b9f0 con 0x5587c69fc5d0 2024-09-16T00:18:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.001+0000 7f55737fe640 1 -- 172.21.15.78:0/1439376167 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f558c0a54a0 con 0x5587c69fc5d0 2024-09-16T00:18:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.001+0000 7f559c8bf740 10 client.25733 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.001+0000 7f559c8bf740 1 -- 172.21.15.78:0/1439376167 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5587c6a15bb0 con 0x5587c69fc5d0 2024-09-16T00:18:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.001+0000 7f55737fe640 1 -- 172.21.15.78:0/1439376167 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558401b9f0 con 0x5587c69fc5d0 2024-09-16T00:18:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.002+0000 7f55737fe640 1 -- 172.21.15.78:0/1439376167 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f558c098ea0 con 0x5587c69fc5d0 2024-09-16T00:18:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.002+0000 7f559c8bf740 10 client.25733 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.002+0000 7f559c8bf740 10 client.25733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.002+0000 7f559c8bf740 20 client.25733 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.002+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5587c6a16fc0 0x5587c6a193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.003+0000 7f559c8bf740 4 client.25733 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:17.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.003+0000 7f559c8bf740 1 -- 172.21.15.78:0/1439376167 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587c696f860 con 0x5587c6a16fc0 2024-09-16T00:18:17.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.003+0000 7f55937fe640 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5587c6a16fc0 0x5587c6a193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:17.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.008+0000 7f55937fe640 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5587c6a16fc0 0x5587c6a193b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee3c7dc9d1f553a server_cookie=8a6e61b57fea7881 in_seq=0 out_seq=0 2024-09-16T00:18:17.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.009+0000 7f55737fe640 10 client.25733 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:17.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.010+0000 7f55737fe640 1 -- 172.21.15.78:0/1439376167 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587c696f860 con 0x5587c6a16fc0 2024-09-16T00:18:17.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.011+0000 7f55737fe640 10 client.25733 handle_command_reply: tid=0 2024-09-16T00:18:17.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.011+0000 7f559c8bf740 1 client.25733 shutdown 2024-09-16T00:18:17.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.011+0000 7f559c8bf740 1 -- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c69fc5d0 msgr2=0x5587c6a06da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:17.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.011+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c69fc5d0 0x5587c6a06da0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f558c09c7c0 tx=0x7f558c09c250 comp rx=0 tx=0).stop 2024-09-16T00:18:17.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.011+0000 7f559c8bf740 1 -- 172.21.15.78:0/1439376167 shutdown_connections 2024-09-16T00:18:17.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.011+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5587c6a16fc0 0x5587c6a193b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.011+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c6a07370 0x5587c6a047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.011+0000 7f55737fe640 0 client.25733 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.012+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fdb40 0x5587c6a04050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.012+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1439376167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c69fc5d0 0x5587c6a06da0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.012+0000 7f559c8bf740 1 -- 172.21.15.78:0/1439376167 >> 172.21.15.78:0/1439376167 conn(0x5587c69c5310 msgr2=0x5587c69fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.012+0000 7f559c8bf740 1 -- 172.21.15.78:0/1439376167 shutdown_connections 2024-09-16T00:18:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.013+0000 7f559c8bf740 1 -- 172.21.15.78:0/1439376167 wait complete. 2024-09-16T00:18:17.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.013+0000 7f559c8bf740 20 client.25733 trim_cache size 0 max 16384 2024-09-16T00:18:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.043+0000 7f559c8bf740 1 Processor -- start 2024-09-16T00:18:17.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.043+0000 7f559c8bf740 1 -- start start 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.044+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a04020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.044+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 0x5587c6a1ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.044+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a1f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.044+0000 7f559c8bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b5260 con 0x5587c69fdb40 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.044+0000 7f559c8bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b6da0 con 0x5587c6a07370 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.044+0000 7f559c8bf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b5930 con 0x5587c69fc5d0 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.045+0000 7f55937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 0x5587c6a1ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.045+0000 7f55937fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 0x5587c6a1ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37952/0 (socket says 172.21.15.78:37952) 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.045+0000 7f55937fe640 1 -- 172.21.15.78:0/1819804377 learned_addr learned my addr 172.21.15.78:0/1819804377 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:17.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.045+0000 7f5593fff640 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a04020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.048+0000 7f5598a83640 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a1f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.048+0000 7f5593fff640 1 -- 172.21.15.78:0/1819804377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 msgr2=0x5587c6a1f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.048+0000 7f5593fff640 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a1f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.048+0000 7f5593fff640 1 -- 172.21.15.78:0/1819804377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 msgr2=0x5587c6a1ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.048+0000 7f5593fff640 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 0x5587c6a1ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.048+0000 7f5593fff640 1 -- 172.21.15.78:0/1819804377 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c6a04b50 con 0x5587c69fc5d0 2024-09-16T00:18:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.048+0000 7f5598a83640 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a1f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:17.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.048+0000 7f5593fff640 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a04020 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f558c09c530 tx=0x7f558c09d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f58818761ddf4219 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.050+0000 7f5579ffb640 1 -- 172.21.15.78:0/1819804377 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0a0be0 con 0x5587c69fc5d0 2024-09-16T00:18:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.050+0000 7f5579ffb640 1 -- 172.21.15.78:0/1819804377 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c09ab30 con 0x5587c69fc5d0 2024-09-16T00:18:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.050+0000 7f5579ffb640 1 -- 172.21.15.78:0/1819804377 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0a5010 con 0x5587c69fc5d0 2024-09-16T00:18:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.051+0000 7f559c8bf740 1 -- 172.21.15.78:0/1819804377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 msgr2=0x5587c6a04020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:17.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.051+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a04020 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f558c09c530 tx=0x7f558c09d070 comp rx=0 tx=0).stop 2024-09-16T00:18:17.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.055+0000 7f559c8bf740 1 -- 172.21.15.78:0/1819804377 shutdown_connections 2024-09-16T00:18:17.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.055+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a1f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.055+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 0x5587c6a1ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.055+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1819804377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a04020 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.055+0000 7f559c8bf740 1 -- 172.21.15.78:0/1819804377 >> 172.21.15.78:0/1819804377 conn(0x5587c69c5310 msgr2=0x5587c6a03910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:17.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.059+0000 7f559c8bf740 1 -- 172.21.15.78:0/1819804377 shutdown_connections 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.060+0000 7f559c8bf740 1 -- 172.21.15.78:0/1819804377 wait complete. 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.060+0000 7f559c8bf740 1 -- start start 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.060+0000 7f559c8bf740 1 Processor -- start 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a0b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 0x5587c6a0b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f559c8bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a0bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f559c8bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b66d0 con 0x5587c69fdb40 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f559c8bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b6000 con 0x5587c6a07370 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f559c8bf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55941b3240 con 0x5587c69fc5d0 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f559c8bf740 10 client.0 _ll_register_callbacks cb 0x7f558c0d0870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f5598a83640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a0bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f5598a83640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a0bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60034/0 (socket says 172.21.15.78:60034) 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f5598a83640 1 -- 172.21.15.78:0/1231270797 learned_addr learned my addr 172.21.15.78:0/1231270797 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.061+0000 7f5593fff640 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a0b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.063+0000 7f5593fff640 1 -- 172.21.15.78:0/1231270797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 msgr2=0x5587c6a0bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.063+0000 7f5593fff640 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a0bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.063+0000 7f5593fff640 1 -- 172.21.15.78:0/1231270797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 msgr2=0x5587c6a0b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.063+0000 7f5593fff640 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 0x5587c6a0b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.063+0000 7f5593fff640 1 -- 172.21.15.78:0/1231270797 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c6a91eb0 con 0x5587c69fc5d0 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.064+0000 7f5593fff640 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a0b450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f558c098c90 tx=0x7f558c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=700ceb20800f5413 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.064+0000 7f55737fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.064+0000 7f55737fe640 1 -- 172.21.15.78:0/1231270797 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0bb250 con 0x5587c69fc5d0 2024-09-16T00:18:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.064+0000 7f55737fe640 1 -- 172.21.15.78:0/1231270797 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f558c0bb050 con 0x5587c69fc5d0 2024-09-16T00:18:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.065+0000 7f559c8bf740 1 -- 172.21.15.78:0/1231270797 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c09ab30 con 0x5587c69fc5d0 2024-09-16T00:18:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.066+0000 7f55737fe640 1 -- 172.21.15.78:0/1231270797 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f558c0a0be0 con 0x5587c69fc5d0 2024-09-16T00:18:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.067+0000 7f559c8bf740 10 client.25621 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.067+0000 7f559c8bf740 1 -- 172.21.15.78:0/1231270797 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5587c6a79e60 con 0x5587c69fc5d0 2024-09-16T00:18:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f55737fe640 1 -- 172.21.15.78:0/1231270797 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f558c09ab30 con 0x5587c69fc5d0 2024-09-16T00:18:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f55737fe640 1 -- 172.21.15.78:0/1231270797 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f558c09db60 con 0x5587c69fc5d0 2024-09-16T00:18:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f559c8bf740 10 client.25621 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:17.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f559c8bf740 10 client.25621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f559c8bf740 20 client.25621 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5587c6a16fc0 0x5587c6a7c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f559c8bf740 4 client.25621 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f559c8bf740 1 -- 172.21.15.78:0/1231270797 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587c6969d70 con 0x5587c6a16fc0 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.068+0000 7f55937fe640 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5587c6a16fc0 0x5587c6a7c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.069+0000 7f55937fe640 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5587c6a16fc0 0x5587c6a7c2b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d913551da43c647 server_cookie=e5118fd433b0cc62 in_seq=0 out_seq=0 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.069+0000 7f55737fe640 10 client.25621 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.069+0000 7f55737fe640 1 -- 172.21.15.78:0/1231270797 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587c6969d70 con 0x5587c6a16fc0 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.069+0000 7f55737fe640 10 client.25621 handle_command_reply: tid=0 2024-09-16T00:18:17.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.069+0000 7f559c8bf740 1 client.25621 shutdown 2024-09-16T00:18:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 -- 172.21.15.78:0/1231270797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 msgr2=0x5587c6a0b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a0b450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f558c098c90 tx=0x7f558c000f40 comp rx=0 tx=0).stop 2024-09-16T00:18:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 -- 172.21.15.78:0/1231270797 shutdown_connections 2024-09-16T00:18:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5587c6a16fc0 0x5587c6a7c2b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5587c6a07370 0x5587c6a0bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5587c69fdb40 0x5587c6a0b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 --2- 172.21.15.78:0/1231270797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c69fc5d0 0x5587c6a0b450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f55737fe640 0 client.25621 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 -- 172.21.15.78:0/1231270797 >> 172.21.15.78:0/1231270797 conn(0x5587c69c5310 msgr2=0x5587c69fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 -- 172.21.15.78:0/1231270797 shutdown_connections 2024-09-16T00:18:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.074+0000 7f559c8bf740 1 -- 172.21.15.78:0/1231270797 wait complete. 2024-09-16T00:18:17.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.075+0000 7f559c8bf740 20 client.25621 trim_cache size 0 max 16384 2024-09-16T00:18:17.077 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:18:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.077+0000 7f559a4e6640 1 -- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f557c0a32c0 msgr2=0x7f557c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.077+0000 7f559a4e6640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f557c0a32c0 0x7f557c0a5780 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f558401d000 tx=0x7f558400ff70 comp rx=0 tx=0).stop 2024-09-16T00:18:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.077+0000 7f559a4e6640 1 -- 172.21.15.78:0/3605743342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5594108450 msgr2=0x7f55941a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:17.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.077+0000 7f559a4e6640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5594108450 0x7f55941a46a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f558c00eb00 tx=0x7f558c00c2d0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.078+0000 7f559a4e6640 1 -- 172.21.15.78:0/3605743342 shutdown_connections 2024-09-16T00:18:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.078+0000 7f559a4e6640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f557c0a32c0 0x7f557c0a5780 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.078+0000 7f559a4e6640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f559410d530 0x7f559419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.078+0000 7f559a4e6640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5594108e00 0x7f55941a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.078+0000 7f559a4e6640 1 --2- 172.21.15.78:0/3605743342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5594108450 0x7f55941a46a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.078+0000 7f559a4e6640 1 -- 172.21.15.78:0/3605743342 >> 172.21.15.78:0/3605743342 conn(0x7f5594077ba0 msgr2=0x7f5594078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.078+0000 7f559a4e6640 1 -- 172.21.15.78:0/3605743342 shutdown_connections 2024-09-16T00:18:17.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:17.078+0000 7f559a4e6640 1 -- 172.21.15.78:0/3605743342 wait complete. 2024-09-16T00:18:17.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:17 smithi078 ceph-mon[31157]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.7 MiB/s wr, 671 op/s 2024-09-16T00:18:17.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:17 smithi193 ceph-mon[35484]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.7 MiB/s wr, 671 op/s 2024-09-16T00:18:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:17 smithi148 ceph-mon[35599]: pgmap v507: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.7 MiB/s wr, 671 op/s 2024-09-16T00:18:19.911 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:19 smithi078 ceph-mon[31157]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.3 MiB/s wr, 663 op/s 2024-09-16T00:18:19.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:19 smithi193 ceph-mon[35484]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.3 MiB/s wr, 663 op/s 2024-09-16T00:18:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:19 smithi148 ceph-mon[35599]: pgmap v508: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.3 MiB/s wr, 663 op/s 2024-09-16T00:18:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 260.0 s, 1077.0 tps, lat 93.050 ms stddev 115.244 2024-09-16T00:18:20.954 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:18:20.955 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:18:21.359 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:21.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:21 smithi078 ceph-mon[31157]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-09-16T00:18:21.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:21 smithi193 ceph-mon[35484]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-09-16T00:18:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:21 smithi148 ceph-mon[35599]: pgmap v509: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.1 MiB/s wr, 558 op/s 2024-09-16T00:18:22.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 -- 172.21.15.78:0/4135096382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4f8105560 msgr2=0x7fa4f810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 --2- 172.21.15.78:0/4135096382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4f8105560 0x7fa4f810d0e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000adb0 tx=0x7fa4f002b7a0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 -- 172.21.15.78:0/4135096382 shutdown_connections 2024-09-16T00:18:22.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 --2- 172.21.15.78:0/4135096382 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4f8105560 0x7fa4f810d0e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 --2- 172.21.15.78:0/4135096382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100de0 0x7fa4f8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 --2- 172.21.15.78:0/4135096382 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f8100430 0x7fa4f8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 -- 172.21.15.78:0/4135096382 >> 172.21.15.78:0/4135096382 conn(0x7fa4f8077230 msgr2=0x7fa4f80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:22.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 -- 172.21.15.78:0/4135096382 shutdown_connections 2024-09-16T00:18:22.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.457+0000 7fa4fda61640 1 -- 172.21.15.78:0/4135096382 wait complete. 2024-09-16T00:18:22.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.458+0000 7fa4fda61640 1 Processor -- start 2024-09-16T00:18:22.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.458+0000 7fa4fda61640 1 -- start start 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4fda61640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4f8100430 0x7fa4f81961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4fda61640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100de0 0x7fa4f8196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4fda61640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f8105560 0x7fa4f819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4f8100430 0x7fa4f81961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4f67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100de0 0x7fa4f8196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4f67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100de0 0x7fa4f8196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:54066/0 (socket says 172.21.15.78:54066) 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4fda61640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f810fd90 con 0x7fa4f8100430 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4f67fc640 1 -- 172.21.15.78:0/3639010618 learned_addr learned my addr 172.21.15.78:0/3639010618 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f810fc10 con 0x7fa4f8105560 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f810ff10 con 0x7fa4f8100de0 2024-09-16T00:18:22.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f8105560 0x7fa4f819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.459+0000 7fa4f67fc640 1 -- 172.21.15.78:0/3639010618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f8105560 msgr2=0x7fa4f819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.460+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f8105560 0x7fa4f819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.460+0000 7fa4f67fc640 1 -- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4f8100430 msgr2=0x7fa4f81961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.460+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4f8100430 0x7fa4f81961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.460+0000 7fa4f67fc640 1 -- 172.21.15.78:0/3639010618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f819b1a0 con 0x7fa4f8100de0 2024-09-16T00:18:22.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.460+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100de0 0x7fa4f8196730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009f70 tx=0x7fa4ec00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:22.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.461+0000 7fa4fca5f640 1 -- 172.21.15.78:0/3639010618 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00cd00 con 0x7fa4f8100de0 2024-09-16T00:18:22.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.461+0000 7fa4fca5f640 1 -- 172.21.15.78:0/3639010618 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec005500 con 0x7fa4f8100de0 2024-09-16T00:18:22.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.461+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f819b490 con 0x7fa4f8100de0 2024-09-16T00:18:22.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.462+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f81a2d70 con 0x7fa4f8100de0 2024-09-16T00:18:22.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.462+0000 7fa4fca5f640 1 -- 172.21.15.78:0/3639010618 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec011830 con 0x7fa4f8100de0 2024-09-16T00:18:22.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.465+0000 7fa4fca5f640 1 -- 172.21.15.78:0/3639010618 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa4ec0046c0 con 0x7fa4f8100de0 2024-09-16T00:18:22.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.465+0000 7fa4ffe3a740 1 Processor -- start 2024-09-16T00:18:22.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.465+0000 7fa4ffe3a740 1 -- start start 2024-09-16T00:18:22.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266ef70 0x562c7266f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72679c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c7267c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c725d0280 con 0x562c7266f890 2024-09-16T00:18:22.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562c72577b10 con 0x562c7267a150 2024-09-16T00:18:22.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562c725cc240 con 0x562c7266ef70 2024-09-16T00:18:22.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c7267c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c7267c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60058/0 (socket says 172.21.15.78:60058) 2024-09-16T00:18:22.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f67fc640 1 -- 172.21.15.78:0/2654100732 learned_addr learned my addr 172.21.15.78:0/2654100732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:22.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f6ffd640 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72679c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266ef70 0x562c7266f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4fca5f640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa4e00a3390 0x7fa4e00a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f67fc640 1 -- 172.21.15.78:0/2654100732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266ef70 msgr2=0x562c7266f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:22.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266ef70 0x562c7266f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f67fc640 1 -- 172.21.15.78:0/2654100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 msgr2=0x562c72679c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:22.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72679c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4fca5f640 1 -- 172.21.15.78:0/3639010618 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa4ec0d1ce0 con 0x7fa4f8100de0 2024-09-16T00:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f67fc640 1 -- 172.21.15.78:0/2654100732 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c725841f0 con 0x562c7267a150 2024-09-16T00:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.466+0000 7fa4f6ffd640 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72679c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.467+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c7267c540 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec09a490 tx=0x7fa4ec053470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62470d3bc450f2c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.467+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa4e00a3390 0x7fa4e00a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.467+0000 7fa4ddffb640 1 -- 172.21.15.78:0/2654100732 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec09de00 con 0x562c7267a150 2024-09-16T00:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.467+0000 7fa4ddffb640 1 -- 172.21.15.78:0/2654100732 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec0d8070 con 0x562c7267a150 2024-09-16T00:18:22.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.467+0000 7fa4ddffb640 1 -- 172.21.15.78:0/2654100732 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec09d990 con 0x562c7267a150 2024-09-16T00:18:22.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.468+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2654100732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 msgr2=0x562c7267c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.468+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c7267c540 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec09a490 tx=0x7fa4ec053470 comp rx=0 tx=0).stop 2024-09-16T00:18:22.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.468+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa4e00a3390 0x7fa4e00a5850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa4f002c010 tx=0x7fa4f000a6d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:22.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.468+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2654100732 shutdown_connections 2024-09-16T00:18:22.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.468+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c7267c540 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.468+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72679c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.468+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2654100732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266ef70 0x562c7266f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.468+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2654100732 >> 172.21.15.78:0/2654100732 conn(0x562c7266b500 msgr2=0x562c7266b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.469+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2654100732 shutdown_connections 2024-09-16T00:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.469+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2654100732 wait complete. 2024-09-16T00:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.469+0000 7fa4ffe3a740 1 -- start start 2024-09-16T00:18:22.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.469+0000 7fa4ffe3a740 1 Processor -- start 2024-09-16T00:18:22.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266ef70 0x562c72673230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72673770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c72673cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562c72666c80 con 0x562c7266f890 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562c7258af10 con 0x562c7266ef70 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562c72666e00 con 0x562c7267a150 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4ffe3a740 10 client.0 _ll_register_callbacks cb 0x562c72553a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72673770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266ef70 0x562c72673230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72673770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37992/0 (socket says 172.21.15.78:37992) 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 -- 172.21.15.78:0/2544511107 learned_addr learned my addr 172.21.15.78:0/2544511107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c72673cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 -- 172.21.15.78:0/2544511107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 msgr2=0x562c72673cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c72673cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 -- 172.21.15.78:0/2544511107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266ef70 msgr2=0x562c72673230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266ef70 0x562c72673230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 -- 172.21.15.78:0/2544511107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c723416a0 con 0x562c7266f890 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72673770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec09dfc0 tx=0x7fa4ec0a2430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c1f29d493a8840 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.470+0000 7fa4d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.471+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/2544511107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec09db70 con 0x562c7266f890 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.471+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2544511107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ec0d8070 con 0x562c7266f890 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.471+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/2544511107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa4ec0a6960 con 0x562c7266f890 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.471+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/2544511107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa4ec09de00 con 0x562c7266f890 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.471+0000 7fa4ffe3a740 10 client.16305 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.471+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2544511107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562c7268cd50 con 0x562c7266f890 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.471+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/2544511107 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec09e030 con 0x562c7266f890 2024-09-16T00:18:22.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.471+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/2544511107 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa4ec053ce0 con 0x562c7266f890 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4ffe3a740 10 client.16305 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4ffe3a740 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4ffe3a740 20 client.16305 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c7268e0a0 0x562c72690490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4ffe3a740 4 client.16305 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2544511107 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa4ec0d8070 con 0x562c7268e0a0 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4f6ffd640 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c7268e0a0 0x562c72690490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4f6ffd640 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c7268e0a0 0x562c72690490 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b2c7e81bf998584 server_cookie=4a1971aef088bc31 in_seq=0 out_seq=0 2024-09-16T00:18:22.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.472+0000 7fa4d6ffd640 10 client.16305 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/2544511107 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa4ec0d8070 con 0x562c7268e0a0 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4d6ffd640 10 client.16305 handle_command_reply: tid=0 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 client.16305 shutdown 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2544511107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 msgr2=0x562c72673770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72673770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec09dfc0 tx=0x7fa4ec0a2430 comp rx=0 tx=0).stop 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2544511107 shutdown_connections 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c7268e0a0 0x562c72690490 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c72673cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266f890 0x562c72673770 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/2544511107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266ef70 0x562c72673230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4d6ffd640 0 client.16305 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:22.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2544511107 >> 172.21.15.78:0/2544511107 conn(0x562c7266b500 msgr2=0x562c72666270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:22.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.473+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2544511107 shutdown_connections 2024-09-16T00:18:22.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.474+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/2544511107 wait complete. 2024-09-16T00:18:22.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.474+0000 7fa4ffe3a740 20 client.16305 trim_cache size 0 max 16384 2024-09-16T00:18:22.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.498+0000 7fa4ffe3a740 1 Processor -- start 2024-09-16T00:18:22.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.498+0000 7fa4ffe3a740 1 -- start start 2024-09-16T00:18:22.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 0x562c726721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266f890 0x562c72673b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c726710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81aca50 con 0x562c7266ef70 2024-09-16T00:18:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81ae590 con 0x562c7267a150 2024-09-16T00:18:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4ffe3a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81ad120 con 0x562c7266f890 2024-09-16T00:18:22.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 0x562c726721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 0x562c726721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38004/0 (socket says 172.21.15.78:38004) 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4f6ffd640 1 -- 172.21.15.78:0/3268292139 learned_addr learned my addr 172.21.15.78:0/3268292139 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c726710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.499+0000 7fa4f67fc640 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266f890 0x562c72673b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.500+0000 7fa4f77fe640 1 -- 172.21.15.78:0/3268292139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266f890 msgr2=0x562c72673b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.500+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266f890 0x562c72673b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.500+0000 7fa4f77fe640 1 -- 172.21.15.78:0/3268292139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 msgr2=0x562c726721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.500+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 0x562c726721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.500+0000 7fa4f77fe640 1 -- 172.21.15.78:0/3268292139 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7266ab50 con 0x562c7267a150 2024-09-16T00:18:22.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.500+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c726710d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0004af0 tx=0x7fa4f0043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8361a5c0f241073 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.502+0000 7fa4dd7fa640 1 -- 172.21.15.78:0/3268292139 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0030710 con 0x562c7267a150 2024-09-16T00:18:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.502+0000 7fa4dd7fa640 1 -- 172.21.15.78:0/3268292139 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0032a20 con 0x562c7267a150 2024-09-16T00:18:22.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.502+0000 7fa4dd7fa640 1 -- 172.21.15.78:0/3268292139 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f003ec40 con 0x562c7267a150 2024-09-16T00:18:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.503+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/3268292139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 msgr2=0x562c726710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.503+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c726710d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0004af0 tx=0x7fa4f0043000 comp rx=0 tx=0).stop 2024-09-16T00:18:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.506+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/3268292139 shutdown_connections 2024-09-16T00:18:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.506+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7267a150 0x562c726710d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.506+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7266f890 0x562c72673b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.506+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/3268292139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 0x562c726721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.506+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/3268292139 >> 172.21.15.78:0/3268292139 conn(0x562c7266b500 msgr2=0x562c726709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.506+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/3268292139 shutdown_connections 2024-09-16T00:18:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.507+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/3268292139 wait complete. 2024-09-16T00:18:22.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.507+0000 7fa4ffe3a740 1 -- start start 2024-09-16T00:18:22.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.508+0000 7fa4ffe3a740 1 Processor -- start 2024-09-16T00:18:22.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.508+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 0x562c72688870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.508+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266f890 0x562c72688db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.508+0000 7fa4ffe3a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c726f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.509+0000 7fa4f67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266f890 0x562c72688db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.509+0000 7fa4f67fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266f890 0x562c72688db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60084/0 (socket says 172.21.15.78:60084) 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.509+0000 7fa4f67fc640 1 -- 172.21.15.78:0/392898013 learned_addr learned my addr 172.21.15.78:0/392898013 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.509+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c726f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.509+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81adec0 con 0x562c7266ef70 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.509+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81ad7f0 con 0x562c7266f890 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.509+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81aaa30 con 0x562c7267a150 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.509+0000 7fa4ffe3a740 10 client.0 _ll_register_callbacks cb 0x7fa4e80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.510+0000 7fa4f77fe640 1 -- 172.21.15.78:0/392898013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266f890 msgr2=0x562c72688db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.510+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266f890 0x562c72688db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.510+0000 7fa4f77fe640 1 -- 172.21.15.78:0/392898013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 msgr2=0x562c72688870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:22.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.510+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 0x562c72688870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.510+0000 7fa4f77fe640 1 -- 172.21.15.78:0/392898013 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c726f7f20 con 0x562c7267a150 2024-09-16T00:18:22.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.510+0000 7fa4f77fe640 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c726f79e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4f002bd90 tx=0x7fa4f003eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b2f646cfc23ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:22.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.510+0000 7fa4d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:18:22.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.518+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/392898013 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0059070 con 0x562c7267a150 2024-09-16T00:18:22.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.518+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/392898013 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa4f0030710 con 0x562c7267a150 2024-09-16T00:18:22.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.518+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f0032a20 con 0x562c7267a150 2024-09-16T00:18:22.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.518+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/392898013 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa4f0030950 con 0x562c7267a150 2024-09-16T00:18:22.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.519+0000 7fa4ffe3a740 10 client.25639 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:22.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.519+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562c726f9140 con 0x562c7267a150 2024-09-16T00:18:22.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.522+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/392898013 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0032a20 con 0x562c7267a150 2024-09-16T00:18:22.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.522+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/392898013 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa4f005eaa0 con 0x562c7267a150 2024-09-16T00:18:22.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.524+0000 7fa4ffe3a740 10 client.25639 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:22.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.524+0000 7fa4ffe3a740 10 client.25639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:22.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.524+0000 7fa4ffe3a740 20 client.25639 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:22.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.524+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c726f9710 0x562c726fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:22.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.524+0000 7fa4ffe3a740 4 client.25639 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:18:22.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.524+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c725dd7f0 con 0x562c726f9710 2024-09-16T00:18:22.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.527+0000 7fa4f6ffd640 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c726f9710 0x562c726fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:22.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.528+0000 7fa4f6ffd640 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c726f9710 0x562c726fbb30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9fbd628e1a3de9 server_cookie=c578b4b85961c03b in_seq=0 out_seq=0 2024-09-16T00:18:22.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.528+0000 7fa4d6ffd640 10 client.25639 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:22.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.530+0000 7fa4d6ffd640 1 -- 172.21.15.78:0/392898013 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c725dd7f0 con 0x562c726f9710 2024-09-16T00:18:22.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.530+0000 7fa4d6ffd640 10 client.25639 handle_command_reply: tid=0 2024-09-16T00:18:22.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.531+0000 7fa4ffe3a740 1 client.25639 shutdown 2024-09-16T00:18:22.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.538+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 msgr2=0x562c726f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.538+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c726f79e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa4f002bd90 tx=0x7fa4f003eb50 comp rx=0 tx=0).stop 2024-09-16T00:18:22.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.539+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 shutdown_connections 2024-09-16T00:18:22.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.539+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562c726f9710 0x562c726fbb30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.539+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562c7267a150 0x562c726f79e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.539+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562c7266f890 0x562c72688db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.539+0000 7fa4ffe3a740 1 --2- 172.21.15.78:0/392898013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562c7266ef70 0x562c72688870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.539+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 >> 172.21.15.78:0/392898013 conn(0x562c7266b500 msgr2=0x562c726770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:22.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.539+0000 7fa4d6ffd640 0 client.25639 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:22.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.543+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 shutdown_connections 2024-09-16T00:18:22.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.544+0000 7fa4ffe3a740 1 -- 172.21.15.78:0/392898013 wait complete. 2024-09-16T00:18:22.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.545+0000 7fa4ffe3a740 20 client.25639 trim_cache size 0 max 16384 2024-09-16T00:18:22.546 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:18:22.547 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:18:22.547 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "82a3d117-acac-4a13-9546-bba110bcf977", 2024-09-16T00:18:22.547 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:18:22.547 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:18:22.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.550+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa4e00a3390 msgr2=0x7fa4e00a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.550+0000 7fa4fda61640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa4e00a3390 0x7fa4e00a5850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa4f002c010 tx=0x7fa4f000a6d0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.550+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100de0 msgr2=0x7fa4f8196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:22.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.550+0000 7fa4fda61640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100de0 0x7fa4f8196730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009f70 tx=0x7fa4ec00ee50 comp rx=0 tx=0).stop 2024-09-16T00:18:22.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.552+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 shutdown_connections 2024-09-16T00:18:22.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.552+0000 7fa4fda61640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa4e00a3390 0x7fa4e00a5850 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.552+0000 7fa4fda61640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa4f8105560 0x7fa4f819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.552+0000 7fa4fda61640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4f8100de0 0x7fa4f8196730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.552+0000 7fa4fda61640 1 --2- 172.21.15.78:0/3639010618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4f8100430 0x7fa4f81961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:22.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.552+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 >> 172.21.15.78:0/3639010618 conn(0x7fa4f8077230 msgr2=0x7fa4f8103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:22.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.554+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 shutdown_connections 2024-09-16T00:18:22.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:22.555+0000 7fa4fda61640 1 -- 172.21.15.78:0/3639010618 wait complete. 2024-09-16T00:18:23.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:23 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.899 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:23.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:18:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:18:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-09-16T00:18:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:18:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:18:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:23 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:18:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:18:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: pgmap v510: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.3 MiB/s wr, 715 op/s 2024-09-16T00:18:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:18:23.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:18:23.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:23.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:23 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:18:24.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.548+0000 7f963f5a6640 1 -- 172.21.15.78:0/3019580271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638105560 msgr2=0x7f963810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.548+0000 7f963f5a6640 1 --2- 172.21.15.78:0/3019580271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638105560 0x7f963810d0e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9630009ca0 tx=0x7f963002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.549+0000 7f963f5a6640 1 -- 172.21.15.78:0/3019580271 shutdown_connections 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.549+0000 7f963f5a6640 1 --2- 172.21.15.78:0/3019580271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638105560 0x7f963810d0e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.549+0000 7f963f5a6640 1 --2- 172.21.15.78:0/3019580271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9638100de0 0x7f9638104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.549+0000 7f963f5a6640 1 --2- 172.21.15.78:0/3019580271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9638100430 0x7f9638100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.549+0000 7f963f5a6640 1 -- 172.21.15.78:0/3019580271 >> 172.21.15.78:0/3019580271 conn(0x7f9638077230 msgr2=0x7f96380fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.549+0000 7f963f5a6640 1 -- 172.21.15.78:0/3019580271 shutdown_connections 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.549+0000 7f963f5a6640 1 -- 172.21.15.78:0/3019580271 wait complete. 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.550+0000 7f963f5a6640 1 Processor -- start 2024-09-16T00:18:24.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.550+0000 7f963f5a6640 1 -- start start 2024-09-16T00:18:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.550+0000 7f963f5a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638100430 0x7f9638196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.550+0000 7f963f5a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9638100de0 0x7f9638196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.550+0000 7f963f5a6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9638105560 0x7f963819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.550+0000 7f963d342640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638100430 0x7f9638196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.550+0000 7f963d342640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638100430 0x7f9638196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38034/0 (socket says 172.21.15.78:38034) 2024-09-16T00:18:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.550+0000 7f963cb41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9638100de0 0x7f9638196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963cb41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9638100de0 0x7f9638196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60098/0 (socket says 172.21.15.78:60098) 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963cb41640 1 -- 172.21.15.78:0/2686635637 learned_addr learned my addr 172.21.15.78:0/2686635637 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f963810fd90 con 0x7f9638100430 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f963810fc10 con 0x7f9638100de0 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f963810ff10 con 0x7f9638105560 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963db43640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9638105560 0x7f963819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963db43640 1 -- 172.21.15.78:0/2686635637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9638100de0 msgr2=0x7f9638196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963db43640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9638100de0 0x7f9638196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963db43640 1 -- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638100430 msgr2=0x7f9638196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963db43640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638100430 0x7f9638196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963db43640 1 -- 172.21.15.78:0/2686635637 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963819b000 con 0x7f9638105560 2024-09-16T00:18:24.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.551+0000 7f963d342640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638100430 0x7f9638196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:24.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.552+0000 7f963db43640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9638105560 0x7f963819a900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9630009bf0 tx=0x7f9630003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:24.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.552+0000 7f962e7fc640 1 -- 172.21.15.78:0/2686635637 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630031750 con 0x7f9638105560 2024-09-16T00:18:24.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.552+0000 7f962e7fc640 1 -- 172.21.15.78:0/2686635637 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96300318f0 con 0x7f9638105560 2024-09-16T00:18:24.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.552+0000 7f962e7fc640 1 -- 172.21.15.78:0/2686635637 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963002fc90 con 0x7f9638105560 2024-09-16T00:18:24.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.552+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963819b290 con 0x7f9638105560 2024-09-16T00:18:24.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.553+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96381a4dc0 con 0x7f9638105560 2024-09-16T00:18:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f962e7fc640 1 -- 172.21.15.78:0/2686635637 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9630004040 con 0x7f9638105560 2024-09-16T00:18:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f964197f740 1 Processor -- start 2024-09-16T00:18:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f964197f740 1 -- start start 2024-09-16T00:18:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f964197f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc003a20 0x561afc002340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f964197f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc002cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f964197f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc018110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f964197f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561afbf682e0 con 0x561afc015d20 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f964197f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561afbf0fce0 con 0x561afc003a20 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.556+0000 7f964197f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561afbf642a0 con 0x561afc002880 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc003a20 0x561afc002340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc003a20 0x561afc002340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60108/0 (socket says 172.21.15.78:60108) 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 -- 172.21.15.78:0/1694435821 learned_addr learned my addr 172.21.15.78:0/1694435821 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963d342640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc002cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 --2- 172.21.15.78:0/1694435821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc018110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 -- 172.21.15.78:0/1694435821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 msgr2=0x561afc002cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 --2- 172.21.15.78:0/1694435821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc002cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 -- 172.21.15.78:0/1694435821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 msgr2=0x561afc018110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 --2- 172.21.15.78:0/1694435821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc018110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 -- 172.21.15.78:0/1694435821 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afbef4440 con 0x561afc003a20 2024-09-16T00:18:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f962e7fc640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96240a3410 0x7f96240a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.557+0000 7f963cb41640 1 --2- 172.21.15.78:0/1694435821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc003a20 0x561afc002340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9628010980 tx=0x7f962800b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b79454c3fbd9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.558+0000 7f962e7fc640 1 -- 172.21.15.78:0/2686635637 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f96300f0df0 con 0x7f9638105560 2024-09-16T00:18:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.558+0000 7f96237fe640 1 -- 172.21.15.78:0/1694435821 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628010110 con 0x561afc003a20 2024-09-16T00:18:24.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.558+0000 7f96237fe640 1 -- 172.21.15.78:0/1694435821 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96280102b0 con 0x561afc003a20 2024-09-16T00:18:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.558+0000 7f96237fe640 1 -- 172.21.15.78:0/1694435821 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962800da40 con 0x561afc003a20 2024-09-16T00:18:24.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.558+0000 7f963d342640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96240a3410 0x7f96240a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.559+0000 7f964197f740 1 -- 172.21.15.78:0/1694435821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc003a20 msgr2=0x561afc002340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.559+0000 7f964197f740 1 --2- 172.21.15.78:0/1694435821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc003a20 0x561afc002340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9628010980 tx=0x7f962800b800 comp rx=0 tx=0).stop 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.560+0000 7f964197f740 1 -- 172.21.15.78:0/1694435821 shutdown_connections 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.560+0000 7f964197f740 1 --2- 172.21.15.78:0/1694435821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc018110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.560+0000 7f964197f740 1 --2- 172.21.15.78:0/1694435821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc002cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.560+0000 7f964197f740 1 --2- 172.21.15.78:0/1694435821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc003a20 0x561afc002340 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9628010980 tx=0x7f962800b800 comp rx=0 tx=0).stop 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.560+0000 7f964197f740 1 -- 172.21.15.78:0/1694435821 >> 172.21.15.78:0/1694435821 conn(0x561afbffea30 msgr2=0x561afbffee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.561+0000 7f964197f740 1 -- 172.21.15.78:0/1694435821 shutdown_connections 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.561+0000 7f963d342640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96240a3410 0x7f96240a58d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x561afbfff890 tx=0x7f9634005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:24.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.561+0000 7f964197f740 1 -- 172.21.15.78:0/1694435821 wait complete. 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 1 -- start start 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 1 Processor -- start 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc015d20 0x561afc006590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc006ad0 0x561afc021f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561afc018a50 con 0x561afc006ad0 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561afbf23050 con 0x561afc015d20 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561afc018bd0 con 0x561afc002880 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f964197f740 10 client.0 _ll_register_callbacks cb 0x561afbf63160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f963cb41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc015d20 0x561afc006590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f963d342640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f963db43640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc006ad0 0x561afc021f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f963d342640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc006050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:54134/0 (socket says 172.21.15.78:54134) 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.562+0000 7f963d342640 1 -- 172.21.15.78:0/330394893 learned_addr learned my addr 172.21.15.78:0/330394893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f963cb41640 1 -- 172.21.15.78:0/330394893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 msgr2=0x561afc006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f963cb41640 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f963cb41640 1 -- 172.21.15.78:0/330394893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc006ad0 msgr2=0x561afc021f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f963cb41640 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc006ad0 0x561afc021f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f963cb41640 1 -- 172.21.15.78:0/330394893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afbcd93c0 con 0x561afc015d20 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f963db43640 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc006ad0 0x561afc021f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f963cb41640 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc015d20 0x561afc006590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9628017000 tx=0x7f962800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f752b2eacf7faeb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f9620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f9620ff9640 1 -- 172.21.15.78:0/330394893 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9628010110 con 0x561afc015d20 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f964197f740 1 -- 172.21.15.78:0/330394893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561afbf75850 con 0x561afc015d20 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.563+0000 7f9620ff9640 1 -- 172.21.15.78:0/330394893 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9628004220 con 0x561afc015d20 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f9620ff9640 1 -- 172.21.15.78:0/330394893 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9628031050 con 0x561afc015d20 2024-09-16T00:18:24.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f964197f740 10 client.25766 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f964197f740 1 -- 172.21.15.78:0/330394893 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561afc024af0 con 0x561afc015d20 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f9620ff9640 1 -- 172.21.15.78:0/330394893 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561afbf75850 con 0x561afc015d20 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f9620ff9640 1 -- 172.21.15.78:0/330394893 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f962800b040 con 0x561afc015d20 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f964197f740 10 client.25766 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f964197f740 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f964197f740 20 client.25766 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f964197f740 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561afc025ea0 0x561afc028290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f964197f740 4 client.25766 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f964197f740 1 -- 172.21.15.78:0/330394893 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561afbf6fd60 con 0x561afc025ea0 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.564+0000 7f963d342640 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561afc025ea0 0x561afc028290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.565+0000 7f963d342640 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561afc025ea0 0x561afc028290 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a7dbf65c173045 server_cookie=5e0b64f303f6784 in_seq=0 out_seq=0 2024-09-16T00:18:24.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.565+0000 7f9620ff9640 10 client.25766 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.565+0000 7f9620ff9640 1 -- 172.21.15.78:0/330394893 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561afbf6fd60 con 0x561afc025ea0 2024-09-16T00:18:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.565+0000 7f9620ff9640 10 client.25766 handle_command_reply: tid=0 2024-09-16T00:18:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 client.25766 shutdown 2024-09-16T00:18:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 -- 172.21.15.78:0/330394893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc015d20 msgr2=0x561afc006590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc015d20 0x561afc006590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9628017000 tx=0x7f962800da60 comp rx=0 tx=0).stop 2024-09-16T00:18:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 -- 172.21.15.78:0/330394893 shutdown_connections 2024-09-16T00:18:24.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561afc025ea0 0x561afc028290 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f9620ff9640 0 client.25766 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc006ad0 0x561afc021f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc015d20 0x561afc006590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 --2- 172.21.15.78:0/330394893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc002880 0x561afc006050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 -- 172.21.15.78:0/330394893 >> 172.21.15.78:0/330394893 conn(0x561afbffea30 msgr2=0x561afc004750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 -- 172.21.15.78:0/330394893 shutdown_connections 2024-09-16T00:18:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.566+0000 7f964197f740 1 -- 172.21.15.78:0/330394893 wait complete. 2024-09-16T00:18:24.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.567+0000 7f964197f740 20 client.25766 trim_cache size 0 max 16384 2024-09-16T00:18:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.590+0000 7f964197f740 1 Processor -- start 2024-09-16T00:18:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.590+0000 7f964197f740 1 -- start start 2024-09-16T00:18:24.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f964197f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc002880 0x561afc012410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f964197f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc006ad0 0x561afc021d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f964197f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc022260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f964197f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96381ac9b0 con 0x561afc015d20 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f964197f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96381ae4f0 con 0x561afc002880 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f964197f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96381ad080 con 0x561afc006ad0 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f963d342640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc002880 0x561afc012410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f963d342640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc002880 0x561afc012410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60118/0 (socket says 172.21.15.78:60118) 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f963d342640 1 -- 172.21.15.78:0/874435353 learned_addr learned my addr 172.21.15.78:0/874435353 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f963db43640 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc022260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.591+0000 7f963cb41640 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc006ad0 0x561afc021d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f963db43640 1 -- 172.21.15.78:0/874435353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc006ad0 msgr2=0x561afc021d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f963db43640 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc006ad0 0x561afc021d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f963db43640 1 -- 172.21.15.78:0/874435353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc002880 msgr2=0x561afc012410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f963db43640 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc002880 0x561afc012410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f963db43640 1 -- 172.21.15.78:0/874435353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afc003290 con 0x561afc015d20 2024-09-16T00:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f963d342640 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc002880 0x561afc012410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f963cb41640 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc006ad0 0x561afc021d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f963db43640 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc022260 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f96300ee530 tx=0x7f96300b8bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be46029953941d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f9622ffd640 1 -- 172.21.15.78:0/874435353 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300bd750 con 0x561afc015d20 2024-09-16T00:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f9622ffd640 1 -- 172.21.15.78:0/874435353 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630036040 con 0x561afc015d20 2024-09-16T00:18:24.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.592+0000 7f9622ffd640 1 -- 172.21.15.78:0/874435353 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300dd0b0 con 0x561afc015d20 2024-09-16T00:18:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.596+0000 7f964197f740 1 -- 172.21.15.78:0/874435353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 msgr2=0x561afc022260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.596+0000 7f964197f740 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc022260 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f96300ee530 tx=0x7f96300b8bd0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.597+0000 7f964197f740 1 -- 172.21.15.78:0/874435353 shutdown_connections 2024-09-16T00:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.597+0000 7f964197f740 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc015d20 0x561afc022260 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.597+0000 7f964197f740 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc006ad0 0x561afc021d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.597+0000 7f964197f740 1 --2- 172.21.15.78:0/874435353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc002880 0x561afc012410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.597+0000 7f964197f740 1 -- 172.21.15.78:0/874435353 >> 172.21.15.78:0/874435353 conn(0x561afbffea30 msgr2=0x561afc011d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:24.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.597+0000 7f964197f740 1 -- 172.21.15.78:0/874435353 shutdown_connections 2024-09-16T00:18:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.601+0000 7f964197f740 1 -- 172.21.15.78:0/874435353 wait complete. 2024-09-16T00:18:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.601+0000 7f964197f740 1 -- start start 2024-09-16T00:18:24.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.601+0000 7f964197f740 1 Processor -- start 2024-09-16T00:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.602+0000 7f964197f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc002880 0x561afc01e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.602+0000 7f964197f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc006ad0 0x561afc01e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.602+0000 7f964197f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc015d20 0x561afc0167c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.602+0000 7f964197f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96381ade20 con 0x561afc002880 2024-09-16T00:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.602+0000 7f964197f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96381ad750 con 0x561afc006ad0 2024-09-16T00:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.602+0000 7f964197f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96381aa990 con 0x561afc015d20 2024-09-16T00:18:24.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.602+0000 7f964197f740 10 client.0 _ll_register_callbacks cb 0x7f9628010470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.607+0000 7f963db43640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc015d20 0x561afc0167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.607+0000 7f963db43640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc015d20 0x561afc0167c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:54158/0 (socket says 172.21.15.78:54158) 2024-09-16T00:18:24.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.607+0000 7f963db43640 1 -- 172.21.15.78:0/661881911 learned_addr learned my addr 172.21.15.78:0/661881911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.608+0000 7f963cb41640 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc006ad0 0x561afc01e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.608+0000 7f963d342640 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc002880 0x561afc01e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.613+0000 7f963db43640 1 -- 172.21.15.78:0/661881911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc006ad0 msgr2=0x561afc01e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.613+0000 7f963db43640 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc006ad0 0x561afc01e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.613+0000 7f963db43640 1 -- 172.21.15.78:0/661881911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc002880 msgr2=0x561afc01e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.613+0000 7f963db43640 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc002880 0x561afc01e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.613+0000 7f963db43640 1 -- 172.21.15.78:0/661881911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afc016fc0 con 0x561afc015d20 2024-09-16T00:18:24.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.626+0000 7f963db43640 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc015d20 0x561afc0167c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f96300c5790 tx=0x7f96300c1400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f846ab17edfe8dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:24.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.626+0000 7f9620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:18:24.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.626+0000 7f9620ff9640 1 -- 172.21.15.78:0/661881911 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300bd050 con 0x561afc015d20 2024-09-16T00:18:24.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.626+0000 7f964197f740 1 -- 172.21.15.78:0/661881911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9630036040 con 0x561afc015d20 2024-09-16T00:18:24.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.626+0000 7f9620ff9640 1 -- 172.21.15.78:0/661881911 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9630009890 con 0x561afc015d20 2024-09-16T00:18:24.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.627+0000 7f9620ff9640 1 -- 172.21.15.78:0/661881911 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f96300bb0f0 con 0x561afc015d20 2024-09-16T00:18:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.628+0000 7f964197f740 10 client.25657 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.628+0000 7f964197f740 1 -- 172.21.15.78:0/661881911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561afc01da60 con 0x561afc015d20 2024-09-16T00:18:24.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.630+0000 7f9620ff9640 1 -- 172.21.15.78:0/661881911 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630036040 con 0x561afc015d20 2024-09-16T00:18:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.631+0000 7f9620ff9640 1 -- 172.21.15.78:0/661881911 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f96300d8070 con 0x561afc015d20 2024-09-16T00:18:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.631+0000 7f964197f740 10 client.25657 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.631+0000 7f964197f740 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.631+0000 7f964197f740 20 client.25657 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.631+0000 7f964197f740 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561afc025ea0 0x561afc092c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.631+0000 7f964197f740 4 client.25657 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:18:24.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.631+0000 7f964197f740 1 -- 172.21.15.78:0/661881911 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561afbeec210 con 0x561afc025ea0 2024-09-16T00:18:24.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.633+0000 7f963d342640 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561afc025ea0 0x561afc092c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:24.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.637+0000 7f963d342640 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561afc025ea0 0x561afc092c60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=380ee75cd4b434af server_cookie=5d3d6b50e1acdea7 in_seq=0 out_seq=0 2024-09-16T00:18:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.637+0000 7f9620ff9640 10 client.25657 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.637+0000 7f9620ff9640 1 -- 172.21.15.78:0/661881911 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561afbeec210 con 0x561afc025ea0 2024-09-16T00:18:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.637+0000 7f9620ff9640 10 client.25657 handle_command_reply: tid=0 2024-09-16T00:18:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.640+0000 7f964197f740 1 client.25657 shutdown 2024-09-16T00:18:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.643+0000 7f964197f740 1 -- 172.21.15.78:0/661881911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc015d20 msgr2=0x561afc0167c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.643+0000 7f964197f740 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc015d20 0x561afc0167c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f96300c5790 tx=0x7f96300c1400 comp rx=0 tx=0).stop 2024-09-16T00:18:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.644+0000 7f964197f740 1 -- 172.21.15.78:0/661881911 shutdown_connections 2024-09-16T00:18:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.644+0000 7f964197f740 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561afc025ea0 0x561afc092c60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.644+0000 7f964197f740 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afc015d20 0x561afc0167c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.644+0000 7f964197f740 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561afc006ad0 0x561afc01e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.644+0000 7f964197f740 1 --2- 172.21.15.78:0/661881911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561afc002880 0x561afc01e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.644+0000 7f964197f740 1 -- 172.21.15.78:0/661881911 >> 172.21.15.78:0/661881911 conn(0x561afbffea30 msgr2=0x561afbffd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.644+0000 7f9620ff9640 0 client.25657 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:24.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.648+0000 7f964197f740 1 -- 172.21.15.78:0/661881911 shutdown_connections 2024-09-16T00:18:24.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.650+0000 7f964197f740 1 -- 172.21.15.78:0/661881911 wait complete. 2024-09-16T00:18:24.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.650+0000 7f964197f740 20 client.25657 trim_cache size 0 max 16384 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout: "82a3d117-acac-4a13-9546-bba110bcf977": { 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "82a3d117-acac-4a13-9546-bba110bcf977", 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:24.651 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:18:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96240a3410 msgr2=0x7f96240a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96240a3410 0x7f96240a58d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x561afbfff890 tx=0x7f9634005f90 comp rx=0 tx=0).stop 2024-09-16T00:18:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9638105560 msgr2=0x7f963819a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9638105560 0x7f963819a900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9630009bf0 tx=0x7f9630003f80 comp rx=0 tx=0).stop 2024-09-16T00:18:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 shutdown_connections 2024-09-16T00:18:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96240a3410 0x7f96240a58d0 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x561afbfff890 tx=0x7f9634005f90 comp rx=0 tx=0).stop 2024-09-16T00:18:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9638105560 0x7f963819a900 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9638100de0 0x7f9638196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 --2- 172.21.15.78:0/2686635637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9638100430 0x7f9638196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:24.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.655+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 >> 172.21.15.78:0/2686635637 conn(0x7f9638077230 msgr2=0x7f9638104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:24.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.658+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 shutdown_connections 2024-09-16T00:18:24.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:24.658+0000 7f963f5a6640 1 -- 172.21.15.78:0/2686635637 wait complete. 2024-09-16T00:18:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 265.0 s, 1306.6 tps, lat 76.684 ms stddev 82.088 2024-09-16T00:18:25.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:25 smithi193 ceph-mon[35484]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 764 op/s 2024-09-16T00:18:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:25 smithi148 ceph-mon[35599]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 764 op/s 2024-09-16T00:18:26.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:25 smithi078 ceph-mon[31157]: pgmap v511: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 MiB/s wr, 764 op/s 2024-09-16T00:18:27.017 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:18:27.017 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-16T00:18:27.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:18:27.025 INFO:tasks.cephfs.filesystem:scrub status for tag:82a3d117-acac-4a13-9546-bba110bcf977 - {'path': '/', 'tag': '82a3d117-acac-4a13-9546-bba110bcf977', 'options': 'recursive,force'} 2024-09-16T00:18:27.417 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:27.860 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:27 smithi078 ceph-mon[31157]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 MiB/s wr, 708 op/s 2024-09-16T00:18:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:27 smithi193 ceph-mon[35484]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 MiB/s wr, 708 op/s 2024-09-16T00:18:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:27 smithi148 ceph-mon[35599]: pgmap v512: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.3 MiB/s wr, 708 op/s 2024-09-16T00:18:27.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.982+0000 7f84227fc640 1 -- 172.21.15.78:0/3175089616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c02e320 con 0x7f8424074eb0 2024-09-16T00:18:27.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.982+0000 7f8429c0a640 1 -- 172.21.15.78:0/3175089616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8424074eb0 msgr2=0x7f8424112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:27.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.982+0000 7f8429c0a640 1 --2- 172.21.15.78:0/3175089616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8424074eb0 0x7f8424112650 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f841c009ca0 tx=0x7f841c02a970 comp rx=0 tx=0).stop 2024-09-16T00:18:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.985+0000 7f8429c0a640 1 -- 172.21.15.78:0/3175089616 shutdown_connections 2024-09-16T00:18:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.985+0000 7f8429c0a640 1 --2- 172.21.15.78:0/3175089616 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424112b90 0x7f8424114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.985+0000 7f8429c0a640 1 --2- 172.21.15.78:0/3175089616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8424074eb0 0x7f8424112650 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.985+0000 7f8429c0a640 1 --2- 172.21.15.78:0/3175089616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8424074590 0x7f8424074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.985+0000 7f8429c0a640 1 -- 172.21.15.78:0/3175089616 >> 172.21.15.78:0/3175089616 conn(0x7f84240fd8c0 msgr2=0x7f84240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.986+0000 7f8429c0a640 1 -- 172.21.15.78:0/3175089616 shutdown_connections 2024-09-16T00:18:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.986+0000 7f8429c0a640 1 -- 172.21.15.78:0/3175089616 wait complete. 2024-09-16T00:18:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.987+0000 7f8429c0a640 1 Processor -- start 2024-09-16T00:18:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.987+0000 7f8429c0a640 1 -- start start 2024-09-16T00:18:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8429c0a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8424074590 0x7f84241a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:27.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8429c0a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8424074eb0 0x7f84241a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:27.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8429c0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424112b90 0x7f842419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:27.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8429c0a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8424118630 con 0x7f8424074590 2024-09-16T00:18:27.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8429c0a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f84241184b0 con 0x7f8424112b90 2024-09-16T00:18:27.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8429c0a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f84241187b0 con 0x7f8424074eb0 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8423fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424112b90 0x7f842419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8423fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424112b90 0x7f842419e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39142/0 (socket says 172.21.15.78:39142) 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8423fff640 1 -- 172.21.15.78:0/490743964 learned_addr learned my addr 172.21.15.78:0/490743964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.988+0000 7f8422ffd640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8424074eb0 0x7f84241a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.989+0000 7f84237fe640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8424074590 0x7f84241a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.989+0000 7f8423fff640 1 -- 172.21.15.78:0/490743964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8424074eb0 msgr2=0x7f84241a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.989+0000 7f8423fff640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8424074eb0 0x7f84241a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.989+0000 7f8423fff640 1 -- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8424074590 msgr2=0x7f84241a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:27.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.989+0000 7f8423fff640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8424074590 0x7f84241a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:27.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.989+0000 7f8423fff640 1 -- 172.21.15.78:0/490743964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842419f090 con 0x7f8424112b90 2024-09-16T00:18:27.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.994+0000 7f8423fff640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424112b90 0x7f842419e960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f841400e900 tx=0x7f841400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:27.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.995+0000 7f8420ff9640 1 -- 172.21.15.78:0/490743964 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841400cd80 con 0x7f8424112b90 2024-09-16T00:18:27.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.995+0000 7f8420ff9640 1 -- 172.21.15.78:0/490743964 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84140035f0 con 0x7f8424112b90 2024-09-16T00:18:27.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.995+0000 7f8420ff9640 1 -- 172.21.15.78:0/490743964 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414010670 con 0x7f8424112b90 2024-09-16T00:18:27.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.996+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842419f380 con 0x7f8424112b90 2024-09-16T00:18:27.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:27.996+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84241ab670 con 0x7f8424112b90 2024-09-16T00:18:28.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.001+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8424074970 con 0x7f8424112b90 2024-09-16T00:18:28.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.002+0000 7f8420ff9640 1 -- 172.21.15.78:0/490743964 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f84140037a0 con 0x7f8424112b90 2024-09-16T00:18:28.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.003+0000 7f8420ff9640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f840c0a3450 0x7f840c0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:28.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.003+0000 7f8420ff9640 1 -- 172.21.15.78:0/490743964 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f84140d02b0 con 0x7f8424112b90 2024-09-16T00:18:28.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.004+0000 7f84237fe640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f840c0a3450 0x7f840c0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:28.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.005+0000 7f84237fe640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f840c0a3450 0x7f840c0a5910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8418006fd0 tx=0x7f8418008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:28.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.009+0000 7f8420ff9640 1 -- 172.21.15.78:0/490743964 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f841401d060 con 0x7f8424112b90 2024-09-16T00:18:28.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.344+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8424104fd0 con 0x7f8424112b90 2024-09-16T00:18:28.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.345+0000 7f8420ff9640 1 -- 172.21.15.78:0/490743964 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f841401f2f0 con 0x7f8424112b90 2024-09-16T00:18:28.346 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:18:28.346 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:18:28.347 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:18:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.348+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f840c0a3450 msgr2=0x7f840c0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.348+0000 7f8429c0a640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f840c0a3450 0x7f840c0a5910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8418006fd0 tx=0x7f8418008040 comp rx=0 tx=0).stop 2024-09-16T00:18:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.348+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424112b90 msgr2=0x7f842419e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.348+0000 7f8429c0a640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424112b90 0x7f842419e960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f841400e900 tx=0x7f841400edc0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.349+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 shutdown_connections 2024-09-16T00:18:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.349+0000 7f8429c0a640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f840c0a3450 0x7f840c0a5910 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.349+0000 7f8429c0a640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8424112b90 0x7f842419e960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.349+0000 7f8429c0a640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8424074eb0 0x7f84241a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.349+0000 7f8429c0a640 1 --2- 172.21.15.78:0/490743964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8424074590 0x7f84241a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.349+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 >> 172.21.15.78:0/490743964 conn(0x7f84240fd8c0 msgr2=0x7f84240ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.349+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 shutdown_connections 2024-09-16T00:18:28.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:28.349+0000 7f8429c0a640 1 -- 172.21.15.78:0/490743964 wait complete. 2024-09-16T00:18:28.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:28 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/490743964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:28.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:28 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/490743964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:29.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:28 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/490743964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:29.285 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-16T00:18:29.285 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-16T00:18:29.727 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:29.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:29 smithi193 ceph-mon[35484]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 716 op/s 2024-09-16T00:18:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:29 smithi148 ceph-mon[35599]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 716 op/s 2024-09-16T00:18:30.007 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:29 smithi078 ceph-mon[31157]: pgmap v513: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 716 op/s 2024-09-16T00:18:30.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.353+0000 7f39ccb23640 1 -- 172.21.15.78:0/245118746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39c8109450 msgr2=0x7f39c8109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:30.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.353+0000 7f39ccb23640 1 --2- 172.21.15.78:0/245118746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39c8109450 0x7f39c8109830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f39b80074e0 tx=0x7f39b802b0a0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.355+0000 7f39ccb23640 1 -- 172.21.15.78:0/245118746 shutdown_connections 2024-09-16T00:18:30.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.355+0000 7f39ccb23640 1 --2- 172.21.15.78:0/245118746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c810ea10 0x7f39c8115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.355+0000 7f39ccb23640 1 --2- 172.21.15.78:0/245118746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c8109e00 0x7f39c810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.355+0000 7f39ccb23640 1 --2- 172.21.15.78:0/245118746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39c8109450 0x7f39c8109830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.355+0000 7f39ccb23640 1 -- 172.21.15.78:0/245118746 >> 172.21.15.78:0/245118746 conn(0x7f39c8073660 msgr2=0x7f39c8073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:30.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.355+0000 7f39ccb23640 1 -- 172.21.15.78:0/245118746 shutdown_connections 2024-09-16T00:18:30.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.357+0000 7f39ccb23640 1 -- 172.21.15.78:0/245118746 wait complete. 2024-09-16T00:18:30.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.357+0000 7f39ccb23640 1 Processor -- start 2024-09-16T00:18:30.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39ccb23640 1 -- start start 2024-09-16T00:18:30.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39ccb23640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39c8109450 0x7f39c810f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39ccb23640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8109e00 0x7f39c810f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39ccb23640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c810ea10 0x7f39c810fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39ccb23640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81183b0 con 0x7f39c8109e00 2024-09-16T00:18:30.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39ccb23640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8118230 con 0x7f39c810ea10 2024-09-16T00:18:30.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39ccb23640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8118530 con 0x7f39c8109450 2024-09-16T00:18:30.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c810ea10 0x7f39c810fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.358+0000 7f39c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c810ea10 0x7f39c810fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39160/0 (socket says 172.21.15.78:39160) 2024-09-16T00:18:30.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.359+0000 7f39c7fff640 1 -- 172.21.15.78:0/626565271 learned_addr learned my addr 172.21.15.78:0/626565271 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.359+0000 7f39c7fff640 1 -- 172.21.15.78:0/626565271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39c8109450 msgr2=0x7f39c810f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.359+0000 7f39c7fff640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39c8109450 0x7f39c810f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.359+0000 7f39c7fff640 1 -- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8109e00 msgr2=0x7f39c810f940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:18:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.359+0000 7f39c7fff640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8109e00 0x7f39c810f940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.359+0000 7f39c7fff640 1 -- 172.21.15.78:0/626565271 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c81b34f0 con 0x7f39c810ea10 2024-09-16T00:18:30.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.359+0000 7f39c7fff640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c810ea10 0x7f39c810fe80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c720 tx=0x7f39bc00cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:30.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.362+0000 7f39c4ff9640 1 -- 172.21.15.78:0/626565271 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc007bb0 con 0x7f39c810ea10 2024-09-16T00:18:30.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.362+0000 7f39c4ff9640 1 -- 172.21.15.78:0/626565271 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39bc00ce30 con 0x7f39c810ea10 2024-09-16T00:18:30.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.362+0000 7f39c4ff9640 1 -- 172.21.15.78:0/626565271 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00f450 con 0x7f39c810ea10 2024-09-16T00:18:30.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.362+0000 7f39ccb23640 1 -- 172.21.15.78:0/626565271 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c81b3720 con 0x7f39c810ea10 2024-09-16T00:18:30.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.362+0000 7f39ccb23640 1 -- 172.21.15.78:0/626565271 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81b3c10 con 0x7f39c810ea10 2024-09-16T00:18:30.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.365+0000 7f39d015e740 1 Processor -- start 2024-09-16T00:18:30.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.366+0000 7f39c4ff9640 1 -- 172.21.15.78:0/626565271 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f39bc004eb0 con 0x7f39c810ea10 2024-09-16T00:18:30.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39d015e740 1 -- start start 2024-09-16T00:18:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39d015e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 0x55f84e658e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39d015e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39d015e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e665ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39d015e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f84e5bd160 con 0x55f84e658a20 2024-09-16T00:18:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39d015e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f84e564720 con 0x55f84e659340 2024-09-16T00:18:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39d015e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f84e5b9120 con 0x55f84e663c00 2024-09-16T00:18:30.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 0x55f84e658e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 0x55f84e658e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56018/0 (socket says 172.21.15.78:56018) 2024-09-16T00:18:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39176/0 (socket says 172.21.15.78:39176) 2024-09-16T00:18:30.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.367+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2255356141 learned_addr learned my addr 172.21.15.78:0/2255356141 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.368+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e665ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.368+0000 7f39c77fe640 1 -- 172.21.15.78:0/2255356141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 msgr2=0x55f84e665ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.368+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e665ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.368+0000 7f39c77fe640 1 -- 172.21.15.78:0/2255356141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 msgr2=0x55f84e658e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.368+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 0x55f84e658e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.368+0000 7f39c77fe640 1 -- 172.21.15.78:0/2255356141 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f84e32eba0 con 0x55f84e659340 2024-09-16T00:18:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.369+0000 7f39c4ff9640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f39b00a33d0 0x7f39b00a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.369+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e665ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.369+0000 7f39c4ff9640 1 -- 172.21.15.78:0/626565271 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f39bc0cf870 con 0x7f39c810ea10 2024-09-16T00:18:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.369+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6636c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39b80074b0 tx=0x7f39b8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10851ea4e56249a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.369+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f39b00a33d0 0x7f39b00a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.369+0000 7f39adffb640 1 -- 172.21.15.78:0/2255356141 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8004780 con 0x55f84e659340 2024-09-16T00:18:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.369+0000 7f39adffb640 1 -- 172.21.15.78:0/2255356141 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b802b9e0 con 0x55f84e659340 2024-09-16T00:18:30.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.369+0000 7f39adffb640 1 -- 172.21.15.78:0/2255356141 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b80035f0 con 0x55f84e659340 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 -- 172.21.15.78:0/2255356141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 msgr2=0x55f84e6636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f39b00a33d0 0x7f39b00a5890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55f84e656330 tx=0x7f39c000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6636c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39b80074b0 tx=0x7f39b8005bd0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 -- 172.21.15.78:0/2255356141 shutdown_connections 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e665ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6636c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 --2- 172.21.15.78:0/2255356141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 0x55f84e658e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 -- 172.21.15.78:0/2255356141 >> 172.21.15.78:0/2255356141 conn(0x55f84e654fb0 msgr2=0x55f84e655390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 -- 172.21.15.78:0/2255356141 shutdown_connections 2024-09-16T00:18:30.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.370+0000 7f39d015e740 1 -- 172.21.15.78:0/2255356141 wait complete. 2024-09-16T00:18:30.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.371+0000 7f39d015e740 1 -- start start 2024-09-16T00:18:30.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.371+0000 7f39d015e740 1 Processor -- start 2024-09-16T00:18:30.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39d015e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e660a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39d015e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e659340 0x55f84e660f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39d015e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e663c00 0x55f84e6614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39d015e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f84e666770 con 0x55f84e659340 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39d015e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f84e545900 con 0x55f84e663c00 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39d015e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f84e6668f0 con 0x55f84e658a20 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39d015e740 10 client.0 _ll_register_callbacks cb 0x55f84e543780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e659340 0x55f84e660f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e659340 0x55f84e660f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56024/0 (socket says 172.21.15.78:56024) 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2305626807 learned_addr learned my addr 172.21.15.78:0/2305626807 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e660a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.372+0000 7f39c7fff640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e663c00 0x55f84e6614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2305626807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 msgr2=0x55f84e660a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e660a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e663c00 msgr2=0x55f84e6614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e663c00 0x55f84e6614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2305626807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f84e585c50 con 0x55f84e659340 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e660a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39c7fff640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e663c00 0x55f84e6614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e659340 0x55f84e660f80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f39c000b010 tx=0x7f39c0009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7caa8c255203b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2305626807 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0028060 con 0x55f84e659340 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.373+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2305626807 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f39c0012070 con 0x55f84e659340 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.374+0000 7f39d015e740 1 -- 172.21.15.78:0/2305626807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39b802b9e0 con 0x55f84e659340 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.374+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2305626807 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f39c0023420 con 0x55f84e659340 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.374+0000 7f39d015e740 10 client.16338 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:30.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.374+0000 7f39d015e740 1 -- 172.21.15.78:0/2305626807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f84e6723a0 con 0x55f84e659340 2024-09-16T00:18:30.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.375+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2305626807 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b802b9e0 con 0x55f84e659340 2024-09-16T00:18:30.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.375+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2305626807 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f39c0008540 con 0x55f84e659340 2024-09-16T00:18:30.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.375+0000 7f39d015e740 10 client.16338 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:30.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.375+0000 7f39d015e740 10 client.16338 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:18:30.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.375+0000 7f39d015e740 20 client.16338 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:30.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.375+0000 7f39d015e740 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f84e6737b0 0x55f84e675bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.375+0000 7f39d015e740 4 client.16338 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:30.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.375+0000 7f39d015e740 1 -- 172.21.15.78:0/2305626807 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f84e5ca890 con 0x55f84e6737b0 2024-09-16T00:18:30.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.376+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f84e6737b0 0x55f84e675bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.376+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f84e6737b0 0x55f84e675bd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350c796eb7bfa0cd server_cookie=abf4998c5f4a05b0 in_seq=0 out_seq=0 2024-09-16T00:18:30.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.376+0000 7f39a6ffd640 10 client.16338 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:30.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.377+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2305626807 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f84e5ca890 con 0x55f84e6737b0 2024-09-16T00:18:30.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.377+0000 7f39a6ffd640 10 client.16338 handle_command_reply: tid=0 2024-09-16T00:18:30.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.377+0000 7f39d015e740 1 client.16338 shutdown 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 -- 172.21.15.78:0/2305626807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e659340 msgr2=0x55f84e660f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e659340 0x55f84e660f80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f39c000b010 tx=0x7f39c0009770 comp rx=0 tx=0).stop 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 -- 172.21.15.78:0/2305626807 shutdown_connections 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f84e6737b0 0x55f84e675bd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e663c00 0x55f84e6614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e659340 0x55f84e660f80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 --2- 172.21.15.78:0/2305626807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e660a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 -- 172.21.15.78:0/2305626807 >> 172.21.15.78:0/2305626807 conn(0x55f84e654fb0 msgr2=0x55f84e65ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39a6ffd640 0 client.16338 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 -- 172.21.15.78:0/2305626807 shutdown_connections 2024-09-16T00:18:30.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.379+0000 7f39d015e740 1 -- 172.21.15.78:0/2305626807 wait complete. 2024-09-16T00:18:30.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.380+0000 7f39d015e740 20 client.16338 trim_cache size 0 max 16384 2024-09-16T00:18:30.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.406+0000 7f39d015e740 1 Processor -- start 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.406+0000 7f39d015e740 1 -- start start 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.406+0000 7f39d015e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e67b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.406+0000 7f39d015e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e67b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.406+0000 7f39d015e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e663c00 0x55f84e65dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39d015e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81bd9f0 con 0x55f84e663c00 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e67b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e663c00 0x55f84e65dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e67b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39200/0 (socket says 172.21.15.78:39200) 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e663c00 0x55f84e65dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56028/0 (socket says 172.21.15.78:56028) 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c6ffd640 1 -- 172.21.15.78:0/162142406 learned_addr learned my addr 172.21.15.78:0/162142406 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c77fe640 1 --2- 172.21.15.78:0/162142406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e67b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39d015e740 1 -- 172.21.15.78:0/162142406 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81bf530 con 0x55f84e659340 2024-09-16T00:18:30.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39d015e740 1 -- 172.21.15.78:0/162142406 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81be0c0 con 0x55f84e658a20 2024-09-16T00:18:30.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c7fff640 1 -- 172.21.15.78:0/162142406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 msgr2=0x55f84e67b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:30.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c7fff640 1 --2- 172.21.15.78:0/162142406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e67b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c7fff640 1 -- 172.21.15.78:0/162142406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 msgr2=0x55f84e67b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:30.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c7fff640 1 --2- 172.21.15.78:0/162142406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e67b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c7fff640 1 -- 172.21.15.78:0/162142406 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f84e659170 con 0x55f84e663c00 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.407+0000 7f39c7fff640 1 --2- 172.21.15.78:0/162142406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e663c00 0x55f84e65dca0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0cd060 tx=0x7f39bc09ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b536c88d6a94159 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.408+0000 7f39ad7fa640 1 -- 172.21.15.78:0/162142406 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc01d030 con 0x55f84e663c00 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.408+0000 7f39ad7fa640 1 -- 172.21.15.78:0/162142406 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39bc099e30 con 0x55f84e663c00 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.408+0000 7f39ad7fa640 1 -- 172.21.15.78:0/162142406 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc09ce60 con 0x55f84e663c00 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.408+0000 7f39d015e740 1 -- 172.21.15.78:0/162142406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e663c00 msgr2=0x55f84e65dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 --2- 172.21.15.78:0/162142406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e663c00 0x55f84e65dca0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0cd060 tx=0x7f39bc09ba30 comp rx=0 tx=0).stop 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 -- 172.21.15.78:0/162142406 shutdown_connections 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 --2- 172.21.15.78:0/162142406 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e663c00 0x55f84e65dca0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 --2- 172.21.15.78:0/162142406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e67b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 --2- 172.21.15.78:0/162142406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e658a20 0x55f84e67b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 -- 172.21.15.78:0/162142406 >> 172.21.15.78:0/162142406 conn(0x55f84e654fb0 msgr2=0x55f84e65f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 -- 172.21.15.78:0/162142406 shutdown_connections 2024-09-16T00:18:30.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 -- 172.21.15.78:0/162142406 wait complete. 2024-09-16T00:18:30.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 -- start start 2024-09-16T00:18:30.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.409+0000 7f39d015e740 1 Processor -- start 2024-09-16T00:18:30.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39d015e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 0x55f84e66c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39d015e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39d015e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e668a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39d015e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81bee60 con 0x55f84e658a20 2024-09-16T00:18:30.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39d015e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81be790 con 0x55f84e659340 2024-09-16T00:18:30.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39d015e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81bb9d0 con 0x55f84e663c00 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39d015e740 10 client.0 _ll_register_callbacks cb 0x7f39c00099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39214/0 (socket says 172.21.15.78:39214) 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2371334565 learned_addr learned my addr 172.21.15.78:0/2371334565 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c7fff640 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e668a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2371334565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 msgr2=0x55f84e668a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e668a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2371334565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 msgr2=0x55f84e66c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 0x55f84e66c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c6ffd640 1 -- 172.21.15.78:0/2371334565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f84e669110 con 0x55f84e659340 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.410+0000 7f39c7fff640 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e668a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:30.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.411+0000 7f39c6ffd640 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6684e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39c000f9b0 tx=0x7f39c0004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce03269aa9ad9bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.411+0000 7f39a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.411+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2371334565 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0032080 con 0x55f84e659340 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.411+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2371334565 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f39c0003a10 con 0x55f84e659340 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.411+0000 7f39d015e740 1 -- 172.21.15.78:0/2371334565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39bc099e30 con 0x55f84e659340 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.411+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2371334565 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f39c0042050 con 0x55f84e659340 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.411+0000 7f39d015e740 10 client.25793 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.411+0000 7f39d015e740 1 -- 172.21.15.78:0/2371334565 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f84e6df330 con 0x55f84e659340 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.412+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2371334565 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39bc099e30 con 0x55f84e659340 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.412+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2371334565 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f39c00045a0 con 0x55f84e659340 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.412+0000 7f39d015e740 10 client.25793 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.412+0000 7f39d015e740 10 client.25793 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.412+0000 7f39d015e740 20 client.25793 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.412+0000 7f39d015e740 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f84e6dfaf0 0x55f84e6e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.412+0000 7f39d015e740 4 client.25793 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-16T00:18:30.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.412+0000 7f39d015e740 1 -- 172.21.15.78:0/2371334565 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x55f84e5c4be0 con 0x55f84e6dfaf0 2024-09-16T00:18:30.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.413+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f84e6dfaf0 0x55f84e6e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:30.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.415+0000 7f39c77fe640 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f84e6dfaf0 0x55f84e6e1f10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d8bb258c6e19b4 server_cookie=11bcf490d1c538c3 in_seq=0 out_seq=0 2024-09-16T00:18:30.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.415+0000 7f39a6ffd640 10 client.25793 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39a6ffd640 1 -- 172.21.15.78:0/2371334565 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f84e5c4be0 con 0x55f84e6dfaf0 2024-09-16T00:18:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39a6ffd640 10 client.25793 handle_command_reply: tid=0 2024-09-16T00:18:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 client.25793 shutdown 2024-09-16T00:18:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 -- 172.21.15.78:0/2371334565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 msgr2=0x55f84e6684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6684e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39c000f9b0 tx=0x7f39c0004b60 comp rx=0 tx=0).stop 2024-09-16T00:18:30.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 -- 172.21.15.78:0/2371334565 shutdown_connections 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f84e6dfaf0 0x55f84e6e1f10 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f84e663c00 0x55f84e668a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39a6ffd640 0 client.25793 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f84e659340 0x55f84e6684e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 --2- 172.21.15.78:0/2371334565 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f84e658a20 0x55f84e66c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 -- 172.21.15.78:0/2371334565 >> 172.21.15.78:0/2371334565 conn(0x55f84e654fb0 msgr2=0x55f84e6622a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 -- 172.21.15.78:0/2371334565 shutdown_connections 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 1 -- 172.21.15.78:0/2371334565 wait complete. 2024-09-16T00:18:30.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.665+0000 7f39d015e740 20 client.25793 trim_cache size 0 max 16384 2024-09-16T00:18:30.668 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:18:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:18:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:18:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 18, 2024-09-16T00:18:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:18:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "b802905": { 2024-09-16T00:18:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 18, 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 145.3, 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:18:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:18:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:18:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:30.670 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:18:30.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 -- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f39b00a33d0 msgr2=0x7f39b00a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:30.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f39b00a33d0 0x7f39b00a5890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x55f84e656330 tx=0x7f39c000b040 comp rx=0 tx=0).stop 2024-09-16T00:18:30.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 -- 172.21.15.78:0/626565271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c810ea10 msgr2=0x7f39c810fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c810ea10 0x7f39c810fe80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c720 tx=0x7f39bc00cbe0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 -- 172.21.15.78:0/626565271 shutdown_connections 2024-09-16T00:18:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f39b00a33d0 0x7f39b00a5890 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39c810ea10 0x7f39c810fe80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c8109e00 0x7f39c810f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 --2- 172.21.15.78:0/626565271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39c8109450 0x7f39c810f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:30.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 -- 172.21.15.78:0/626565271 >> 172.21.15.78:0/626565271 conn(0x7f39c8073660 msgr2=0x7f39c81139d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:30.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 -- 172.21.15.78:0/626565271 shutdown_connections 2024-09-16T00:18:30.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:30.666+0000 7f39ccb23640 1 -- 172.21.15.78:0/626565271 wait complete. 2024-09-16T00:18:30.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 270.0 s, 981.5 tps, lat 91.816 ms stddev 101.835 2024-09-16T00:18:31.506 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b802905', quiesce took 4.5 seconds. Will release after: 20.85 2024-09-16T00:18:31.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:31 smithi078 ceph-mon[31157]: pgmap v514: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 729 op/s 2024-09-16T00:18:31.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:31 smithi193 ceph-mon[35484]: pgmap v514: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 729 op/s 2024-09-16T00:18:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:31 smithi148 ceph-mon[35599]: pgmap v514: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 729 op/s 2024-09-16T00:18:33.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:33 smithi193 ceph-mon[35484]: pgmap v515: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 MiB/s wr, 734 op/s 2024-09-16T00:18:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:33 smithi148 ceph-mon[35599]: pgmap v515: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 MiB/s wr, 734 op/s 2024-09-16T00:18:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:33 smithi078 ceph-mon[31157]: pgmap v515: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.9 MiB/s wr, 734 op/s 2024-09-16T00:18:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:34 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:18:34.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:34 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:18:34.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:34 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:18:35.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:34 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:18:35.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:34 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:18:35.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:34 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:18:35.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:34 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:18:35.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:34 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:18:35.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:34 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:18:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 275.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:18:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:35 smithi148 ceph-mon[35599]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 560 op/s 2024-09-16T00:18:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:35 smithi078 ceph-mon[31157]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 560 op/s 2024-09-16T00:18:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:35 smithi193 ceph-mon[35484]: pgmap v516: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 560 op/s 2024-09-16T00:18:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:37 smithi148 ceph-mon[35599]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-16T00:18:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:37 smithi078 ceph-mon[31157]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-16T00:18:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:37 smithi193 ceph-mon[35484]: pgmap v517: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 480 op/s 2024-09-16T00:18:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:39 smithi148 ceph-mon[35599]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 443 op/s 2024-09-16T00:18:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:39 smithi078 ceph-mon[31157]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 443 op/s 2024-09-16T00:18:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:39 smithi193 ceph-mon[35484]: pgmap v518: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 443 op/s 2024-09-16T00:18:40.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 280.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:18:41.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:41 smithi148 ceph-mon[35599]: pgmap v519: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 536 op/s 2024-09-16T00:18:41.982 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:41 smithi193 ceph-mon[35484]: pgmap v519: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 536 op/s 2024-09-16T00:18:42.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:41 smithi078 ceph-mon[31157]: pgmap v519: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 536 op/s 2024-09-16T00:18:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:43 smithi148 ceph-mon[35599]: pgmap v520: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 877 op/s 2024-09-16T00:18:44.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:43 smithi078 ceph-mon[31157]: pgmap v520: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 877 op/s 2024-09-16T00:18:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:43 smithi193 ceph-mon[35484]: pgmap v520: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 877 op/s 2024-09-16T00:18:45.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 285.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:18:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:45 smithi148 ceph-mon[35599]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-09-16T00:18:46.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:45 smithi078 ceph-mon[31157]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-09-16T00:18:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:45 smithi193 ceph-mon[35484]: pgmap v521: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 1.06k op/s 2024-09-16T00:18:48.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:47 smithi078 ceph-mon[31157]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-09-16T00:18:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:47 smithi193 ceph-mon[35484]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-09-16T00:18:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:47 smithi148 ceph-mon[35599]: pgmap v522: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2024-09-16T00:18:50.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:49 smithi078 ceph-mon[31157]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 970 op/s 2024-09-16T00:18:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:49 smithi193 ceph-mon[35484]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 970 op/s 2024-09-16T00:18:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:49 smithi148 ceph-mon[35599]: pgmap v523: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 13 MiB/s wr, 970 op/s 2024-09-16T00:18:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 290.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:18:52.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:51 smithi078 ceph-mon[31157]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 13 MiB/s wr, 926 op/s 2024-09-16T00:18:52.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:51 smithi193 ceph-mon[35484]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 13 MiB/s wr, 926 op/s 2024-09-16T00:18:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:51 smithi148 ceph-mon[35599]: pgmap v524: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 13 MiB/s wr, 926 op/s 2024-09-16T00:18:52.357 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:18:52.613 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 -- 172.21.15.78:0/2712051463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 msgr2=0x7f75141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 --2- 172.21.15.78:0/2712051463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 0x7f75141021c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7504009900 tx=0x7f750402a410 comp rx=0 tx=0).stop 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 -- 172.21.15.78:0/2712051463 shutdown_connections 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 --2- 172.21.15.78:0/2712051463 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7514106f10 0x7f751410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 --2- 172.21.15.78:0/2712051463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514102790 0x7f75141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 --2- 172.21.15.78:0/2712051463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 0x7f75141021c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 -- 172.21.15.78:0/2712051463 >> 172.21.15.78:0/2712051463 conn(0x7f75140fd8c0 msgr2=0x7f75140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 -- 172.21.15.78:0/2712051463 shutdown_connections 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 -- 172.21.15.78:0/2712051463 wait complete. 2024-09-16T00:18:53.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.116+0000 7f75197d9640 1 Processor -- start 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f75197d9640 1 -- start start 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f75197d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 0x7f7514198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f75197d9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514102790 0x7f7514198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f75197d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7514106f10 0x7f751419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f75197d9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7514111f30 con 0x7f7514101de0 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f75197d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7514111db0 con 0x7f7514106f10 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f75197d9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75141120b0 con 0x7f7514102790 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f7512ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 0x7f7514198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f7512ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 0x7f7514198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55322/0 (socket says 172.21.15.78:55322) 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f7512ffd640 1 -- 172.21.15.78:0/4195266107 learned_addr learned my addr 172.21.15.78:0/4195266107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f75137fe640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7514106f10 0x7f751419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f750bfff640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514102790 0x7f7514198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:53.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f7512ffd640 1 -- 172.21.15.78:0/4195266107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514102790 msgr2=0x7f7514198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f7512ffd640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514102790 0x7f7514198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f7512ffd640 1 -- 172.21.15.78:0/4195266107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7514106f10 msgr2=0x7f751419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f7512ffd640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7514106f10 0x7f751419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.117+0000 7f7512ffd640 1 -- 172.21.15.78:0/4195266107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751419d2c0 con 0x7f7514101de0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.118+0000 7f7512ffd640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 0x7f7514198310 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7504005ab0 tx=0x7f75040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.118+0000 7f7510ff9640 1 -- 172.21.15.78:0/4195266107 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750403f030 con 0x7f7514101de0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.118+0000 7f7510ff9640 1 -- 172.21.15.78:0/4195266107 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504003ce0 con 0x7f7514101de0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.118+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751419d550 con 0x7f7514101de0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.118+0000 7f7510ff9640 1 -- 172.21.15.78:0/4195266107 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504035070 con 0x7f7514101de0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.118+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75141a4e90 con 0x7f7514101de0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.119+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75141021c0 con 0x7f7514101de0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.120+0000 7f7510ff9640 1 -- 172.21.15.78:0/4195266107 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7504042030 con 0x7f7514101de0 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.121+0000 7f7510ff9640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f75000a3240 0x7f75000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:53.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.121+0000 7f750bfff640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f75000a3240 0x7f75000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:53.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.121+0000 7f7510ff9640 1 -- 172.21.15.78:0/4195266107 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f75040f0860 con 0x7f7514101de0 2024-09-16T00:18:53.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.121+0000 7f750bfff640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f75000a3240 0x7f75000a5700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f75141088e0 tx=0x7f74fc005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:53.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.128+0000 7f7510ff9640 1 -- 172.21.15.78:0/4195266107 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75040bd220 con 0x7f7514101de0 2024-09-16T00:18:53.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.336+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75141a5060 con 0x7f7514101de0 2024-09-16T00:18:53.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.337+0000 7f7510ff9640 1 -- 172.21.15.78:0/4195266107 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f75040bcb20 con 0x7f7514101de0 2024-09-16T00:18:53.338 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:18:53.339 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:18:53.340 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:18:53.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f75000a3240 msgr2=0x7f75000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:53.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f75000a3240 0x7f75000a5700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f75141088e0 tx=0x7f74fc005cc0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 msgr2=0x7f7514198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:53.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 0x7f7514198310 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7504005ab0 tx=0x7f75040042f0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 shutdown_connections 2024-09-16T00:18:53.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f75000a3240 0x7f75000a5700 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7514106f10 0x7f751419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7514102790 0x7f7514198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 --2- 172.21.15.78:0/4195266107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7514101de0 0x7f7514198310 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:53.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 >> 172.21.15.78:0/4195266107 conn(0x7f75140fd8c0 msgr2=0x7f75140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:53.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 shutdown_connections 2024-09-16T00:18:53.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:53.339+0000 7f75197d9640 1 -- 172.21.15.78:0/4195266107 wait complete. 2024-09-16T00:18:54.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:53 smithi078 ceph-mon[31157]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.6 MiB/s wr, 724 op/s 2024-09-16T00:18:54.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:53 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4195266107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:54.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:53 smithi193 ceph-mon[35484]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.6 MiB/s wr, 724 op/s 2024-09-16T00:18:54.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:53 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4195266107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:53 smithi148 ceph-mon[35599]: pgmap v525: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 9.6 MiB/s wr, 724 op/s 2024-09-16T00:18:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:53 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4195266107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:54.214 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id b802905 --release --await' 2024-09-16T00:18:54.214 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id b802905 --release --await 2024-09-16T00:18:54.466 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 -- 172.21.15.78:0/541895474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0010c160 msgr2=0x7f2f0010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/541895474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0010c160 0x7f2f0010e5e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009900 tx=0x7f2ee802a480 comp rx=0 tx=0).stop 2024-09-16T00:18:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 -- 172.21.15.78:0/541895474 shutdown_connections 2024-09-16T00:18:55.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/541895474 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0010c160 0x7f2f0010e5e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/541895474 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f001036a0 0x7f2f0010bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/541895474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f00102d80 0x7f2f00103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 -- 172.21.15.78:0/541895474 >> 172.21.15.78:0/541895474 conn(0x7f2f000fc780 msgr2=0x7f2f000feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 -- 172.21.15.78:0/541895474 shutdown_connections 2024-09-16T00:18:55.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 -- 172.21.15.78:0/541895474 wait complete. 2024-09-16T00:18:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 Processor -- start 2024-09-16T00:18:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.978+0000 7f2f06a6c640 1 -- start start 2024-09-16T00:18:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f06a6c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00102d80 0x7f2f001009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f06a6c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f001036a0 0x7f2f00100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f06a6c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f0010c160 0x7f2f00101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f06a6c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00111f40 con 0x7f2f00102d80 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f06a6c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00111dc0 con 0x7f2f001036a0 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f06a6c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001120c0 con 0x7f2f0010c160 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00102d80 0x7f2f001009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00102d80 0x7f2f001009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55340/0 (socket says 172.21.15.78:55340) 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 -- 172.21.15.78:0/1490715118 learned_addr learned my addr 172.21.15.78:0/1490715118 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2efffff640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f001036a0 0x7f2f00100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 -- 172.21.15.78:0/1490715118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f0010c160 msgr2=0x7f2f00101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f0010c160 0x7f2f00101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 -- 172.21.15.78:0/1490715118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f001036a0 msgr2=0x7f2f00100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f001036a0 0x7f2f00100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 -- 172.21.15.78:0/1490715118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f001a6890 con 0x7f2f00102d80 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2efffff640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f001036a0 0x7f2f00100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:55.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f04808640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00102d80 0x7f2f001009e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800d770 tx=0x7f2ef800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:55.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2efdffb640 1 -- 172.21.15.78:0/1490715118 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8017070 con 0x7f2f00102d80 2024-09-16T00:18:55.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2efdffb640 1 -- 172.21.15.78:0/1490715118 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef8005500 con 0x7f2f00102d80 2024-09-16T00:18:55.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2efdffb640 1 -- 172.21.15.78:0/1490715118 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8004e00 con 0x7f2f00102d80 2024-09-16T00:18:55.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.979+0000 7f2f06a6c640 1 -- 172.21.15.78:0/1490715118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f001a6b80 con 0x7f2f00102d80 2024-09-16T00:18:55.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.980+0000 7f2f06a6c640 1 -- 172.21.15.78:0/1490715118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f001a7070 con 0x7f2f00102d80 2024-09-16T00:18:55.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.982+0000 7f2efdffb640 1 -- 172.21.15.78:0/1490715118 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2ef8004fa0 con 0x7f2f00102d80 2024-09-16T00:18:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f08e45740 1 Processor -- start 2024-09-16T00:18:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f08e45740 1 -- start start 2024-09-16T00:18:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c75440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c82630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f08e45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557136bda620 con 0x557136c75980 2024-09-16T00:18:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f08e45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557136b81e50 con 0x557136c80240 2024-09-16T00:18:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f08e45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557136bd65e0 con 0x557136c75060 2024-09-16T00:18:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2efdffb640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2eec0a3240 0x7f2eec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f04808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f04808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55352/0 (socket says 172.21.15.78:55352) 2024-09-16T00:18:55.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f04808640 1 -- 172.21.15.78:0/2256865439 learned_addr learned my addr 172.21.15.78:0/2256865439 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:55.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f05009640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2eec0a3240 0x7f2eec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2efffff640 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c82630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.983+0000 7f2f05009640 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c75440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2efdffb640 1 -- 172.21.15.78:0/1490715118 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2ef80cf7a0 con 0x7f2f00102d80 2024-09-16T00:18:55.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2f04808640 1 -- 172.21.15.78:0/2256865439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 msgr2=0x557136c75440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:55.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2f04808640 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c75440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2f04808640 1 -- 172.21.15.78:0/2256865439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 msgr2=0x557136c82630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2f04808640 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c82630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2f04808640 1 -- 172.21.15.78:0/2256865439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557136b72d40 con 0x557136c75980 2024-09-16T00:18:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2f05009640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2eec0a3240 0x7f2eec0a5700 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80096c0 tx=0x7f2ee8009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2f04808640 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7fd00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80a41a0 tx=0x7f2ef80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfe2eb689633f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2ef6ffd640 1 -- 172.21.15.78:0/2256865439 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8020070 con 0x557136c75980 2024-09-16T00:18:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2ef6ffd640 1 -- 172.21.15.78:0/2256865439 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef80046c0 con 0x557136c75980 2024-09-16T00:18:55.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.984+0000 7f2ef6ffd640 1 -- 172.21.15.78:0/2256865439 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8005320 con 0x557136c75980 2024-09-16T00:18:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 -- 172.21.15.78:0/2256865439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 msgr2=0x557136c7fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7fd00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80a41a0 tx=0x7f2ef80048a0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 -- 172.21.15.78:0/2256865439 shutdown_connections 2024-09-16T00:18:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c82630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7fd00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 --2- 172.21.15.78:0/2256865439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c75440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 -- 172.21.15.78:0/2256865439 >> 172.21.15.78:0/2256865439 conn(0x557136c715f0 msgr2=0x557136c719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 -- 172.21.15.78:0/2256865439 shutdown_connections 2024-09-16T00:18:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 -- 172.21.15.78:0/2256865439 wait complete. 2024-09-16T00:18:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 -- start start 2024-09-16T00:18:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.985+0000 7f2f08e45740 1 Processor -- start 2024-09-16T00:18:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c7dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f08e45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557136c82db0 con 0x557136c75980 2024-09-16T00:18:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f08e45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557136b62f10 con 0x557136c80240 2024-09-16T00:18:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f08e45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557136c82f30 con 0x557136c75060 2024-09-16T00:18:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f08e45740 10 client.0 _ll_register_callbacks cb 0x557136b86db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55356/0 (socket says 172.21.15.78:55356) 2024-09-16T00:18:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 -- 172.21.15.78:0/1401873414 learned_addr learned my addr 172.21.15.78:0/1401873414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:55.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f05009640 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c7dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2f04808640 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 -- 172.21.15.78:0/1401873414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 msgr2=0x557136c7d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c7d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 -- 172.21.15.78:0/1401873414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 msgr2=0x557136c7dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c7dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 -- 172.21.15.78:0/1401873414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571368e9580 con 0x557136c75980 2024-09-16T00:18:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2efffff640 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7d660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00116c0 tx=0x7f2ef0011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83052f8806d4a740 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:55.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2edffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.986+0000 7f2edffff640 1 -- 172.21.15.78:0/1401873414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000fb40 con 0x557136c75980 2024-09-16T00:18:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 1 -- 172.21.15.78:0/1401873414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557136be7b90 con 0x557136c75980 2024-09-16T00:18:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2edffff640 1 -- 172.21.15.78:0/1401873414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2ef000fce0 con 0x557136c75980 2024-09-16T00:18:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2edffff640 1 -- 172.21.15.78:0/1401873414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2ef0026050 con 0x557136c75980 2024-09-16T00:18:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 10 client.16359 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 1 -- 172.21.15.78:0/1401873414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5571369b3960 con 0x557136c75980 2024-09-16T00:18:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2edffff640 1 -- 172.21.15.78:0/1401873414 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557136be7b90 con 0x557136c75980 2024-09-16T00:18:55.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2edffff640 1 -- 172.21.15.78:0/1401873414 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2ef00125a0 con 0x557136c75980 2024-09-16T00:18:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 10 client.16359 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 10 client.16359 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:18:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 20 client.16359 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557136c8fe80 0x557136c922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 4 client.16359 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f08e45740 1 -- 172.21.15.78:0/1401873414 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557136be20a0 con 0x557136c8fe80 2024-09-16T00:18:55.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.987+0000 7f2f04808640 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557136c8fe80 0x557136c922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.988+0000 7f2edffff640 10 client.16359 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.988+0000 7f2f04808640 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557136c8fe80 0x557136c922c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8298192394d6c043 server_cookie=6ee6bcd921575bba in_seq=0 out_seq=0 2024-09-16T00:18:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2edffff640 1 -- 172.21.15.78:0/1401873414 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557136be20a0 con 0x557136c8fe80 2024-09-16T00:18:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2edffff640 10 client.16359 handle_command_reply: tid=0 2024-09-16T00:18:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 client.16359 shutdown 2024-09-16T00:18:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 -- 172.21.15.78:0/1401873414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 msgr2=0x557136c7d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7d660 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00116c0 tx=0x7f2ef0011b80 comp rx=0 tx=0).stop 2024-09-16T00:18:55.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 -- 172.21.15.78:0/1401873414 shutdown_connections 2024-09-16T00:18:55.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557136c8fe80 0x557136c922c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c7dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c7d660 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1401873414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c7d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 -- 172.21.15.78:0/1401873414 >> 172.21.15.78:0/1401873414 conn(0x557136c715f0 msgr2=0x557136c772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:55.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2edffff640 0 client.16359 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:55.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 -- 172.21.15.78:0/1401873414 shutdown_connections 2024-09-16T00:18:55.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 1 -- 172.21.15.78:0/1401873414 wait complete. 2024-09-16T00:18:55.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:54.989+0000 7f2f08e45740 20 client.16359 trim_cache size 0 max 16384 2024-09-16T00:18:55.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.005+0000 7f2f08e45740 1 Processor -- start 2024-09-16T00:18:55.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f08e45740 1 -- start start 2024-09-16T00:18:55.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c97b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c980d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f08e45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001b0e20 con 0x557136c75980 2024-09-16T00:18:55.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f08e45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001b2960 con 0x557136c80240 2024-09-16T00:18:55.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f08e45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001b14f0 con 0x557136c75060 2024-09-16T00:18:55.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c97b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c97b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55370/0 (socket says 172.21.15.78:55370) 2024-09-16T00:18:55.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 -- 172.21.15.78:0/1850429336 learned_addr learned my addr 172.21.15.78:0/1850429336 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:55.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 -- 172.21.15.78:0/1850429336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 msgr2=0x557136c771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f05009640 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 -- 172.21.15.78:0/1850429336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 msgr2=0x557136c980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 -- 172.21.15.78:0/1850429336 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557136c92940 con 0x557136c75980 2024-09-16T00:18:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2f05009640 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:55.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2efffff640 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c97b90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0002230 tx=0x7f2ef0009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27f168c41498076 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.006+0000 7f2ef67fc640 1 -- 172.21.15.78:0/1850429336 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000f730 con 0x557136c75980 2024-09-16T00:18:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2ef67fc640 1 -- 172.21.15.78:0/1850429336 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef0012be0 con 0x557136c75980 2024-09-16T00:18:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2ef67fc640 1 -- 172.21.15.78:0/1850429336 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0003ca0 con 0x557136c75980 2024-09-16T00:18:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 -- 172.21.15.78:0/1850429336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 msgr2=0x557136c97b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c97b90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0002230 tx=0x7f2ef0009520 comp rx=0 tx=0).stop 2024-09-16T00:18:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 -- 172.21.15.78:0/1850429336 shutdown_connections 2024-09-16T00:18:55.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75980 0x557136c97b90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 --2- 172.21.15.78:0/1850429336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75060 0x557136c771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 -- 172.21.15.78:0/1850429336 >> 172.21.15.78:0/1850429336 conn(0x557136c715f0 msgr2=0x557136c76ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 -- 172.21.15.78:0/1850429336 shutdown_connections 2024-09-16T00:18:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.007+0000 7f2f08e45740 1 -- 172.21.15.78:0/1850429336 wait complete. 2024-09-16T00:18:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 1 -- start start 2024-09-16T00:18:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 1 Processor -- start 2024-09-16T00:18:55.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75060 0x557136c881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75980 0x557136c88700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c84670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001b2290 con 0x557136c75060 2024-09-16T00:18:55.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001b1bc0 con 0x557136c80240 2024-09-16T00:18:55.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001aee00 con 0x557136c75980 2024-09-16T00:18:55.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.008+0000 7f2f08e45740 10 client.0 _ll_register_callbacks cb 0x7f2ef0009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f04808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75060 0x557136c881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f04808640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75060 0x557136c881c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55386/0 (socket says 172.21.15.78:55386) 2024-09-16T00:18:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f04808640 1 -- 172.21.15.78:0/201411354 learned_addr learned my addr 172.21.15.78:0/201411354 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f05009640 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c84670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2efffff640 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75980 0x557136c88700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f04808640 1 -- 172.21.15.78:0/201411354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75980 msgr2=0x557136c88700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:55.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f04808640 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75980 0x557136c88700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f04808640 1 -- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 msgr2=0x557136c84670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f04808640 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c84670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f04808640 1 -- 172.21.15.78:0/201411354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557136c84df0 con 0x557136c75060 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.009+0000 7f2f05009640 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c84670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f04808640 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75060 0x557136c881c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ccf90 tx=0x7f2ef8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e030eb5ad6970fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2edffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2edffff640 1 -- 172.21.15.78:0/201411354 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8020040 con 0x557136c75060 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2edffff640 1 -- 172.21.15.78:0/201411354 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2ef8099d60 con 0x557136c75060 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 1 -- 172.21.15.78:0/201411354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef0012be0 con 0x557136c75060 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2edffff640 1 -- 172.21.15.78:0/201411354 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2ef80a46d0 con 0x557136c75060 2024-09-16T00:18:55.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 10 client.16365 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 1 -- 172.21.15.78:0/201411354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557136d03d70 con 0x557136c75060 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2edffff640 1 -- 172.21.15.78:0/201411354 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef0012be0 con 0x557136c75060 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2edffff640 1 -- 172.21.15.78:0/201411354 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2ef80980d0 con 0x557136c75060 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 10 client.16365 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 10 client.16365 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 20 client.16365 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557136d040d0 0x557136d064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 4 client.16365 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b802905", "release": true, "await": true}] 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.010+0000 7f2f08e45740 1 -- 172.21.15.78:0/201411354 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b802905", "release": true, "await": true}) -- 0x557136b5e5a0 con 0x557136d040d0 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.011+0000 7f2efffff640 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557136d040d0 0x557136d064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.011+0000 7f2edffff640 10 client.16365 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:55.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.011+0000 7f2efffff640 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557136d040d0 0x557136d064c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d580f626b54087c4 server_cookie=6d2e4155bacf5b73 in_seq=0 out_seq=0 2024-09-16T00:18:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2edffff640 1 -- 172.21.15.78:0/201411354 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557136b5e5a0 con 0x557136d040d0 2024-09-16T00:18:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2edffff640 10 client.16365 handle_command_reply: tid=0 2024-09-16T00:18:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 client.16365 shutdown 2024-09-16T00:18:55.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 -- 172.21.15.78:0/201411354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75060 msgr2=0x557136c881c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75060 0x557136c881c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80ccf90 tx=0x7f2ef8009c30 comp rx=0 tx=0).stop 2024-09-16T00:18:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 -- 172.21.15.78:0/201411354 shutdown_connections 2024-09-16T00:18:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557136d040d0 0x557136d064c0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557136c80240 0x557136c84670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557136c75980 0x557136c88700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 --2- 172.21.15.78:0/201411354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557136c75060 0x557136c881c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2edffff640 0 client.16365 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.055+0000 7f2f08e45740 1 -- 172.21.15.78:0/201411354 >> 172.21.15.78:0/201411354 conn(0x557136c715f0 msgr2=0x557136c6ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.056+0000 7f2f08e45740 1 -- 172.21.15.78:0/201411354 shutdown_connections 2024-09-16T00:18:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.056+0000 7f2f08e45740 1 -- 172.21.15.78:0/201411354 wait complete. 2024-09-16T00:18:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.056+0000 7f2f08e45740 20 client.16365 trim_cache size 0 max 16384 2024-09-16T00:18:55.061 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 20, 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "b802905": { 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 20, 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 145.3, 2024-09-16T00:18:55.062 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:18:55.063 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:18:55.063 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:18:55.063 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:18:55.063 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:18:55.063 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:18:55.063 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:55.063 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:55.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:55.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:55.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:18:55.064 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:18:55.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 -- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2eec0a3240 msgr2=0x7f2eec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2eec0a3240 0x7f2eec0a5700 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80096c0 tx=0x7f2ee8009610 comp rx=0 tx=0).stop 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 -- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00102d80 msgr2=0x7f2f001009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00102d80 0x7f2f001009e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800d770 tx=0x7f2ef800dc30 comp rx=0 tx=0).stop 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 -- 172.21.15.78:0/1490715118 shutdown_connections 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2eec0a3240 0x7f2eec0a5700 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f0010c160 0x7f2f00101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f001036a0 0x7f2f00100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 --2- 172.21.15.78:0/1490715118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00102d80 0x7f2f001009e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.058+0000 7f2f06a6c640 1 -- 172.21.15.78:0/1490715118 >> 172.21.15.78:0/1490715118 conn(0x7f2f000fc780 msgr2=0x7f2f000fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.063+0000 7f2f06a6c640 1 -- 172.21.15.78:0/1490715118 shutdown_connections 2024-09-16T00:18:55.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:55.063+0000 7f2f06a6c640 1 -- 172.21.15.78:0/1490715118 wait complete. 2024-09-16T00:18:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 295.0 s, 187.5 tps, lat 2716.834 ms stddev 7668.977 2024-09-16T00:18:55.882 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b802905', total seconds elapsed: 28.9 2024-09-16T00:18:55.882 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-16T00:18:55.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:18:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:55 smithi078 ceph-mon[31157]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-09-16T00:18:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:55 smithi193 ceph-mon[35484]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-09-16T00:18:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:55 smithi148 ceph-mon[35599]: pgmap v526: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-09-16T00:18:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:56.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.969+0000 7f34c0a75640 1 -- 172.21.15.78:0/92458265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34bc109450 msgr2=0x7f34bc109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.969+0000 7f34c0a75640 1 --2- 172.21.15.78:0/92458265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34bc109450 0x7f34bc109830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b880 tx=0x7f34ac005f90 comp rx=0 tx=0).stop 2024-09-16T00:18:56.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.969+0000 7f34c0a75640 1 -- 172.21.15.78:0/92458265 shutdown_connections 2024-09-16T00:18:56.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.969+0000 7f34c0a75640 1 --2- 172.21.15.78:0/92458265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc074990 0x7f34bc07b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.969+0000 7f34c0a75640 1 --2- 172.21.15.78:0/92458265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34bc109e00 0x7f34bc0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.969+0000 7f34c0a75640 1 --2- 172.21.15.78:0/92458265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34bc109450 0x7f34bc109830 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b880 tx=0x7f34ac005f90 comp rx=0 tx=0).stop 2024-09-16T00:18:56.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.969+0000 7f34c0a75640 1 -- 172.21.15.78:0/92458265 >> 172.21.15.78:0/92458265 conn(0x7f34bc06e140 msgr2=0x7f34bc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:56.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.973+0000 7f34c0a75640 1 -- 172.21.15.78:0/92458265 shutdown_connections 2024-09-16T00:18:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.974+0000 7f34c0a75640 1 -- 172.21.15.78:0/92458265 wait complete. 2024-09-16T00:18:56.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.975+0000 7f34c0a75640 1 Processor -- start 2024-09-16T00:18:56.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.975+0000 7f34c0a75640 1 -- start start 2024-09-16T00:18:56.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.975+0000 7f34c0a75640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc074990 0x7f34bc07b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.975+0000 7f34c0a75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34bc109e00 0x7f34bc076440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.975+0000 7f34c0a75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34bc076ae0 0x7f34bc1bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.975+0000 7f34c0a75640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc07e780 con 0x7f34bc109e00 2024-09-16T00:18:56.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.975+0000 7f34c0a75640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc07e600 con 0x7f34bc074990 2024-09-16T00:18:56.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.975+0000 7f34c0a75640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc07e900 con 0x7f34bc076ae0 2024-09-16T00:18:56.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.976+0000 7f34b9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34bc109e00 0x7f34bc076440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:56.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.976+0000 7f34b9d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34bc109e00 0x7f34bc076440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35296/0 (socket says 172.21.15.78:35296) 2024-09-16T00:18:56.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.976+0000 7f34b9d9b640 1 -- 172.21.15.78:0/3443936972 learned_addr learned my addr 172.21.15.78:0/3443936972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:56.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.977+0000 7f34bad9d640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34bc076ae0 0x7f34bc1bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:56.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.979+0000 7f34b9d9b640 1 -- 172.21.15.78:0/3443936972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34bc076ae0 msgr2=0x7f34bc1bbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.979+0000 7f34b9d9b640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34bc076ae0 0x7f34bc1bbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.979+0000 7f34b9d9b640 1 -- 172.21.15.78:0/3443936972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc074990 msgr2=0x7f34bc07b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:56.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.979+0000 7f34b9d9b640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc074990 0x7f34bc07b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:56.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.979+0000 7f34b9d9b640 1 -- 172.21.15.78:0/3443936972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc1bc4e0 con 0x7f34bc109e00 2024-09-16T00:18:56.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.982+0000 7f34b9d9b640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34bc109e00 0x7f34bc076440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f34b400ec50 tx=0x7f34b400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:56.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.989+0000 7f34ab7fe640 1 -- 172.21.15.78:0/3443936972 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b400ee60 con 0x7f34bc109e00 2024-09-16T00:18:56.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.989+0000 7f34ab7fe640 1 -- 172.21.15.78:0/3443936972 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b401a070 con 0x7f34bc109e00 2024-09-16T00:18:56.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.989+0000 7f34ab7fe640 1 -- 172.21.15.78:0/3443936972 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4013870 con 0x7f34bc109e00 2024-09-16T00:18:56.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.989+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc1bc7d0 con 0x7f34bc109e00 2024-09-16T00:18:56.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.989+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1bccc0 con 0x7f34bc109e00 2024-09-16T00:18:56.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.993+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc073b00 con 0x7f34bc109e00 2024-09-16T00:18:56.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.997+0000 7f34ab7fe640 1 -- 172.21.15.78:0/3443936972 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f34b4014070 con 0x7f34bc109e00 2024-09-16T00:18:56.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.997+0000 7f34ab7fe640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34a40a3450 0x7f34a40a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:56.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.998+0000 7f34ab7fe640 1 -- 172.21.15.78:0/3443936972 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f34b40cf210 con 0x7f34bc109e00 2024-09-16T00:18:56.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:56.999+0000 7f34ba59c640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34a40a3450 0x7f34a40a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:57.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.000+0000 7f34ba59c640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34a40a3450 0x7f34a40a5910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b610 tx=0x7f34ac00ad60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:57.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.004+0000 7f34ab7fe640 1 -- 172.21.15.78:0/3443936972 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34b409bb70 con 0x7f34bc109e00 2024-09-16T00:18:57.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:18:57.076 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:56 smithi078 ceph-mon[31157]: pgmap v527: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 532 KiB/s wr, 42 op/s 2024-09-16T00:18:57.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:56 smithi193 ceph-mon[35484]: pgmap v527: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 532 KiB/s wr, 42 op/s 2024-09-16T00:18:57.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:56 smithi148 ceph-mon[35599]: pgmap v527: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 532 KiB/s wr, 42 op/s 2024-09-16T00:18:57.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.439+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34bc080650 con 0x7f34bc109e00 2024-09-16T00:18:57.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.440+0000 7f34ab7fe640 1 -- 172.21.15.78:0/3443936972 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f34b409b470 con 0x7f34bc109e00 2024-09-16T00:18:57.447 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:18:57.447 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:18:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:18:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.448+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34a40a3450 msgr2=0x7f34a40a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.448+0000 7f34c0a75640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34a40a3450 0x7f34a40a5910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00b610 tx=0x7f34ac00ad60 comp rx=0 tx=0).stop 2024-09-16T00:18:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.448+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34bc109e00 msgr2=0x7f34bc076440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.448+0000 7f34c0a75640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34bc109e00 0x7f34bc076440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f34b400ec50 tx=0x7f34b400c480 comp rx=0 tx=0).stop 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.454+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 shutdown_connections 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.454+0000 7f34c0a75640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f34a40a3450 0x7f34a40a5910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.454+0000 7f34c0a75640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34bc076ae0 0x7f34bc1bbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.454+0000 7f34c0a75640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34bc109e00 0x7f34bc076440 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.454+0000 7f34c0a75640 1 --2- 172.21.15.78:0/3443936972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34bc074990 0x7f34bc07b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.454+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 >> 172.21.15.78:0/3443936972 conn(0x7f34bc06e140 msgr2=0x7f34bc0725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.454+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 shutdown_connections 2024-09-16T00:18:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:57.455+0000 7f34c0a75640 1 -- 172.21.15.78:0/3443936972 wait complete. 2024-09-16T00:18:58.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:57 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3443936972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:57 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3443936972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:58.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:57 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3443936972' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:18:58.437 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:58.487 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-16T00:18:58.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-16T00:18:59.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:18:58 smithi078 ceph-mon[31157]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-09-16T00:18:59.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.059+0000 7f6f9e10a640 1 -- 172.21.15.78:0/2272278228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98101de0 msgr2=0x7f6f981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:59.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.059+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/2272278228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98101de0 0x7f6f981021c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009900 tx=0x7f6f8c02a3f0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.059+0000 7f6f9e10a640 1 -- 172.21.15.78:0/2272278228 shutdown_connections 2024-09-16T00:18:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.059+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/2272278228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98106f10 0x7f6f9810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.059+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/2272278228 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98102790 0x7f6f981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.059+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/2272278228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98101de0 0x7f6f981021c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.059+0000 7f6f9e10a640 1 -- 172.21.15.78:0/2272278228 >> 172.21.15.78:0/2272278228 conn(0x7f6f980fd8e0 msgr2=0x7f6f980ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.060+0000 7f6f9e10a640 1 -- 172.21.15.78:0/2272278228 shutdown_connections 2024-09-16T00:18:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.060+0000 7f6f9e10a640 1 -- 172.21.15.78:0/2272278228 wait complete. 2024-09-16T00:18:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.060+0000 7f6f9e10a640 1 Processor -- start 2024-09-16T00:18:59.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f9e10a640 1 -- start start 2024-09-16T00:18:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f9e10a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98101de0 0x7f6f98196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f9e10a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98102790 0x7f6f981967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f9e10a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98106f10 0x7f6f9819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f9e10a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810fdd0 con 0x7f6f98101de0 2024-09-16T00:18:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f9e10a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810fc50 con 0x7f6f98106f10 2024-09-16T00:18:59.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f9e10a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810ff50 con 0x7f6f98102790 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98106f10 0x7f6f9819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98106f10 0x7f6f9819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46684/0 (socket says 172.21.15.78:46684) 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.061+0000 7f6f97fff640 1 -- 172.21.15.78:0/1213494629 learned_addr learned my addr 172.21.15.78:0/1213494629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f977fe640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98101de0 0x7f6f98196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f97fff640 1 -- 172.21.15.78:0/1213494629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98102790 msgr2=0x7f6f981967d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f97fff640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98102790 0x7f6f981967d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f97fff640 1 -- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98101de0 msgr2=0x7f6f98196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f97fff640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98101de0 0x7f6f98196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f97fff640 1 -- 172.21.15.78:0/1213494629 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9819b240 con 0x7f6f98106f10 2024-09-16T00:18:59.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f97fff640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98106f10 0x7f6f9819ab60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800b510 tx=0x7f6f8800b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f94ff9640 1 -- 172.21.15.78:0/1213494629 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88005020 con 0x7f6f98106f10 2024-09-16T00:18:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.062+0000 7f6f94ff9640 1 -- 172.21.15.78:0/1213494629 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f880047a0 con 0x7f6f98106f10 2024-09-16T00:18:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.063+0000 7f6f94ff9640 1 -- 172.21.15.78:0/1213494629 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f880088c0 con 0x7f6f98106f10 2024-09-16T00:18:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.063+0000 7f6f9e10a640 1 -- 172.21.15.78:0/1213494629 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9819b530 con 0x7f6f98106f10 2024-09-16T00:18:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.064+0000 7f6f9e10a640 1 -- 172.21.15.78:0/1213494629 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f981a2e10 con 0x7f6f98106f10 2024-09-16T00:18:59.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.067+0000 7f6f94ff9640 1 -- 172.21.15.78:0/1213494629 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6f88025020 con 0x7f6f98106f10 2024-09-16T00:18:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6fa04e3740 1 Processor -- start 2024-09-16T00:18:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6f94ff9640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f800a3110 0x7f6f800a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6fa04e3740 1 -- start start 2024-09-16T00:18:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f94a0 0x5598c23f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f800a3110 0x7f6f800a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c2404140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 0x5598c2406a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6fa04e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5598c235f2e0 con 0x5598c2404680 2024-09-16T00:18:59.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6fa04e3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5598c2306ce0 con 0x5598c23f9dc0 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.069+0000 7f6fa04e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5598c235b2a0 con 0x5598c23f94a0 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c2404140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c2404140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46692/0 (socket says 172.21.15.78:46692) 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 -- 172.21.15.78:0/536796966 learned_addr learned my addr 172.21.15.78:0/536796966 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f94ff9640 1 -- 172.21.15.78:0/1213494629 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6f880d7e70 con 0x7f6f98106f10 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f97fff640 1 --2- 172.21.15.78:0/536796966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 0x5598c2406a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f800a3110 0x7f6f800a55d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009fd0 tx=0x7f6f90014a50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 -- 172.21.15.78:0/536796966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f94a0 msgr2=0x5598c23f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/536796966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f94a0 0x5598c23f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 -- 172.21.15.78:0/536796966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 msgr2=0x5598c2406a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/536796966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 0x5598c2406a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.070+0000 7f6f96ffd640 1 -- 172.21.15.78:0/536796966 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c22eb440 con 0x5598c23f9dc0 2024-09-16T00:18:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.071+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/536796966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c2404140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6f900117a0 tx=0x7f6f9001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=debec6ea02f2b29c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:59.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.071+0000 7f6f7dffb640 1 -- 172.21.15.78:0/536796966 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90014030 con 0x5598c23f9dc0 2024-09-16T00:18:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.071+0000 7f6f7dffb640 1 -- 172.21.15.78:0/536796966 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f90003e50 con 0x5598c23f9dc0 2024-09-16T00:18:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.071+0000 7f6f7dffb640 1 -- 172.21.15.78:0/536796966 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90004280 con 0x5598c23f9dc0 2024-09-16T00:18:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.072+0000 7f6fa04e3740 1 -- 172.21.15.78:0/536796966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 msgr2=0x5598c2404140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.072+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/536796966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c2404140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6f900117a0 tx=0x7f6f9001b000 comp rx=0 tx=0).stop 2024-09-16T00:18:59.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.072+0000 7f6fa04e3740 1 -- 172.21.15.78:0/536796966 shutdown_connections 2024-09-16T00:18:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.072+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/536796966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 0x5598c2406a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.072+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/536796966 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c2404140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.072+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/536796966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f94a0 0x5598c23f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.072+0000 7f6fa04e3740 1 -- 172.21.15.78:0/536796966 >> 172.21.15.78:0/536796966 conn(0x5598c23f5a30 msgr2=0x5598c23f5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.074+0000 7f6fa04e3740 1 -- 172.21.15.78:0/536796966 shutdown_connections 2024-09-16T00:18:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.074+0000 7f6fa04e3740 1 -- 172.21.15.78:0/536796966 wait complete. 2024-09-16T00:18:59.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.074+0000 7f6fa04e3740 1 -- start start 2024-09-16T00:18:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.074+0000 7f6fa04e3740 1 Processor -- start 2024-09-16T00:18:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f94a0 0x5598c2403f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c2401230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 0x5598c2401770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f94a0 0x5598c2403f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f94a0 0x5598c2403f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46702/0 (socket says 172.21.15.78:46702) 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f977fe640 1 -- 172.21.15.78:0/1866746958 learned_addr learned my addr 172.21.15.78:0/1866746958 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5598c24071f0 con 0x5598c2404680 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5598c231a050 con 0x5598c23f94a0 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5598c2407370 con 0x5598c23f9dc0 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f97fff640 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 0x5598c2401770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c2401230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6fa04e3740 10 client.0 _ll_register_callbacks cb 0x5598c235a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f977fe640 1 -- 172.21.15.78:0/1866746958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 msgr2=0x5598c2401230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f977fe640 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c2401230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f977fe640 1 -- 172.21.15.78:0/1866746958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 msgr2=0x5598c2401770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f977fe640 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 0x5598c2401770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f977fe640 1 -- 172.21.15.78:0/1866746958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c20d03c0 con 0x5598c23f94a0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.075+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c2401230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6f977fe640 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f94a0 0x5598c2403f80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c005ab0 tx=0x7f6f8c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36477a6ec450ec10 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6f76ffd640 1 -- 172.21.15.78:0/1866746958 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c035030 con 0x5598c23f94a0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6f76ffd640 1 -- 172.21.15.78:0/1866746958 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6f8c0046c0 con 0x5598c23f94a0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f90003e50 con 0x5598c23f94a0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6f76ffd640 1 -- 172.21.15.78:0/1866746958 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6f8c005d50 con 0x5598c23f94a0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6fa04e3740 10 client.25823 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5598c2412b40 con 0x5598c23f94a0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6f76ffd640 1 -- 172.21.15.78:0/1866746958 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c004da0 con 0x5598c23f94a0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.076+0000 7f6f76ffd640 1 -- 172.21.15.78:0/1866746958 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6f8c003c70 con 0x5598c23f94a0 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.077+0000 7f6fa04e3740 10 client.25823 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:59.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.077+0000 7f6fa04e3740 10 client.25823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.077+0000 7f6fa04e3740 20 client.25823 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.077+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5598c2414000 0x5598c2416450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.077+0000 7f6fa04e3740 4 client.25823 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.077+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6f90003e50 con 0x5598c2414000 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.077+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5598c2414000 0x5598c2416450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.078+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5598c2414000 0x5598c2416450 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a86eb54756d0342 server_cookie=e883e02431d4f70b in_seq=0 out_seq=0 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.078+0000 7f6f76ffd640 10 client.25823 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.078+0000 7f6f76ffd640 1 -- 172.21.15.78:0/1866746958 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6f90003e50 con 0x5598c2414000 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.078+0000 7f6f76ffd640 10 client.25823 handle_command_reply: tid=0 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.078+0000 7f6fa04e3740 1 client.25823 shutdown 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f94a0 msgr2=0x5598c2403f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f94a0 0x5598c2403f80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c005ab0 tx=0x7f6f8c005270 comp rx=0 tx=0).stop 2024-09-16T00:18:59.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 shutdown_connections 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5598c2414000 0x5598c2416450 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c2404680 0x5598c2401770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6f76ffd640 0 client.25823 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c2401230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/1866746958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f94a0 0x5598c2403f80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 >> 172.21.15.78:0/1866746958 conn(0x5598c23f5a30 msgr2=0x5598c23fb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 shutdown_connections 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 1 -- 172.21.15.78:0/1866746958 wait complete. 2024-09-16T00:18:59.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.079+0000 7f6fa04e3740 20 client.25823 trim_cache size 0 max 16384 2024-09-16T00:18:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.103+0000 7f6fa04e3740 1 Processor -- start 2024-09-16T00:18:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.103+0000 7f6fa04e3740 1 -- start start 2024-09-16T00:18:59.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.103+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c23f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.103+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c23f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.103+0000 7f6f977fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c23f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35334/0 (socket says 172.21.15.78:35334) 2024-09-16T00:18:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.103+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c241bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.103+0000 7f6f977fe640 1 -- 172.21.15.78:0/3937022526 learned_addr learned my addr 172.21.15.78:0/3937022526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.103+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c2404680 0x5598c241c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6fa04e3740 1 -- 172.21.15.78:0/3937022526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981ac710 con 0x5598c23f94a0 2024-09-16T00:18:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f97fff640 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c2404680 0x5598c241c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c241bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6fa04e3740 1 -- 172.21.15.78:0/3937022526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981ae250 con 0x5598c23f9dc0 2024-09-16T00:18:59.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6fa04e3740 1 -- 172.21.15.78:0/3937022526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981acde0 con 0x5598c2404680 2024-09-16T00:18:59.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f977fe640 1 -- 172.21.15.78:0/3937022526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c2404680 msgr2=0x5598c241c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:59.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f977fe640 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c2404680 0x5598c241c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f977fe640 1 -- 172.21.15.78:0/3937022526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 msgr2=0x5598c241bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:59.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f977fe640 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c241bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f977fe640 1 -- 172.21.15.78:0/3937022526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c24001e0 con 0x5598c23f94a0 2024-09-16T00:18:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f97fff640 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c2404680 0x5598c241c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:18:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.104+0000 7f6f977fe640 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c23f5e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0058e0 tx=0x7f6f8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e594e95b91f510 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:59.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.105+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3937022526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c035030 con 0x5598c23f94a0 2024-09-16T00:18:59.110 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:18:59.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.105+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3937022526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c055070 con 0x5598c23f94a0 2024-09-16T00:18:59.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.105+0000 7f6f7d7fa640 1 -- 172.21.15.78:0/3937022526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0035f0 con 0x5598c23f94a0 2024-09-16T00:18:59.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.106+0000 7f6fa04e3740 1 -- 172.21.15.78:0/3937022526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 msgr2=0x5598c23f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:59.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.106+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c23f5e10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0058e0 tx=0x7f6f8c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.106+0000 7f6fa04e3740 1 -- 172.21.15.78:0/3937022526 shutdown_connections 2024-09-16T00:18:59.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.106+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c2404680 0x5598c241c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.106+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c23f9dc0 0x5598c241bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.106+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/3937022526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c23f5e10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.106+0000 7f6fa04e3740 1 -- 172.21.15.78:0/3937022526 >> 172.21.15.78:0/3937022526 conn(0x5598c23f5a30 msgr2=0x5598c23f4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.107+0000 7f6fa04e3740 1 -- 172.21.15.78:0/3937022526 shutdown_connections 2024-09-16T00:18:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.107+0000 7f6fa04e3740 1 -- 172.21.15.78:0/3937022526 wait complete. 2024-09-16T00:18:59.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.107+0000 7f6fa04e3740 1 -- start start 2024-09-16T00:18:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.108+0000 7f6fa04e3740 1 Processor -- start 2024-09-16T00:18:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.108+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c240a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.108+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c240ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.108+0000 7f6fa04e3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c2404680 0x5598c240b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.108+0000 7f6fa04e3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981adb80 con 0x5598c23f94a0 2024-09-16T00:18:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.108+0000 7f6fa04e3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981ad4b0 con 0x5598c2404680 2024-09-16T00:18:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.108+0000 7f6fa04e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981aa6f0 con 0x5598c23f9dc0 2024-09-16T00:18:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c240ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f96ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c240ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:50420/0 (socket says 172.21.15.78:50420) 2024-09-16T00:18:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f96ffd640 1 -- 172.21.15.78:0/2486031077 learned_addr learned my addr 172.21.15.78:0/2486031077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:18:59.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6fa04e3740 10 client.0 _ll_register_callbacks cb 0x7f6f88005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f97fff640 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c2404680 0x5598c240b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f96ffd640 1 -- 172.21.15.78:0/2486031077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c2404680 msgr2=0x5598c240b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c2404680 0x5598c240b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f96ffd640 1 -- 172.21.15.78:0/2486031077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 msgr2=0x5598c240a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c240a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.109+0000 7f6f96ffd640 1 -- 172.21.15.78:0/2486031077 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c24042f0 con 0x5598c23f9dc0 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.110+0000 7f6f96ffd640 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c240ab20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6f900113a0 tx=0x7f6f9000fcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c92179cac50332d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.110+0000 7f6f76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:18:59.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.110+0000 7f6f76ffd640 1 -- 172.21.15.78:0/2486031077 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000f6d0 con 0x5598c23f9dc0 2024-09-16T00:18:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.110+0000 7f6f76ffd640 1 -- 172.21.15.78:0/2486031077 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6f9000f870 con 0x5598c23f9dc0 2024-09-16T00:18:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.110+0000 7f6fa04e3740 1 -- 172.21.15.78:0/2486031077 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f8c055070 con 0x5598c23f9dc0 2024-09-16T00:18:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.110+0000 7f6f76ffd640 1 -- 172.21.15.78:0/2486031077 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6f9000fa70 con 0x5598c23f9dc0 2024-09-16T00:18:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.110+0000 7f6fa04e3740 10 client.25693 fetch_fsmap learned FSMap version 34 2024-09-16T00:18:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.110+0000 7f6fa04e3740 1 -- 172.21.15.78:0/2486031077 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5598c247f0c0 con 0x5598c23f9dc0 2024-09-16T00:18:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6f76ffd640 1 -- 172.21.15.78:0/2486031077 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c055070 con 0x5598c23f9dc0 2024-09-16T00:18:59.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6f76ffd640 1 -- 172.21.15.78:0/2486031077 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6f90004090 con 0x5598c23f9dc0 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6fa04e3740 10 client.25693 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6fa04e3740 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6fa04e3740 20 client.25693 populate_metadata read hostname 'smithi078' 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5598c2414000 0x5598c2481540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6fa04e3740 4 client.25693 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6fa04e3740 1 -- 172.21.15.78:0/2486031077 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5598c236c850 con 0x5598c2414000 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.111+0000 7f6f977fe640 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5598c2414000 0x5598c2481540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.112+0000 7f6f977fe640 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5598c2414000 0x5598c2481540 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7d7e6dfdfa9ec3 server_cookie=afbff3ce8741e6a8 in_seq=0 out_seq=0 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.112+0000 7f6f76ffd640 10 client.25693 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.112+0000 7f6f76ffd640 1 -- 172.21.15.78:0/2486031077 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5598c236c850 con 0x5598c2414000 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.112+0000 7f6f76ffd640 10 client.25693 handle_command_reply: tid=0 2024-09-16T00:18:59.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.112+0000 7f6fa04e3740 1 client.25693 shutdown 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.112+0000 7f6fa04e3740 1 -- 172.21.15.78:0/2486031077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 msgr2=0x5598c240ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.112+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c240ab20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6f900113a0 tx=0x7f6f9000fcb0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 1 -- 172.21.15.78:0/2486031077 shutdown_connections 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5598c2414000 0x5598c2481540 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598c2404680 0x5598c240b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598c23f9dc0 0x5598c240ab20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 1 --2- 172.21.15.78:0/2486031077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598c23f94a0 0x5598c240a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 1 -- 172.21.15.78:0/2486031077 >> 172.21.15.78:0/2486031077 conn(0x5598c23f5a30 msgr2=0x5598c2402a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6f76ffd640 0 client.25693 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 1 -- 172.21.15.78:0/2486031077 shutdown_connections 2024-09-16T00:18:59.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 1 -- 172.21.15.78:0/2486031077 wait complete. 2024-09-16T00:18:59.126 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:18:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:18:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:18:59.126 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.113+0000 7f6fa04e3740 20 client.25693 trim_cache size 0 max 16384 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 -- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f800a3110 msgr2=0x7f6f800a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f800a3110 0x7f6f800a55d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009fd0 tx=0x7f6f90014a50 comp rx=0 tx=0).stop 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 -- 172.21.15.78:0/1213494629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98106f10 msgr2=0x7f6f9819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98106f10 0x7f6f9819ab60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800b510 tx=0x7f6f8800b9d0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 -- 172.21.15.78:0/1213494629 shutdown_connections 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6f800a3110 0x7f6f800a55d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f98106f10 0x7f6f9819ab60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f98102790 0x7f6f981967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 --2- 172.21.15.78:0/1213494629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f98101de0 0x7f6f98196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:18:59.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 -- 172.21.15.78:0/1213494629 >> 172.21.15.78:0/1213494629 conn(0x7f6f980fd8e0 msgr2=0x7f6f980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:18:59.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 -- 172.21.15.78:0/1213494629 shutdown_connections 2024-09-16T00:18:59.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:18:59.114+0000 7f6f9e10a640 1 -- 172.21.15.78:0/1213494629 wait complete. 2024-09-16T00:18:59.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:18:58 smithi193 ceph-mon[35484]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-09-16T00:18:59.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:18:58 smithi148 ceph-mon[35599]: pgmap v528: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.8 MiB/s wr, 82 op/s 2024-09-16T00:19:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.055+0000 7f28c5191640 1 -- 172.21.15.78:0/2534478388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c01055a0 msgr2=0x7f28c010d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.055+0000 7f28c5191640 1 --2- 172.21.15.78:0/2534478388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c01055a0 0x7f28c010d100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009be0 tx=0x7f28b802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.056+0000 7f28c5191640 1 -- 172.21.15.78:0/2534478388 shutdown_connections 2024-09-16T00:19:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.056+0000 7f28c5191640 1 --2- 172.21.15.78:0/2534478388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c01055a0 0x7f28c010d100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.056+0000 7f28c5191640 1 --2- 172.21.15.78:0/2534478388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28c01012d0 0x7f28c0104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.056+0000 7f28c5191640 1 --2- 172.21.15.78:0/2534478388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c0100920 0x7f28c0100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.056+0000 7f28c5191640 1 -- 172.21.15.78:0/2534478388 >> 172.21.15.78:0/2534478388 conn(0x7f28c0075f70 msgr2=0x7f28c0076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.057+0000 7f28c5191640 1 -- 172.21.15.78:0/2534478388 shutdown_connections 2024-09-16T00:19:00.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.058+0000 7f28c5191640 1 -- 172.21.15.78:0/2534478388 wait complete. 2024-09-16T00:19:00.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.059+0000 7f28c5191640 1 Processor -- start 2024-09-16T00:19:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.059+0000 7f28c5191640 1 -- start start 2024-09-16T00:19:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28c5191640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c0100920 0x7f28c0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28c5191640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c01012d0 0x7f28c0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28c5191640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28c01055a0 0x7f28c019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28c5191640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28c0110620 con 0x7f28c01055a0 2024-09-16T00:19:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28c01055a0 0x7f28c019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28c01055a0 0x7f28c019b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35360/0 (socket says 172.21.15.78:35360) 2024-09-16T00:19:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28bf59e640 1 -- 172.21.15.78:0/917955853 learned_addr learned my addr 172.21.15.78:0/917955853 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28bed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c0100920 0x7f28c0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01104a0 con 0x7f28c01012d0 2024-09-16T00:19:00.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.060+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28c01107a0 con 0x7f28c0100920 2024-09-16T00:19:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.061+0000 7f28bed9d640 1 -- 172.21.15.78:0/917955853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c01012d0 msgr2=0x7f28c0197650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:19:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.061+0000 7f28bed9d640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c01012d0 0x7f28c0197650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.061+0000 7f28bed9d640 1 -- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28c01055a0 msgr2=0x7f28c019b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.061+0000 7f28bed9d640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28c01055a0 0x7f28c019b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.061+0000 7f28bed9d640 1 -- 172.21.15.78:0/917955853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c019c0d0 con 0x7f28c0100920 2024-09-16T00:19:00.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.062+0000 7f28bed9d640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c0100920 0x7f28c0197110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009e90 tx=0x7f28b000b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.066+0000 7f28a7fff640 1 -- 172.21.15.78:0/917955853 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0022070 con 0x7f28c0100920 2024-09-16T00:19:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.066+0000 7f28a7fff640 1 -- 172.21.15.78:0/917955853 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b000be40 con 0x7f28c0100920 2024-09-16T00:19:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.066+0000 7f28a7fff640 1 -- 172.21.15.78:0/917955853 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b001da70 con 0x7f28c0100920 2024-09-16T00:19:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.066+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c019c3c0 con 0x7f28c0100920 2024-09-16T00:19:00.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.066+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c0110d80 con 0x7f28c0100920 2024-09-16T00:19:00.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.071+0000 7f28c756a740 1 Processor -- start 2024-09-16T00:19:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.072+0000 7f28a7fff640 1 -- 172.21.15.78:0/917955853 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f28b0025070 con 0x7f28c0100920 2024-09-16T00:19:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.076+0000 7f28c756a740 1 -- start start 2024-09-16T00:19:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.076+0000 7f28c756a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dcecbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.076+0000 7f28c756a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 0x5607dced64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.076+0000 7f28c756a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 0x5607dced8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.076+0000 7f28c756a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607dce30160 con 0x5607dced6a30 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.076+0000 7f28c756a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5607dcdd7720 con 0x5607dcecb850 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.076+0000 7f28c756a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607dce2c120 con 0x5607dcecc170 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dcecbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dcecbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46748/0 (socket says 172.21.15.78:46748) 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28bf59e640 1 -- 172.21.15.78:0/3885542402 learned_addr learned my addr 172.21.15.78:0/3885542402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28be59c640 1 --2- 172.21.15.78:0/3885542402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 0x5607dced8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28bf59e640 1 -- 172.21.15.78:0/3885542402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 msgr2=0x5607dced64f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28bf59e640 1 --2- 172.21.15.78:0/3885542402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 0x5607dced64f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28bf59e640 1 -- 172.21.15.78:0/3885542402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 msgr2=0x5607dced8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28bf59e640 1 --2- 172.21.15.78:0/3885542402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 0x5607dced8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.079+0000 7f28bf59e640 1 -- 172.21.15.78:0/3885542402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607dcba1ba0 con 0x5607dcecb850 2024-09-16T00:19:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.081+0000 7f28a7fff640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28a80a33d0 0x7f28a80a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.082+0000 7f28a7fff640 1 -- 172.21.15.78:0/917955853 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f28b00d7520 con 0x7f28c0100920 2024-09-16T00:19:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.082+0000 7f28bf59e640 1 --2- 172.21.15.78:0/3885542402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dcecbc30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28b8034040 tx=0x7f28b802abf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd399045277a13e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:00.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.082+0000 7f28bed9d640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28a80a33d0 0x7f28a80a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.082+0000 7f28a57fa640 1 -- 172.21.15.78:0/3885542402 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8036040 con 0x5607dcecb850 2024-09-16T00:19:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.083+0000 7f28a57fa640 1 -- 172.21.15.78:0/3885542402 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8030df0 con 0x5607dcecb850 2024-09-16T00:19:00.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.083+0000 7f28a57fa640 1 -- 172.21.15.78:0/3885542402 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8004360 con 0x5607dcecb850 2024-09-16T00:19:00.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.084+0000 7f28c756a740 1 -- 172.21.15.78:0/3885542402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 msgr2=0x5607dcecbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:00.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.084+0000 7f28c756a740 1 --2- 172.21.15.78:0/3885542402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dcecbc30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28b8034040 tx=0x7f28b802abf0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.086+0000 7f28bed9d640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28a80a33d0 0x7f28a80a5890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f28b00a3820 tx=0x7f28b000bcc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:00.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.089+0000 7f28c756a740 1 -- 172.21.15.78:0/3885542402 shutdown_connections 2024-09-16T00:19:00.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.089+0000 7f28c756a740 1 --2- 172.21.15.78:0/3885542402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 0x5607dced8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.089+0000 7f28c756a740 1 --2- 172.21.15.78:0/3885542402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 0x5607dced64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.089+0000 7f28c756a740 1 --2- 172.21.15.78:0/3885542402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dcecbc30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.089+0000 7f28c756a740 1 -- 172.21.15.78:0/3885542402 >> 172.21.15.78:0/3885542402 conn(0x5607dcec7de0 msgr2=0x5607dcec81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:00.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.090+0000 7f28c756a740 1 -- 172.21.15.78:0/3885542402 shutdown_connections 2024-09-16T00:19:00.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.090+0000 7f28c756a740 1 -- 172.21.15.78:0/3885542402 wait complete. 2024-09-16T00:19:00.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.090+0000 7f28c756a740 1 -- start start 2024-09-16T00:19:00.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28c756a740 1 Processor -- start 2024-09-16T00:19:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28c756a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecb850 0x5607dced3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28c756a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dced3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28c756a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 0x5607dced4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28c756a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607dced9600 con 0x5607dced6a30 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28c756a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5607dcdb8900 con 0x5607dcecc170 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28c756a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607dced9780 con 0x5607dcecb850 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28c756a740 10 client.0 _ll_register_callbacks cb 0x5607dcdb6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 0x5607dced4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28be59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dced3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28be59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dced3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46750/0 (socket says 172.21.15.78:46750) 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28be59c640 1 -- 172.21.15.78:0/68808355 learned_addr learned my addr 172.21.15.78:0/68808355 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.091+0000 7f28bf59e640 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecb850 0x5607dced3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28be59c640 1 -- 172.21.15.78:0/68808355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecb850 msgr2=0x5607dced3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28be59c640 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecb850 0x5607dced3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28be59c640 1 -- 172.21.15.78:0/68808355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 msgr2=0x5607dced4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:00.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28be59c640 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 0x5607dced4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28be59c640 1 -- 172.21.15.78:0/68808355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607dcdf8c50 con 0x5607dcecc170 2024-09-16T00:19:00.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28bf59e640 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecb850 0x5607dced3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:00.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28be59c640 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dced3e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e680 tx=0x7f28b400eb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a73fefcc1159d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:00.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:19:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28a2ff5640 1 -- 172.21.15.78:0/68808355 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b400aa90 con 0x5607dcecc170 2024-09-16T00:19:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.092+0000 7f28a2ff5640 1 -- 172.21.15.78:0/68808355 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f28b400ac30 con 0x5607dcecc170 2024-09-16T00:19:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 1 -- 172.21.15.78:0/68808355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b8030df0 con 0x5607dcecc170 2024-09-16T00:19:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28a2ff5640 1 -- 172.21.15.78:0/68808355 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f28b400ee80 con 0x5607dcecc170 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 10 client.25838 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 1 -- 172.21.15.78:0/68808355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607dcee53b0 con 0x5607dcecc170 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28a2ff5640 1 -- 172.21.15.78:0/68808355 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b8030df0 con 0x5607dcecc170 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28a2ff5640 1 -- 172.21.15.78:0/68808355 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f28b4010550 con 0x5607dcecc170 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 10 client.25838 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 10 client.25838 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 20 client.25838 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5607dcee6860 0x5607dcee8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 4 client.25838 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.093+0000 7f28c756a740 1 -- 172.21.15.78:0/68808355 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607dce3d890 con 0x5607dcee6860 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.094+0000 7f28bf59e640 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5607dcee6860 0x5607dcee8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.094+0000 7f28bf59e640 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5607dcee6860 0x5607dcee8c80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad179c251dbc4d3 server_cookie=31ccc40ead84ce12 in_seq=0 out_seq=0 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.094+0000 7f28a2ff5640 10 client.25838 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28a2ff5640 1 -- 172.21.15.78:0/68808355 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607dce3d890 con 0x5607dcee6860 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28a2ff5640 10 client.25838 handle_command_reply: tid=0 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 client.25838 shutdown 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 -- 172.21.15.78:0/68808355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 msgr2=0x5607dced3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dced3e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e680 tx=0x7f28b400eb40 comp rx=0 tx=0).stop 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 -- 172.21.15.78:0/68808355 shutdown_connections 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5607dcee6860 0x5607dcee8c80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dced6a30 0x5607dced4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dced3e40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28a2ff5640 0 client.25838 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 --2- 172.21.15.78:0/68808355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecb850 0x5607dced3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 -- 172.21.15.78:0/68808355 >> 172.21.15.78:0/68808355 conn(0x5607dcec7de0 msgr2=0x5607dcecdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 -- 172.21.15.78:0/68808355 shutdown_connections 2024-09-16T00:19:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 1 -- 172.21.15.78:0/68808355 wait complete. 2024-09-16T00:19:00.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.095+0000 7f28c756a740 20 client.25838 trim_cache size 0 max 16384 2024-09-16T00:19:00.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28c756a740 1 Processor -- start 2024-09-16T00:19:00.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28c756a740 1 -- start start 2024-09-16T00:19:00.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28c756a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcec7de0 0x5607dceee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28c756a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dceeea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28c756a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 0x5607dced0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28c756a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac00faa0 con 0x5607dcec7de0 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28c756a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac0115e0 con 0x5607dcecb850 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28c756a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac010170 con 0x5607dcecc170 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcec7de0 0x5607dceee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcec7de0 0x5607dceee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35400/0 (socket says 172.21.15.78:35400) 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.117+0000 7f28bf59e640 1 -- 172.21.15.78:0/1012449671 learned_addr learned my addr 172.21.15.78:0/1012449671 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28be59c640 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dceeea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28bf59e640 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 0x5607dced0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28bf59e640 1 -- 172.21.15.78:0/1012449671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 msgr2=0x5607dced0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28bf59e640 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 0x5607dced0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28bf59e640 1 -- 172.21.15.78:0/1012449671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 msgr2=0x5607dceeea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28bf59e640 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dceeea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28bf59e640 1 -- 172.21.15.78:0/1012449671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607dcee92e0 con 0x5607dcec7de0 2024-09-16T00:19:00.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28bf59e640 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcec7de0 0x5607dceee510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f28b802ac60 tx=0x7f28b8003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22733ee4ee7fc22f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:00.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28a4ff9640 1 -- 172.21.15.78:0/1012449671 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8036040 con 0x5607dcec7de0 2024-09-16T00:19:00.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28a4ff9640 1 -- 172.21.15.78:0/1012449671 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b805a030 con 0x5607dcec7de0 2024-09-16T00:19:00.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.118+0000 7f28a4ff9640 1 -- 172.21.15.78:0/1012449671 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b804f040 con 0x5607dcec7de0 2024-09-16T00:19:00.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 -- 172.21.15.78:0/1012449671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcec7de0 msgr2=0x5607dceee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:00.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcec7de0 0x5607dceee510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f28b802ac60 tx=0x7f28b8003e90 comp rx=0 tx=0).stop 2024-09-16T00:19:00.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 -- 172.21.15.78:0/1012449671 shutdown_connections 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcecc170 0x5607dced0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecb850 0x5607dceeea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 --2- 172.21.15.78:0/1012449671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcec7de0 0x5607dceee510 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 -- 172.21.15.78:0/1012449671 >> 172.21.15.78:0/1012449671 conn(0x5607dcecf2f0 msgr2=0x5607dced21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 -- 172.21.15.78:0/1012449671 shutdown_connections 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 -- 172.21.15.78:0/1012449671 wait complete. 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.119+0000 7f28c756a740 1 -- start start 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.120+0000 7f28c756a740 1 Processor -- start 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.120+0000 7f28c756a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcec7de0 0x5607dcedb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.120+0000 7f28c756a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcecb850 0x5607dcedb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcec7de0 0x5607dcedb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28bf59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcec7de0 0x5607dcedb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:50458/0 (socket says 172.21.15.78:50458) 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28bf59e640 1 -- 172.21.15.78:0/2853100518 learned_addr learned my addr 172.21.15.78:0/2853100518 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28c756a740 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dcf504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28be59c640 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcecb850 0x5607dcedb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac010f10 con 0x5607dcecb850 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac010840 con 0x5607dcecc170 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac00da80 con 0x5607dcec7de0 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.121+0000 7f28bf59e640 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dcf504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28bf59e640 1 -- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 msgr2=0x5607dcf504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28bf59e640 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dcf504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28bf59e640 1 -- 172.21.15.78:0/2853100518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcecb850 msgr2=0x5607dcedb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28bf59e640 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcecb850 0x5607dcedb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28bf59e640 1 -- 172.21.15.78:0/2853100518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607dcec8a70 con 0x5607dcec7de0 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28be59c640 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcecb850 0x5607dcedb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:19:00.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28c756a740 10 client.0 _ll_register_callbacks cb 0x7f28b4005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b805a030 con 0x5607dcec7de0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28bf59e640 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcec7de0 0x5607dcedb020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f28b8031040 tx=0x7f28b8009b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3d0c50e4f5c35b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28a2ff5640 10 client.25708 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28a2ff5640 1 -- 172.21.15.78:0/2853100518 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8067060 con 0x5607dcec7de0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28a2ff5640 1 -- 172.21.15.78:0/2853100518 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f28b8030720 con 0x5607dcec7de0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28a2ff5640 1 -- 172.21.15.78:0/2853100518 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f28b8036070 con 0x5607dcec7de0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28c756a740 10 client.25708 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.122+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5607dcec33f0 con 0x5607dcec7de0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.123+0000 7f28a2ff5640 1 -- 172.21.15.78:0/2853100518 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f28b806a050 con 0x5607dcec7de0 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.123+0000 7f28c756a740 10 client.25708 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.123+0000 7f28c756a740 10 client.25708 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:19:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.123+0000 7f28c756a740 20 client.25708 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.123+0000 7f28c756a740 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5607dced6a30 0x5607dcf53af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.123+0000 7f28c756a740 4 client.25708 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-16T00:19:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.123+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x5607dce37be0 con 0x5607dced6a30 2024-09-16T00:19:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.123+0000 7f28be59c640 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5607dced6a30 0x5607dcf53af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.124+0000 7f28be59c640 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5607dced6a30 0x5607dcf53af0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed01080209691bd2 server_cookie=e70224285f22aaa7 in_seq=0 out_seq=0 2024-09-16T00:19:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.124+0000 7f28a2ff5640 10 client.25708 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:00.354 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:19:00.354 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:19:00.354 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:19:00.354 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 22, 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "ceb33b92": { 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 22, 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 145.3, 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:19:00.355 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:00.356 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28a2ff5640 1 -- 172.21.15.78:0/2853100518 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5607dce37be0 con 0x5607dced6a30 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28a2ff5640 10 client.25708 handle_command_reply: tid=0 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 client.25708 shutdown 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcec7de0 msgr2=0x5607dcedb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcec7de0 0x5607dcedb020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f28b8031040 tx=0x7f28b8009b40 comp rx=0 tx=0).stop 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 shutdown_connections 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5607dced6a30 0x5607dcf53af0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607dcecc170 0x5607dcf504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607dcecb850 0x5607dcedb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 --2- 172.21.15.78:0/2853100518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607dcec7de0 0x5607dcedb020 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28a2ff5640 0 client.25708 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.351+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 >> 172.21.15.78:0/2853100518 conn(0x5607dcecf2f0 msgr2=0x5607dcec66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.352+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 shutdown_connections 2024-09-16T00:19:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.352+0000 7f28c756a740 1 -- 172.21.15.78:0/2853100518 wait complete. 2024-09-16T00:19:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.352+0000 7f28c756a740 20 client.25708 trim_cache size 0 max 16384 2024-09-16T00:19:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28a80a33d0 msgr2=0x7f28a80a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28a80a33d0 0x7f28a80a5890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f28b00a3820 tx=0x7f28b000bcc0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c0100920 msgr2=0x7f28c0197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c0100920 0x7f28c0197110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009e90 tx=0x7f28b000b690 comp rx=0 tx=0).stop 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 shutdown_connections 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28a80a33d0 0x7f28a80a5890 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28c01055a0 0x7f28c019b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28c01012d0 0x7f28c0197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 --2- 172.21.15.78:0/917955853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28c0100920 0x7f28c0197110 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 >> 172.21.15.78:0/917955853 conn(0x7f28c0075f70 msgr2=0x7f28c00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.353+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 shutdown_connections 2024-09-16T00:19:00.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:00.354+0000 7f28c5191640 1 -- 172.21.15.78:0/917955853 wait complete. 2024-09-16T00:19:00.378 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:19:00.379 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:19:00.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 300.0 s, 733.1 tps, lat 118.629 ms stddev 146.918 2024-09-16T00:19:01.058 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ceb33b92', quiesce took 5.2 seconds. Will release after: 20.150000000000002 2024-09-16T00:19:01.064 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 -- 172.21.15.78:0/2923980428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c106f10 msgr2=0x7fa67c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 --2- 172.21.15.78:0/2923980428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c106f10 0x7fa67c10ea90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa674009ca0 tx=0x7fa67402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 -- 172.21.15.78:0/2923980428 shutdown_connections 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 --2- 172.21.15.78:0/2923980428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c106f10 0x7fa67c10ea90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 --2- 172.21.15.78:0/2923980428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa67c102790 0x7fa67c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 --2- 172.21.15.78:0/2923980428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa67c101de0 0x7fa67c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 -- 172.21.15.78:0/2923980428 >> 172.21.15.78:0/2923980428 conn(0x7fa67c0fd8c0 msgr2=0x7fa67c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 -- 172.21.15.78:0/2923980428 shutdown_connections 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.555+0000 7fa68252c640 1 -- 172.21.15.78:0/2923980428 wait complete. 2024-09-16T00:19:01.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa68252c640 1 Processor -- start 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa68252c640 1 -- start start 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa68252c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c101de0 0x7fa67c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa68252c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa67c102790 0x7fa67c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa68252c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa67c106f10 0x7fa67c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa68252c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c112010 con 0x7fa67c101de0 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa68252c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c111e90 con 0x7fa67c106f10 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa68252c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c112190 con 0x7fa67c102790 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c101de0 0x7fa67c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c101de0 0x7fa67c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35420/0 (socket says 172.21.15.78:35420) 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67bfff640 1 -- 172.21.15.78:0/4036626123 learned_addr learned my addr 172.21.15.78:0/4036626123 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67b7fe640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa67c102790 0x7fa67c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:01.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa680ac9640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa67c106f10 0x7fa67c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67bfff640 1 -- 172.21.15.78:0/4036626123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa67c102790 msgr2=0x7fa67c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67bfff640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa67c102790 0x7fa67c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67bfff640 1 -- 172.21.15.78:0/4036626123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa67c106f10 msgr2=0x7fa67c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67bfff640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa67c106f10 0x7fa67c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.556+0000 7fa67bfff640 1 -- 172.21.15.78:0/4036626123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c1a6ab0 con 0x7fa67c101de0 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.557+0000 7fa67bfff640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c101de0 0x7fa67c0744f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00b4a0 tx=0x7fa66c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.557+0000 7fa6797fa640 1 -- 172.21.15.78:0/4036626123 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c014070 con 0x7fa67c101de0 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.557+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c1a6da0 con 0x7fa67c101de0 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.557+0000 7fa6797fa640 1 -- 172.21.15.78:0/4036626123 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c010af0 con 0x7fa67c101de0 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.557+0000 7fa6797fa640 1 -- 172.21.15.78:0/4036626123 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c017d10 con 0x7fa67c101de0 2024-09-16T00:19:01.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.557+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c1a7250 con 0x7fa67c101de0 2024-09-16T00:19:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.558+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa67c102240 con 0x7fa67c101de0 2024-09-16T00:19:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.559+0000 7fa6797fa640 1 -- 172.21.15.78:0/4036626123 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa66c017070 con 0x7fa67c101de0 2024-09-16T00:19:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.559+0000 7fa6797fa640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6680a32c0 0x7fa6680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.560+0000 7fa67b7fe640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6680a32c0 0x7fa6680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.560+0000 7fa6797fa640 1 -- 172.21.15.78:0/4036626123 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa66c0cf070 con 0x7fa67c101de0 2024-09-16T00:19:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.560+0000 7fa67b7fe640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6680a32c0 0x7fa6680a5780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa67c075f80 tx=0x7fa664008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:01.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.564+0000 7fa6797fa640 1 -- 172.21.15.78:0/4036626123 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa66c09ba30 con 0x7fa67c101de0 2024-09-16T00:19:01.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.765+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa67c10a0f0 con 0x7fa67c101de0 2024-09-16T00:19:01.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.766+0000 7fa6797fa640 1 -- 172.21.15.78:0/4036626123 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fa66c09b330 con 0x7fa67c101de0 2024-09-16T00:19:01.767 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:19:01.767 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:19:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6680a32c0 msgr2=0x7fa6680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6680a32c0 0x7fa6680a5780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa67c075f80 tx=0x7fa664008040 comp rx=0 tx=0).stop 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c101de0 msgr2=0x7fa67c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c101de0 0x7fa67c0744f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00b4a0 tx=0x7fa66c00b960 comp rx=0 tx=0).stop 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 shutdown_connections 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6680a32c0 0x7fa6680a5780 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa67c106f10 0x7fa67c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa67c102790 0x7fa67c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 --2- 172.21.15.78:0/4036626123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa67c101de0 0x7fa67c0744f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 >> 172.21.15.78:0/4036626123 conn(0x7fa67c0fd8c0 msgr2=0x7fa67c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 shutdown_connections 2024-09-16T00:19:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:01.768+0000 7fa68252c640 1 -- 172.21.15.78:0/4036626123 wait complete. 2024-09-16T00:19:01.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:01 smithi193 ceph-mon[35484]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.0 MiB/s wr, 191 op/s 2024-09-16T00:19:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:01 smithi148 ceph-mon[35599]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.0 MiB/s wr, 191 op/s 2024-09-16T00:19:02.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:01 smithi078 ceph-mon[31157]: pgmap v529: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.0 MiB/s wr, 191 op/s 2024-09-16T00:19:02.470 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:19:02.761 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:02.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:02 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4036626123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:02 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4036626123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:02.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:02 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4036626123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 -- 172.21.15.78:0/3293091656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680101420 msgr2=0x7fb68010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 --2- 172.21.15.78:0/3293091656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680101420 0x7fb68010d950 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb678009ca0 tx=0x7fb67802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 -- 172.21.15.78:0/3293091656 shutdown_connections 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 --2- 172.21.15.78:0/3293091656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680101420 0x7fb68010d950 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 --2- 172.21.15.78:0/3293091656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb680100a80 0x7fb680100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 --2- 172.21.15.78:0/3293091656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb680106a80 0x7fb680106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 -- 172.21.15.78:0/3293091656 >> 172.21.15.78:0/3293091656 conn(0x7fb6800fc7a0 msgr2=0x7fb6800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 -- 172.21.15.78:0/3293091656 shutdown_connections 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.385+0000 7fb6866ed640 1 -- 172.21.15.78:0/3293091656 wait complete. 2024-09-16T00:19:03.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb6866ed640 1 Processor -- start 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb6866ed640 1 -- start start 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb6866ed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb680100a80 0x7fb68010e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb6866ed640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb680101420 0x7fb68010e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb6866ed640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680106a80 0x7fb68010eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb6866ed640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb680111fa0 con 0x7fb680106a80 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb6866ed640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb680111e20 con 0x7fb680101420 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb6866ed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb680112120 con 0x7fb680100a80 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb684c8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680106a80 0x7fb68010eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb684c8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680106a80 0x7fb68010eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35438/0 (socket says 172.21.15.78:35438) 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb684c8a640 1 -- 172.21.15.78:0/892442013 learned_addr learned my addr 172.21.15.78:0/892442013 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb680101420 0x7fb68010e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb67ffff640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb680100a80 0x7fb68010e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb684c8a640 1 -- 172.21.15.78:0/892442013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb680100a80 msgr2=0x7fb68010e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb684c8a640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb680100a80 0x7fb68010e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb684c8a640 1 -- 172.21.15.78:0/892442013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb680101420 msgr2=0x7fb68010e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb684c8a640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb680101420 0x7fb68010e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.386+0000 7fb684c8a640 1 -- 172.21.15.78:0/892442013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6801a6880 con 0x7fb680106a80 2024-09-16T00:19:03.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.387+0000 7fb684c8a640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680106a80 0x7fb68010eb60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb678031a60 tx=0x7fb678031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.387+0000 7fb67d7fa640 1 -- 172.21.15.78:0/892442013 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678003b70 con 0x7fb680106a80 2024-09-16T00:19:03.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.387+0000 7fb67d7fa640 1 -- 172.21.15.78:0/892442013 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb678003d10 con 0x7fb680106a80 2024-09-16T00:19:03.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.387+0000 7fb6866ed640 1 -- 172.21.15.78:0/892442013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6801a6b10 con 0x7fb680106a80 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.387+0000 7fb67d7fa640 1 -- 172.21.15.78:0/892442013 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678004770 con 0x7fb680106a80 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.387+0000 7fb6866ed640 1 -- 172.21.15.78:0/892442013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6801a7050 con 0x7fb680106a80 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.389+0000 7fb67d7fa640 1 -- 172.21.15.78:0/892442013 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb67802f790 con 0x7fb680106a80 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.389+0000 7fb688ac6740 1 Processor -- start 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.389+0000 7fb688ac6740 1 -- start start 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 0x5599c34e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34eb370 0x5599c34ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb688ac6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5599c34462f0 con 0x5599c34e1b40 2024-09-16T00:19:03.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb688ac6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5599c33edcf0 con 0x5599c34e05d0 2024-09-16T00:19:03.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb688ac6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5599c34422b0 con 0x5599c34eb370 2024-09-16T00:19:03.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35448/0 (socket says 172.21.15.78:35448) 2024-09-16T00:19:03.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 -- 172.21.15.78:0/32895905 learned_addr learned my addr 172.21.15.78:0/32895905 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:03.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67ffff640 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 0x5599c34e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34eb370 0x5599c34ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67d7fa640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6680a3240 0x7fb6680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 -- 172.21.15.78:0/32895905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34eb370 msgr2=0x5599c34ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:03.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34eb370 0x5599c34ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 -- 172.21.15.78:0/32895905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 msgr2=0x5599c34e0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:03.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 0x5599c34e0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 -- 172.21.15.78:0/32895905 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c33d23a0 con 0x5599c34e1b40 2024-09-16T00:19:03.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb684c8a640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6680a3240 0x7fb6680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67d7fa640 1 -- 172.21.15.78:0/892442013 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb6780f0fa0 con 0x7fb680106a80 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.390+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e0090 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb674013560 tx=0x7fb67400a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5999cd419ab9f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.391+0000 7fb6667fc640 1 -- 172.21.15.78:0/32895905 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674019030 con 0x5599c34e1b40 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.391+0000 7fb684c8a640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6680a3240 0x7fb6680a5700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb6780b96c0 tx=0x7fb67808cff0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.391+0000 7fb6667fc640 1 -- 172.21.15.78:0/32895905 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6740110d0 con 0x5599c34e1b40 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.391+0000 7fb6667fc640 1 -- 172.21.15.78:0/32895905 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67400f8e0 con 0x5599c34e1b40 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.391+0000 7fb688ac6740 1 -- 172.21.15.78:0/32895905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 msgr2=0x5599c34e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.391+0000 7fb688ac6740 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e0090 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb674013560 tx=0x7fb67400a800 comp rx=0 tx=0).stop 2024-09-16T00:19:03.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 -- 172.21.15.78:0/32895905 shutdown_connections 2024-09-16T00:19:03.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34eb370 0x5599c34ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 0x5599c34e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 --2- 172.21.15.78:0/32895905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e0090 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 -- 172.21.15.78:0/32895905 >> 172.21.15.78:0/32895905 conn(0x5599c34a9310 msgr2=0x5599c34dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:03.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 -- 172.21.15.78:0/32895905 shutdown_connections 2024-09-16T00:19:03.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 -- 172.21.15.78:0/32895905 wait complete. 2024-09-16T00:19:03.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 -- start start 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 Processor -- start 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34ead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e1b40 0x5599c34e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c34e8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5599c34ee070 con 0x5599c34eb370 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5599c3401060 con 0x5599c34e1b40 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5599c34ee1f0 con 0x5599c34e05d0 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.392+0000 7fb688ac6740 10 client.0 _ll_register_callbacks cb 0x5599c31548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c34e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c34e8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35456/0 (socket says 172.21.15.78:35456) 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 -- 172.21.15.78:0/3355104705 learned_addr learned my addr 172.21.15.78:0/3355104705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:03.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67ffff640 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e1b40 0x5599c34e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 -- 172.21.15.78:0/3355104705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 msgr2=0x5599c34ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 -- 172.21.15.78:0/3355104705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e1b40 msgr2=0x5599c34e7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e1b40 0x5599c34e7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 -- 172.21.15.78:0/3355104705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c31b73c0 con 0x5599c34eb370 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c34e8560 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb67400f460 tx=0x7fb674011280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c738235d506ab0a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb65f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3355104705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674019040 con 0x5599c34eb370 2024-09-16T00:19:03.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3355104705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb674005250 con 0x5599c34eb370 2024-09-16T00:19:03.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb688ac6740 1 -- 172.21.15.78:0/3355104705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6740110d0 con 0x5599c34eb370 2024-09-16T00:19:03.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3355104705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb674031050 con 0x5599c34eb370 2024-09-16T00:19:03.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb688ac6740 10 client.16419 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:03.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb688ac6740 1 -- 172.21.15.78:0/3355104705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599c34f9af0 con 0x5599c34eb370 2024-09-16T00:19:03.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.393+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3355104705 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6740110d0 con 0x5599c34eb370 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3355104705 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb67400f660 con 0x5599c34eb370 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb688ac6740 10 client.16419 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb688ac6740 10 client.16419 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb688ac6740 20 client.16419 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5599c34faef0 0x5599c34fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb688ac6740 4 client.16419 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb688ac6740 1 -- 172.21.15.78:0/3355104705 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599c3453860 con 0x5599c34faef0 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb67ffff640 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5599c34faef0 0x5599c34fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb65f7fe640 10 client.16419 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.394+0000 7fb67ffff640 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5599c34faef0 0x5599c34fd2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=26c8c60af3618b43 server_cookie=19b234e49e94d535 in_seq=0 out_seq=0 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3355104705 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599c3453860 con 0x5599c34faef0 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb65f7fe640 10 client.16419 handle_command_reply: tid=0 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 client.16419 shutdown 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 -- 172.21.15.78:0/3355104705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 msgr2=0x5599c34e8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c34e8560 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb67400f460 tx=0x7fb674011280 comp rx=0 tx=0).stop 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 -- 172.21.15.78:0/3355104705 shutdown_connections 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c34e8560 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e1b40 0x5599c34e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5599c34faef0 0x5599c34fd2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3355104705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 -- 172.21.15.78:0/3355104705 >> 172.21.15.78:0/3355104705 conn(0x5599c34a9310 msgr2=0x5599c34e2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb65f7fe640 0 client.16419 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 -- 172.21.15.78:0/3355104705 shutdown_connections 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.395+0000 7fb688ac6740 1 -- 172.21.15.78:0/3355104705 wait complete. 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.396+0000 7fb688ac6740 20 client.16419 trim_cache size 0 max 16384 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.408+0000 7fb688ac6740 1 Processor -- start 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.408+0000 7fb688ac6740 1 -- start start 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.408+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 0x5599c34f7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.408+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e1b40 0x5599c34e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.408+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c3502a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.408+0000 7fb688ac6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801b0d30 con 0x5599c34eb370 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.408+0000 7fb688ac6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801b2870 con 0x5599c34e05d0 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.408+0000 7fb688ac6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801b1400 con 0x5599c34e1b40 2024-09-16T00:19:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c3502a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c3502a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35464/0 (socket says 172.21.15.78:35464) 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 -- 172.21.15.78:0/2808776263 learned_addr learned my addr 172.21.15.78:0/2808776263 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67ffff640 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e1b40 0x5599c34e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 0x5599c34f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 -- 172.21.15.78:0/2808776263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e1b40 msgr2=0x5599c34e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e1b40 0x5599c34e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 -- 172.21.15.78:0/2808776263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 msgr2=0x5599c34f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 0x5599c34f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 -- 172.21.15.78:0/2808776263 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c34e3120 con 0x5599c34eb370 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67ffff640 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e1b40 0x5599c34e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c3502a70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb67400b010 tx=0x7fb674000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18afa3ab03862b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb665ffb640 1 -- 172.21.15.78:0/2808776263 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674019070 con 0x5599c34eb370 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb665ffb640 1 -- 172.21.15.78:0/2808776263 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb674011d80 con 0x5599c34eb370 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.409+0000 7fb665ffb640 1 -- 172.21.15.78:0/2808776263 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6740048c0 con 0x5599c34eb370 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 -- 172.21.15.78:0/2808776263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 msgr2=0x5599c3502a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c3502a70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb67400b010 tx=0x7fb674000bc0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 -- 172.21.15.78:0/2808776263 shutdown_connections 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34eb370 0x5599c3502a70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e1b40 0x5599c34e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 --2- 172.21.15.78:0/2808776263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34e05d0 0x5599c34f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 -- 172.21.15.78:0/2808776263 >> 172.21.15.78:0/2808776263 conn(0x5599c34a9310 msgr2=0x5599c34e6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 -- 172.21.15.78:0/2808776263 shutdown_connections 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 -- 172.21.15.78:0/2808776263 wait complete. 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 -- start start 2024-09-16T00:19:03.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.410+0000 7fb688ac6740 1 Processor -- start 2024-09-16T00:19:03.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb688ac6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34eb370 0x5599c34e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb688ac6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801b21a0 con 0x5599c34e1b40 2024-09-16T00:19:03.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb688ac6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801b1ad0 con 0x5599c34eb370 2024-09-16T00:19:03.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb688ac6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801aed10 con 0x5599c34e05d0 2024-09-16T00:19:03.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb688ac6740 10 client.0 _ll_register_callbacks cb 0x7fb67000a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35472/0 (socket says 172.21.15.78:35472) 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 -- 172.21.15.78:0/3959522954 learned_addr learned my addr 172.21.15.78:0/3959522954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34eb370 0x5599c34e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 -- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 msgr2=0x5599c34f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 -- 172.21.15.78:0/3959522954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34eb370 msgr2=0x5599c34e5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34eb370 0x5599c34e5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 -- 172.21.15.78:0/3959522954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c34e56c0 con 0x5599c34e1b40 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:03.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb67ffff640 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e4b40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb670004810 tx=0x7fb67000afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8246f0627585f380 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb65f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3959522954 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6700059c0 con 0x5599c34e1b40 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3959522954 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb670005b60 con 0x5599c34e1b40 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.411+0000 7fb688ac6740 1 -- 172.21.15.78:0/3959522954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb674011d80 con 0x5599c34e1b40 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3959522954 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb67002c050 con 0x5599c34e1b40 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb688ac6740 10 client.16425 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb688ac6740 1 -- 172.21.15.78:0/3959522954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599c3565900 con 0x5599c34e1b40 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3959522954 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb674011d80 con 0x5599c34e1b40 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3959522954 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb670024500 con 0x5599c34e1b40 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb688ac6740 10 client.16425 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb688ac6740 10 client.16425 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:19:03.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb688ac6740 20 client.16425 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5599c3566060 0x5599c3568480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb688ac6740 4 client.16425 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb688ac6740 1 -- 172.21.15.78:0/3959522954 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599c344dd70 con 0x5599c3566060 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.412+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5599c3566060 0x5599c3568480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb65f7fe640 10 client.16425 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb67f7fe640 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5599c3566060 0x5599c3568480 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51189d8f89bf53c3 server_cookie=b7120faaa93fd168 in_seq=0 out_seq=0 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb65f7fe640 1 -- 172.21.15.78:0/3959522954 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599c344dd70 con 0x5599c3566060 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb65f7fe640 10 client.16425 handle_command_reply: tid=0 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 client.16425 shutdown 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 -- 172.21.15.78:0/3959522954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 msgr2=0x5599c34e4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e4b40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb670004810 tx=0x7fb67000afc0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 -- 172.21.15.78:0/3959522954 shutdown_connections 2024-09-16T00:19:03.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5599c34eb370 0x5599c34e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5599c34e1b40 0x5599c34e4b40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5599c3566060 0x5599c3568480 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 --2- 172.21.15.78:0/3959522954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599c34e05d0 0x5599c34f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb65f7fe640 0 client.16425 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:19:03.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 -- 172.21.15.78:0/3959522954 >> 172.21.15.78:0/3959522954 conn(0x5599c34a9310 msgr2=0x5599c34e70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:03.443 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 -- 172.21.15.78:0/3959522954 shutdown_connections 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 1 -- 172.21.15.78:0/3959522954 wait complete. 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.413+0000 7fb688ac6740 20 client.16425 trim_cache size 0 max 16384 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 -- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6680a3240 msgr2=0x7fb6680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6680a3240 0x7fb6680a5700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb6780b96c0 tx=0x7fb67808cff0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 -- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680106a80 msgr2=0x7fb68010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680106a80 0x7fb68010eb60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb678031a60 tx=0x7fb678031a90 comp rx=0 tx=0).stop 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 -- 172.21.15.78:0/892442013 shutdown_connections 2024-09-16T00:19:03.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6680a3240 0x7fb6680a5700 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb680106a80 0x7fb68010eb60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb680101420 0x7fb68010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 --2- 172.21.15.78:0/892442013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb680100a80 0x7fb68010e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 -- 172.21.15.78:0/892442013 >> 172.21.15.78:0/892442013 conn(0x7fb6800fc7a0 msgr2=0x7fb6800fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 -- 172.21.15.78:0/892442013 shutdown_connections 2024-09-16T00:19:03.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:03.414+0000 7fb6866ed640 1 -- 172.21.15.78:0/892442013 wait complete. 2024-09-16T00:19:03.760 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:03 smithi078 ceph-mon[31157]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.4 MiB/s wr, 291 op/s 2024-09-16T00:19:03.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:03 smithi193 ceph-mon[35484]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.4 MiB/s wr, 291 op/s 2024-09-16T00:19:03.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:03 smithi148 ceph-mon[35599]: pgmap v530: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.4 MiB/s wr, 291 op/s 2024-09-16T00:19:04.566 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:19:04.820 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:05.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 -- 172.21.15.78:0/1202876358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50102790 msgr2=0x7f8b501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 --2- 172.21.15.78:0/1202876358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50102790 0x7f8b501067e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009900 tx=0x7f8b4402a410 comp rx=0 tx=0).stop 2024-09-16T00:19:05.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 -- 172.21.15.78:0/1202876358 shutdown_connections 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 --2- 172.21.15.78:0/1202876358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b50106f10 0x7f8b5010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 --2- 172.21.15.78:0/1202876358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50102790 0x7f8b501067e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 --2- 172.21.15.78:0/1202876358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b50101de0 0x7f8b501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 -- 172.21.15.78:0/1202876358 >> 172.21.15.78:0/1202876358 conn(0x7f8b500fd8c0 msgr2=0x7f8b500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 -- 172.21.15.78:0/1202876358 shutdown_connections 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.302+0000 7f8b562c9640 1 -- 172.21.15.78:0/1202876358 wait complete. 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b562c9640 1 Processor -- start 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b562c9640 1 -- start start 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b562c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101de0 0x7f8b500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b562c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b50102790 0x7f8b50074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b562c9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b50106f10 0x7f8b50074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b562c9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50111f90 con 0x7f8b50101de0 2024-09-16T00:19:05.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b562c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50111e10 con 0x7f8b50102790 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b562c9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b50112110 con 0x7f8b50106f10 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101de0 0x7f8b500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101de0 0x7f8b500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35498/0 (socket says 172.21.15.78:35498) 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4ffff640 1 -- 172.21.15.78:0/2762257769 learned_addr learned my addr 172.21.15.78:0/2762257769 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b50102790 0x7f8b50074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b54866640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b50106f10 0x7f8b50074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4ffff640 1 -- 172.21.15.78:0/2762257769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b50106f10 msgr2=0x7f8b50074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b50106f10 0x7f8b50074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4ffff640 1 -- 172.21.15.78:0/2762257769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b50102790 msgr2=0x7f8b50074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b50102790 0x7f8b50074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b4ffff640 1 -- 172.21.15.78:0/2762257769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b501a6930 con 0x7f8b50101de0 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.303+0000 7f8b54866640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b50106f10 0x7f8b50074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:05.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.304+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101de0 0x7f8b500744f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000d950 tx=0x7f8b4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:05.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.304+0000 7f8b4d7fa640 1 -- 172.21.15.78:0/2762257769 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40017070 con 0x7f8b50101de0 2024-09-16T00:19:05.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.305+0000 7f8b562c9640 1 -- 172.21.15.78:0/2762257769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b501a6c20 con 0x7f8b50101de0 2024-09-16T00:19:05.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.305+0000 7f8b562c9640 1 -- 172.21.15.78:0/2762257769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b501a71b0 con 0x7f8b50101de0 2024-09-16T00:19:05.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.305+0000 7f8b4d7fa640 1 -- 172.21.15.78:0/2762257769 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b40009c30 con 0x7f8b50101de0 2024-09-16T00:19:05.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.305+0000 7f8b4d7fa640 1 -- 172.21.15.78:0/2762257769 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000f6b0 con 0x7f8b50101de0 2024-09-16T00:19:05.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b586a2740 1 Processor -- start 2024-09-16T00:19:05.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b4d7fa640 1 -- 172.21.15.78:0/2762257769 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8b4000f850 con 0x7f8b50101de0 2024-09-16T00:19:05.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b586a2740 1 -- start start 2024-09-16T00:19:05.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7db40 0x55c1bbe7c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe7ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe89760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b586a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c1bbde22f0 con 0x55c1bbe87370 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b586a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c1bbd89cf0 con 0x55c1bbe7db40 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.307+0000 7f8b586a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c1bbdde2b0 con 0x55c1bbe7c5d0 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe89760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe89760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35512/0 (socket says 172.21.15.78:35512) 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 -- 172.21.15.78:0/2135062414 learned_addr learned my addr 172.21.15.78:0/2135062414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b54866640 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7db40 0x55c1bbe7c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe7ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 -- 172.21.15.78:0/2135062414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 msgr2=0x55c1bbe7ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe7ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 -- 172.21.15.78:0/2135062414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7db40 msgr2=0x55c1bbe7c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7db40 0x55c1bbe7c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 -- 172.21.15.78:0/2135062414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bbd6e3a0 con 0x55c1bbe87370 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4d7fa640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b380a3240 0x7f8b380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe89760 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009900 tx=0x7f8b44032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9330be33fd6df98e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b54866640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b380a3240 0x7f8b380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b4d7fa640 1 -- 172.21.15.78:0/2762257769 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8b400cf8f0 con 0x7f8b50101de0 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b367fc640 1 -- 172.21.15.78:0/2135062414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4403f070 con 0x55c1bbe87370 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b367fc640 1 -- 172.21.15.78:0/2135062414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b440317b0 con 0x55c1bbe87370 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.308+0000 7f8b367fc640 1 -- 172.21.15.78:0/2135062414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b44035070 con 0x55c1bbe87370 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 -- 172.21.15.78:0/2135062414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 msgr2=0x55c1bbe89760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b54866640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b380a3240 0x7f8b380a5700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800f4c0 tx=0x7f8b48007740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe89760 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009900 tx=0x7f8b44032ce0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 -- 172.21.15.78:0/2135062414 shutdown_connections 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe89760 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe7ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 --2- 172.21.15.78:0/2135062414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7db40 0x55c1bbe7c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 -- 172.21.15.78:0/2135062414 >> 172.21.15.78:0/2135062414 conn(0x55c1bbe45310 msgr2=0x55c1bbe78c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 -- 172.21.15.78:0/2135062414 shutdown_connections 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.309+0000 7f8b586a2740 1 -- 172.21.15.78:0/2135062414 wait complete. 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 1 -- start start 2024-09-16T00:19:05.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 1 Processor -- start 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe86ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 0x55c1bbe83f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe84490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c1bbe8a010 con 0x55c1bbe87370 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c1bbd9d060 con 0x55c1bbe7c5d0 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c1bbe8a190 con 0x55c1bbe7db40 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b586a2740 10 client.0 _ll_register_callbacks cb 0x55c1bbaf08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b54866640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe84490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b54866640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe84490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35528/0 (socket says 172.21.15.78:35528) 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.310+0000 7f8b54866640 1 -- 172.21.15.78:0/3706051639 learned_addr learned my addr 172.21.15.78:0/3706051639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:05.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe86ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 0x55c1bbe83f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b54866640 1 -- 172.21.15.78:0/3706051639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 msgr2=0x55c1bbe83f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b54866640 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 0x55c1bbe83f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b54866640 1 -- 172.21.15.78:0/3706051639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 msgr2=0x55c1bbe86ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b54866640 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe86ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b54866640 1 -- 172.21.15.78:0/3706051639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bbb533c0 con 0x55c1bbe87370 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe86ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b54866640 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe84490 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8b48010eb0 tx=0x7f8b48004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1380f12c9f3ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3706051639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48026680 con 0x55c1bbe87370 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3706051639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8b48026820 con 0x55c1bbe87370 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b586a2740 1 -- 172.21.15.78:0/3706051639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b440317b0 con 0x55c1bbe87370 2024-09-16T00:19:05.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3706051639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8b4802f050 con 0x55c1bbe87370 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b586a2740 10 client.16437 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b586a2740 1 -- 172.21.15.78:0/3706051639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c1bbe95b20 con 0x55c1bbe87370 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.311+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3706051639 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b440317b0 con 0x55c1bbe87370 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3706051639 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8b4801dc10 con 0x55c1bbe87370 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b586a2740 10 client.16437 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b586a2740 10 client.16437 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b586a2740 20 client.16437 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55c1bbe970b0 0x55c1bbe99570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b586a2740 4 client.16437 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b586a2740 1 -- 172.21.15.78:0/3706051639 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1bbdef860 con 0x55c1bbe970b0 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55c1bbe970b0 0x55c1bbe99570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55c1bbe970b0 0x55c1bbe99570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f91ef6b2db554bd server_cookie=82ed7028d909770c in_seq=0 out_seq=0 2024-09-16T00:19:05.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.312+0000 7f8b2f7fe640 10 client.16437 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3706051639 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1bbdef860 con 0x55c1bbe970b0 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b2f7fe640 10 client.16437 handle_command_reply: tid=0 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 client.16437 shutdown 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 -- 172.21.15.78:0/3706051639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 msgr2=0x55c1bbe84490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe84490 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8b48010eb0 tx=0x7f8b48004b50 comp rx=0 tx=0).stop 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 -- 172.21.15.78:0/3706051639 shutdown_connections 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe84490 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 0x55c1bbe83f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55c1bbe970b0 0x55c1bbe99570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3706051639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe86ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 -- 172.21.15.78:0/3706051639 >> 172.21.15.78:0/3706051639 conn(0x55c1bbe45310 msgr2=0x55c1bbe7e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:05.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b2f7fe640 0 client.16437 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:19:05.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 -- 172.21.15.78:0/3706051639 shutdown_connections 2024-09-16T00:19:05.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.313+0000 7f8b586a2740 1 -- 172.21.15.78:0/3706051639 wait complete. 2024-09-16T00:19:05.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.314+0000 7f8b586a2740 20 client.16437 trim_cache size 0 max 16384 2024-09-16T00:19:05.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.326+0000 7f8b586a2740 1 Processor -- start 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.326+0000 7f8b586a2740 1 -- start start 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.326+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe93340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.326+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 0x55c1bbe78700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.326+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe78c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.326+0000 7f8b586a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c00faa0 con 0x55c1bbe87370 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.326+0000 7f8b586a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c0115e0 con 0x55c1bbe7c5d0 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.326+0000 7f8b586a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c010170 con 0x55c1bbe7db40 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe78c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe78c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35534/0 (socket says 172.21.15.78:35534) 2024-09-16T00:19:05.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 -- 172.21.15.78:0/1054865459 learned_addr learned my addr 172.21.15.78:0/1054865459 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:05.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe93340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 0x55c1bbe78700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 -- 172.21.15.78:0/1054865459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 msgr2=0x55c1bbe78700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 0x55c1bbe78700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 -- 172.21.15.78:0/1054865459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 msgr2=0x55c1bbe93340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe93340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 -- 172.21.15.78:0/1054865459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bbe79dd0 con 0x55c1bbe87370 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe93340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b54866640 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe78c40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8b48010900 tx=0x7f8b48007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc75df6f4b9604e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b35ffb640 1 -- 172.21.15.78:0/1054865459 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48037070 con 0x55c1bbe87370 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b35ffb640 1 -- 172.21.15.78:0/1054865459 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4801ccd0 con 0x55c1bbe87370 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.327+0000 7f8b35ffb640 1 -- 172.21.15.78:0/1054865459 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4802c040 con 0x55c1bbe87370 2024-09-16T00:19:05.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 -- 172.21.15.78:0/1054865459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 msgr2=0x55c1bbe78c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe78c40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8b48010900 tx=0x7f8b48007830 comp rx=0 tx=0).stop 2024-09-16T00:19:05.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 -- 172.21.15.78:0/1054865459 shutdown_connections 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe87370 0x55c1bbe78c40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7db40 0x55c1bbe78700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 --2- 172.21.15.78:0/1054865459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe93340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 -- 172.21.15.78:0/1054865459 >> 172.21.15.78:0/1054865459 conn(0x55c1bbe45310 msgr2=0x55c1bbe83f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 -- 172.21.15.78:0/1054865459 shutdown_connections 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 -- 172.21.15.78:0/1054865459 wait complete. 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.328+0000 7f8b586a2740 1 -- start start 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b586a2740 1 Processor -- start 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe8f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe7db40 0x55c1bbe8fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b586a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe87370 0x55c1bbe80c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b586a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c010f10 con 0x55c1bbe7db40 2024-09-16T00:19:05.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b586a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c010840 con 0x55c1bbe87370 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b586a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3c00da80 con 0x55c1bbe7c5d0 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b586a2740 10 client.0 _ll_register_callbacks cb 0x7f8b48007cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe7db40 0x55c1bbe8fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe7db40 0x55c1bbe8fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35540/0 (socket says 172.21.15.78:35540) 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 -- 172.21.15.78:0/3668864301 learned_addr learned my addr 172.21.15.78:0/3668864301 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b54866640 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe87370 0x55c1bbe80c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe8f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 -- 172.21.15.78:0/3668864301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 msgr2=0x55c1bbe8f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe8f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 -- 172.21.15.78:0/3668864301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe87370 msgr2=0x55c1bbe80c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe87370 0x55c1bbe80c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 -- 172.21.15.78:0/3668864301 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bbe81440 con 0x55c1bbe7db40 2024-09-16T00:19:05.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b4f7fe640 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe7db40 0x55c1bbe8fda0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8b4402a930 tx=0x7f8b44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279769d4eced427b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.329+0000 7f8b2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3668864301 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4404b060 con 0x55c1bbe7db40 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3668864301 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8b44005240 con 0x55c1bbe7db40 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 1 -- 172.21.15.78:0/3668864301 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4801ccd0 con 0x55c1bbe7db40 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3668864301 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8b44052050 con 0x55c1bbe7db40 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 10 client.16443 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 1 -- 172.21.15.78:0/3668864301 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c1bbf01ab0 con 0x55c1bbe7db40 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3668864301 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4801ccd0 con 0x55c1bbe7db40 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3668864301 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8b440035f0 con 0x55c1bbe7db40 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 10 client.16443 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 10 client.16443 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 20 client.16443 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:05.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55c1bbf01d90 0x55c1bbf04180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 4 client.16443 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.330+0000 7f8b586a2740 1 -- 172.21.15.78:0/3668864301 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1bbde9d70 con 0x55c1bbf01d90 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.331+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55c1bbf01d90 0x55c1bbf04180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.331+0000 7f8b2f7fe640 10 client.16443 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.331+0000 7f8b4ffff640 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55c1bbf01d90 0x55c1bbf04180 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=840b2abde21ba643 server_cookie=950399ac6ab015c2 in_seq=0 out_seq=0 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.331+0000 7f8b2f7fe640 1 -- 172.21.15.78:0/3668864301 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1bbde9d70 con 0x55c1bbf01d90 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.331+0000 7f8b2f7fe640 10 client.16443 handle_command_reply: tid=0 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.331+0000 7f8b586a2740 1 client.16443 shutdown 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.331+0000 7f8b586a2740 1 -- 172.21.15.78:0/3668864301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe7db40 msgr2=0x55c1bbe8fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.331+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe7db40 0x55c1bbe8fda0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8b4402a930 tx=0x7f8b44004270 comp rx=0 tx=0).stop 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 1 -- 172.21.15.78:0/3668864301 shutdown_connections 2024-09-16T00:19:05.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55c1bbf01d90 0x55c1bbf04180 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c1bbe87370 0x55c1bbe80c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c1bbe7db40 0x55c1bbe8fda0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 1 --2- 172.21.15.78:0/3668864301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1bbe7c5d0 0x55c1bbe8f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 1 -- 172.21.15.78:0/3668864301 >> 172.21.15.78:0/3668864301 conn(0x55c1bbe45310 msgr2=0x55c1bbe78700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:05.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b2f7fe640 0 client.16443 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:19:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 1 -- 172.21.15.78:0/3668864301 shutdown_connections 2024-09-16T00:19:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 1 -- 172.21.15.78:0/3668864301 wait complete. 2024-09-16T00:19:05.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b586a2740 20 client.16443 trim_cache size 0 max 16384 2024-09-16T00:19:05.345 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:19:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b562c9640 1 -- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b380a3240 msgr2=0x7f8b380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b562c9640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b380a3240 0x7f8b380a5700 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800f4c0 tx=0x7f8b48007740 comp rx=0 tx=0).stop 2024-09-16T00:19:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b562c9640 1 -- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101de0 msgr2=0x7f8b500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.332+0000 7f8b562c9640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101de0 0x7f8b500744f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000d950 tx=0x7f8b4000de10 comp rx=0 tx=0).stop 2024-09-16T00:19:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.333+0000 7f8b562c9640 1 -- 172.21.15.78:0/2762257769 shutdown_connections 2024-09-16T00:19:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.333+0000 7f8b562c9640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b380a3240 0x7f8b380a5700 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.333+0000 7f8b562c9640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b50106f10 0x7f8b50074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.333+0000 7f8b562c9640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b50102790 0x7f8b50074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.333+0000 7f8b562c9640 1 --2- 172.21.15.78:0/2762257769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b50101de0 0x7f8b500744f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:05.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.333+0000 7f8b562c9640 1 -- 172.21.15.78:0/2762257769 >> 172.21.15.78:0/2762257769 conn(0x7f8b500fd8c0 msgr2=0x7f8b500fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:05.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.333+0000 7f8b562c9640 1 -- 172.21.15.78:0/2762257769 shutdown_connections 2024-09-16T00:19:05.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:05.333+0000 7f8b562c9640 1 -- 172.21.15.78:0/2762257769 wait complete. 2024-09-16T00:19:05.861 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 305.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:19:05.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:05 smithi193 ceph-mon[35484]: pgmap v531: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.3 MiB/s wr, 320 op/s 2024-09-16T00:19:05.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:05 smithi148 ceph-mon[35599]: pgmap v531: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.3 MiB/s wr, 320 op/s 2024-09-16T00:19:06.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:05 smithi078 ceph-mon[31157]: pgmap v531: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.3 MiB/s wr, 320 op/s 2024-09-16T00:19:06.586 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:19:06.849 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.335+0000 7fea261ca640 1 -- 172.21.15.78:0/3307176962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 msgr2=0x7fea20100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.335+0000 7fea261ca640 1 --2- 172.21.15.78:0/3307176962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 0x7fea20100220 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fea10009930 tx=0x7fea1002a440 comp rx=0 tx=0).stop 2024-09-16T00:19:07.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.335+0000 7fea261ca640 1 -- 172.21.15.78:0/3307176962 shutdown_connections 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.335+0000 7fea261ca640 1 --2- 172.21.15.78:0/3307176962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea20106fc0 0x7fea2010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.335+0000 7fea261ca640 1 --2- 172.21.15.78:0/3307176962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea201007f0 0x7fea20106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.335+0000 7fea261ca640 1 --2- 172.21.15.78:0/3307176962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 0x7fea20100220 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.335+0000 7fea261ca640 1 -- 172.21.15.78:0/3307176962 >> 172.21.15.78:0/3307176962 conn(0x7fea200fba90 msgr2=0x7fea200fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 -- 172.21.15.78:0/3307176962 shutdown_connections 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 -- 172.21.15.78:0/3307176962 wait complete. 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 Processor -- start 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 -- start start 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 0x7fea20198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea201007f0 0x7fea20198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea20106fc0 0x7fea2019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea20112020 con 0x7fea200ffe40 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea20111ea0 con 0x7fea201007f0 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea261ca640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea201121a0 con 0x7fea20106fc0 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 0x7fea20198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 0x7fea20198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56860/0 (socket says 172.21.15.78:56860) 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.336+0000 7fea1f7fe640 1 -- 172.21.15.78:0/846901471 learned_addr learned my addr 172.21.15.78:0/846901471 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea17fff640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea201007f0 0x7fea20198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1ffff640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea20106fc0 0x7fea2019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1f7fe640 1 -- 172.21.15.78:0/846901471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea20106fc0 msgr2=0x7fea2019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea20106fc0 0x7fea2019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1f7fe640 1 -- 172.21.15.78:0/846901471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea201007f0 msgr2=0x7fea20198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea201007f0 0x7fea20198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1f7fe640 1 -- 172.21.15.78:0/846901471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2019d5d0 con 0x7fea200ffe40 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 0x7fea20198620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fea10009a50 tx=0x7fea10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1d7fa640 1 -- 172.21.15.78:0/846901471 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1003f030 con 0x7fea200ffe40 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea261ca640 1 -- 172.21.15.78:0/846901471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2019d860 con 0x7fea200ffe40 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1d7fa640 1 -- 172.21.15.78:0/846901471 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea10003ce0 con 0x7fea200ffe40 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea1d7fa640 1 -- 172.21.15.78:0/846901471 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10035070 con 0x7fea200ffe40 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.337+0000 7fea261ca640 1 -- 172.21.15.78:0/846901471 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201a51a0 con 0x7fea200ffe40 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.339+0000 7fea1d7fa640 1 -- 172.21.15.78:0/846901471 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fea10042030 con 0x7fea200ffe40 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea285a3740 1 Processor -- start 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea285a3740 1 -- start start 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea285a3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 0x55a98e37b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea285a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 0x55a98e37ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea285a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e386370 0x55a98e388760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea285a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a98e2e12f0 con 0x55a98e386370 2024-09-16T00:19:07.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea285a3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a98e288cf0 con 0x55a98e37cb40 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea285a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a98e2dd2b0 con 0x55a98e37b5d0 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea17fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e386370 0x55a98e388760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea17fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e386370 0x55a98e388760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56864/0 (socket says 172.21.15.78:56864) 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea1d7fa640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea0c0a3240 0x7fea0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea17fff640 1 -- 172.21.15.78:0/812944366 learned_addr learned my addr 172.21.15.78:0/812944366 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.340+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/812944366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 0x55a98e37ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea1ffff640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea0c0a3240 0x7fea0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea17fff640 1 -- 172.21.15.78:0/812944366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 msgr2=0x55a98e37ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea17fff640 1 --2- 172.21.15.78:0/812944366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 0x55a98e37ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea17fff640 1 -- 172.21.15.78:0/812944366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 msgr2=0x55a98e37b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea17fff640 1 --2- 172.21.15.78:0/812944366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 0x55a98e37b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea17fff640 1 -- 172.21.15.78:0/812944366 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98e26d3a0 con 0x55a98e386370 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea1d7fa640 1 -- 172.21.15.78:0/846901471 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fea100f0320 con 0x7fea200ffe40 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea17fff640 1 --2- 172.21.15.78:0/812944366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e386370 0x55a98e388760 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b790 tx=0x7fea0800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8253e2c5994ceed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea1ffff640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea0c0a3240 0x7fea0c0a5700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea18009eb0 tx=0x7fea18009540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea15ffb640 1 -- 172.21.15.78:0/812944366 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0800d950 con 0x55a98e386370 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea15ffb640 1 -- 172.21.15.78:0/812944366 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea08018070 con 0x55a98e386370 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.341+0000 7fea15ffb640 1 -- 172.21.15.78:0/812944366 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea080123e0 con 0x55a98e386370 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 -- 172.21.15.78:0/812944366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e386370 msgr2=0x55a98e388760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 --2- 172.21.15.78:0/812944366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e386370 0x55a98e388760 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b790 tx=0x7fea0800bc50 comp rx=0 tx=0).stop 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 -- 172.21.15.78:0/812944366 shutdown_connections 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 --2- 172.21.15.78:0/812944366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e386370 0x55a98e388760 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 --2- 172.21.15.78:0/812944366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 0x55a98e37ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 --2- 172.21.15.78:0/812944366 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 0x55a98e37b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 -- 172.21.15.78:0/812944366 >> 172.21.15.78:0/812944366 conn(0x55a98e344310 msgr2=0x55a98e377c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 -- 172.21.15.78:0/812944366 shutdown_connections 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 -- 172.21.15.78:0/812944366 wait complete. 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 -- start start 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.342+0000 7fea285a3740 1 Processor -- start 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea285a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e385ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea285a3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 0x55a98e382f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea285a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e386370 0x55a98e3834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea285a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a98e389070 con 0x55a98e37b5d0 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea285a3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a98e29c060 con 0x55a98e37cb40 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea285a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a98e3891f0 con 0x55a98e386370 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea285a3740 10 client.0 _ll_register_callbacks cb 0x55a98dfef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e385ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e385ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56872/0 (socket says 172.21.15.78:56872) 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 -- 172.21.15.78:0/90223936 learned_addr learned my addr 172.21.15.78:0/90223936 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1ffff640 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e386370 0x55a98e3834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea17fff640 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 0x55a98e382f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 -- 172.21.15.78:0/90223936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e386370 msgr2=0x55a98e3834f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e386370 0x55a98e3834f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 -- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 msgr2=0x55a98e382f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 0x55a98e382f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 -- 172.21.15.78:0/90223936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98e0523c0 con 0x55a98e37b5d0 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea17fff640 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 0x55a98e382f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e385ce0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fea100b8f50 tx=0x7fea1003b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dde2c8b5304a31d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fe9feffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fe9feffd640 1 -- 172.21.15.78:0/90223936 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10049040 con 0x55a98e37b5d0 2024-09-16T00:19:07.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fe9feffd640 1 -- 172.21.15.78:0/90223936 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fea100b8bf0 con 0x55a98e37b5d0 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.343+0000 7fea285a3740 1 -- 172.21.15.78:0/90223936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea08018070 con 0x55a98e37b5d0 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fe9feffd640 1 -- 172.21.15.78:0/90223936 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea100bbee0 con 0x55a98e37b5d0 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea285a3740 10 client.16455 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea285a3740 1 -- 172.21.15.78:0/90223936 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a98e394a80 con 0x55a98e37b5d0 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fe9feffd640 1 -- 172.21.15.78:0/90223936 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea08018070 con 0x55a98e37b5d0 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fe9feffd640 1 -- 172.21.15.78:0/90223936 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fea100c4d60 con 0x55a98e37b5d0 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea285a3740 10 client.16455 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea285a3740 10 client.16455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea285a3740 20 client.16455 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea285a3740 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a98e395e70 0x55a98e3982c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea285a3740 4 client.16455 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea285a3740 1 -- 172.21.15.78:0/90223936 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a98e2ee860 con 0x55a98e395e70 2024-09-16T00:19:07.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.344+0000 7fea17fff640 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a98e395e70 0x55a98e3982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.345+0000 7fe9feffd640 10 client.16455 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.345+0000 7fea17fff640 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a98e395e70 0x55a98e3982c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5a3365d06031bfb server_cookie=c2413fb58a79a2b6 in_seq=0 out_seq=0 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.345+0000 7fe9feffd640 1 -- 172.21.15.78:0/90223936 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a98e2ee860 con 0x55a98e395e70 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.345+0000 7fe9feffd640 10 client.16455 handle_command_reply: tid=0 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 client.16455 shutdown 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 -- 172.21.15.78:0/90223936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 msgr2=0x55a98e385ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e385ce0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fea100b8f50 tx=0x7fea1003b790 comp rx=0 tx=0).stop 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 -- 172.21.15.78:0/90223936 shutdown_connections 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e386370 0x55a98e3834f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a98e395e70 0x55a98e3982c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e37cb40 0x55a98e382f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 --2- 172.21.15.78:0/90223936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e385ce0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fe9feffd640 0 client.16455 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:19:07.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 -- 172.21.15.78:0/90223936 >> 172.21.15.78:0/90223936 conn(0x55a98e344310 msgr2=0x55a98e37d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:07.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 -- 172.21.15.78:0/90223936 shutdown_connections 2024-09-16T00:19:07.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 1 -- 172.21.15.78:0/90223936 wait complete. 2024-09-16T00:19:07.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.346+0000 7fea285a3740 20 client.16455 trim_cache size 0 max 16384 2024-09-16T00:19:07.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea285a3740 1 Processor -- start 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea285a3740 1 -- start start 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea285a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 0x55a98e377e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea285a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37cb40 0x55a98e392310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea285a3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 0x55a98e39dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea285a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea201aec60 con 0x55a98e37cb40 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea285a3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea201b07a0 con 0x55a98e386370 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea285a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea201af330 con 0x55a98e37b5d0 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea17fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37cb40 0x55a98e392310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea17fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37cb40 0x55a98e392310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56886/0 (socket says 172.21.15.78:56886) 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea17fff640 1 -- 172.21.15.78:0/3693185292 learned_addr learned my addr 172.21.15.78:0/3693185292 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 0x55a98e377e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea1ffff640 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 0x55a98e39dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea17fff640 1 -- 172.21.15.78:0/3693185292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 msgr2=0x55a98e377e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea17fff640 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 0x55a98e377e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea17fff640 1 -- 172.21.15.78:0/3693185292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 msgr2=0x55a98e39dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea17fff640 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 0x55a98e39dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea17fff640 1 -- 172.21.15.78:0/3693185292 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98e37c760 con 0x55a98e37cb40 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.359+0000 7fea1ffff640 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 0x55a98e39dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea17fff640 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37cb40 0x55a98e392310 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b6e0 tx=0x7fea08013a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b523638af82dae2c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea157fa640 1 -- 172.21.15.78:0/3693185292 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08028070 con 0x55a98e37cb40 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea157fa640 1 -- 172.21.15.78:0/3693185292 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea08018040 con 0x55a98e37cb40 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea157fa640 1 -- 172.21.15.78:0/3693185292 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08023c00 con 0x55a98e37cb40 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea285a3740 1 -- 172.21.15.78:0/3693185292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37cb40 msgr2=0x55a98e392310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea285a3740 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37cb40 0x55a98e392310 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b6e0 tx=0x7fea08013a50 comp rx=0 tx=0).stop 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea285a3740 1 -- 172.21.15.78:0/3693185292 shutdown_connections 2024-09-16T00:19:07.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea285a3740 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 0x55a98e39dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea285a3740 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37cb40 0x55a98e392310 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea285a3740 1 --2- 172.21.15.78:0/3693185292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37b5d0 0x55a98e377e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.360+0000 7fea285a3740 1 -- 172.21.15.78:0/3693185292 >> 172.21.15.78:0/3693185292 conn(0x55a98e344310 msgr2=0x55a98e377700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 -- 172.21.15.78:0/3693185292 shutdown_connections 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 -- 172.21.15.78:0/3693185292 wait complete. 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 -- start start 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 Processor -- start 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e38a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37cb40 0x55a98e38a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 0x55a98e38ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea201b00d0 con 0x55a98e37b5d0 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fea201afa00 con 0x55a98e386370 2024-09-16T00:19:07.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea201acc40 con 0x55a98e37cb40 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea285a3740 10 client.0 _ll_register_callbacks cb 0x7fea10009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e38a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea1f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e38a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56902/0 (socket says 172.21.15.78:56902) 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.361+0000 7fea1f7fe640 1 -- 172.21.15.78:0/879729343 learned_addr learned my addr 172.21.15.78:0/879729343 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea17fff640 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37cb40 0x55a98e38a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea1f7fe640 1 -- 172.21.15.78:0/879729343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37cb40 msgr2=0x55a98e38a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37cb40 0x55a98e38a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea1f7fe640 1 -- 172.21.15.78:0/879729343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 msgr2=0x55a98e38ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 0x55a98e38ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea1f7fe640 1 -- 172.21.15.78:0/879729343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98e3ffd00 con 0x55a98e37b5d0 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea1f7fe640 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e38a310 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fea100bc280 tx=0x7fea100b8e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d9e5f229aab5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:07.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fe9feffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fe9feffd640 1 -- 172.21.15.78:0/879729343 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10049040 con 0x55a98e37b5d0 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fe9feffd640 1 -- 172.21.15.78:0/879729343 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fea100d54f0 con 0x55a98e37b5d0 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 1 -- 172.21.15.78:0/879729343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea08018040 con 0x55a98e37b5d0 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fe9feffd640 1 -- 172.21.15.78:0/879729343 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea100bc350 con 0x55a98e37b5d0 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 10 client.16461 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 1 -- 172.21.15.78:0/879729343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a98e400fa0 con 0x55a98e37b5d0 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fe9feffd640 1 -- 172.21.15.78:0/879729343 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea08018040 con 0x55a98e37b5d0 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fe9feffd640 1 -- 172.21.15.78:0/879729343 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fea100b82d0 con 0x55a98e37b5d0 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 10 client.16461 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 10 client.16461 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 20 client.16461 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a98e401530 0x55a98e403920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:07.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 4 client.16461 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:19:07.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.362+0000 7fea285a3740 1 -- 172.21.15.78:0/879729343 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a98e2e8d70 con 0x55a98e401530 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea17fff640 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a98e401530 0x55a98e403920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fe9feffd640 10 client.16461 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea17fff640 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a98e401530 0x55a98e403920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25faf0156364cf53 server_cookie=d1398276e8660280 in_seq=0 out_seq=0 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fe9feffd640 1 -- 172.21.15.78:0/879729343 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a98e2e8d70 con 0x55a98e401530 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fe9feffd640 10 client.16461 handle_command_reply: tid=0 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 client.16461 shutdown 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 -- 172.21.15.78:0/879729343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 msgr2=0x55a98e38a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e38a310 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fea100bc280 tx=0x7fea100b8e20 comp rx=0 tx=0).stop 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 -- 172.21.15.78:0/879729343 shutdown_connections 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a98e401530 0x55a98e403920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a98e386370 0x55a98e38ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a98e37cb40 0x55a98e38a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 --2- 172.21.15.78:0/879729343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a98e37b5d0 0x55a98e38a310 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.363+0000 7fea285a3740 1 -- 172.21.15.78:0/879729343 >> 172.21.15.78:0/879729343 conn(0x55a98e344310 msgr2=0x55a98e37d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fe9feffd640 0 client.16461 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:19:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea285a3740 1 -- 172.21.15.78:0/879729343 shutdown_connections 2024-09-16T00:19:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea285a3740 1 -- 172.21.15.78:0/879729343 wait complete. 2024-09-16T00:19:07.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea285a3740 20 client.16461 trim_cache size 0 max 16384 2024-09-16T00:19:07.382 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:19:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 -- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea0c0a3240 msgr2=0x7fea0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea0c0a3240 0x7fea0c0a5700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea18009eb0 tx=0x7fea18009540 comp rx=0 tx=0).stop 2024-09-16T00:19:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 -- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 msgr2=0x7fea20198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 0x7fea20198620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fea10009a50 tx=0x7fea10004270 comp rx=0 tx=0).stop 2024-09-16T00:19:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 -- 172.21.15.78:0/846901471 shutdown_connections 2024-09-16T00:19:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fea0c0a3240 0x7fea0c0a5700 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea20106fc0 0x7fea2019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fea201007f0 0x7fea20198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 --2- 172.21.15.78:0/846901471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200ffe40 0x7fea20198620 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 -- 172.21.15.78:0/846901471 >> 172.21.15.78:0/846901471 conn(0x7fea200fba90 msgr2=0x7fea200fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 -- 172.21.15.78:0/846901471 shutdown_connections 2024-09-16T00:19:07.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:07.364+0000 7fea261ca640 1 -- 172.21.15.78:0/846901471 wait complete. 2024-09-16T00:19:07.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:07 smithi193 ceph-mon[35484]: pgmap v532: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.3 MiB/s wr, 322 op/s 2024-09-16T00:19:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:07 smithi148 ceph-mon[35599]: pgmap v532: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.3 MiB/s wr, 322 op/s 2024-09-16T00:19:08.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:07 smithi078 ceph-mon[31157]: pgmap v532: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.3 MiB/s wr, 322 op/s 2024-09-16T00:19:08.208 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:19:08.454 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 -- 172.21.15.78:0/3333868885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c102570 msgr2=0x7f649c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/3333868885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c102570 0x7f649c1029b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6484009900 tx=0x7f648402a480 comp rx=0 tx=0).stop 2024-09-16T00:19:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 -- 172.21.15.78:0/3333868885 shutdown_connections 2024-09-16T00:19:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/3333868885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f649c102ef0 0x7f649c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/3333868885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c102570 0x7f649c1029b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/3333868885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c100e20 0x7f649c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 -- 172.21.15.78:0/3333868885 >> 172.21.15.78:0/3333868885 conn(0x7f649c0fc7d0 msgr2=0x7f649c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 -- 172.21.15.78:0/3333868885 shutdown_connections 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.284+0000 7f64a0c3f640 1 -- 172.21.15.78:0/3333868885 wait complete. 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f64a0c3f640 1 Processor -- start 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f64a0c3f640 1 -- start start 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f64a0c3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c100e20 0x7f649c197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f64a0c3f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c102570 0x7f649c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f64a0c3f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f649c102ef0 0x7f649c19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f64a0c3f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f649c111020 con 0x7f649c100e20 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f64a0c3f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f649c110ea0 con 0x7f649c102ef0 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f64a0c3f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1111a0 con 0x7f649c102570 2024-09-16T00:19:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c100e20 0x7f649c197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c100e20 0x7f649c197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56922/0 (socket says 172.21.15.78:56922) 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649a59c640 1 -- 172.21.15.78:0/4212457969 learned_addr learned my addr 172.21.15.78:0/4212457969 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649ad9d640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f649c102ef0 0x7f649c19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649a59c640 1 -- 172.21.15.78:0/4212457969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c102570 msgr2=0x7f649c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649a59c640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c102570 0x7f649c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649a59c640 1 -- 172.21.15.78:0/4212457969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f649c102ef0 msgr2=0x7f649c19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649a59c640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f649c102ef0 0x7f649c19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.285+0000 7f649a59c640 1 -- 172.21.15.78:0/4212457969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c19c4f0 con 0x7f649c100e20 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.286+0000 7f649a59c640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c100e20 0x7f649c197540 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f648c00b4a0 tx=0x7f648c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.286+0000 7f64937fe640 1 -- 172.21.15.78:0/4212457969 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c014070 con 0x7f649c100e20 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.286+0000 7f64a0c3f640 1 -- 172.21.15.78:0/4212457969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c19c7e0 con 0x7f649c100e20 2024-09-16T00:19:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.286+0000 7f64937fe640 1 -- 172.21.15.78:0/4212457969 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f648c010af0 con 0x7f649c100e20 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.286+0000 7f64937fe640 1 -- 172.21.15.78:0/4212457969 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c017d10 con 0x7f649c100e20 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.286+0000 7f64a0c3f640 1 -- 172.21.15.78:0/4212457969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c1a40c0 con 0x7f649c100e20 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.288+0000 7f64937fe640 1 -- 172.21.15.78:0/4212457969 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f648c017070 con 0x7f649c100e20 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.288+0000 7f64a3018740 1 Processor -- start 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64a3018740 1 -- start start 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64a3018740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c42f50 0x5564a8c414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64a3018740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 0x5564a8c41e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64a3018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c4c780 0x5564a8c4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64a3018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5564a8ba32f0 con 0x5564a8c4c780 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64a3018740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5564a8b4acf0 con 0x5564a8c419e0 2024-09-16T00:19:09.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64a3018740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564a8b9f2b0 con 0x5564a8c42f50 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64937fe640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64880a3240 0x7f64880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f6493fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c4c780 0x5564a8c4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f6493fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c4c780 0x5564a8c4eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56936/0 (socket says 172.21.15.78:56936) 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f6493fff640 1 -- 172.21.15.78:0/737600472 learned_addr learned my addr 172.21.15.78:0/737600472 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f649ad9d640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64880a3240 0x7f64880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f64937fe640 1 -- 172.21.15.78:0/4212457969 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f648c0cf150 con 0x7f649c100e20 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f649a59c640 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 0x5564a8c41e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f649ad9d640 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c42f50 0x5564a8c414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f6493fff640 1 -- 172.21.15.78:0/737600472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c42f50 msgr2=0x5564a8c414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f6493fff640 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c42f50 0x5564a8c414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f6493fff640 1 -- 172.21.15.78:0/737600472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 msgr2=0x5564a8c41e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f6493fff640 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 0x5564a8c41e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.289+0000 7f6493fff640 1 -- 172.21.15.78:0/737600472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564a8b2f3a0 con 0x5564a8c4c780 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.290+0000 7f649ad9d640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64880a3240 0x7f64880a5700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6494009fd0 tx=0x7f6494009a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.290+0000 7f6493fff640 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c4c780 0x5564a8c4eb70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6484009a20 tx=0x7f64840040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0369beb2195b8fe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.290+0000 7f6490ff9640 1 -- 172.21.15.78:0/737600472 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648403e030 con 0x5564a8c4c780 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.290+0000 7f6490ff9640 1 -- 172.21.15.78:0/737600472 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64840042e0 con 0x5564a8c4c780 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.290+0000 7f6490ff9640 1 -- 172.21.15.78:0/737600472 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484035040 con 0x5564a8c4c780 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- 172.21.15.78:0/737600472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c4c780 msgr2=0x5564a8c4eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c4c780 0x5564a8c4eb70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6484009a20 tx=0x7f64840040c0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- 172.21.15.78:0/737600472 shutdown_connections 2024-09-16T00:19:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c4c780 0x5564a8c4eb70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 0x5564a8c41e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 --2- 172.21.15.78:0/737600472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c42f50 0x5564a8c414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- 172.21.15.78:0/737600472 >> 172.21.15.78:0/737600472 conn(0x5564a8c06310 msgr2=0x5564a8c3e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- 172.21.15.78:0/737600472 shutdown_connections 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- 172.21.15.78:0/737600472 wait complete. 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- start start 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 Processor -- start 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c419e0 0x5564a8c4c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c42f50 0x5564a8c493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 0x5564a8c49900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5564a8c4f420 con 0x5564a8c419e0 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5564a8b5e060 con 0x5564a8c42f50 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.291+0000 7f64a3018740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564a8c4f5a0 con 0x5564a8c4c780 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c419e0 0x5564a8c4c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f64a3018740 10 client.0 _ll_register_callbacks cb 0x5564a88b18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c419e0 0x5564a8c4c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56938/0 (socket says 172.21.15.78:56938) 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 -- 172.21.15.78:0/3690008427 learned_addr learned my addr 172.21.15.78:0/3690008427 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f6493fff640 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c42f50 0x5564a8c493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649ad9d640 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 0x5564a8c49900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 -- 172.21.15.78:0/3690008427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 msgr2=0x5564a8c49900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 0x5564a8c49900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 -- 172.21.15.78:0/3690008427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c42f50 msgr2=0x5564a8c493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c42f50 0x5564a8c493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 -- 172.21.15.78:0/3690008427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564a89143c0 con 0x5564a8c419e0 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f6493fff640 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c42f50 0x5564a8c493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f649a59c640 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c419e0 0x5564a8c4c0f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f648c09b0b0 tx=0x7f648c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8de171fb6131f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f6481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f6481ffb640 1 -- 172.21.15.78:0/3690008427 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c01e040 con 0x5564a8c419e0 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f6481ffb640 1 -- 172.21.15.78:0/3690008427 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f648c097a20 con 0x5564a8c419e0 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f64a3018740 1 -- 172.21.15.78:0/3690008427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64840042e0 con 0x5564a8c419e0 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f6481ffb640 1 -- 172.21.15.78:0/3690008427 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f648c09f760 con 0x5564a8c419e0 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f64a3018740 10 client.16473 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f64a3018740 1 -- 172.21.15.78:0/3690008427 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5564a8c5ae30 con 0x5564a8c419e0 2024-09-16T00:19:09.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.292+0000 7f6481ffb640 1 -- 172.21.15.78:0/3690008427 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64840042e0 con 0x5564a8c419e0 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f6481ffb640 1 -- 172.21.15.78:0/3690008427 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f648c09a800 con 0x5564a8c419e0 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f64a3018740 10 client.16473 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f64a3018740 10 client.16473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f64a3018740 20 client.16473 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f64a3018740 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5564a8c5c2f0 0x5564a8c5e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f64a3018740 4 client.16473 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f64a3018740 1 -- 172.21.15.78:0/3690008427 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564a8bb0860 con 0x5564a8c5c2f0 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f6493fff640 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5564a8c5c2f0 0x5564a8c5e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f6481ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.293+0000 7f6493fff640 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5564a8c5c2f0 0x5564a8c5e780 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44f32aa05d1b7f92 server_cookie=c3beeb8647b90002 in_seq=0 out_seq=0 2024-09-16T00:19:09.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f6481ffb640 1 -- 172.21.15.78:0/3690008427 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564a8bb0860 con 0x5564a8c5c2f0 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f6481ffb640 10 client.16473 handle_command_reply: tid=0 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 client.16473 shutdown 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 -- 172.21.15.78:0/3690008427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c419e0 msgr2=0x5564a8c4c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c419e0 0x5564a8c4c0f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f648c09b0b0 tx=0x7f648c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 -- 172.21.15.78:0/3690008427 shutdown_connections 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5564a8c5c2f0 0x5564a8c5e780 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 0x5564a8c49900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c42f50 0x5564a8c493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 --2- 172.21.15.78:0/3690008427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c419e0 0x5564a8c4c0f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f6481ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:19:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 -- 172.21.15.78:0/3690008427 >> 172.21.15.78:0/3690008427 conn(0x5564a8c06310 msgr2=0x5564a8c43d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 -- 172.21.15.78:0/3690008427 shutdown_connections 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.294+0000 7f64a3018740 1 -- 172.21.15.78:0/3690008427 wait complete. 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.295+0000 7f64a3018740 20 client.16473 trim_cache size 0 max 16384 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.307+0000 7f64a3018740 1 Processor -- start 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.307+0000 7f64a3018740 1 -- start start 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.307+0000 7f64a3018740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c419e0 0x5564a8c586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.307+0000 7f64a3018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c48c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.307+0000 7f64a3018740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c4c780 0x5564a8c4db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.307+0000 7f64a3018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1acad0 con 0x5564a8c42f50 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.307+0000 7f64a3018740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1ae610 con 0x5564a8c4c780 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.307+0000 7f64a3018740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1ad1a0 con 0x5564a8c419e0 2024-09-16T00:19:09.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c48c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c48c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56950/0 (socket says 172.21.15.78:56950) 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 -- 172.21.15.78:0/255350373 learned_addr learned my addr 172.21.15.78:0/255350373 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f649a59c640 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c419e0 0x5564a8c586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f649ad9d640 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c4c780 0x5564a8c4db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 -- 172.21.15.78:0/255350373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c419e0 msgr2=0x5564a8c586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c419e0 0x5564a8c586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 -- 172.21.15.78:0/255350373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c4c780 msgr2=0x5564a8c4db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c4c780 0x5564a8c4db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 -- 172.21.15.78:0/255350373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564a8c4bd10 con 0x5564a8c42f50 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f649a59c640 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c419e0 0x5564a8c586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:09.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6493fff640 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c48c60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f64840096c0 tx=0x7f6484031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72c251c09c3ddc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6483fff640 1 -- 172.21.15.78:0/255350373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648403e040 con 0x5564a8c42f50 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6483fff640 1 -- 172.21.15.78:0/255350373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6484050070 con 0x5564a8c42f50 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.308+0000 7f6483fff640 1 -- 172.21.15.78:0/255350373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484035040 con 0x5564a8c42f50 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 -- 172.21.15.78:0/255350373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 msgr2=0x5564a8c48c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c48c60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f64840096c0 tx=0x7f6484031650 comp rx=0 tx=0).stop 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 -- 172.21.15.78:0/255350373 shutdown_connections 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c4c780 0x5564a8c4db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c48c60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 --2- 172.21.15.78:0/255350373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c419e0 0x5564a8c586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 -- 172.21.15.78:0/255350373 >> 172.21.15.78:0/255350373 conn(0x5564a8c06310 msgr2=0x5564a8c47c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:09.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 -- 172.21.15.78:0/255350373 shutdown_connections 2024-09-16T00:19:09.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 -- 172.21.15.78:0/255350373 wait complete. 2024-09-16T00:19:09.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 -- start start 2024-09-16T00:19:09.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.309+0000 7f64a3018740 1 Processor -- start 2024-09-16T00:19:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f64a3018740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 0x5564a8c46eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f64a3018740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c54830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f64a3018740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 0x5564a8c54d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f64a3018740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1adf40 con 0x5564a8c42f50 2024-09-16T00:19:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f64a3018740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1ad870 con 0x5564a8c419e0 2024-09-16T00:19:09.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f64a3018740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f649c1aaab0 con 0x5564a8c4c780 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f64a3018740 10 client.0 _ll_register_callbacks cb 0x7f648c097740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c54830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c54830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56956/0 (socket says 172.21.15.78:56956) 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 -- 172.21.15.78:0/1013257 learned_addr learned my addr 172.21.15.78:0/1013257 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f649a59c640 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 0x5564a8c46eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f649ad9d640 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 0x5564a8c54d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 -- 172.21.15.78:0/1013257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 msgr2=0x5564a8c54d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 0x5564a8c54d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 -- 172.21.15.78:0/1013257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 msgr2=0x5564a8c46eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 0x5564a8c46eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 -- 172.21.15.78:0/1013257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564a8c63b00 con 0x5564a8c42f50 2024-09-16T00:19:09.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6493fff640 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c54830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6484031620 tx=0x7f64840316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ade75f3baab68c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6481ffb640 1 -- 172.21.15.78:0/1013257 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648403e070 con 0x5564a8c42f50 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f6481ffb640 1 -- 172.21.15.78:0/1013257 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6484003f50 con 0x5564a8c42f50 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.310+0000 7f64a3018740 1 -- 172.21.15.78:0/1013257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6484050070 con 0x5564a8c42f50 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f6481ffb640 1 -- 172.21.15.78:0/1013257 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6484035040 con 0x5564a8c42f50 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f64a3018740 10 client.16479 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f64a3018740 1 -- 172.21.15.78:0/1013257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5564a8cc7190 con 0x5564a8c42f50 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f6481ffb640 1 -- 172.21.15.78:0/1013257 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6484050070 con 0x5564a8c42f50 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f6481ffb640 1 -- 172.21.15.78:0/1013257 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f64840038f0 con 0x5564a8c42f50 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f64a3018740 10 client.16479 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:09.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f64a3018740 10 client.16479 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f64a3018740 20 client.16479 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f64a3018740 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5564a8cc7400 0x5564a8cc97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f64a3018740 4 client.16479 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f64a3018740 1 -- 172.21.15.78:0/1013257 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564a8baad70 con 0x5564a8cc7400 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.311+0000 7f649a59c640 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5564a8cc7400 0x5564a8cc97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f649a59c640 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5564a8cc7400 0x5564a8cc97f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47477bf4af1941e3 server_cookie=cb386e6a1ed38995 in_seq=0 out_seq=0 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f6481ffb640 10 client.16479 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f6481ffb640 1 -- 172.21.15.78:0/1013257 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564a8baad70 con 0x5564a8cc7400 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f6481ffb640 10 client.16479 handle_command_reply: tid=0 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 client.16479 shutdown 2024-09-16T00:19:09.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 -- 172.21.15.78:0/1013257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 msgr2=0x5564a8c54830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c54830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6484031620 tx=0x7f64840316c0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 -- 172.21.15.78:0/1013257 shutdown_connections 2024-09-16T00:19:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5564a8cc7400 0x5564a8cc97f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564a8c4c780 0x5564a8c54d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f6481ffb640 0 client.16479 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:19:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564a8c42f50 0x5564a8c54830 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 --2- 172.21.15.78:0/1013257 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564a8c419e0 0x5564a8c46eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 -- 172.21.15.78:0/1013257 >> 172.21.15.78:0/1013257 conn(0x5564a8c06310 msgr2=0x5564a8c47c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:09.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 -- 172.21.15.78:0/1013257 shutdown_connections 2024-09-16T00:19:09.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 1 -- 172.21.15.78:0/1013257 wait complete. 2024-09-16T00:19:09.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.312+0000 7f64a3018740 20 client.16479 trim_cache size 0 max 16384 2024-09-16T00:19:09.332 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:19:09.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 -- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64880a3240 msgr2=0x7f64880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64880a3240 0x7f64880a5700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6494009fd0 tx=0x7f6494009a10 comp rx=0 tx=0).stop 2024-09-16T00:19:09.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 -- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c100e20 msgr2=0x7f649c197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:09.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c100e20 0x7f649c197540 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f648c00b4a0 tx=0x7f648c00b960 comp rx=0 tx=0).stop 2024-09-16T00:19:09.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 -- 172.21.15.78:0/4212457969 shutdown_connections 2024-09-16T00:19:09.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64880a3240 0x7f64880a5700 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f649c102ef0 0x7f649c19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c102570 0x7f649c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 --2- 172.21.15.78:0/4212457969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f649c100e20 0x7f649c197540 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:09.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 -- 172.21.15.78:0/4212457969 >> 172.21.15.78:0/4212457969 conn(0x7f649c0fc7d0 msgr2=0x7f649c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:09.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 -- 172.21.15.78:0/4212457969 shutdown_connections 2024-09-16T00:19:09.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:09.313+0000 7f64a0c3f640 1 -- 172.21.15.78:0/4212457969 wait complete. 2024-09-16T00:19:09.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:09 smithi193 ceph-mon[35484]: pgmap v533: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.8 MiB/s wr, 299 op/s 2024-09-16T00:19:09.958 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:09 smithi148 ceph-mon[35599]: pgmap v533: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.8 MiB/s wr, 299 op/s 2024-09-16T00:19:10.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:09 smithi078 ceph-mon[31157]: pgmap v533: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.8 MiB/s wr, 299 op/s 2024-09-16T00:19:10.181 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:19:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:10.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 310.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:19:11.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 -- 172.21.15.78:0/4090319714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 msgr2=0x7f70dc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4090319714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 0x7f70dc10ec40 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009cd0 tx=0x7f70cc02a910 comp rx=0 tx=0).stop 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 -- 172.21.15.78:0/4090319714 shutdown_connections 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4090319714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 0x7f70dc10ec40 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4090319714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70dc1036e0 0x7f70dc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4090319714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc102d30 0x7f70dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 -- 172.21.15.78:0/4090319714 >> 172.21.15.78:0/4090319714 conn(0x7f70dc0fc7d0 msgr2=0x7f70dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 -- 172.21.15.78:0/4090319714 shutdown_connections 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 -- 172.21.15.78:0/4090319714 wait complete. 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.251+0000 7f70e3afb640 1 Processor -- start 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e3afb640 1 -- start start 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e3afb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70dc102d30 0x7f70dc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e3afb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc1036e0 0x7f70dc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e3afb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 0x7f70dc19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e3afb640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1120e0 con 0x7f70dc1082a0 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e3afb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc111f60 con 0x7f70dc1036e0 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e3afb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc112260 con 0x7f70dc102d30 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 0x7f70dc19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 0x7f70dc19cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56968/0 (socket says 172.21.15.78:56968) 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 -- 172.21.15.78:0/4190810459 learned_addr learned my addr 172.21.15.78:0/4190810459 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e1897640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70dc102d30 0x7f70dc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e1096640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc1036e0 0x7f70dc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 -- 172.21.15.78:0/4190810459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70dc102d30 msgr2=0x7f70dc198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70dc102d30 0x7f70dc198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 -- 172.21.15.78:0/4190810459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc1036e0 msgr2=0x7f70dc198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc1036e0 0x7f70dc198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 -- 172.21.15.78:0/4190810459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc19d5d0 con 0x7f70dc1082a0 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e1897640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70dc102d30 0x7f70dc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e1096640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc1036e0 0x7f70dc198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.252+0000 7f70e2098640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 0x7f70dc19cef0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005cc0 tx=0x7f70cc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.253+0000 7f70daffd640 1 -- 172.21.15.78:0/4190810459 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc031660 con 0x7f70dc1082a0 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.253+0000 7f70daffd640 1 -- 172.21.15.78:0/4190810459 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70cc031800 con 0x7f70dc1082a0 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.253+0000 7f70e3afb640 1 -- 172.21.15.78:0/4190810459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc19d860 con 0x7f70dc1082a0 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.253+0000 7f70daffd640 1 -- 172.21.15.78:0/4190810459 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc02fb50 con 0x7f70dc1082a0 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.253+0000 7f70e3afb640 1 -- 172.21.15.78:0/4190810459 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc1a51a0 con 0x7f70dc1082a0 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70daffd640 1 -- 172.21.15.78:0/4190810459 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f70cc02fd70 con 0x7f70dc1082a0 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70e5ed4740 1 Processor -- start 2024-09-16T00:19:11.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70e5ed4740 1 -- start start 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 0x562ad11f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad11f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad1204760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562ad115d2f0 con 0x562ad11f75d0 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ad1104cf0 con 0x562ad11f8b40 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.255+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ad11592b0 con 0x562ad1202370 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70daffd640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70c40a3240 0x7f70c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad11f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad11f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56976/0 (socket says 172.21.15.78:56976) 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 -- 172.21.15.78:0/1728072878 learned_addr learned my addr 172.21.15.78:0/1728072878 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1096640 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 0x562ad11f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70daffd640 1 -- 172.21.15.78:0/4190810459 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f70cc0f11a0 con 0x7f70dc1082a0 2024-09-16T00:19:11.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e2098640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70c40a3240 0x7f70c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1096640 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad1204760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 -- 172.21.15.78:0/1728072878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 msgr2=0x562ad1204760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad1204760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 -- 172.21.15.78:0/1728072878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 msgr2=0x562ad11f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 0x562ad11f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 -- 172.21.15.78:0/1728072878 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ad10e93a0 con 0x562ad11f75d0 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e1897640 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad11f7a10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f70d400bd40 tx=0x7f70d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a19c0a41933ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70e2098640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70c40a3240 0x7f70c40a5700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0b9e30 tx=0x7f70cc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70c3fff640 1 -- 172.21.15.78:0/1728072878 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4015070 con 0x562ad11f75d0 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70c3fff640 1 -- 172.21.15.78:0/1728072878 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d400be30 con 0x562ad11f75d0 2024-09-16T00:19:11.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.256+0000 7f70c3fff640 1 -- 172.21.15.78:0/1728072878 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4011a00 con 0x562ad11f75d0 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 -- 172.21.15.78:0/1728072878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 msgr2=0x562ad11f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad11f7a10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f70d400bd40 tx=0x7f70d400f730 comp rx=0 tx=0).stop 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 -- 172.21.15.78:0/1728072878 shutdown_connections 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad1204760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad11f7a10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/1728072878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 0x562ad11f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 -- 172.21.15.78:0/1728072878 >> 172.21.15.78:0/1728072878 conn(0x562ad11c0310 msgr2=0x562ad11f3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 -- 172.21.15.78:0/1728072878 shutdown_connections 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 -- 172.21.15.78:0/1728072878 wait complete. 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.257+0000 7f70e5ed4740 1 -- start start 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e5ed4740 1 Processor -- start 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 0x562ad1201d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f8b40 0x562ad11feff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562ad1205070 con 0x562ad11f8b40 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ad1118060 con 0x562ad11f75d0 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ad12051f0 con 0x562ad1202370 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e5ed4740 10 client.0 _ll_register_callbacks cb 0x562ad0e6b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1897640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f8b40 0x562ad11feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1897640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f8b40 0x562ad11feff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56978/0 (socket says 172.21.15.78:56978) 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1897640 1 -- 172.21.15.78:0/433008137 learned_addr learned my addr 172.21.15.78:0/433008137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1096640 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1096640 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 0x562ad1201d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1897640 1 -- 172.21.15.78:0/433008137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 msgr2=0x562ad11ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1897640 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1897640 1 -- 172.21.15.78:0/433008137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 msgr2=0x562ad1201d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:11.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1897640 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 0x562ad1201d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1897640 1 -- 172.21.15.78:0/433008137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ad0ece3c0 con 0x562ad11f8b40 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.258+0000 7f70e1096640 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e1897640 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f8b40 0x562ad11feff0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f70d4011620 tx=0x7f70d400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422a13efa85e9c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70c17fa640 1 -- 172.21.15.78:0/433008137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4028060 con 0x562ad11f8b40 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70c17fa640 1 -- 172.21.15.78:0/433008137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f70d4005310 con 0x562ad11f8b40 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 1 -- 172.21.15.78:0/433008137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d400be30 con 0x562ad11f8b40 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70c17fa640 1 -- 172.21.15.78:0/433008137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f70d402f050 con 0x562ad11f8b40 2024-09-16T00:19:11.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 10 client.16491 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 1 -- 172.21.15.78:0/433008137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562ad1210ae0 con 0x562ad11f8b40 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70c17fa640 1 -- 172.21.15.78:0/433008137 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d400be30 con 0x562ad11f8b40 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70c17fa640 1 -- 172.21.15.78:0/433008137 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f70d40049c0 con 0x562ad11f8b40 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 10 client.16491 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 20 client.16491 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ad1211ee0 0x562ad12142d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 4 client.16491 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.259+0000 7f70e5ed4740 1 -- 172.21.15.78:0/433008137 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ad116a860 con 0x562ad1211ee0 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.260+0000 7f70e1096640 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ad1211ee0 0x562ad12142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.260+0000 7f70e1096640 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ad1211ee0 0x562ad12142d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59a97a4da4cb04a server_cookie=dc32d4b0298b3ee1 in_seq=0 out_seq=0 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.260+0000 7f70c17fa640 10 client.16491 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:11.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70c17fa640 1 -- 172.21.15.78:0/433008137 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ad116a860 con 0x562ad1211ee0 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70c17fa640 10 client.16491 handle_command_reply: tid=0 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 client.16491 shutdown 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 -- 172.21.15.78:0/433008137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f8b40 msgr2=0x562ad11feff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f8b40 0x562ad11feff0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f70d4011620 tx=0x7f70d400bab0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 -- 172.21.15.78:0/433008137 shutdown_connections 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ad1211ee0 0x562ad12142d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f8b40 0x562ad11feff0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/433008137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 0x562ad1201d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70c17fa640 0 client.16491 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 -- 172.21.15.78:0/433008137 >> 172.21.15.78:0/433008137 conn(0x562ad11c0310 msgr2=0x562ad11f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 -- 172.21.15.78:0/433008137 shutdown_connections 2024-09-16T00:19:11.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 1 -- 172.21.15.78:0/433008137 wait complete. 2024-09-16T00:19:11.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.261+0000 7f70e5ed4740 20 client.16491 trim_cache size 0 max 16384 2024-09-16T00:19:11.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e5ed4740 1 Processor -- start 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e5ed4740 1 -- start start 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad1200f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 0x562ad11fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11fa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1aec10 con 0x562ad11f75d0 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b0750 con 0x562ad11f8b40 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1af2e0 con 0x562ad1202370 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad1200f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad1200f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56980/0 (socket says 172.21.15.78:56980) 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 -- 172.21.15.78:0/2347048186 learned_addr learned my addr 172.21.15.78:0/2347048186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1897640 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 0x562ad11fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 -- 172.21.15.78:0/2347048186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 msgr2=0x562ad11fa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11fa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 -- 172.21.15.78:0/2347048186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 msgr2=0x562ad11fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 0x562ad11fa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.275+0000 7f70e1096640 1 -- 172.21.15.78:0/2347048186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ad11ff950 con 0x562ad11f75d0 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.276+0000 7f70e1096640 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad1200f60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f70d0004700 tx=0x7f70d000a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aae82510323889f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.276+0000 7f70c37fe640 1 -- 172.21.15.78:0/2347048186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0027440 con 0x562ad11f75d0 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.276+0000 7f70c37fe640 1 -- 172.21.15.78:0/2347048186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d00168b0 con 0x562ad11f75d0 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.276+0000 7f70c37fe640 1 -- 172.21.15.78:0/2347048186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0014730 con 0x562ad11f75d0 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2347048186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 msgr2=0x562ad1200f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad1200f60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f70d0004700 tx=0x7f70d000a2b0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2347048186 shutdown_connections 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad1202370 0x562ad11fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f8b40 0x562ad11fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2347048186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad11f75d0 0x562ad1200f60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2347048186 >> 172.21.15.78:0/2347048186 conn(0x562ad11c0310 msgr2=0x562ad1200850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2347048186 shutdown_connections 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2347048186 wait complete. 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 -- start start 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.277+0000 7f70e5ed4740 1 Processor -- start 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 0x562ad120ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad11f8b40 0x562ad11fbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e5ed4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad1202370 0x562ad11fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b0080 con 0x562ad1202370 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1af9b0 con 0x562ad11f75d0 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e5ed4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1acbf0 con 0x562ad11f8b40 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e5ed4740 10 client.0 _ll_register_callbacks cb 0x7f70d000a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:11.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad1202370 0x562ad11fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad1202370 0x562ad11fc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56992/0 (socket says 172.21.15.78:56992) 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 -- 172.21.15.78:0/2001153429 learned_addr learned my addr 172.21.15.78:0/2001153429 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 0x562ad120ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1897640 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad11f8b40 0x562ad11fbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 -- 172.21.15.78:0/2001153429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad11f8b40 msgr2=0x562ad11fbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad11f8b40 0x562ad11fbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 -- 172.21.15.78:0/2001153429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 msgr2=0x562ad120ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 0x562ad120ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 -- 172.21.15.78:0/2001153429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ad11fcad0 con 0x562ad1202370 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70e1096640 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad1202370 0x562ad11fc2a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f70d000aee0 tx=0x7f70d0006da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5014aa73b933b8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:11.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.278+0000 7f70c17fa640 1 -- 172.21.15.78:0/2001153429 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d000a7f0 con 0x562ad1202370 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70c17fa640 1 -- 172.21.15.78:0/2001153429 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f70d0016d20 con 0x562ad1202370 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2001153429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d00168b0 con 0x562ad1202370 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70c17fa640 1 -- 172.21.15.78:0/2001153429 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f70d0027440 con 0x562ad1202370 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 10 client.16497 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2001153429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562ad127c8c0 con 0x562ad1202370 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70c17fa640 1 -- 172.21.15.78:0/2001153429 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d00168b0 con 0x562ad1202370 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70c17fa640 1 -- 172.21.15.78:0/2001153429 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f70d0025020 con 0x562ad1202370 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 10 client.16497 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 20 client.16497 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ad127cfa0 0x562ad127f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 4 client.16497 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:19:11.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2001153429 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ad1164d70 con 0x562ad127cfa0 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.279+0000 7f70e1897640 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ad127cfa0 0x562ad127f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70c17fa640 10 client.16497 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e1897640 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ad127cfa0 0x562ad127f390 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb22a270b14d43f server_cookie=5220d9660cbf6453 in_seq=0 out_seq=0 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70c17fa640 1 -- 172.21.15.78:0/2001153429 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ad1164d70 con 0x562ad127cfa0 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70c17fa640 10 client.16497 handle_command_reply: tid=0 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 client.16497 shutdown 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2001153429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad1202370 msgr2=0x562ad11fc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad1202370 0x562ad11fc2a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f70d000aee0 tx=0x7f70d0006da0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2001153429 shutdown_connections 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ad127cfa0 0x562ad127f390 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ad1202370 0x562ad11fc2a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ad11f8b40 0x562ad11fbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70c17fa640 0 client.16497 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 --2- 172.21.15.78:0/2001153429 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ad11f75d0 0x562ad120ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2001153429 >> 172.21.15.78:0/2001153429 conn(0x562ad11c0310 msgr2=0x562ad1201070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2001153429 shutdown_connections 2024-09-16T00:19:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.280+0000 7f70e5ed4740 1 -- 172.21.15.78:0/2001153429 wait complete. 2024-09-16T00:19:11.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e5ed4740 20 client.16497 trim_cache size 0 max 16384 2024-09-16T00:19:11.294 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:19:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 -- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70c40a3240 msgr2=0x7f70c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70c40a3240 0x7f70c40a5700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0b9e30 tx=0x7f70cc0095c0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 -- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 msgr2=0x7f70dc19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 0x7f70dc19cef0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f70cc005cc0 tx=0x7f70cc003f90 comp rx=0 tx=0).stop 2024-09-16T00:19:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 -- 172.21.15.78:0/4190810459 shutdown_connections 2024-09-16T00:19:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70c40a3240 0x7f70c40a5700 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f70dc1082a0 0x7f70dc19cef0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f70dc1036e0 0x7f70dc198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 --2- 172.21.15.78:0/4190810459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70dc102d30 0x7f70dc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:11.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 -- 172.21.15.78:0/4190810459 >> 172.21.15.78:0/4190810459 conn(0x7f70dc0fc7d0 msgr2=0x7f70dc106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:11.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 -- 172.21.15.78:0/4190810459 shutdown_connections 2024-09-16T00:19:11.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:11.281+0000 7f70e3afb640 1 -- 172.21.15.78:0/4190810459 wait complete. 2024-09-16T00:19:11.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:11 smithi193 ceph-mon[35484]: pgmap v534: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 250 op/s 2024-09-16T00:19:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:11 smithi148 ceph-mon[35599]: pgmap v534: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 250 op/s 2024-09-16T00:19:12.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:11 smithi078 ceph-mon[31157]: pgmap v534: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 250 op/s 2024-09-16T00:19:13.109 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:19:13.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:19:13.364 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:13.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:13 smithi078 ceph-mon[31157]: pgmap v535: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.2 MiB/s wr, 142 op/s 2024-09-16T00:19:13.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 -- 172.21.15.78:0/1941919450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c100cc0 msgr2=0x7fc42c106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 --2- 172.21.15.78:0/1941919450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c100cc0 0x7fc42c106800 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc414009900 tx=0x7fc41402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 -- 172.21.15.78:0/1941919450 shutdown_connections 2024-09-16T00:19:13.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 --2- 172.21.15.78:0/1941919450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc42c106f30 0x7fc42c10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 --2- 172.21.15.78:0/1941919450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c100cc0 0x7fc42c106800 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 --2- 172.21.15.78:0/1941919450 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42c100310 0x7fc42c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 -- 172.21.15.78:0/1941919450 >> 172.21.15.78:0/1941919450 conn(0x7fc42c077210 msgr2=0x7fc42c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 -- 172.21.15.78:0/1941919450 shutdown_connections 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.895+0000 7fc431611640 1 -- 172.21.15.78:0/1941919450 wait complete. 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc431611640 1 Processor -- start 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc431611640 1 -- start start 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc431611640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc42c100310 0x7fc42c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc431611640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42c100cc0 0x7fc42c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc431611640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c106f30 0x7fc42c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc431611640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c111f50 con 0x7fc42c106f30 2024-09-16T00:19:13.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc431611640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c111dd0 con 0x7fc42c100cc0 2024-09-16T00:19:13.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc431611640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1120d0 con 0x7fc42c100310 2024-09-16T00:19:13.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc42b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c106f30 0x7fc42c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc42b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c106f30 0x7fc42c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57014/0 (socket says 172.21.15.78:57014) 2024-09-16T00:19:13.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc42b7fe640 1 -- 172.21.15.78:0/773033744 learned_addr learned my addr 172.21.15.78:0/773033744 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:13.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.896+0000 7fc42affd640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc42c100310 0x7fc42c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42c100cc0 0x7fc42c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc42b7fe640 1 -- 172.21.15.78:0/773033744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc42c100310 msgr2=0x7fc42c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc42b7fe640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc42c100310 0x7fc42c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc42b7fe640 1 -- 172.21.15.78:0/773033744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42c100cc0 msgr2=0x7fc42c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc42b7fe640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42c100cc0 0x7fc42c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc42b7fe640 1 -- 172.21.15.78:0/773033744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c19d330 con 0x7fc42c106f30 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42c100cc0 0x7fc42c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc42b7fe640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c106f30 0x7fc42c19cc50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c970 tx=0x7fc41c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc423fff640 1 -- 172.21.15.78:0/773033744 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c007bb0 con 0x7fc42c106f30 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc423fff640 1 -- 172.21.15.78:0/773033744 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc41c007d50 con 0x7fc42c106f30 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc431611640 1 -- 172.21.15.78:0/773033744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c19d620 con 0x7fc42c106f30 2024-09-16T00:19:13.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc423fff640 1 -- 172.21.15.78:0/773033744 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c018800 con 0x7fc42c106f30 2024-09-16T00:19:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.897+0000 7fc431611640 1 -- 172.21.15.78:0/773033744 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c1a4f00 con 0x7fc42c106f30 2024-09-16T00:19:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.899+0000 7fc423fff640 1 -- 172.21.15.78:0/773033744 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc41c00fa50 con 0x7fc42c106f30 2024-09-16T00:19:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.899+0000 7fc4339ea740 1 Processor -- start 2024-09-16T00:19:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.899+0000 7fc4339ea740 1 -- start start 2024-09-16T00:19:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f4b60 0x5571b84f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8502130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc4339ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5571b845a280 con 0x5571b84ffd40 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc4339ea740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5571b8401b10 con 0x5571b84f5480 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc4339ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5571b8456240 con 0x5571b84f4b60 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8502130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8502130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57016/0 (socket says 172.21.15.78:57016) 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 -- 172.21.15.78:0/546319048 learned_addr learned my addr 172.21.15.78:0/546319048 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42affd640 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc423fff640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc40c0a3240 0x7fc40c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f4b60 0x5571b84f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42b7fe640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc40c0a3240 0x7fc40c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 -- 172.21.15.78:0/546319048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f4b60 msgr2=0x5571b84f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f4b60 0x5571b84f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 -- 172.21.15.78:0/546319048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 msgr2=0x5571b84ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42a7fc640 1 -- 172.21.15.78:0/546319048 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b840e1f0 con 0x5571b84ffd40 2024-09-16T00:19:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.900+0000 7fc42affd640 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:19:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.901+0000 7fc423fff640 1 -- 172.21.15.78:0/773033744 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc41c0cfd10 con 0x7fc42c106f30 2024-09-16T00:19:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.901+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8502130 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc414005ec0 tx=0x7fc414000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53939e28b2c59a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.901+0000 7fc4217fa640 1 -- 172.21.15.78:0/546319048 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414045070 con 0x5571b84ffd40 2024-09-16T00:19:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.901+0000 7fc4217fa640 1 -- 172.21.15.78:0/546319048 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc414031a00 con 0x5571b84ffd40 2024-09-16T00:19:13.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.901+0000 7fc4217fa640 1 -- 172.21.15.78:0/546319048 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414035040 con 0x5571b84ffd40 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- 172.21.15.78:0/546319048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 msgr2=0x5571b8502130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8502130 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc414005ec0 tx=0x7fc414000bc0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc42b7fe640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc40c0a3240 0x7fc40c0a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc41c09bf60 tx=0x7fc41c06dda0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- 172.21.15.78:0/546319048 shutdown_connections 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8502130 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 --2- 172.21.15.78:0/546319048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f4b60 0x5571b84f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- 172.21.15.78:0/546319048 >> 172.21.15.78:0/546319048 conn(0x5571b84f10f0 msgr2=0x5571b84f14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- 172.21.15.78:0/546319048 shutdown_connections 2024-09-16T00:19:13.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- 172.21.15.78:0/546319048 wait complete. 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- start start 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 Processor -- start 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b84fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 0x5571b84fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5571b8502910 con 0x5571b84f4b60 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5571b8414f10 con 0x5571b84f5480 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.902+0000 7fc4339ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5571b8502a90 con 0x5571b84ffd40 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc4339ea740 10 client.0 _ll_register_callbacks cb 0x5571b83dda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b84fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b84fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57030/0 (socket says 172.21.15.78:57030) 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 -- 172.21.15.78:0/4043865895 learned_addr learned my addr 172.21.15.78:0/4043865895 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42affd640 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 0x5571b84fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 -- 172.21.15.78:0/4043865895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 msgr2=0x5571b84fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 0x5571b84fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 -- 172.21.15.78:0/4043865895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 msgr2=0x5571b84fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 -- 172.21.15.78:0/4043865895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b81cb6a0 con 0x5571b84f4b60 2024-09-16T00:19:13.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b84fcca0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc414043040 tx=0x7fc414005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb6ef061d32e38b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc41a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc41a7fc640 1 -- 172.21.15.78:0/4043865895 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414051060 con 0x5571b84f4b60 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc41a7fc640 1 -- 172.21.15.78:0/4043865895 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc4140035f0 con 0x5571b84f4b60 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc4339ea740 1 -- 172.21.15.78:0/4043865895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc414031a00 con 0x5571b84f4b60 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc41a7fc640 1 -- 172.21.15.78:0/4043865895 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc414058050 con 0x5571b84f4b60 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc4339ea740 10 client.16509 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.903+0000 7fc4339ea740 1 -- 172.21.15.78:0/4043865895 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5571b850e590 con 0x5571b84f4b60 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc41a7fc640 1 -- 172.21.15.78:0/4043865895 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc414031a00 con 0x5571b84f4b60 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc41a7fc640 1 -- 172.21.15.78:0/4043865895 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc41404b030 con 0x5571b84f4b60 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc4339ea740 10 client.16509 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:13.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc4339ea740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc4339ea740 20 client.16509 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc4339ea740 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5571b850f8e0 0x5571b8511cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc4339ea740 4 client.16509 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc4339ea740 1 -- 172.21.15.78:0/4043865895 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571b84677f0 con 0x5571b850f8e0 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.904+0000 7fc42affd640 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5571b850f8e0 0x5571b8511cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc42affd640 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5571b850f8e0 0x5571b8511cd0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba016aa0fdfc0186 server_cookie=1750a614024c9759 in_seq=0 out_seq=0 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc41a7fc640 10 client.16509 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc41a7fc640 1 -- 172.21.15.78:0/4043865895 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571b84677f0 con 0x5571b850f8e0 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc41a7fc640 10 client.16509 handle_command_reply: tid=0 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 client.16509 shutdown 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 -- 172.21.15.78:0/4043865895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 msgr2=0x5571b84fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b84fcca0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc414043040 tx=0x7fc414005670 comp rx=0 tx=0).stop 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 -- 172.21.15.78:0/4043865895 shutdown_connections 2024-09-16T00:19:13.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5571b850f8e0 0x5571b8511cd0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 0x5571b84fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b84fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 --2- 172.21.15.78:0/4043865895 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b84fcca0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.905+0000 7fc4339ea740 1 -- 172.21.15.78:0/4043865895 >> 172.21.15.78:0/4043865895 conn(0x5571b84f10f0 msgr2=0x5571b84f6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:13.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.906+0000 7fc41a7fc640 0 client.16509 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:13.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.906+0000 7fc4339ea740 1 -- 172.21.15.78:0/4043865895 shutdown_connections 2024-09-16T00:19:13.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.906+0000 7fc4339ea740 1 -- 172.21.15.78:0/4043865895 wait complete. 2024-09-16T00:19:13.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.906+0000 7fc4339ea740 20 client.16509 trim_cache size 0 max 16384 2024-09-16T00:19:13.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc4339ea740 1 Processor -- start 2024-09-16T00:19:13.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc4339ea740 1 -- start start 2024-09-16T00:19:13.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f4b60 0x5571b84feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f5480 0x5571b8514d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8515240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc4339ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1aeae0 con 0x5571b84ffd40 2024-09-16T00:19:13.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc4339ea740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1b0620 con 0x5571b84f4b60 2024-09-16T00:19:13.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc4339ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1af1b0 con 0x5571b84f5480 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8515240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8515240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57038/0 (socket says 172.21.15.78:57038) 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 -- 172.21.15.78:0/1036289963 learned_addr learned my addr 172.21.15.78:0/1036289963 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f4b60 0x5571b84feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42affd640 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f5480 0x5571b8514d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 -- 172.21.15.78:0/1036289963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f5480 msgr2=0x5571b8514d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f5480 0x5571b8514d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 -- 172.21.15.78:0/1036289963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f4b60 msgr2=0x5571b84feb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f4b60 0x5571b84feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.923+0000 7fc42a7fc640 1 -- 172.21.15.78:0/1036289963 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b84ec830 con 0x5571b84ffd40 2024-09-16T00:19:13.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.924+0000 7fc42affd640 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f5480 0x5571b8514d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.924+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8515240 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc4140092a0 tx=0x7fc4140056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfb753363d129 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.924+0000 7fc420ff9640 1 -- 172.21.15.78:0/1036289963 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414062070 con 0x5571b84ffd40 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.924+0000 7fc420ff9640 1 -- 172.21.15.78:0/1036289963 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc414004420 con 0x5571b84ffd40 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.924+0000 7fc420ff9640 1 -- 172.21.15.78:0/1036289963 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414051030 con 0x5571b84ffd40 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.924+0000 7fc4339ea740 1 -- 172.21.15.78:0/1036289963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 msgr2=0x5571b8515240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.924+0000 7fc4339ea740 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8515240 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc4140092a0 tx=0x7fc4140056e0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 -- 172.21.15.78:0/1036289963 shutdown_connections 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84ffd40 0x5571b8515240 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84f5480 0x5571b8514d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 --2- 172.21.15.78:0/1036289963 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f4b60 0x5571b84feb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 -- 172.21.15.78:0/1036289963 >> 172.21.15.78:0/1036289963 conn(0x5571b84f10f0 msgr2=0x5571b84fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 -- 172.21.15.78:0/1036289963 shutdown_connections 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 -- 172.21.15.78:0/1036289963 wait complete. 2024-09-16T00:19:13.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 -- start start 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 Processor -- start 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b85723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b8572910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 0x5571b8576810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1aff50 con 0x5571b84f4b60 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1af880 con 0x5571b84f5480 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c1acac0 con 0x5571b84ffd40 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc4339ea740 10 client.0 _ll_register_callbacks cb 0x7fc414043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc42a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b85723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc42a7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b85723d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57054/0 (socket says 172.21.15.78:57054) 2024-09-16T00:19:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.925+0000 7fc42a7fc640 1 -- 172.21.15.78:0/316039468 learned_addr learned my addr 172.21.15.78:0/316039468 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc42a7fc640 1 -- 172.21.15.78:0/316039468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 msgr2=0x5571b8576810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc42affd640 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b8572910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 0x5571b8576810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc42a7fc640 1 -- 172.21.15.78:0/316039468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 msgr2=0x5571b8572910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b8572910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc42a7fc640 1 -- 172.21.15.78:0/316039468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b8576e80 con 0x5571b84f4b60 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc42a7fc640 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b85723d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc414060040 tx=0x7fc4140045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9652afddc4132c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc41a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc41a7fc640 1 -- 172.21.15.78:0/316039468 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414062040 con 0x5571b84f4b60 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc41a7fc640 1 -- 172.21.15.78:0/316039468 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc4140315b0 con 0x5571b84f4b60 2024-09-16T00:19:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc4339ea740 1 -- 172.21.15.78:0/316039468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc414004420 con 0x5571b84f4b60 2024-09-16T00:19:13.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc41a7fc640 1 -- 172.21.15.78:0/316039468 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc414051040 con 0x5571b84f4b60 2024-09-16T00:19:13.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc4339ea740 10 client.16515 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:13.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc4339ea740 1 -- 172.21.15.78:0/316039468 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5571b857f3b0 con 0x5571b84f4b60 2024-09-16T00:19:13.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.926+0000 7fc41a7fc640 1 -- 172.21.15.78:0/316039468 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc414004420 con 0x5571b84f4b60 2024-09-16T00:19:13.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc41a7fc640 1 -- 172.21.15.78:0/316039468 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc414065070 con 0x5571b84f4b60 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc4339ea740 10 client.16515 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc4339ea740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc4339ea740 20 client.16515 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc4339ea740 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5571b857f8d0 0x5571b8581cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc4339ea740 4 client.16515 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc4339ea740 1 -- 172.21.15.78:0/316039468 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571b8461d00 con 0x5571b857f8d0 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc42affd640 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5571b857f8d0 0x5571b8581cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc42affd640 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5571b857f8d0 0x5571b8581cc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ec8b2e759c74ea server_cookie=6d37feb639887ae2 in_seq=0 out_seq=0 2024-09-16T00:19:13.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.927+0000 7fc41a7fc640 10 client.16515 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:13.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:13 smithi193 ceph-mon[35484]: pgmap v535: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.2 MiB/s wr, 142 op/s 2024-09-16T00:19:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.928+0000 7fc41a7fc640 1 -- 172.21.15.78:0/316039468 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571b8461d00 con 0x5571b857f8d0 2024-09-16T00:19:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.928+0000 7fc41a7fc640 10 client.16515 handle_command_reply: tid=0 2024-09-16T00:19:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.928+0000 7fc4339ea740 1 client.16515 shutdown 2024-09-16T00:19:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.928+0000 7fc4339ea740 1 -- 172.21.15.78:0/316039468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 msgr2=0x5571b85723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.928+0000 7fc4339ea740 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b85723d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc414060040 tx=0x7fc4140045d0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.928+0000 7fc4339ea740 1 -- 172.21.15.78:0/316039468 shutdown_connections 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc4339ea740 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5571b857f8d0 0x5571b8581cc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc4339ea740 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b84ffd40 0x5571b8576810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc4339ea740 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5571b84f5480 0x5571b8572910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc41a7fc640 0 client.16515 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc4339ea740 1 --2- 172.21.15.78:0/316039468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5571b84f4b60 0x5571b85723d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc4339ea740 1 -- 172.21.15.78:0/316039468 >> 172.21.15.78:0/316039468 conn(0x5571b84f10f0 msgr2=0x5571b84f0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc4339ea740 1 -- 172.21.15.78:0/316039468 shutdown_connections 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc4339ea740 1 -- 172.21.15.78:0/316039468 wait complete. 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc4339ea740 20 client.16515 trim_cache size 0 max 16384 2024-09-16T00:19:13.937 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:19:13.938 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:19:13.938 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "30ec6092-d039-44e0-8550-618c14ea0ef3", 2024-09-16T00:19:13.938 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:19:13.938 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:19:13.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 -- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc40c0a3240 msgr2=0x7fc40c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc40c0a3240 0x7fc40c0a5700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc41c09bf60 tx=0x7fc41c06dda0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 -- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c106f30 msgr2=0x7fc42c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c106f30 0x7fc42c19cc50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c970 tx=0x7fc41c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 -- 172.21.15.78:0/773033744 shutdown_connections 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc40c0a3240 0x7fc40c0a5700 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc42c106f30 0x7fc42c19cc50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc42c100cc0 0x7fc42c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 --2- 172.21.15.78:0/773033744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc42c100310 0x7fc42c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.929+0000 7fc431611640 1 -- 172.21.15.78:0/773033744 >> 172.21.15.78:0/773033744 conn(0x7fc42c077210 msgr2=0x7fc42c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.930+0000 7fc431611640 1 -- 172.21.15.78:0/773033744 shutdown_connections 2024-09-16T00:19:13.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:13.930+0000 7fc431611640 1 -- 172.21.15.78:0/773033744 wait complete. 2024-09-16T00:19:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:13 smithi148 ceph-mon[35599]: pgmap v535: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.2 MiB/s wr, 142 op/s 2024-09-16T00:19:14.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:14 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:19:14.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:14 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:19:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:14 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:14 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:14.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:14 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:14 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:19:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:14 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:19:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:14 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:14 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:14.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:14 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:15.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:14 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:19:15.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:14 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:19:15.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:14 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:14 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:15.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:14 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:19:15.203 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:19:15.474 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:15.864 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 315.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:19:15.913 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:15 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:19:15.913 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:15 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:15.914 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:15 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:19:15.914 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:15 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:15.914 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:15 smithi078 ceph-mon[31157]: pgmap v536: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 932 KiB/s wr, 42 op/s 2024-09-16T00:19:15.914 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:15.914 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:15.914 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:19:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:15 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:19:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:15 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:15 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:19:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:15 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:15 smithi193 ceph-mon[35484]: pgmap v536: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 932 KiB/s wr, 42 op/s 2024-09-16T00:19:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:19:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:15 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:19:15.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:15 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:15.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:15 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:19:15.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:15 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:19:15.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:15 smithi148 ceph-mon[35599]: pgmap v536: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 932 KiB/s wr, 42 op/s 2024-09-16T00:19:15.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:19:15.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:19:15.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:19:16.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 -- 172.21.15.78:0/2414492353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c102790 msgr2=0x7f413c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 --2- 172.21.15.78:0/2414492353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c102790 0x7f413c1067e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4128009900 tx=0x7f412802a410 comp rx=0 tx=0).stop 2024-09-16T00:19:16.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 -- 172.21.15.78:0/2414492353 shutdown_connections 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 --2- 172.21.15.78:0/2414492353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413c106f10 0x7f413c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 --2- 172.21.15.78:0/2414492353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c102790 0x7f413c1067e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 --2- 172.21.15.78:0/2414492353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c101de0 0x7f413c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 -- 172.21.15.78:0/2414492353 >> 172.21.15.78:0/2414492353 conn(0x7f413c0fd8c0 msgr2=0x7f413c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 -- 172.21.15.78:0/2414492353 shutdown_connections 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.994+0000 7f414313f640 1 -- 172.21.15.78:0/2414492353 wait complete. 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f414313f640 1 Processor -- start 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f414313f640 1 -- start start 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f414313f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c101de0 0x7f413c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f414313f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102790 0x7f413c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f414313f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413c106f10 0x7f413c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f414313f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f413c111f90 con 0x7f413c101de0 2024-09-16T00:19:16.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f414313f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f413c111e10 con 0x7f413c106f10 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f414313f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f413c112110 con 0x7f413c102790 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c101de0 0x7f413c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c101de0 0x7f413c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57070/0 (socket says 172.21.15.78:57070) 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 -- 172.21.15.78:0/1999838635 learned_addr learned my addr 172.21.15.78:0/1999838635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f41416dc640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413c106f10 0x7f413c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f413bfff640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102790 0x7f413c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 -- 172.21.15.78:0/1999838635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102790 msgr2=0x7f413c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102790 0x7f413c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 -- 172.21.15.78:0/1999838635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413c106f10 msgr2=0x7f413c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413c106f10 0x7f413c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 -- 172.21.15.78:0/1999838635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c19d440 con 0x7f413c101de0 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f41416dc640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413c106f10 0x7f413c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.995+0000 7f4140edb640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c101de0 0x7f413c198490 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f413000b4a0 tx=0x7f413000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.996+0000 7f4139ffb640 1 -- 172.21.15.78:0/1999838635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130014070 con 0x7f413c101de0 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.996+0000 7f414313f640 1 -- 172.21.15.78:0/1999838635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c19d730 con 0x7f413c101de0 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.996+0000 7f4139ffb640 1 -- 172.21.15.78:0/1999838635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4130010af0 con 0x7f413c101de0 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.996+0000 7f4139ffb640 1 -- 172.21.15.78:0/1999838635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130017d10 con 0x7f413c101de0 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.996+0000 7f414313f640 1 -- 172.21.15.78:0/1999838635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c1a5010 con 0x7f413c101de0 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.998+0000 7f4139ffb640 1 -- 172.21.15.78:0/1999838635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4130017070 con 0x7f413c101de0 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4145518740 1 Processor -- start 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4145518740 1 -- start start 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4145518740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82934c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4145518740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934cdc0 0x55f829357140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4145518740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f829357680 0x55f829359a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4145518740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f8292b22e0 con 0x55f829357680 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4145518740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f829259ce0 con 0x55f82934c4a0 2024-09-16T00:19:16.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4145518740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f8292ae2a0 con 0x55f82934cdc0 2024-09-16T00:19:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4139ffb640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f412c0a3240 0x7f412c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f413bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f829357680 0x55f829359a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f413bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f829357680 0x55f829359a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57078/0 (socket says 172.21.15.78:57078) 2024-09-16T00:19:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f413bfff640 1 -- 172.21.15.78:0/1823633705 learned_addr learned my addr 172.21.15.78:0/1823633705 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f41416dc640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f412c0a3240 0x7f412c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4139ffb640 1 -- 172.21.15.78:0/1999838635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f41300cf100 con 0x7f413c101de0 2024-09-16T00:19:16.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f41416dc640 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82934c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:15.999+0000 7f4140edb640 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934cdc0 0x55f829357140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f413bfff640 1 -- 172.21.15.78:0/1823633705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934cdc0 msgr2=0x55f829357140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f413bfff640 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934cdc0 0x55f829357140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f413bfff640 1 -- 172.21.15.78:0/1823633705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 msgr2=0x55f82934c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f413bfff640 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82934c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f413bfff640 1 -- 172.21.15.78:0/1823633705 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f82923e440 con 0x55f829357680 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f41416dc640 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82934c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f413bfff640 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f829357680 0x55f829359a70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4128009900 tx=0x7f4128032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6bd920bb4c7da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f41416dc640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f412c0a3240 0x7f412c0a5700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4134005c40 tx=0x7f41340095b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f4126ffd640 1 -- 172.21.15.78:0/1823633705 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412803f070 con 0x55f829357680 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f4126ffd640 1 -- 172.21.15.78:0/1823633705 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41280317b0 con 0x55f829357680 2024-09-16T00:19:16.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.000+0000 7f4126ffd640 1 -- 172.21.15.78:0/1823633705 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128035070 con 0x55f829357680 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 -- 172.21.15.78:0/1823633705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f829357680 msgr2=0x55f829359a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f829357680 0x55f829359a70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4128009900 tx=0x7f4128032ce0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 -- 172.21.15.78:0/1823633705 shutdown_connections 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f829357680 0x55f829359a70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934cdc0 0x55f829357140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 --2- 172.21.15.78:0/1823633705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82934c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 -- 172.21.15.78:0/1823633705 >> 172.21.15.78:0/1823633705 conn(0x55f829348a30 msgr2=0x55f829348e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 -- 172.21.15.78:0/1823633705 shutdown_connections 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 -- 172.21.15.78:0/1823633705 wait complete. 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 -- start start 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.001+0000 7f4145518740 1 Processor -- start 2024-09-16T00:19:16.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f829356f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829357680 0x55f829354720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f82935a250 con 0x55f82934cdc0 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f82926d050 con 0x55f82934c4a0 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f82935a3d0 con 0x55f829357680 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 10 client.0 _ll_register_callbacks cb 0x55f8292ad160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57080/0 (socket says 172.21.15.78:57080) 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 -- 172.21.15.78:0/2025098395 learned_addr learned my addr 172.21.15.78:0/2025098395 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4140edb640 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f829356f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f41416dc640 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829357680 0x55f829354720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 -- 172.21.15.78:0/2025098395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829357680 msgr2=0x55f829354720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:16.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829357680 0x55f829354720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 -- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 msgr2=0x55f829356f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f829356f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 -- 172.21.15.78:0/2025098395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8290233c0 con 0x55f82934cdc0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4140edb640 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f829356f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f413bfff640 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293541e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f41280052e0 tx=0x7f4128004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc8a67b5d33c1a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f411bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f411bfff640 1 -- 172.21.15.78:0/2025098395 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412804a020 con 0x55f82934cdc0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f411bfff640 1 -- 172.21.15.78:0/2025098395 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f41280035f0 con 0x55f82934cdc0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 1 -- 172.21.15.78:0/2025098395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41280317b0 con 0x55f82934cdc0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f411bfff640 1 -- 172.21.15.78:0/2025098395 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4128055050 con 0x55f82934cdc0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 10 client.16527 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.002+0000 7f4145518740 1 -- 172.21.15.78:0/2025098395 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f829365b70 con 0x55f82934cdc0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f411bfff640 1 -- 172.21.15.78:0/2025098395 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41280317b0 con 0x55f82934cdc0 2024-09-16T00:19:16.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f411bfff640 1 -- 172.21.15.78:0/2025098395 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4128004cc0 con 0x55f82934cdc0 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f4145518740 10 client.16527 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f4145518740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f4145518740 20 client.16527 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f4145518740 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f829366ec0 0x55f8293692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f4145518740 4 client.16527 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f4145518740 1 -- 172.21.15.78:0/2025098395 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8292bf850 con 0x55f829366ec0 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.003+0000 7f4140edb640 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f829366ec0 0x55f8293692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f411bfff640 10 client.16527 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f4140edb640 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f829366ec0 0x55f8293692b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e149570026bbdf39 server_cookie=c0518f99541329a in_seq=0 out_seq=0 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f411bfff640 1 -- 172.21.15.78:0/2025098395 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8292bf850 con 0x55f829366ec0 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f411bfff640 10 client.16527 handle_command_reply: tid=0 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f4145518740 1 client.16527 shutdown 2024-09-16T00:19:16.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f4145518740 1 -- 172.21.15.78:0/2025098395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 msgr2=0x55f8293541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f4145518740 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293541e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f41280052e0 tx=0x7f4128004040 comp rx=0 tx=0).stop 2024-09-16T00:19:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f4145518740 1 -- 172.21.15.78:0/2025098395 shutdown_connections 2024-09-16T00:19:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.004+0000 7f4145518740 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f829366ec0 0x55f8293692b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.005+0000 7f4145518740 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829357680 0x55f829354720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.005+0000 7f4145518740 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293541e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.005+0000 7f4145518740 1 --2- 172.21.15.78:0/2025098395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f829356f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.005+0000 7f4145518740 1 -- 172.21.15.78:0/2025098395 >> 172.21.15.78:0/2025098395 conn(0x55f829348a30 msgr2=0x55f82934e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:16.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.005+0000 7f411bfff640 0 client.16527 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:16.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.005+0000 7f4145518740 1 -- 172.21.15.78:0/2025098395 shutdown_connections 2024-09-16T00:19:16.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.005+0000 7f4145518740 1 -- 172.21.15.78:0/2025098395 wait complete. 2024-09-16T00:19:16.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.005+0000 7f4145518740 20 client.16527 trim_cache size 0 max 16384 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.021+0000 7f4145518740 1 Processor -- start 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.021+0000 7f4145518740 1 -- start start 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f4145518740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f829348a30 0x55f8293634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f4145518740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934c4a0 0x55f82934f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f4145518740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f4145518740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1aeb50 con 0x55f82934cdc0 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f4145518740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1b0690 con 0x55f829348a30 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f4145518740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1af220 con 0x55f82934c4a0 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57084/0 (socket says 172.21.15.78:57084) 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 -- 172.21.15.78:0/1448620409 learned_addr learned my addr 172.21.15.78:0/1448620409 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:16.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f4140edb640 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f829348a30 0x55f8293634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f413bfff640 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934c4a0 0x55f82934f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 -- 172.21.15.78:0/1448620409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934c4a0 msgr2=0x55f82934f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934c4a0 0x55f82934f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 -- 172.21.15.78:0/1448620409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f829348a30 msgr2=0x55f8293634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f829348a30 0x55f8293634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 -- 172.21.15.78:0/1448620409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f82935a380 con 0x55f82934cdc0 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f4140edb640 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f829348a30 0x55f8293634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41416dc640 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293531c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4134009850 tx=0x7f4134005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cffee6fb93423f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41267fc640 1 -- 172.21.15.78:0/1448620409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413402c070 con 0x55f82934cdc0 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41267fc640 1 -- 172.21.15.78:0/1448620409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413401ddd0 con 0x55f82934cdc0 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.022+0000 7f41267fc640 1 -- 172.21.15.78:0/1448620409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134027a60 con 0x55f82934cdc0 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 -- 172.21.15.78:0/1448620409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 msgr2=0x55f8293531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293531c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4134009850 tx=0x7f4134005bd0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 -- 172.21.15.78:0/1448620409 shutdown_connections 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f8293531c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f82934c4a0 0x55f82934f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 --2- 172.21.15.78:0/1448620409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f829348a30 0x55f8293634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 -- 172.21.15.78:0/1448620409 >> 172.21.15.78:0/1448620409 conn(0x55f829354880 msgr2=0x55f829352ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 -- 172.21.15.78:0/1448620409 shutdown_connections 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 -- 172.21.15.78:0/1448620409 wait complete. 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.023+0000 7f4145518740 1 -- start start 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4145518740 1 Processor -- start 2024-09-16T00:19:16.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4145518740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829348a30 0x55f82935f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4145518740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82935fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4145518740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f82935a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4145518740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1affc0 con 0x55f82934cdc0 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4145518740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1af8f0 con 0x55f82934c4a0 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4145518740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1acb30 con 0x55f829348a30 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4145518740 10 client.0 _ll_register_callbacks cb 0x7f4128032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f41416dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f82935a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f41416dc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f82935a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57092/0 (socket says 172.21.15.78:57092) 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f41416dc640 1 -- 172.21.15.78:0/3465887964 learned_addr learned my addr 172.21.15.78:0/3465887964 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4140edb640 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829348a30 0x55f82935f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f413bfff640 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82935fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4140edb640 1 -- 172.21.15.78:0/3465887964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 msgr2=0x55f82935fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4140edb640 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82935fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4140edb640 1 -- 172.21.15.78:0/3465887964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 msgr2=0x55f82935a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4140edb640 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f82935a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f4140edb640 1 -- 172.21.15.78:0/3465887964 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f829348860 con 0x55f829348a30 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.024+0000 7f41416dc640 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f82935a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f4140edb640 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829348a30 0x55f82935f630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f413009b0c0 tx=0x7f4130017f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadbc6d01a4748b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f411bfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f411bfff640 1 -- 172.21.15.78:0/3465887964 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413001e040 con 0x55f829348a30 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f411bfff640 1 -- 172.21.15.78:0/3465887964 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4130017360 con 0x55f829348a30 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f4145518740 1 -- 172.21.15.78:0/3465887964 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f413401ddd0 con 0x55f829348a30 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f411bfff640 1 -- 172.21.15.78:0/3465887964 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4130097a30 con 0x55f829348a30 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f4145518740 10 client.25783 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f4145518740 1 -- 172.21.15.78:0/3465887964 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f8293d1e30 con 0x55f829348a30 2024-09-16T00:19:16.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f411bfff640 1 -- 172.21.15.78:0/3465887964 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413401ddd0 con 0x55f829348a30 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f411bfff640 1 -- 172.21.15.78:0/3465887964 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f413009a810 con 0x55f829348a30 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f4145518740 10 client.25783 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f4145518740 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.025+0000 7f4145518740 20 client.25783 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f829357680 0x55f8293d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 4 client.25783 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 -- 172.21.15.78:0/3465887964 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8292b9d60 con 0x55f829357680 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f413bfff640 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f829357680 0x55f8293d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f411bfff640 10 client.25783 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f413bfff640 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f829357680 0x55f8293d4540 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2080ac263d49d99f server_cookie=35b2c56680b13d82 in_seq=0 out_seq=0 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f411bfff640 1 -- 172.21.15.78:0/3465887964 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f8292b9d60 con 0x55f829357680 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f411bfff640 10 client.25783 handle_command_reply: tid=0 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 client.25783 shutdown 2024-09-16T00:19:16.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 -- 172.21.15.78:0/3465887964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829348a30 msgr2=0x55f82935f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829348a30 0x55f82935f630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f413009b0c0 tx=0x7f4130017f90 comp rx=0 tx=0).stop 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 -- 172.21.15.78:0/3465887964 shutdown_connections 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f829357680 0x55f8293d4540 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f82934cdc0 0x55f82935a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f411bfff640 0 client.25783 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f82934c4a0 0x55f82935fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 --2- 172.21.15.78:0/3465887964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f829348a30 0x55f82935f630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.026+0000 7f4145518740 1 -- 172.21.15.78:0/3465887964 >> 172.21.15.78:0/3465887964 conn(0x55f829354880 msgr2=0x55f82934f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f4145518740 1 -- 172.21.15.78:0/3465887964 shutdown_connections 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f4145518740 1 -- 172.21.15.78:0/3465887964 wait complete. 2024-09-16T00:19:16.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f4145518740 20 client.25783 trim_cache size 0 max 16384 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout: "30ec6092-d039-44e0-8550-618c14ea0ef3": { 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "30ec6092-d039-44e0-8550-618c14ea0ef3", 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:16.037 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:19:16.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 -- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f412c0a3240 msgr2=0x7f412c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f412c0a3240 0x7f412c0a5700 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4134005c40 tx=0x7f41340095b0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 -- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c101de0 msgr2=0x7f413c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:16.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c101de0 0x7f413c198490 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f413000b4a0 tx=0x7f413000b960 comp rx=0 tx=0).stop 2024-09-16T00:19:16.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 -- 172.21.15.78:0/1999838635 shutdown_connections 2024-09-16T00:19:16.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f412c0a3240 0x7f412c0a5700 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f413c106f10 0x7f413c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102790 0x7f413c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 --2- 172.21.15.78:0/1999838635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f413c101de0 0x7f413c198490 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:16.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 -- 172.21.15.78:0/1999838635 >> 172.21.15.78:0/1999838635 conn(0x7f413c0fd8c0 msgr2=0x7f413c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:16.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 -- 172.21.15.78:0/1999838635 shutdown_connections 2024-09-16T00:19:16.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:16.027+0000 7f414313f640 1 -- 172.21.15.78:0/1999838635 wait complete. 2024-09-16T00:19:17.052 INFO:tasks.cephfs.filesystem:scrub status for tag:30ec6092-d039-44e0-8550-618c14ea0ef3 - {'path': '/', 'tag': '30ec6092-d039-44e0-8550-618c14ea0ef3', 'options': 'recursive,force'} 2024-09-16T00:19:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:17 smithi193 ceph-mon[35484]: pgmap v537: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-16T00:19:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:17 smithi148 ceph-mon[35599]: pgmap v537: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-16T00:19:18.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:17 smithi078 ceph-mon[31157]: pgmap v537: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.6 KiB/s wr, 13 op/s 2024-09-16T00:19:19.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:19 smithi193 ceph-mon[35484]: pgmap v538: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:19:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:19 smithi148 ceph-mon[35599]: pgmap v538: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:19:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:19 smithi078 ceph-mon[31157]: pgmap v538: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:19:20.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 320.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:19:21.209 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:19:21.471 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:21.889 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:21 smithi078 ceph-mon[31157]: pgmap v539: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-16T00:19:21.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:21 smithi193 ceph-mon[35484]: pgmap v539: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-16T00:19:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:21 smithi148 ceph-mon[35599]: pgmap v539: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.968+0000 7f1accf8a640 1 -- 172.21.15.78:0/3768747977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 msgr2=0x7f1ac810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.968+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3768747977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 0x7f1ac810ea80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009ca0 tx=0x7f1ac002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.968+0000 7f1accf8a640 1 -- 172.21.15.78:0/3768747977 shutdown_connections 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.968+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3768747977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 0x7f1ac810ea80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.968+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3768747977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8102780 0x7f1ac81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.968+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3768747977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8101dd0 0x7f1ac81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.969+0000 7f1accf8a640 1 -- 172.21.15.78:0/3768747977 >> 172.21.15.78:0/3768747977 conn(0x7f1ac80fd8b0 msgr2=0x7f1ac80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.969+0000 7f1accf8a640 1 -- 172.21.15.78:0/3768747977 shutdown_connections 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.969+0000 7f1accf8a640 1 -- 172.21.15.78:0/3768747977 wait complete. 2024-09-16T00:19:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.969+0000 7f1accf8a640 1 Processor -- start 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.969+0000 7f1accf8a640 1 -- start start 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.969+0000 7f1accf8a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8101dd0 0x7f1ac81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1accf8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102780 0x7f1ac8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1accf8a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 0x7f1ac819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1accf8a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111f60 con 0x7f1ac8106f00 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1accf8a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111de0 con 0x7f1ac8102780 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1accf8a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81120e0 con 0x7f1ac8101dd0 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 0x7f1ac819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 0x7f1ac819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48190/0 (socket says 172.21.15.78:48190) 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 -- 172.21.15.78:0/3080290691 learned_addr learned my addr 172.21.15.78:0/3080290691 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac5d9b640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102780 0x7f1ac8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac659c640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8101dd0 0x7f1ac81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 -- 172.21.15.78:0/3080290691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8101dd0 msgr2=0x7f1ac81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8101dd0 0x7f1ac81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 -- 172.21.15.78:0/3080290691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102780 msgr2=0x7f1ac8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102780 0x7f1ac8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 -- 172.21.15.78:0/3080290691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac819d390 con 0x7f1ac8106f00 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ac6d9d640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 0x7f1ac819ccb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0031a20 tx=0x7f1ac0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ab77fe640 1 -- 172.21.15.78:0/3080290691 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0003c30 con 0x7f1ac8106f00 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac819d620 con 0x7f1ac8106f00 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ab77fe640 1 -- 172.21.15.78:0/3080290691 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac0003dd0 con 0x7f1ac8106f00 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1ab77fe640 1 -- 172.21.15.78:0/3080290691 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac002fc40 con 0x7f1ac8106f00 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.970+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a4f60 con 0x7f1ac8106f00 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.971+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac81021b0 con 0x7f1ac8106f00 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.973+0000 7f1ab77fe640 1 -- 172.21.15.78:0/3080290691 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1ac0004050 con 0x7f1ac8106f00 2024-09-16T00:19:22.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.973+0000 7f1ab77fe640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab00a3240 0x7f1ab00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.973+0000 7f1ac659c640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab00a3240 0x7f1ab00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.973+0000 7f1ab77fe640 1 -- 172.21.15.78:0/3080290691 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1ac00f0ff0 con 0x7f1ac8106f00 2024-09-16T00:19:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.974+0000 7f1ac659c640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab00a3240 0x7f1ab00a5700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009b60 tx=0x7f1ab8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:21.976+0000 7f1ab77fe640 1 -- 172.21.15.78:0/3080290691 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ac00bda40 con 0x7f1ac8106f00 2024-09-16T00:19:22.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.178+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ac810a140 con 0x7f1ac8106f00 2024-09-16T00:19:22.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.179+0000 7f1ab77fe640 1 -- 172.21.15.78:0/3080290691 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f1ac00bd340 con 0x7f1ac8106f00 2024-09-16T00:19:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:19:22.180 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:19:22.180 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab00a3240 msgr2=0x7f1ab00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab00a3240 0x7f1ab00a5700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009b60 tx=0x7f1ab8005f90 comp rx=0 tx=0).stop 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 msgr2=0x7f1ac819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 0x7f1ac819ccb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0031a20 tx=0x7f1ac0031a50 comp rx=0 tx=0).stop 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 shutdown_connections 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ab00a3240 0x7f1ab00a5700 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac8106f00 0x7f1ac819ccb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ac8102780 0x7f1ac8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:22.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 --2- 172.21.15.78:0/3080290691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ac8101dd0 0x7f1ac81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:22.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 >> 172.21.15.78:0/3080290691 conn(0x7f1ac80fd8b0 msgr2=0x7f1ac80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:22.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 shutdown_connections 2024-09-16T00:19:22.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:22.181+0000 7f1accf8a640 1 -- 172.21.15.78:0/3080290691 wait complete. 2024-09-16T00:19:22.928 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:22 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3080290691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:22.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:22 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3080290691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:22.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:22 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3080290691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:22.962 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id ceb33b92 --release --await' 2024-09-16T00:19:22.962 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id ceb33b92 --release --await 2024-09-16T00:19:23.212 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:23.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 -- 172.21.15.78:0/1461705931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 msgr2=0x7f2f2810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/1461705931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 0x7f2f2810eaa0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009ca0 tx=0x7f2f2002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 -- 172.21.15.78:0/1461705931 shutdown_connections 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/1461705931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 0x7f2f2810eaa0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/1461705931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f281027a0 0x7f2f281067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/1461705931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f28101df0 0x7f2f281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 -- 172.21.15.78:0/1461705931 >> 172.21.15.78:0/1461705931 conn(0x7f2f280fd8f0 msgr2=0x7f2f280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 -- 172.21.15.78:0/1461705931 shutdown_connections 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.768+0000 7f2f2cc30640 1 -- 172.21.15.78:0/1461705931 wait complete. 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2cc30640 1 Processor -- start 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2cc30640 1 -- start start 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2cc30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f28101df0 0x7f2f28198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2cc30640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f281027a0 0x7f2f28198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2cc30640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 0x7f2f2819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2cc30640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28111fa0 con 0x7f2f28106f20 2024-09-16T00:19:23.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2cc30640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28111e20 con 0x7f2f28101df0 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2cc30640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28112120 con 0x7f2f281027a0 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f26d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 0x7f2f2819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f26d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 0x7f2f2819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48218/0 (socket says 172.21.15.78:48218) 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f26d9d640 1 -- 172.21.15.78:0/2988422756 learned_addr learned my addr 172.21.15.78:0/2988422756 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f2659c640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f28101df0 0x7f2f28198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f281027a0 0x7f2f28198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f26d9d640 1 -- 172.21.15.78:0/2988422756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f281027a0 msgr2=0x7f2f28198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f281027a0 0x7f2f28198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f26d9d640 1 -- 172.21.15.78:0/2988422756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f28101df0 msgr2=0x7f2f28198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f28101df0 0x7f2f28198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f26d9d640 1 -- 172.21.15.78:0/2988422756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2819d3f0 con 0x7f2f28106f20 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.769+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f281027a0 0x7f2f28198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:23.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.770+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 0x7f2f2819cd10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2f20031a20 tx=0x7f2f20031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.770+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/2988422756 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20003c30 con 0x7f2f28106f20 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.770+0000 7f2f2cc30640 1 -- 172.21.15.78:0/2988422756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2819d680 con 0x7f2f28106f20 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.770+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/2988422756 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f20003dd0 con 0x7f2f28106f20 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.770+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/2988422756 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2002fcf0 con 0x7f2f28106f20 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.770+0000 7f2f2cc30640 1 -- 172.21.15.78:0/2988422756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f281a4fc0 con 0x7f2f28106f20 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.772+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/2988422756 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2f20004050 con 0x7f2f28106f20 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.772+0000 7f2f2f009740 1 Processor -- start 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2f009740 1 -- start start 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f1f7fe640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f140a3240 0x7f2f140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b022320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b031110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b033a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2f009740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a92af87620 con 0x55a92b021f40 2024-09-16T00:19:23.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2f009740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a92af2ee50 con 0x55a92b031650 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2f009740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a92af835e0 con 0x55a92b022860 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f140a3240 0x7f2f140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f1f7fe640 1 -- 172.21.15.78:0/2988422756 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2f200f1180 con 0x7f2f28106f20 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b022320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f26d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b033a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b022320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48234/0 (socket says 172.21.15.78:48234) 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2659c640 1 -- 172.21.15.78:0/3900085732 learned_addr learned my addr 172.21.15.78:0/3900085732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b031110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2659c640 1 -- 172.21.15.78:0/3900085732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 msgr2=0x55a92b031110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2659c640 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b031110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2659c640 1 -- 172.21.15.78:0/3900085732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 msgr2=0x55a92b033a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2659c640 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b033a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f2659c640 1 -- 172.21.15.78:0/3900085732 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92af1fd40 con 0x55a92b021f40 2024-09-16T00:19:23.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.773+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b033a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.774+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f140a3240 0x7f2f140a5700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006fd0 tx=0x7f2f10007870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.774+0000 7f2f2659c640 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b022320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800bd10 tx=0x7f2f180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54b5ee6f95c6aed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.774+0000 7f2f1cff9640 1 -- 172.21.15.78:0/3900085732 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18004030 con 0x55a92b021f40 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.774+0000 7f2f1cff9640 1 -- 172.21.15.78:0/3900085732 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f180041d0 con 0x55a92b021f40 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.774+0000 7f2f1cff9640 1 -- 172.21.15.78:0/3900085732 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18011800 con 0x55a92b021f40 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 -- 172.21.15.78:0/3900085732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 msgr2=0x55a92b022320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b022320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800bd10 tx=0x7f2f180107a0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 -- 172.21.15.78:0/3900085732 shutdown_connections 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b033a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b031110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 --2- 172.21.15.78:0/3900085732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b022320 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 -- 172.21.15.78:0/3900085732 >> 172.21.15.78:0/3900085732 conn(0x55a92b01e5f0 msgr2=0x55a92b01e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 -- 172.21.15.78:0/3900085732 shutdown_connections 2024-09-16T00:19:23.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 -- 172.21.15.78:0/3900085732 wait complete. 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 -- start start 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.775+0000 7f2f2f009740 1 Processor -- start 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b02e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b022860 0x55a92b02eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b031650 0x55a92b02eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2f009740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a92b034220 con 0x55a92b021f40 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2f009740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a92af0ff10 con 0x55a92b022860 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2f009740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a92b0343a0 con 0x55a92b031650 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2f009740 10 client.0 _ll_register_callbacks cb 0x55a92af33db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b02e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b02e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48246/0 (socket says 172.21.15.78:48246) 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 -- 172.21.15.78:0/1254091758 learned_addr learned my addr 172.21.15.78:0/1254091758 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b022860 0x55a92b02eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b031650 0x55a92b02eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 -- 172.21.15.78:0/1254091758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b031650 msgr2=0x55a92b02eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b031650 0x55a92b02eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 -- 172.21.15.78:0/1254091758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b022860 msgr2=0x55a92b02eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b022860 0x55a92b02eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 -- 172.21.15.78:0/1254091758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92ac96580 con 0x55a92b021f40 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b031650 0x55a92b02eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2659c640 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b02e570 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2f18000c00 tx=0x7f2f18004010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ea77bcbb0fd0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1254091758 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18029060 con 0x55a92b021f40 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1254091758 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2f18005040 con 0x55a92b021f40 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.776+0000 7f2f2f009740 1 -- 172.21.15.78:0/1254091758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f180041d0 con 0x55a92b021f40 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1254091758 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f18030050 con 0x55a92b021f40 2024-09-16T00:19:23.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f2f009740 10 client.16551 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f2f009740 1 -- 172.21.15.78:0/1254091758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a92ad60960 con 0x55a92b021f40 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1254091758 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f180041d0 con 0x55a92b021f40 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1254091758 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2f180046c0 con 0x55a92b021f40 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f2f009740 10 client.16551 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f2f009740 10 client.16551 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f2f009740 20 client.16551 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a92b0411d0 0x55a92b0435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f2f009740 4 client.16551 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f2f009740 1 -- 172.21.15.78:0/1254091758 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a92af94b90 con 0x55a92b0411d0 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.777+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a92b0411d0 0x55a92b0435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f0dffb640 10 client.16551 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a92b0411d0 0x55a92b0435c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769e59c0d1b576b5 server_cookie=76ba80df41dd6123 in_seq=0 out_seq=0 2024-09-16T00:19:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1254091758 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a92af94b90 con 0x55a92b0411d0 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f0dffb640 10 client.16551 handle_command_reply: tid=0 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f2f009740 1 client.16551 shutdown 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f2f009740 1 -- 172.21.15.78:0/1254091758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 msgr2=0x55a92b02e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b02e570 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2f18000c00 tx=0x7f2f18004010 comp rx=0 tx=0).stop 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f2f009740 1 -- 172.21.15.78:0/1254091758 shutdown_connections 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a92b0411d0 0x55a92b0435c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b031650 0x55a92b02eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b022860 0x55a92b02eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1254091758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b02e570 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.778+0000 7f2f0dffb640 0 client.16551 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.779+0000 7f2f2f009740 1 -- 172.21.15.78:0/1254091758 >> 172.21.15.78:0/1254091758 conn(0x55a92b01e5f0 msgr2=0x55a92b024350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.779+0000 7f2f2f009740 1 -- 172.21.15.78:0/1254091758 shutdown_connections 2024-09-16T00:19:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.779+0000 7f2f2f009740 1 -- 172.21.15.78:0/1254091758 wait complete. 2024-09-16T00:19:23.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.779+0000 7f2f2f009740 20 client.16551 trim_cache size 0 max 16384 2024-09-16T00:19:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.795+0000 7f2f2f009740 1 Processor -- start 2024-09-16T00:19:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.795+0000 7f2f2f009740 1 -- start start 2024-09-16T00:19:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b030440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b048bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b049100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2f009740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ac9f0 con 0x55a92b021f40 2024-09-16T00:19:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2f009740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ae530 con 0x55a92b031650 2024-09-16T00:19:23.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2f009740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ad0c0 con 0x55a92b022860 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b030440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b030440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48260/0 (socket says 172.21.15.78:48260) 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 -- 172.21.15.78:0/761315163 learned_addr learned my addr 172.21.15.78:0/761315163 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b049100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b048bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 -- 172.21.15.78:0/761315163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 msgr2=0x55a92b048bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b048bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 -- 172.21.15.78:0/761315163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 msgr2=0x55a92b049100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b049100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 -- 172.21.15.78:0/761315163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92b01e090 con 0x55a92b021f40 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b048bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f2659c640 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b030440 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2f18004ac0 tx=0x7f2f180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a446b6dfac1aa93 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.796+0000 7f2f0ffff640 1 -- 172.21.15.78:0/761315163 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18039070 con 0x55a92b021f40 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f0ffff640 1 -- 172.21.15.78:0/761315163 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1800b460 con 0x55a92b021f40 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f0ffff640 1 -- 172.21.15.78:0/761315163 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18029040 con 0x55a92b021f40 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 -- 172.21.15.78:0/761315163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 msgr2=0x55a92b030440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b030440 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2f18004ac0 tx=0x7f2f180107a0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 -- 172.21.15.78:0/761315163 shutdown_connections 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b049100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b048bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 --2- 172.21.15.78:0/761315163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b030440 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 -- 172.21.15.78:0/761315163 >> 172.21.15.78:0/761315163 conn(0x55a92b01e5f0 msgr2=0x55a92b01d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 -- 172.21.15.78:0/761315163 shutdown_connections 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.797+0000 7f2f2f009740 1 -- 172.21.15.78:0/761315163 wait complete. 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 1 -- start start 2024-09-16T00:19:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 1 Processor -- start 2024-09-16T00:19:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b039aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b035e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b036370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ade60 con 0x55a92b021f40 2024-09-16T00:19:23.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281ad790 con 0x55a92b031650 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f281aa9d0 con 0x55a92b022860 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2f009740 10 client.0 _ll_register_callbacks cb 0x7f2f1800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b039aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b039aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48270/0 (socket says 172.21.15.78:48270) 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2659c640 1 -- 172.21.15.78:0/1019036867 learned_addr learned my addr 172.21.15.78:0/1019036867 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b036370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b035e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2659c640 1 -- 172.21.15.78:0/1019036867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 msgr2=0x55a92b035e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:23.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2659c640 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b035e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2659c640 1 -- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 msgr2=0x55a92b036370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2659c640 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b036370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f2659c640 1 -- 172.21.15.78:0/1019036867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92b0368b0 con 0x55a92b021f40 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.798+0000 7f2f26d9d640 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b036370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2659c640 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b039aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2f18004130 tx=0x7f2f18004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd441233cb1659b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1019036867 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18039070 con 0x55a92b021f40 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1019036867 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2f180047a0 con 0x55a92b021f40 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 1 -- 172.21.15.78:0/1019036867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f1800b460 con 0x55a92b021f40 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1019036867 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f18029070 con 0x55a92b021f40 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 10 client.16557 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:23.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 1 -- 172.21.15.78:0/1019036867 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a92b0ac8f0 con 0x55a92b021f40 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1019036867 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1800b460 con 0x55a92b021f40 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1019036867 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2f1803eb80 con 0x55a92b021f40 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 10 client.16557 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 10 client.16557 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 20 client.16557 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a92b0acef0 0x55a92b0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 4 client.16557 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ceb33b92", "release": true, "await": true}] 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.799+0000 7f2f2f009740 1 -- 172.21.15.78:0/1019036867 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ceb33b92", "release": true, "await": true}) -- 0x55a92af8f0a0 con 0x55a92b0acef0 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.800+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a92b0acef0 0x55a92b0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.800+0000 7f2f25d9b640 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a92b0acef0 0x55a92b0af2e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84eaea98a1f62d77 server_cookie=f5a32d89817cd7b in_seq=0 out_seq=0 2024-09-16T00:19:23.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.800+0000 7f2f0dffb640 10 client.16557 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:23.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.858+0000 7f2f0dffb640 1 -- 172.21.15.78:0/1019036867 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a92af8f0a0 con 0x55a92b0acef0 2024-09-16T00:19:23.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.858+0000 7f2f0dffb640 10 client.16557 handle_command_reply: tid=0 2024-09-16T00:19:23.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 client.16557 shutdown 2024-09-16T00:19:23.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 -- 172.21.15.78:0/1019036867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 msgr2=0x55a92b039aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b039aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2f18004130 tx=0x7f2f18004160 comp rx=0 tx=0).stop 2024-09-16T00:19:23.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 -- 172.21.15.78:0/1019036867 shutdown_connections 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a92b0acef0 0x55a92b0af2e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a92b031650 0x55a92b036370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a92b022860 0x55a92b035e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 --2- 172.21.15.78:0/1019036867 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a92b021f40 0x55a92b039aa0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 -- 172.21.15.78:0/1019036867 >> 172.21.15.78:0/1019036867 conn(0x55a92b01e5f0 msgr2=0x55a92b023a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f0dffb640 0 client.16557 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 -- 172.21.15.78:0/1019036867 shutdown_connections 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 1 -- 172.21.15.78:0/1019036867 wait complete. 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.859+0000 7f2f2f009740 20 client.16557 trim_cache size 0 max 16384 2024-09-16T00:19:23.861 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 24, 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "ceb33b92": { 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 24, 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 145.3, 2024-09-16T00:19:23.862 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:19:23.863 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:19:23.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 -- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f140a3240 msgr2=0x7f2f140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f140a3240 0x7f2f140a5700 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006fd0 tx=0x7f2f10007870 comp rx=0 tx=0).stop 2024-09-16T00:19:23.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 -- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 msgr2=0x7f2f2819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 0x7f2f2819cd10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2f20031a20 tx=0x7f2f20031a50 comp rx=0 tx=0).stop 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 -- 172.21.15.78:0/2988422756 shutdown_connections 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2f140a3240 0x7f2f140a5700 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f28106f20 0x7f2f2819cd10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f281027a0 0x7f2f28198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 --2- 172.21.15.78:0/2988422756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f28101df0 0x7f2f28198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 -- 172.21.15.78:0/2988422756 >> 172.21.15.78:0/2988422756 conn(0x7f2f280fd8f0 msgr2=0x7f2f280fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 -- 172.21.15.78:0/2988422756 shutdown_connections 2024-09-16T00:19:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:23.860+0000 7f2f2cc30640 1 -- 172.21.15.78:0/2988422756 wait complete. 2024-09-16T00:19:23.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:23 smithi193 ceph-mon[35484]: pgmap v540: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T00:19:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:23 smithi148 ceph-mon[35599]: pgmap v540: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T00:19:23.970 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:23 smithi078 ceph-mon[31157]: pgmap v540: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T00:19:24.782 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ceb33b92', total seconds elapsed: 28.9 2024-09-16T00:19:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 325.0 s, 297.9 tps, lat 1721.407 ms stddev 6110.992 2024-09-16T00:19:25.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:25 smithi193 ceph-mon[35484]: pgmap v541: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-16T00:19:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:25 smithi148 ceph-mon[35599]: pgmap v541: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-16T00:19:26.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:25 smithi078 ceph-mon[31157]: pgmap v541: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-16T00:19:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:27 smithi193 ceph-mon[35484]: pgmap v542: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-16T00:19:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:27 smithi148 ceph-mon[35599]: pgmap v542: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-16T00:19:28.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:27 smithi078 ceph-mon[31157]: pgmap v542: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-16T00:19:29.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:29 smithi193 ceph-mon[35484]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.2 MiB/s wr, 271 op/s 2024-09-16T00:19:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:29 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:19:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:29 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:19:29.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:29 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:19:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:29 smithi148 ceph-mon[35599]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.2 MiB/s wr, 271 op/s 2024-09-16T00:19:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:29 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:19:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:29 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:19:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:29 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:19:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:29 smithi078 ceph-mon[31157]: pgmap v543: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.2 MiB/s wr, 271 op/s 2024-09-16T00:19:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:29 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:19:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:29 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:19:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:29 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:19:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 330.0 s, 1251.2 tps, lat 81.205 ms stddev 109.663 2024-09-16T00:19:31.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:31 smithi193 ceph-mon[35484]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 339 op/s 2024-09-16T00:19:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:31 smithi148 ceph-mon[35599]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 339 op/s 2024-09-16T00:19:32.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:31 smithi078 ceph-mon[31157]: pgmap v544: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.4 MiB/s wr, 339 op/s 2024-09-16T00:19:33.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:33 smithi193 ceph-mon[35484]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 464 op/s 2024-09-16T00:19:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:33 smithi148 ceph-mon[35599]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 464 op/s 2024-09-16T00:19:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:33 smithi078 ceph-mon[31157]: pgmap v545: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 11 MiB/s wr, 464 op/s 2024-09-16T00:19:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 335.0 s, 1505.8 tps, lat 65.765 ms stddev 78.773 2024-09-16T00:19:35.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:35 smithi193 ceph-mon[35484]: pgmap v546: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 566 op/s 2024-09-16T00:19:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:35 smithi148 ceph-mon[35599]: pgmap v546: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 566 op/s 2024-09-16T00:19:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:35 smithi078 ceph-mon[31157]: pgmap v546: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 17 MiB/s wr, 566 op/s 2024-09-16T00:19:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:37 smithi148 ceph-mon[35599]: pgmap v547: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 20 MiB/s wr, 655 op/s 2024-09-16T00:19:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:37 smithi078 ceph-mon[31157]: pgmap v547: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 20 MiB/s wr, 655 op/s 2024-09-16T00:19:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:37 smithi193 ceph-mon[35484]: pgmap v547: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 20 MiB/s wr, 655 op/s 2024-09-16T00:19:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:39 smithi148 ceph-mon[35599]: pgmap v548: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 MiB/s wr, 666 op/s 2024-09-16T00:19:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:39 smithi078 ceph-mon[31157]: pgmap v548: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 MiB/s wr, 666 op/s 2024-09-16T00:19:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:39 smithi193 ceph-mon[35484]: pgmap v548: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 MiB/s wr, 666 op/s 2024-09-16T00:19:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 340.0 s, 1480.0 tps, lat 67.890 ms stddev 81.836 2024-09-16T00:19:41.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:41 smithi148 ceph-mon[35599]: pgmap v549: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 589 op/s 2024-09-16T00:19:41.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:41 smithi193 ceph-mon[35484]: pgmap v549: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 589 op/s 2024-09-16T00:19:42.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:41 smithi078 ceph-mon[31157]: pgmap v549: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 MiB/s wr, 589 op/s 2024-09-16T00:19:43.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:43 smithi148 ceph-mon[35599]: pgmap v550: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 654 op/s 2024-09-16T00:19:44.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:43 smithi078 ceph-mon[31157]: pgmap v550: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 654 op/s 2024-09-16T00:19:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:43 smithi193 ceph-mon[35484]: pgmap v550: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 MiB/s wr, 654 op/s 2024-09-16T00:19:45.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 345.0 s, 1176.8 tps, lat 82.807 ms stddev 97.286 2024-09-16T00:19:45.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:45 smithi148 ceph-mon[35599]: pgmap v551: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 603 op/s 2024-09-16T00:19:46.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:45 smithi078 ceph-mon[31157]: pgmap v551: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 603 op/s 2024-09-16T00:19:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:45 smithi193 ceph-mon[35484]: pgmap v551: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 603 op/s 2024-09-16T00:19:47.053 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:19:47.485 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:47.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:47 smithi148 ceph-mon[35599]: pgmap v552: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 598 op/s 2024-09-16T00:19:48.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:47 smithi078 ceph-mon[31157]: pgmap v552: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 598 op/s 2024-09-16T00:19:48.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:47 smithi193 ceph-mon[35484]: pgmap v552: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 598 op/s 2024-09-16T00:19:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 -- 172.21.15.78:0/2548933327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101470 msgr2=0x7f13e4101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2548933327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101470 0x7f13e4101850 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009900 tx=0x7f13d802a410 comp rx=0 tx=0).stop 2024-09-16T00:19:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 -- 172.21.15.78:0/2548933327 shutdown_connections 2024-09-16T00:19:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2548933327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e4105de0 0x7f13e410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2548933327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13e4101e20 0x7f13e41056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2548933327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101470 0x7f13e4101850 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 -- 172.21.15.78:0/2548933327 >> 172.21.15.78:0/2548933327 conn(0x7f13e4077230 msgr2=0x7f13e40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 -- 172.21.15.78:0/2548933327 shutdown_connections 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.402+0000 7f13eaf80640 1 -- 172.21.15.78:0/2548933327 wait complete. 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.403+0000 7f13eaf80640 1 Processor -- start 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.403+0000 7f13eaf80640 1 -- start start 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.404+0000 7f13eaf80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13e4101470 0x7f13e41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.404+0000 7f13eaf80640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101e20 0x7f13e41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.404+0000 7f13eaf80640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e4105de0 0x7f13e419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.404+0000 7f13eaf80640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4111fb0 con 0x7f13e4101e20 2024-09-16T00:19:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.404+0000 7f13eaf80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4111e30 con 0x7f13e4101470 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.404+0000 7f13eaf80640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4112130 con 0x7f13e4105de0 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.404+0000 7f13e8d1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13e4101470 0x7f13e41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e8d1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13e4101470 0x7f13e41982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:38546/0 (socket says 172.21.15.78:38546) 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e8d1c640 1 -- 172.21.15.78:0/2295910067 learned_addr learned my addr 172.21.15.78:0/2295910067 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e3fff640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101e20 0x7f13e41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e8d1c640 1 -- 172.21.15.78:0/2295910067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e4105de0 msgr2=0x7f13e419cb80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e4105de0 0x7f13e419cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e8d1c640 1 -- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101e20 msgr2=0x7f13e41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101e20 0x7f13e41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e8d1c640 1 -- 172.21.15.78:0/2295910067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e419d260 con 0x7f13e4101470 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e3fff640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101e20 0x7f13e41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:48.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.405+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13e4101470 0x7f13e41982b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009850 tx=0x7f13d80309c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.406+0000 7f13e1ffb640 1 -- 172.21.15.78:0/2295910067 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d803f070 con 0x7f13e4101470 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.406+0000 7f13e1ffb640 1 -- 172.21.15.78:0/2295910067 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d80035f0 con 0x7f13e4101470 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.406+0000 7f13e1ffb640 1 -- 172.21.15.78:0/2295910067 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8035030 con 0x7f13e4101470 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.406+0000 7f13eaf80640 1 -- 172.21.15.78:0/2295910067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e419d4f0 con 0x7f13e4101470 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.406+0000 7f13eaf80640 1 -- 172.21.15.78:0/2295910067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e419d8c0 con 0x7f13e4101470 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13ed359740 1 Processor -- start 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13ed359740 1 -- start start 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13e1ffb640 1 -- 172.21.15.78:0/2295910067 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f13d8042070 con 0x7f13e4101470 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13ed359740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866543880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13ed359740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x55686654e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13ed359740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x556866550a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13ed359740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5568664a92e0 con 0x5568665434a0 2024-09-16T00:19:48.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13ed359740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556866450ce0 con 0x556866543dc0 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.410+0000 7f13ed359740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5568664a52a0 con 0x55686654e680 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x55686654e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x55686654e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:38560/0 (socket says 172.21.15.78:38560) 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 -- 172.21.15.78:0/4240321248 learned_addr learned my addr 172.21.15.78:0/4240321248 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e3fff640 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x556866550a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 -- 172.21.15.78:0/4240321248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 msgr2=0x556866550a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x556866550a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 -- 172.21.15.78:0/4240321248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 msgr2=0x556866543880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866543880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 -- 172.21.15.78:0/4240321248 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556866435440 con 0x556866543dc0 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e3fff640 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x556866550a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.411+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x55686654e140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f13d80f6c10 tx=0x7f13d80f6d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abfe6aa0e8360e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:48.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.412+0000 7f13ceffd640 1 -- 172.21.15.78:0/4240321248 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80ff070 con 0x556866543dc0 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.412+0000 7f13ceffd640 1 -- 172.21.15.78:0/4240321248 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d80f2550 con 0x556866543dc0 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.412+0000 7f13ceffd640 1 -- 172.21.15.78:0/4240321248 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80fabe0 con 0x556866543dc0 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.412+0000 7f13e1ffb640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13d40a3240 0x7f13d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.412+0000 7f13e1ffb640 1 -- 172.21.15.78:0/2295910067 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f13d80f0700 con 0x7f13e4101470 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.412+0000 7f13e951d640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13d40a3240 0x7f13d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13ed359740 1 -- 172.21.15.78:0/4240321248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 msgr2=0x55686654e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13ed359740 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x55686654e140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f13d80f6c10 tx=0x7f13d80f6d00 comp rx=0 tx=0).stop 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13ed359740 1 -- 172.21.15.78:0/4240321248 shutdown_connections 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13ed359740 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x556866550a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13ed359740 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x55686654e140 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13ed359740 1 --2- 172.21.15.78:0/4240321248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866543880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13ed359740 1 -- 172.21.15.78:0/4240321248 >> 172.21.15.78:0/4240321248 conn(0x55686653fa30 msgr2=0x55686653fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13e951d640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13d40a3240 0x7f13d40a5700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00aa90 tx=0x7f13dc009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:48.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.413+0000 7f13ed359740 1 -- 172.21.15.78:0/4240321248 shutdown_connections 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.414+0000 7f13ed359740 1 -- 172.21.15.78:0/4240321248 wait complete. 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.414+0000 7f13ed359740 1 -- start start 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.415+0000 7f13ed359740 1 Processor -- start 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.415+0000 7f13ed359740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x55686654e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.415+0000 7f13ed359740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556866543dc0 0x55686654b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.415+0000 7f13ed359740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55686654e680 0x55686654b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.415+0000 7f13ed359740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556866551010 con 0x5568665434a0 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.415+0000 7f13ed359740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556866464050 con 0x55686654e680 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.415+0000 7f13ed359740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556866551190 con 0x556866543dc0 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.415+0000 7f13ed359740 10 client.0 _ll_register_callbacks cb 0x5568664a4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e951d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55686654e680 0x55686654b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556866543dc0 0x55686654b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e951d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55686654e680 0x55686654b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:38570/0 (socket says 172.21.15.78:38570) 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e951d640 1 -- 172.21.15.78:0/3732802315 learned_addr learned my addr 172.21.15.78:0/3732802315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x55686654e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e3fff640 1 -- 172.21.15.78:0/3732802315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55686654e680 msgr2=0x55686654b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e3fff640 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55686654e680 0x55686654b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e3fff640 1 -- 172.21.15.78:0/3732802315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 msgr2=0x55686654e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e3fff640 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x55686654e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e3fff640 1 -- 172.21.15.78:0/3732802315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55686621a3c0 con 0x556866543dc0 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x55686654e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13e3fff640 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556866543dc0 0x55686654b2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13d0005d00 tx=0x7f13d0010b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=194adf631ff4d8dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.416+0000 7f13c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13c3fff640 1 -- 172.21.15.78:0/3732802315 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0012920 con 0x556866543dc0 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13c3fff640 1 -- 172.21.15.78:0/3732802315 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f13d0012ac0 con 0x556866543dc0 2024-09-16T00:19:48.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13ed359740 1 -- 172.21.15.78:0/3732802315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d80f2550 con 0x556866543dc0 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13c3fff640 1 -- 172.21.15.78:0/3732802315 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f13d0012cc0 con 0x556866543dc0 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13ed359740 10 client.25804 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13ed359740 1 -- 172.21.15.78:0/3732802315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5568665652a0 con 0x556866543dc0 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13c3fff640 1 -- 172.21.15.78:0/3732802315 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d80f2550 con 0x556866543dc0 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13c3fff640 1 -- 172.21.15.78:0/3732802315 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f13d000f040 con 0x556866543dc0 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13ed359740 10 client.25804 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13ed359740 10 client.25804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.417+0000 7f13ed359740 20 client.25804 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.418+0000 7f13ed359740 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556866566be0 0x556866569020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.418+0000 7f13ed359740 4 client.25804 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.418+0000 7f13ed359740 1 -- 172.21.15.78:0/3732802315 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568664b6850 con 0x556866566be0 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.418+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556866566be0 0x556866569020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.418+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556866566be0 0x556866569020 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919f09b879f499bf server_cookie=adddd2ba44f6d74c in_seq=0 out_seq=0 2024-09-16T00:19:48.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.418+0000 7f13c3fff640 10 client.25804 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.419+0000 7f13c3fff640 1 -- 172.21.15.78:0/3732802315 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568664b6850 con 0x556866566be0 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.419+0000 7f13c3fff640 10 client.25804 handle_command_reply: tid=0 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.419+0000 7f13ed359740 1 client.25804 shutdown 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 -- 172.21.15.78:0/3732802315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556866543dc0 msgr2=0x55686654b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556866543dc0 0x55686654b2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13d0005d00 tx=0x7f13d0010b00 comp rx=0 tx=0).stop 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 -- 172.21.15.78:0/3732802315 shutdown_connections 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556866566be0 0x556866569020 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55686654e680 0x55686654b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556866543dc0 0x55686654b2c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 --2- 172.21.15.78:0/3732802315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x55686654e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13c3fff640 0 client.25804 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 -- 172.21.15.78:0/3732802315 >> 172.21.15.78:0/3732802315 conn(0x55686653fa30 msgr2=0x556866545760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:48.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 -- 172.21.15.78:0/3732802315 shutdown_connections 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 1 -- 172.21.15.78:0/3732802315 wait complete. 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.420+0000 7f13ed359740 20 client.25804 trim_cache size 0 max 16384 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.444+0000 7f13ed359740 1 Processor -- start 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.444+0000 7f13ed359740 1 -- start start 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.444+0000 7f13ed359740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568665434a0 0x556866562b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.444+0000 7f13ed359740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556866543dc0 0x55686656e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.444+0000 7f13ed359740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686656ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.445+0000 7f13ed359740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800fb70 con 0x556866543dc0 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.445+0000 7f13ed359740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80116b0 con 0x5568665434a0 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.445+0000 7f13ed359740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8010240 con 0x55686654e680 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.445+0000 7f13e951d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686656ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.445+0000 7f13e951d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686656ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53042/0 (socket says 172.21.15.78:53042) 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.445+0000 7f13e951d640 1 -- 172.21.15.78:0/1297686518 learned_addr learned my addr 172.21.15.78:0/1297686518 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:48.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.445+0000 7f13e3fff640 1 --2- 172.21.15.78:0/1297686518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556866543dc0 0x55686656e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.445+0000 7f13e3fff640 1 -- 172.21.15.78:0/1297686518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 msgr2=0x55686656ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.446+0000 7f13e3fff640 1 --2- 172.21.15.78:0/1297686518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686656ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.446+0000 7f13e3fff640 1 -- 172.21.15.78:0/1297686518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568665434a0 msgr2=0x556866562b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.446+0000 7f13e3fff640 1 --2- 172.21.15.78:0/1297686518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568665434a0 0x556866562b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.446+0000 7f13e3fff640 1 -- 172.21.15.78:0/1297686518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55686654a270 con 0x556866543dc0 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.446+0000 7f13e3fff640 1 --2- 172.21.15.78:0/1297686518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556866543dc0 0x55686656e8f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f13d000e7b0 tx=0x7f13d00096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c3b938c65d7892 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.446+0000 7f13ce7fc640 1 -- 172.21.15.78:0/1297686518 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0012e60 con 0x556866543dc0 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.446+0000 7f13ce7fc640 1 -- 172.21.15.78:0/1297686518 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d0010070 con 0x556866543dc0 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.446+0000 7f13ce7fc640 1 -- 172.21.15.78:0/1297686518 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00052a0 con 0x556866543dc0 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.448+0000 7f13ed359740 1 -- 172.21.15.78:0/1297686518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556866543dc0 msgr2=0x55686656e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.448+0000 7f13ed359740 1 --2- 172.21.15.78:0/1297686518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556866543dc0 0x55686656e8f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f13d000e7b0 tx=0x7f13d00096d0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.449+0000 7f13ed359740 1 -- 172.21.15.78:0/1297686518 shutdown_connections 2024-09-16T00:19:48.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.449+0000 7f13ed359740 1 --2- 172.21.15.78:0/1297686518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686656ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.449+0000 7f13ed359740 1 --2- 172.21.15.78:0/1297686518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556866543dc0 0x55686656e8f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.449+0000 7f13ed359740 1 --2- 172.21.15.78:0/1297686518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5568665434a0 0x556866562b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.449+0000 7f13ed359740 1 -- 172.21.15.78:0/1297686518 >> 172.21.15.78:0/1297686518 conn(0x55686653fa30 msgr2=0x556866546900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.453+0000 7f13ed359740 1 -- 172.21.15.78:0/1297686518 shutdown_connections 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.453+0000 7f13ed359740 1 -- 172.21.15.78:0/1297686518 wait complete. 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.453+0000 7f13ed359740 1 -- start start 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.453+0000 7f13ed359740 1 Processor -- start 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13ed359740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866556fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13ed359740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x556866557520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13ed359740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686655ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13ed359740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8010fe0 con 0x5568665434a0 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13ed359740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8010910 con 0x556866543dc0 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13ed359740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800db50 con 0x55686654e680 2024-09-16T00:19:48.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13ed359740 10 client.0 _ll_register_callbacks cb 0x7f13d0009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13e8d1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866556fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13e8d1c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866556fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59208/0 (socket says 172.21.15.78:59208) 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.454+0000 7f13e8d1c640 1 -- 172.21.15.78:0/2593002653 learned_addr learned my addr 172.21.15.78:0/2593002653 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.455+0000 7f13e951d640 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686655ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.455+0000 7f13e8d1c640 1 -- 172.21.15.78:0/2593002653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 msgr2=0x55686655ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.455+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686655ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.455+0000 7f13e8d1c640 1 -- 172.21.15.78:0/2593002653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 msgr2=0x556866557520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.455+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x556866557520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.455+0000 7f13e8d1c640 1 -- 172.21.15.78:0/2593002653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55686655f530 con 0x5568665434a0 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.455+0000 7f13e8d1c640 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866556fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f13d802a930 tx=0x7f13d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7189513f9646b78f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.456+0000 7f13c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.456+0000 7f13c3fff640 1 -- 172.21.15.78:0/2593002653 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8049070 con 0x5568665434a0 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.456+0000 7f13c3fff640 1 -- 172.21.15.78:0/2593002653 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f13d8003b80 con 0x5568665434a0 2024-09-16T00:19:48.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.460+0000 7f13ed359740 1 -- 172.21.15.78:0/2593002653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d0010070 con 0x5568665434a0 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.463+0000 7f13c3fff640 1 -- 172.21.15.78:0/2593002653 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f13d80ff070 con 0x5568665434a0 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.466+0000 7f13ed359740 10 client.16566 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.466+0000 7f13ed359740 1 -- 172.21.15.78:0/2593002653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5568665530f0 con 0x5568665434a0 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.468+0000 7f13c3fff640 1 -- 172.21.15.78:0/2593002653 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d0010070 con 0x5568665434a0 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.468+0000 7f13c3fff640 1 -- 172.21.15.78:0/2593002653 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f13d80ba0c0 con 0x5568665434a0 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.468+0000 7f13ed359740 10 client.16566 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.468+0000 7f13ed359740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.468+0000 7f13ed359740 20 client.16566 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.468+0000 7f13ed359740 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5568665fcb40 0x5568665fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.468+0000 7f13ed359740 4 client.16566 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.468+0000 7f13ed359740 1 -- 172.21.15.78:0/2593002653 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568664b0d60 con 0x5568665fcb40 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.469+0000 7f13e3fff640 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5568665fcb40 0x5568665fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.475+0000 7f13e3fff640 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5568665fcb40 0x5568665fef30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6af2b2851bbb052 server_cookie=53d12706c16f3999 in_seq=0 out_seq=0 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.475+0000 7f13c3fff640 10 client.16566 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.475+0000 7f13c3fff640 1 -- 172.21.15.78:0/2593002653 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5568664b0d60 con 0x5568665fcb40 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.475+0000 7f13c3fff640 10 client.16566 handle_command_reply: tid=0 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.475+0000 7f13ed359740 1 client.16566 shutdown 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.478+0000 7f13ed359740 1 -- 172.21.15.78:0/2593002653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 msgr2=0x556866556fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.478+0000 7f13ed359740 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866556fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f13d802a930 tx=0x7f13d8005fd0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.484+0000 7f13ed359740 1 -- 172.21.15.78:0/2593002653 shutdown_connections 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.484+0000 7f13ed359740 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5568665fcb40 0x5568665fef30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.484+0000 7f13ed359740 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55686654e680 0x55686655ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.484+0000 7f13ed359740 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556866543dc0 0x556866557520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.484+0000 7f13ed359740 1 --2- 172.21.15.78:0/2593002653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5568665434a0 0x556866556fe0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.484+0000 7f13ed359740 1 -- 172.21.15.78:0/2593002653 >> 172.21.15.78:0/2593002653 conn(0x55686653fa30 msgr2=0x55686654cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.484+0000 7f13c3fff640 0 client.16566 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:19:48.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.486+0000 7f13ed359740 1 -- 172.21.15.78:0/2593002653 shutdown_connections 2024-09-16T00:19:48.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.486+0000 7f13ed359740 1 -- 172.21.15.78:0/2593002653 wait complete. 2024-09-16T00:19:48.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.486+0000 7f13ed359740 20 client.16566 trim_cache size 0 max 16384 2024-09-16T00:19:48.782 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:19:48.782 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:19:48.782 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:19:48.782 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.487+0000 7f13eaf80640 1 -- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13d40a3240 msgr2=0x7f13d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.487+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13d40a3240 0x7f13d40a5700 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00aa90 tx=0x7f13dc009250 comp rx=0 tx=0).stop 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.487+0000 7f13eaf80640 1 -- 172.21.15.78:0/2295910067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13e4101470 msgr2=0x7f13e41982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.487+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13e4101470 0x7f13e41982b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009850 tx=0x7f13d80309c0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.491+0000 7f13eaf80640 1 -- 172.21.15.78:0/2295910067 shutdown_connections 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.491+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13d40a3240 0x7f13d40a5700 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.491+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13e4105de0 0x7f13e419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.491+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e4101e20 0x7f13e41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.491+0000 7f13eaf80640 1 --2- 172.21.15.78:0/2295910067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13e4101470 0x7f13e41982b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.491+0000 7f13eaf80640 1 -- 172.21.15.78:0/2295910067 >> 172.21.15.78:0/2295910067 conn(0x7f13e4077230 msgr2=0x7f13e4105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.491+0000 7f13eaf80640 1 -- 172.21.15.78:0/2295910067 shutdown_connections 2024-09-16T00:19:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:48.491+0000 7f13eaf80640 1 -- 172.21.15.78:0/2295910067 wait complete. 2024-09-16T00:19:49.990 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:49 smithi078 ceph-mon[31157]: pgmap v553: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 605 op/s 2024-09-16T00:19:50.043 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:19:50.043 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:19:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:49 smithi193 ceph-mon[35484]: pgmap v553: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 605 op/s 2024-09-16T00:19:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:49 smithi148 ceph-mon[35599]: pgmap v553: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 MiB/s wr, 605 op/s 2024-09-16T00:19:50.474 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 350.0 s, 1288.4 tps, lat 79.348 ms stddev 96.855 2024-09-16T00:19:51.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 -- 172.21.15.78:0/1142768940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250010c170 msgr2=0x7f250010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 --2- 172.21.15.78:0/1142768940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250010c170 0x7f250010e610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009be0 tx=0x7f24f802a750 comp rx=0 tx=0).stop 2024-09-16T00:19:51.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 -- 172.21.15.78:0/1142768940 shutdown_connections 2024-09-16T00:19:51.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 --2- 172.21.15.78:0/1142768940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250010c170 0x7f250010e610 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 --2- 172.21.15.78:0/1142768940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2500103660 0x7f250010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 --2- 172.21.15.78:0/1142768940 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2500102d40 0x7f2500103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 -- 172.21.15.78:0/1142768940 >> 172.21.15.78:0/1142768940 conn(0x7f25000fc7a0 msgr2=0x7f25000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:51.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 -- 172.21.15.78:0/1142768940 shutdown_connections 2024-09-16T00:19:51.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.105+0000 7f2508174640 1 -- 172.21.15.78:0/1142768940 wait complete. 2024-09-16T00:19:51.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2508174640 1 Processor -- start 2024-09-16T00:19:51.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2508174640 1 -- start start 2024-09-16T00:19:51.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2508174640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2500102d40 0x7f2500198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:51.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2508174640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2500103660 0x7f2500198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:51.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2508174640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250010c170 0x7f250019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:51.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2508174640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2500111f50 con 0x7f2500103660 2024-09-16T00:19:51.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2508174640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2500111dd0 con 0x7f250010c170 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2508174640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25001120d0 con 0x7f2500102d40 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f250570f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2500103660 0x7f2500198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f250570f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2500103660 0x7f2500198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59226/0 (socket says 172.21.15.78:59226) 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f250570f640 1 -- 172.21.15.78:0/1174938505 learned_addr learned my addr 172.21.15.78:0/1174938505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.106+0000 7f2506711640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250010c170 0x7f250019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2505f10640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2500102d40 0x7f2500198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2505f10640 1 -- 172.21.15.78:0/1174938505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250010c170 msgr2=0x7f250019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2505f10640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250010c170 0x7f250019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2505f10640 1 -- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2500103660 msgr2=0x7f2500198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2505f10640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2500103660 0x7f2500198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2505f10640 1 -- 172.21.15.78:0/1174938505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250019d420 con 0x7f2500102d40 2024-09-16T00:19:51.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f250570f640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2500103660 0x7f2500198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:19:51.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2505f10640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2500102d40 0x7f2500198450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009e90 tx=0x7f24fc00b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:51.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f24f6ffd640 1 -- 172.21.15.78:0/1174938505 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc013070 con 0x7f2500102d40 2024-09-16T00:19:51.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250019d710 con 0x7f2500102d40 2024-09-16T00:19:51.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f24f6ffd640 1 -- 172.21.15.78:0/1174938505 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24fc00fb70 con 0x7f2500102d40 2024-09-16T00:19:51.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25001008d0 con 0x7f2500102d40 2024-09-16T00:19:51.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.107+0000 7f24f6ffd640 1 -- 172.21.15.78:0/1174938505 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00e3e0 con 0x7f2500102d40 2024-09-16T00:19:51.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.109+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2500103120 con 0x7f2500102d40 2024-09-16T00:19:51.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.111+0000 7f24f6ffd640 1 -- 172.21.15.78:0/1174938505 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f24fc02d020 con 0x7f2500102d40 2024-09-16T00:19:51.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.112+0000 7f24f6ffd640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24ec0a3380 0x7f24ec0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:51.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.112+0000 7f24f6ffd640 1 -- 172.21.15.78:0/1174938505 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f24fc0e04b0 con 0x7f2500102d40 2024-09-16T00:19:51.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.113+0000 7f250570f640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24ec0a3380 0x7f24ec0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:51.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.114+0000 7f250570f640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24ec0a3380 0x7f24ec0a5840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f25000feda0 tx=0x7f24f000b360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:51.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.117+0000 7f24f6ffd640 1 -- 172.21.15.78:0/1174938505 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24fc0ace10 con 0x7f2500102d40 2024-09-16T00:19:51.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.494+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25001071e0 con 0x7f2500102d40 2024-09-16T00:19:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.497+0000 7f24f6ffd640 1 -- 172.21.15.78:0/1174938505 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f24fc0ac710 con 0x7f2500102d40 2024-09-16T00:19:51.499 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:19:51.499 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:19:51.499 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24ec0a3380 msgr2=0x7f24ec0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24ec0a3380 0x7f24ec0a5840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f25000feda0 tx=0x7f24f000b360 comp rx=0 tx=0).stop 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2500102d40 msgr2=0x7f2500198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2500102d40 0x7f2500198450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009e90 tx=0x7f24fc00b690 comp rx=0 tx=0).stop 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 shutdown_connections 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f24ec0a3380 0x7f24ec0a5840 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f250010c170 0x7f250019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2500103660 0x7f2500198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 --2- 172.21.15.78:0/1174938505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2500102d40 0x7f2500198450 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 >> 172.21.15.78:0/1174938505 conn(0x7f25000fc7a0 msgr2=0x7f25000fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 shutdown_connections 2024-09-16T00:19:51.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:51.501+0000 7f2508174640 1 -- 172.21.15.78:0/1174938505 wait complete. 2024-09-16T00:19:51.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:51 smithi078 ceph-mon[31157]: pgmap v554: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 614 op/s 2024-09-16T00:19:51.770 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:51 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1174938505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:52.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:51 smithi193 ceph-mon[35484]: pgmap v554: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 614 op/s 2024-09-16T00:19:52.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:51 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1174938505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:51 smithi148 ceph-mon[35599]: pgmap v554: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 614 op/s 2024-09-16T00:19:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:51 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1174938505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:19:52.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:19:52.682 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.378+0000 7fb9fffff640 1 -- 172.21.15.78:0/4178287702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 msgr2=0x7fba0006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.378+0000 7fb9fffff640 1 --2- 172.21.15.78:0/4178287702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 0x7fba0006ee70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009900 tx=0x7fb9f402a420 comp rx=0 tx=0).stop 2024-09-16T00:19:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.380+0000 7fb9fffff640 1 -- 172.21.15.78:0/4178287702 shutdown_connections 2024-09-16T00:19:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.380+0000 7fb9fffff640 1 --2- 172.21.15.78:0/4178287702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba0011b140 0x7fba0011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.380+0000 7fb9fffff640 1 --2- 172.21.15.78:0/4178287702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 0x7fba0006ee70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.380+0000 7fb9fffff640 1 --2- 172.21.15.78:0/4178287702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba00110190 0x7fba0006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.380+0000 7fb9fffff640 1 -- 172.21.15.78:0/4178287702 >> 172.21.15.78:0/4178287702 conn(0x7fba00073180 msgr2=0x7fba00073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:53.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.382+0000 7fb9fffff640 1 -- 172.21.15.78:0/4178287702 shutdown_connections 2024-09-16T00:19:53.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.382+0000 7fb9fffff640 1 -- 172.21.15.78:0/4178287702 wait complete. 2024-09-16T00:19:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9fffff640 1 Processor -- start 2024-09-16T00:19:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9fffff640 1 -- start start 2024-09-16T00:19:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9fffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 0x7fba0011a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9fffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba00110190 0x7fba00114910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9feffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 0x7fba0011a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9feffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 0x7fba0011a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53090/0 (socket says 172.21.15.78:53090) 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba00110190 0x7fba00114910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba00110190 0x7fba00114910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59230/0 (socket says 172.21.15.78:59230) 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.383+0000 7fb9feffd640 1 -- 172.21.15.78:0/531161501 learned_addr learned my addr 172.21.15.78:0/531161501 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.384+0000 7fb9fffff640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba0011b140 0x7fba00114e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.384+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba00111330 con 0x7fba00110190 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.384+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fba001111b0 con 0x7fba0011b140 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.384+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fba001114b0 con 0x7fba0006ea30 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.384+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba0011b140 0x7fba00114e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.385+0000 7fb9feffd640 1 -- 172.21.15.78:0/531161501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba0011b140 msgr2=0x7fba00114e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.385+0000 7fb9feffd640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba0011b140 0x7fba00114e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.385+0000 7fb9feffd640 1 -- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba00110190 msgr2=0x7fba00114910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.385+0000 7fb9feffd640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba00110190 0x7fba00114910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.385+0000 7fb9feffd640 1 -- 172.21.15.78:0/531161501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba001155a0 con 0x7fba0006ea30 2024-09-16T00:19:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.389+0000 7fb9feffd640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 0x7fba0011a8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000e9f0 tx=0x7fb9f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.390+0000 7fb9e7fff640 1 -- 172.21.15.78:0/531161501 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f000cd80 con 0x7fba0006ea30 2024-09-16T00:19:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.390+0000 7fb9e7fff640 1 -- 172.21.15.78:0/531161501 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f0010910 con 0x7fba0006ea30 2024-09-16T00:19:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.390+0000 7fb9e7fff640 1 -- 172.21.15.78:0/531161501 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0018990 con 0x7fba0006ea30 2024-09-16T00:19:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.390+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba00074420 con 0x7fba0006ea30 2024-09-16T00:19:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.390+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba000748e0 con 0x7fba0006ea30 2024-09-16T00:19:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.394+0000 7fba07779740 1 Processor -- start 2024-09-16T00:19:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.394+0000 7fba07779740 1 -- start start 2024-09-16T00:19:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.394+0000 7fba07779740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18784270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.394+0000 7fba07779740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 0x55ca18784bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.394+0000 7fba07779740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1878f550 0x55ca18791940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.394+0000 7fba07779740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca186ea2f0 con 0x55ca18785d20 2024-09-16T00:19:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.394+0000 7fba07779740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ca18691cf0 con 0x55ca187847b0 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.394+0000 7fba07779740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ca186e62b0 con 0x55ca1878f550 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9ff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18784270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9ff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18784270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59246/0 (socket says 172.21.15.78:59246) 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/777086773 learned_addr learned my addr 172.21.15.78:0/777086773 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/777086773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1878f550 msgr2=0x55ca18791940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/777086773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1878f550 0x55ca18791940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9e7fff640 1 -- 172.21.15.78:0/531161501 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb9f0018b30 con 0x7fba0006ea30 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/777086773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 msgr2=0x55ca18784bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/777086773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 0x55ca18784bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.395+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/777086773 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca186763a0 con 0x55ca18785d20 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.396+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/777086773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18784270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8007e50 tx=0x7fb9f80317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7653d327994a0189 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.397+0000 7fb9e57fa640 1 -- 172.21.15.78:0/777086773 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8032850 con 0x55ca18785d20 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.397+0000 7fb9e57fa640 1 -- 172.21.15.78:0/777086773 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f803a3f0 con 0x55ca18785d20 2024-09-16T00:19:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.397+0000 7fb9e7fff640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb9e80a3380 0x7fb9e80a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.397+0000 7fb9e57fa640 1 -- 172.21.15.78:0/777086773 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8039600 con 0x55ca18785d20 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.397+0000 7fb9e7fff640 1 -- 172.21.15.78:0/531161501 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb9f00d0440 con 0x7fba0006ea30 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fb9feffd640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb9e80a3380 0x7fb9e80a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 -- 172.21.15.78:0/777086773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 msgr2=0x55ca18784270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 --2- 172.21.15.78:0/777086773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18784270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8007e50 tx=0x7fb9f80317a0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 -- 172.21.15.78:0/777086773 shutdown_connections 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 --2- 172.21.15.78:0/777086773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca1878f550 0x55ca18791940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 --2- 172.21.15.78:0/777086773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 0x55ca18784bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 --2- 172.21.15.78:0/777086773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18784270 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 -- 172.21.15.78:0/777086773 >> 172.21.15.78:0/777086773 conn(0x55ca1874d310 msgr2=0x55ca18780e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fb9feffd640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb9e80a3380 0x7fb9e80a5840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0098be0 tx=0x7fb9f009c310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 -- 172.21.15.78:0/777086773 shutdown_connections 2024-09-16T00:19:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.398+0000 7fba07779740 1 -- 172.21.15.78:0/777086773 wait complete. 2024-09-16T00:19:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.399+0000 7fba07779740 1 -- start start 2024-09-16T00:19:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.400+0000 7fba07779740 1 Processor -- start 2024-09-16T00:19:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.401+0000 7fba07779740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca1878eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.401+0000 7fba07779740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca1878c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.401+0000 7fba07779740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca1878c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.401+0000 7fba07779740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ca18792250 con 0x55ca18785d20 2024-09-16T00:19:53.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.401+0000 7fba07779740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ca186a5060 con 0x55ca1878f550 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.401+0000 7fba07779740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ca187923d0 con 0x55ca187847b0 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.401+0000 7fba07779740 10 client.0 _ll_register_callbacks cb 0x55ca183f88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.402+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca1878c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.402+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca1878c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59260/0 (socket says 172.21.15.78:59260) 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.402+0000 7fb9fe7fc640 1 -- 172.21.15.78:0/4241179289 learned_addr learned my addr 172.21.15.78:0/4241179289 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.402+0000 7fb9fe7fc640 1 -- 172.21.15.78:0/4241179289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 msgr2=0x55ca1878eed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.402+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca1878eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.402+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca1878c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.402+0000 7fb9fe7fc640 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca1878eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.403+0000 7fb9fe7fc640 1 -- 172.21.15.78:0/4241179289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 msgr2=0x55ca1878c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.403+0000 7fb9fe7fc640 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca1878c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.403+0000 7fb9fe7fc640 1 -- 172.21.15.78:0/4241179289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1845b3c0 con 0x55ca18785d20 2024-09-16T00:19:53.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.403+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca1878c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:19:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.403+0000 7fb9fe7fc640 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca1878c180 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4004fd0 tx=0x7fb9f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4eeda8906e20d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.403+0000 7fb9e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.403+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/4241179289 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4041070 con 0x55ca18785d20 2024-09-16T00:19:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.404+0000 7fba07779740 1 -- 172.21.15.78:0/4241179289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f803a3f0 con 0x55ca18785d20 2024-09-16T00:19:53.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.404+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/4241179289 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb9f4005200 con 0x55ca18785d20 2024-09-16T00:19:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.404+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/4241179289 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9f404f050 con 0x55ca18785d20 2024-09-16T00:19:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.404+0000 7fba07779740 10 client.16581 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.404+0000 7fba07779740 1 -- 172.21.15.78:0/4241179289 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ca1879dd60 con 0x55ca18785d20 2024-09-16T00:19:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.405+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/4241179289 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f803a3f0 con 0x55ca18785d20 2024-09-16T00:19:53.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.405+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/4241179289 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb9f4031440 con 0x55ca18785d20 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.405+0000 7fba07779740 10 client.16581 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.405+0000 7fba07779740 10 client.16581 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.405+0000 7fba07779740 20 client.16581 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.405+0000 7fba07779740 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55ca1879f610 0x55ca187a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.405+0000 7fba07779740 4 client.16581 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.405+0000 7fba07779740 1 -- 172.21.15.78:0/4241179289 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca186f7860 con 0x55ca1879f610 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.406+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55ca1879f610 0x55ca187a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.407+0000 7fb9e2ff5640 10 client.16581 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:19:53.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.407+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55ca1879f610 0x55ca187a1a30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6b5ce0980fc86a4f server_cookie=71461e211fb154f9 in_seq=0 out_seq=0 2024-09-16T00:19:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.407+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/4241179289 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca186f7860 con 0x55ca1879f610 2024-09-16T00:19:53.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.407+0000 7fb9e2ff5640 10 client.16581 handle_command_reply: tid=0 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 client.16581 shutdown 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 -- 172.21.15.78:0/4241179289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 msgr2=0x55ca1878c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca1878c180 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4004fd0 tx=0x7fb9f40048f0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 -- 172.21.15.78:0/4241179289 shutdown_connections 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca1878c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca1878c180 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55ca1879f610 0x55ca187a1a30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 --2- 172.21.15.78:0/4241179289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca1878eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 -- 172.21.15.78:0/4241179289 >> 172.21.15.78:0/4241179289 conn(0x55ca1874d310 msgr2=0x55ca18786b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fb9e2ff5640 0 client.16581 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 -- 172.21.15.78:0/4241179289 shutdown_connections 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 1 -- 172.21.15.78:0/4241179289 wait complete. 2024-09-16T00:19:53.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.408+0000 7fba07779740 20 client.16581 trim_cache size 0 max 16384 2024-09-16T00:19:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fba07779740 1 Processor -- start 2024-09-16T00:19:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fba07779740 1 -- start start 2024-09-16T00:19:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fba07779740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 0x55ca18780ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fba07779740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca18785d20 0x55ca1879b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fba07779740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca1878f550 0x55ca1878ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fba07779740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec00fa50 con 0x55ca1878f550 2024-09-16T00:19:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fba07779740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec011590 con 0x55ca187847b0 2024-09-16T00:19:53.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fba07779740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec010120 con 0x55ca18785d20 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fb9ff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca1878f550 0x55ca1878ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca18785d20 0x55ca1879b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fb9fe7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca18785d20 0x55ca1879b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53102/0 (socket says 172.21.15.78:53102) 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fb9ff7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca1878f550 0x55ca1878ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59276/0 (socket says 172.21.15.78:59276) 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.427+0000 7fb9fe7fc640 1 -- 172.21.15.78:0/181426032 learned_addr learned my addr 172.21.15.78:0/181426032 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/181426032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 0x55ca18780ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/181426032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca18785d20 msgr2=0x55ca1879b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/181426032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca18785d20 0x55ca1879b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/181426032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 msgr2=0x55ca18780ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/181426032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 0x55ca18780ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/181426032 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1878b150 con 0x55ca1878f550 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/181426032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca1878f550 0x55ca1878ba40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8031be0 tx=0x7fb9f80317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892106ca0a07ac98 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9e4ff9640 1 -- 172.21.15.78:0/181426032 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f803a6a0 con 0x55ca1878f550 2024-09-16T00:19:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9e4ff9640 1 -- 172.21.15.78:0/181426032 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f8003e50 con 0x55ca1878f550 2024-09-16T00:19:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.428+0000 7fb9e4ff9640 1 -- 172.21.15.78:0/181426032 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8032c50 con 0x55ca1878f550 2024-09-16T00:19:53.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 -- 172.21.15.78:0/181426032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca1878f550 msgr2=0x55ca1878ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 --2- 172.21.15.78:0/181426032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca1878f550 0x55ca1878ba40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8031be0 tx=0x7fb9f80317a0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 -- 172.21.15.78:0/181426032 shutdown_connections 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 --2- 172.21.15.78:0/181426032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca1878f550 0x55ca1878ba40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 --2- 172.21.15.78:0/181426032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca18785d20 0x55ca1879b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 --2- 172.21.15.78:0/181426032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca187847b0 0x55ca18780ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 -- 172.21.15.78:0/181426032 >> 172.21.15.78:0/181426032 conn(0x55ca1874d310 msgr2=0x55ca187808e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 -- 172.21.15.78:0/181426032 shutdown_connections 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.429+0000 7fba07779740 1 -- 172.21.15.78:0/181426032 wait complete. 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 1 -- start start 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 1 Processor -- start 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca18797890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18797dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca18792c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec010ec0 con 0x55ca18785d20 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec0107f0 con 0x55ca1878f550 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9ec00da30 con 0x55ca187847b0 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fba07779740 10 client.0 _ll_register_callbacks cb 0x7fb9f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9ff7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca18797890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9ff7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca18797890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53104/0 (socket says 172.21.15.78:53104) 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/1210060894 learned_addr learned my addr 172.21.15.78:0/1210060894 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9fe7fc640 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18797dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca18792c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/1210060894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 msgr2=0x55ca18792c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:53.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca18792c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/1210060894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 msgr2=0x55ca18797dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.430+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18797dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fb9ff7fe640 1 -- 172.21.15.78:0/1210060894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1878f240 con 0x55ca187847b0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fb9fe7fc640 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18797dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fb9ff7fe640 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca18797890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb9f803afd0 tx=0x7fb9f8031fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a765e7c1090f79b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fb9e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/1210060894 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8003b80 con 0x55ca187847b0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fba07779740 1 -- 172.21.15.78:0/1210060894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f8003e50 con 0x55ca187847b0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/1210060894 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb9f8032650 con 0x55ca187847b0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/1210060894 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9f8032850 con 0x55ca187847b0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fba07779740 10 client.25825 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.431+0000 7fba07779740 1 -- 172.21.15.78:0/1210060894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ca18809c40 con 0x55ca187847b0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/1210060894 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f8003e50 con 0x55ca187847b0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/1210060894 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb9f80058a0 con 0x55ca187847b0 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fba07779740 10 client.25825 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:53.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fba07779740 10 client.25825 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fba07779740 20 client.25825 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fba07779740 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55ca1880a120 0x55ca1880c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fba07779740 4 client.25825 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fba07779740 1 -- 172.21.15.78:0/1210060894 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca186f1d70 con 0x55ca1880a120 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.432+0000 7fb9fe7fc640 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55ca1880a120 0x55ca1880c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fb9fe7fc640 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55ca1880a120 0x55ca1880c540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16250ffb5de6b6d server_cookie=d44b6dc076faa565 in_seq=0 out_seq=0 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fb9e2ff5640 10 client.25825 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fb9e2ff5640 1 -- 172.21.15.78:0/1210060894 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca186f1d70 con 0x55ca1880a120 2024-09-16T00:19:53.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fb9e2ff5640 10 client.25825 handle_command_reply: tid=0 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 client.25825 shutdown 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 -- 172.21.15.78:0/1210060894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 msgr2=0x55ca18797890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca18797890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb9f803afd0 tx=0x7fb9f8031fd0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 -- 172.21.15.78:0/1210060894 shutdown_connections 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ca1878f550 0x55ca18792c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ca18785d20 0x55ca18797dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55ca1880a120 0x55ca1880c540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 --2- 172.21.15.78:0/1210060894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca187847b0 0x55ca18797890 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 -- 172.21.15.78:0/1210060894 >> 172.21.15.78:0/1210060894 conn(0x55ca1874d310 msgr2=0x55ca1877f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fb9e2ff5640 0 client.25825 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.433+0000 7fba07779740 1 -- 172.21.15.78:0/1210060894 shutdown_connections 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fba07779740 1 -- 172.21.15.78:0/1210060894 wait complete. 2024-09-16T00:19:53.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fba07779740 20 client.25825 trim_cache size 0 max 16384 2024-09-16T00:19:53.444 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:19:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb9e80a3380 msgr2=0x7fb9e80a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb9e80a3380 0x7fb9e80a5840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0098be0 tx=0x7fb9f009c310 comp rx=0 tx=0).stop 2024-09-16T00:19:53.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 msgr2=0x7fba0011a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 0x7fba0011a8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000e9f0 tx=0x7fb9f000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 shutdown_connections 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb9e80a3380 0x7fb9e80a5840 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fba0011b140 0x7fba00114e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba00110190 0x7fba00114910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 --2- 172.21.15.78:0/531161501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba0006ea30 0x7fba0011a8b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.434+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 >> 172.21.15.78:0/531161501 conn(0x7fba00073180 msgr2=0x7fba0011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.435+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 shutdown_connections 2024-09-16T00:19:53.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:53.435+0000 7fb9fffff640 1 -- 172.21.15.78:0/531161501 wait complete. 2024-09-16T00:19:54.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:53 smithi078 ceph-mon[31157]: pgmap v555: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 624 op/s 2024-09-16T00:19:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:53 smithi193 ceph-mon[35484]: pgmap v555: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 624 op/s 2024-09-16T00:19:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:53 smithi148 ceph-mon[35599]: pgmap v555: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 MiB/s wr, 624 op/s 2024-09-16T00:19:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 355.0 s, 1090.0 tps, lat 86.757 ms stddev 105.975 2024-09-16T00:19:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:55 smithi078 ceph-mon[31157]: pgmap v556: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 570 op/s 2024-09-16T00:19:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:55 smithi193 ceph-mon[35484]: pgmap v556: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 570 op/s 2024-09-16T00:19:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:55 smithi148 ceph-mon[35599]: pgmap v556: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 570 op/s 2024-09-16T00:19:56.380 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:19:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.643+0000 7f28f3e3e640 1 -- 172.21.15.78:0/1071256618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 msgr2=0x7f28ec106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.643+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/1071256618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 0x7f28ec106e60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009930 tx=0x7f28e802a420 comp rx=0 tx=0).stop 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.643+0000 7f28f3e3e640 1 -- 172.21.15.78:0/1071256618 shutdown_connections 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.643+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/1071256618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec101420 0x7f28ec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.643+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/1071256618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28ec100a80 0x7f28ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.643+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/1071256618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 0x7f28ec106e60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.643+0000 7f28f3e3e640 1 -- 172.21.15.78:0/1071256618 >> 172.21.15.78:0/1071256618 conn(0x7f28ec0fc7a0 msgr2=0x7f28ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.644+0000 7f28f3e3e640 1 -- 172.21.15.78:0/1071256618 shutdown_connections 2024-09-16T00:19:57.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.644+0000 7f28f3e3e640 1 -- 172.21.15.78:0/1071256618 wait complete. 2024-09-16T00:19:57.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.645+0000 7f28f3e3e640 1 Processor -- start 2024-09-16T00:19:57.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.645+0000 7f28f3e3e640 1 -- start start 2024-09-16T00:19:57.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.646+0000 7f28f3e3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec100a80 0x7f28ec073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.646+0000 7f28f3e3e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28ec101420 0x7f28ec06dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.646+0000 7f28f3e3e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 0x7f28ec06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.647+0000 7f28f3e3e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec06c4d0 con 0x7f28ec106a80 2024-09-16T00:19:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.647+0000 7f28f3e3e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec06c350 con 0x7f28ec100a80 2024-09-16T00:19:57.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.647+0000 7f28f3e3e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec06c650 con 0x7f28ec101420 2024-09-16T00:19:57.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.648+0000 7f28f1bda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec100a80 0x7f28ec073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.648+0000 7f28f1bda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec100a80 0x7f28ec073a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59982/0 (socket says 172.21.15.78:59982) 2024-09-16T00:19:57.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.648+0000 7f28f1bda640 1 -- 172.21.15.78:0/3047230170 learned_addr learned my addr 172.21.15.78:0/3047230170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:57.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.650+0000 7f28f23db640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 0x7f28ec06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.651+0000 7f28f23db640 1 -- 172.21.15.78:0/3047230170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28ec101420 msgr2=0x7f28ec06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.651+0000 7f28f23db640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28ec101420 0x7f28ec06dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.651+0000 7f28f23db640 1 -- 172.21.15.78:0/3047230170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec100a80 msgr2=0x7f28ec073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.651+0000 7f28f23db640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec100a80 0x7f28ec073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.651+0000 7f28f23db640 1 -- 172.21.15.78:0/3047230170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec06e5e0 con 0x7f28ec106a80 2024-09-16T00:19:57.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.651+0000 7f28f23db640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 0x7f28ec06dff0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009fd0 tx=0x7f28e400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:57.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.653+0000 7f28e2ffd640 1 -- 172.21.15.78:0/3047230170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e400ce20 con 0x7f28ec106a80 2024-09-16T00:19:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.653+0000 7f28e2ffd640 1 -- 172.21.15.78:0/3047230170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e4005020 con 0x7f28ec106a80 2024-09-16T00:19:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.653+0000 7f28e2ffd640 1 -- 172.21.15.78:0/3047230170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4010720 con 0x7f28ec106a80 2024-09-16T00:19:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.654+0000 7f28f3e3e640 1 -- 172.21.15.78:0/3047230170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec06e8d0 con 0x7f28ec106a80 2024-09-16T00:19:57.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.654+0000 7f28f3e3e640 1 -- 172.21.15.78:0/3047230170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1ac5f0 con 0x7f28ec106a80 2024-09-16T00:19:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.660+0000 7f28f6217740 1 Processor -- start 2024-09-16T00:19:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.665+0000 7f28e2ffd640 1 -- 172.21.15.78:0/3047230170 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f28e4005540 con 0x7f28ec106a80 2024-09-16T00:19:57.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.666+0000 7f28e2ffd640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28d80a3410 0x7f28d80a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.666+0000 7f28e2ffd640 1 -- 172.21.15.78:0/3047230170 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f28e40d0100 con 0x7f28ec106a80 2024-09-16T00:19:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.667+0000 7f28f6217740 1 -- start start 2024-09-16T00:19:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.667+0000 7f28f6217740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683d99c0 0x555e683d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.667+0000 7f28f6217740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e683eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.667+0000 7f28f6217740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.667+0000 7f28f6217740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555e6833e2f0 con 0x555e683eb9d0 2024-09-16T00:19:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.667+0000 7f28f6217740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555e682e5cf0 con 0x555e683da2e0 2024-09-16T00:19:57.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.667+0000 7f28f6217740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e6833a2b0 con 0x555e683d99c0 2024-09-16T00:19:57.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.670+0000 7f28f13d9640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28d80a3410 0x7f28d80a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.670+0000 7f28f13d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e683eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.670+0000 7f28f23db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.670+0000 7f28f23db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58568/0 (socket says 172.21.15.78:58568) 2024-09-16T00:19:57.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.670+0000 7f28f23db640 1 -- 172.21.15.78:0/3929829109 learned_addr learned my addr 172.21.15.78:0/3929829109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.675+0000 7f28f23db640 1 -- 172.21.15.78:0/3929829109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683d99c0 msgr2=0x555e683d9da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:19:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.675+0000 7f28f23db640 1 --2- 172.21.15.78:0/3929829109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683d99c0 0x555e683d9da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.675+0000 7f28f23db640 1 -- 172.21.15.78:0/3929829109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 msgr2=0x555e683eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.675+0000 7f28f23db640 1 --2- 172.21.15.78:0/3929829109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e683eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.675+0000 7f28f23db640 1 -- 172.21.15.78:0/3929829109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e682ca3a0 con 0x555e683eb9d0 2024-09-16T00:19:57.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.675+0000 7f28f23db640 1 --2- 172.21.15.78:0/3929829109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eddc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f28e409c060 tx=0x7f28e4005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2587a35e41cefcf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.677+0000 7f28d7fff640 1 -- 172.21.15.78:0/3929829109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40a0710 con 0x555e683eb9d0 2024-09-16T00:19:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.677+0000 7f28d7fff640 1 -- 172.21.15.78:0/3929829109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e40a4fd0 con 0x555e683eb9d0 2024-09-16T00:19:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.677+0000 7f28d7fff640 1 -- 172.21.15.78:0/3929829109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e409b7b0 con 0x555e683eb9d0 2024-09-16T00:19:57.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.679+0000 7f28f6217740 1 -- 172.21.15.78:0/3929829109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 msgr2=0x555e683eddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.679+0000 7f28f6217740 1 --2- 172.21.15.78:0/3929829109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eddc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f28e409c060 tx=0x7f28e4005940 comp rx=0 tx=0).stop 2024-09-16T00:19:57.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.681+0000 7f28f6217740 1 -- 172.21.15.78:0/3929829109 shutdown_connections 2024-09-16T00:19:57.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.681+0000 7f28f6217740 1 --2- 172.21.15.78:0/3929829109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eddc0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.681+0000 7f28f6217740 1 --2- 172.21.15.78:0/3929829109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e683eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.681+0000 7f28f6217740 1 --2- 172.21.15.78:0/3929829109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683d99c0 0x555e683d9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.681+0000 7f28f6217740 1 -- 172.21.15.78:0/3929829109 >> 172.21.15.78:0/3929829109 conn(0x555e683a1310 msgr2=0x555e683d46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:57.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.685+0000 7f28f13d9640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28d80a3410 0x7f28d80a58d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009460 tx=0x7f28dc009a10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:57.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.689+0000 7f28f6217740 1 -- 172.21.15.78:0/3929829109 shutdown_connections 2024-09-16T00:19:57.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.693+0000 7f28f6217740 1 -- 172.21.15.78:0/3929829109 wait complete. 2024-09-16T00:19:57.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.693+0000 7f28f6217740 1 -- start start 2024-09-16T00:19:57.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.694+0000 7f28f6217740 1 Processor -- start 2024-09-16T00:19:57.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.694+0000 7f28f6217740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683d99c0 0x555e683e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.694+0000 7f28f6217740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683da2e0 0x555e683e5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.694+0000 7f28f6217740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.694+0000 7f28f6217740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555e683dafd0 con 0x555e683eb9d0 2024-09-16T00:19:57.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.694+0000 7f28f6217740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555e682f9060 con 0x555e683d99c0 2024-09-16T00:19:57.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.694+0000 7f28f6217740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e683db150 con 0x555e683da2e0 2024-09-16T00:19:57.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.694+0000 7f28f6217740 10 client.0 _ll_register_callbacks cb 0x555e6804c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:57.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.696+0000 7f28f23db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.696+0000 7f28f23db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58584/0 (socket says 172.21.15.78:58584) 2024-09-16T00:19:57.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.696+0000 7f28f23db640 1 -- 172.21.15.78:0/4268510729 learned_addr learned my addr 172.21.15.78:0/4268510729 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:57.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.697+0000 7f28f13d9640 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683da2e0 0x555e683e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.698+0000 7f28f23db640 1 -- 172.21.15.78:0/4268510729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683da2e0 msgr2=0x555e683e5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:57.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.698+0000 7f28f23db640 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683da2e0 0x555e683e5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.698+0000 7f28f23db640 1 -- 172.21.15.78:0/4268510729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683d99c0 msgr2=0x555e683e53e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:19:57.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.698+0000 7f28f23db640 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683d99c0 0x555e683e53e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.698+0000 7f28f23db640 1 -- 172.21.15.78:0/4268510729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e680af3c0 con 0x555e683eb9d0 2024-09-16T00:19:57.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.702+0000 7f28f23db640 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eb0f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009230 tx=0x7f28e400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93883671593e342f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:57.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.703+0000 7f28d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:19:57.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.703+0000 7f28d57fa640 1 -- 172.21.15.78:0/4268510729 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40051d0 con 0x555e683eb9d0 2024-09-16T00:19:57.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.703+0000 7f28d57fa640 1 -- 172.21.15.78:0/4268510729 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f28e40989d0 con 0x555e683eb9d0 2024-09-16T00:19:57.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.704+0000 7f28f6217740 1 -- 172.21.15.78:0/4268510729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e40a4fd0 con 0x555e683eb9d0 2024-09-16T00:19:57.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.707+0000 7f28d57fa640 1 -- 172.21.15.78:0/4268510729 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f28e409b7b0 con 0x555e683eb9d0 2024-09-16T00:19:57.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.707+0000 7f28f6217740 10 client.16599 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:57.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.707+0000 7f28f6217740 1 -- 172.21.15.78:0/4268510729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e68402c10 con 0x555e683eb9d0 2024-09-16T00:19:57.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.710+0000 7f28d57fa640 1 -- 172.21.15.78:0/4268510729 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e40a4fd0 con 0x555e683eb9d0 2024-09-16T00:19:57.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.710+0000 7f28d57fa640 1 -- 172.21.15.78:0/4268510729 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f28e409b220 con 0x555e683eb9d0 2024-09-16T00:19:57.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.711+0000 7f28f6217740 10 client.16599 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:57.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.711+0000 7f28f6217740 10 client.16599 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:19:57.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.711+0000 7f28f6217740 20 client.16599 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:57.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.711+0000 7f28f6217740 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x555e68403f60 0x555e68406350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.711+0000 7f28f6217740 4 client.16599 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:19:57.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.711+0000 7f28f6217740 1 -- 172.21.15.78:0/4268510729 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e6834b860 con 0x555e68403f60 2024-09-16T00:19:57.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.715+0000 7f28f1bda640 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x555e68403f60 0x555e68406350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.717+0000 7f28f1bda640 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x555e68403f60 0x555e68406350 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4089db85332fcc3e server_cookie=88b80dcfb4478443 in_seq=0 out_seq=0 2024-09-16T00:19:57.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.717+0000 7f28d57fa640 10 client.16599 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:19:57.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.718+0000 7f28d57fa640 1 -- 172.21.15.78:0/4268510729 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e6834b860 con 0x555e68403f60 2024-09-16T00:19:57.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.718+0000 7f28d57fa640 10 client.16599 handle_command_reply: tid=0 2024-09-16T00:19:57.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.721+0000 7f28f6217740 1 client.16599 shutdown 2024-09-16T00:19:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.729+0000 7f28f6217740 1 -- 172.21.15.78:0/4268510729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 msgr2=0x555e683eb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.729+0000 7f28f6217740 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eb0f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009230 tx=0x7f28e400c890 comp rx=0 tx=0).stop 2024-09-16T00:19:57.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28f6217740 1 -- 172.21.15.78:0/4268510729 shutdown_connections 2024-09-16T00:19:57.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28f6217740 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683eb9d0 0x555e683eb0f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28f6217740 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683da2e0 0x555e683e5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28f6217740 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x555e68403f60 0x555e68406350 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28f6217740 1 --2- 172.21.15.78:0/4268510729 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683d99c0 0x555e683e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28f6217740 1 -- 172.21.15.78:0/4268510729 >> 172.21.15.78:0/4268510729 conn(0x555e683a1310 msgr2=0x555e683d3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:57.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28d57fa640 0 client.16599 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:19:57.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28f6217740 1 -- 172.21.15.78:0/4268510729 shutdown_connections 2024-09-16T00:19:57.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.731+0000 7f28f6217740 1 -- 172.21.15.78:0/4268510729 wait complete. 2024-09-16T00:19:57.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.732+0000 7f28f6217740 20 client.16599 trim_cache size 0 max 16384 2024-09-16T00:19:57.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.756+0000 7f28f6217740 1 Processor -- start 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.756+0000 7f28f6217740 1 -- start start 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.756+0000 7f28f6217740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e683eb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.756+0000 7f28f6217740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e6840b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.756+0000 7f28f6217740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683eb9d0 0x555e6840be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.756+0000 7f28f6217740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1b53d0 con 0x555e683d99c0 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.756+0000 7f28f6217740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1b6f10 con 0x555e683da2e0 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.756+0000 7f28f6217740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1b5aa0 con 0x555e683eb9d0 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683eb9d0 0x555e6840be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683eb9d0 0x555e6840be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37352/0 (socket says 172.21.15.78:37352) 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 -- 172.21.15.78:0/3461179486 learned_addr learned my addr 172.21.15.78:0/3461179486 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f1bda640 1 --2- 172.21.15.78:0/3461179486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e683eb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 -- 172.21.15.78:0/3461179486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 msgr2=0x555e6840b920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:19:57.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 --2- 172.21.15.78:0/3461179486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e6840b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 -- 172.21.15.78:0/3461179486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 msgr2=0x555e683eb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 --2- 172.21.15.78:0/3461179486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e683eb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 -- 172.21.15.78:0/3461179486 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e683dec50 con 0x555e683eb9d0 2024-09-16T00:19:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.757+0000 7f28f23db640 1 --2- 172.21.15.78:0/3461179486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683eb9d0 0x555e6840be60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28e4098d30 tx=0x7f28e4005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73d5dfdb6d61c703 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.758+0000 7f28d77fe640 1 -- 172.21.15.78:0/3461179486 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e409bea0 con 0x555e683eb9d0 2024-09-16T00:19:57.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.759+0000 7f28d77fe640 1 -- 172.21.15.78:0/3461179486 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e40ba070 con 0x555e683eb9d0 2024-09-16T00:19:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.760+0000 7f28f6217740 1 -- 172.21.15.78:0/3461179486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683eb9d0 msgr2=0x555e6840be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.760+0000 7f28f6217740 1 --2- 172.21.15.78:0/3461179486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683eb9d0 0x555e6840be60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f28e4098d30 tx=0x7f28e4005f70 comp rx=0 tx=0).stop 2024-09-16T00:19:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.760+0000 7f28f6217740 1 -- 172.21.15.78:0/3461179486 shutdown_connections 2024-09-16T00:19:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.760+0000 7f28f6217740 1 --2- 172.21.15.78:0/3461179486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683eb9d0 0x555e6840be60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.760+0000 7f28f6217740 1 --2- 172.21.15.78:0/3461179486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e6840b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.760+0000 7f28f6217740 1 --2- 172.21.15.78:0/3461179486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e683eb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.760+0000 7f28f6217740 1 -- 172.21.15.78:0/3461179486 >> 172.21.15.78:0/3461179486 conn(0x555e683a1310 msgr2=0x555e683df7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:57.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.764+0000 7f28f6217740 1 -- 172.21.15.78:0/3461179486 shutdown_connections 2024-09-16T00:19:57.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.768+0000 7f28f6217740 1 -- 172.21.15.78:0/3461179486 wait complete. 2024-09-16T00:19:57.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.768+0000 7f28f6217740 1 -- start start 2024-09-16T00:19:57.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.768+0000 7f28f6217740 1 Processor -- start 2024-09-16T00:19:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.769+0000 7f28f6217740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e68412bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.769+0000 7f28f6217740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e684130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.769+0000 7f28f6217740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683f2410 0x555e683f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.769+0000 7f28f6217740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1b6840 con 0x555e683d99c0 2024-09-16T00:19:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.769+0000 7f28f6217740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1b6170 con 0x555e683da2e0 2024-09-16T00:19:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.769+0000 7f28f6217740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1b33b0 con 0x555e683f2410 2024-09-16T00:19:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.769+0000 7f28f6217740 10 client.0 _ll_register_callbacks cb 0x555e683eb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:19:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.770+0000 7f28f1bda640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e68412bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.770+0000 7f28f1bda640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e68412bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58616/0 (socket says 172.21.15.78:58616) 2024-09-16T00:19:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.770+0000 7f28f1bda640 1 -- 172.21.15.78:0/917090478 learned_addr learned my addr 172.21.15.78:0/917090478 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:19:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.770+0000 7f28f13d9640 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e684130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.771+0000 7f28f23db640 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683f2410 0x555e683f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.776+0000 7f28f23db640 1 -- 172.21.15.78:0/917090478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 msgr2=0x555e684130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.776+0000 7f28f23db640 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e684130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.776+0000 7f28f23db640 1 -- 172.21.15.78:0/917090478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 msgr2=0x555e68412bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:19:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.776+0000 7f28f23db640 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e68412bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.776+0000 7f28f23db640 1 -- 172.21.15.78:0/917090478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e683f4e60 con 0x555e683f2410 2024-09-16T00:19:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.780+0000 7f28f23db640 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683f2410 0x555e683f48a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f28e40987c0 tx=0x7f28e40986c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aee6c6daa712b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:19:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.780+0000 7f28d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:19:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.781+0000 7f28d57fa640 1 -- 172.21.15.78:0/917090478 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e409bea0 con 0x555e683f2410 2024-09-16T00:19:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.781+0000 7f28d57fa640 1 -- 172.21.15.78:0/917090478 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f28e409b220 con 0x555e683f2410 2024-09-16T00:19:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.781+0000 7f28f6217740 1 -- 172.21.15.78:0/917090478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e68345d70 con 0x555e683f2410 2024-09-16T00:19:57.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.782+0000 7f28d57fa640 1 -- 172.21.15.78:0/917090478 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f28e40a4fd0 con 0x555e683f2410 2024-09-16T00:19:57.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.785+0000 7f28f6217740 10 client.25834 fetch_fsmap learned FSMap version 34 2024-09-16T00:19:57.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.785+0000 7f28f6217740 1 -- 172.21.15.78:0/917090478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555e683f4e60 con 0x555e683f2410 2024-09-16T00:19:57.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.786+0000 7f28d57fa640 1 -- 172.21.15.78:0/917090478 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555e68345d70 con 0x555e683f2410 2024-09-16T00:19:57.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.786+0000 7f28d57fa640 1 -- 172.21.15.78:0/917090478 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f28e40bc040 con 0x555e683f2410 2024-09-16T00:19:57.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.786+0000 7f28f6217740 10 client.25834 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:19:57.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.786+0000 7f28f6217740 10 client.25834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:19:57.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.786+0000 7f28f6217740 20 client.25834 populate_metadata read hostname 'smithi078' 2024-09-16T00:19:57.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.786+0000 7f28f6217740 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x555e683f63b0 0x555e683f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:19:57.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.786+0000 7f28f6217740 4 client.25834 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:19:57.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.786+0000 7f28f6217740 1 -- 172.21.15.78:0/917090478 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e682c2170 con 0x555e683f63b0 2024-09-16T00:19:57.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.787+0000 7f28f1bda640 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x555e683f63b0 0x555e683f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:19:57.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.789+0000 7f28f1bda640 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x555e683f63b0 0x555e683f87a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=187fa322e5606be6 server_cookie=b992bbf03a993090 in_seq=0 out_seq=0 2024-09-16T00:19:57.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.789+0000 7f28d57fa640 10 client.25834 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:19:57.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.789+0000 7f28d57fa640 1 -- 172.21.15.78:0/917090478 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e682c2170 con 0x555e683f63b0 2024-09-16T00:19:57.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.789+0000 7f28d57fa640 10 client.25834 handle_command_reply: tid=0 2024-09-16T00:19:57.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.789+0000 7f28f6217740 1 client.25834 shutdown 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 -- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683f2410 msgr2=0x555e683f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683f2410 0x555e683f48a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f28e40987c0 tx=0x7f28e40986c0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 -- 172.21.15.78:0/917090478 shutdown_connections 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e683f2410 0x555e683f48a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x555e683f63b0 0x555e683f87a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555e683da2e0 0x555e684130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 --2- 172.21.15.78:0/917090478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555e683d99c0 0x555e68412bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 -- 172.21.15.78:0/917090478 >> 172.21.15.78:0/917090478 conn(0x555e683a1310 msgr2=0x555e683d86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:57.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28d57fa640 0 client.25834 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:19:57.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.793+0000 7f28f6217740 1 -- 172.21.15.78:0/917090478 shutdown_connections 2024-09-16T00:19:57.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.794+0000 7f28f6217740 1 -- 172.21.15.78:0/917090478 wait complete. 2024-09-16T00:19:57.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.794+0000 7f28f6217740 20 client.25834 trim_cache size 0 max 16384 2024-09-16T00:19:57.797 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 -- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28d80a3410 msgr2=0x7f28d80a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28d80a3410 0x7f28d80a58d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009460 tx=0x7f28dc009a10 comp rx=0 tx=0).stop 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 -- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 msgr2=0x7f28ec06dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 0x7f28ec06dff0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009fd0 tx=0x7f28e400ef10 comp rx=0 tx=0).stop 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 -- 172.21.15.78:0/3047230170 shutdown_connections 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f28d80a3410 0x7f28d80a58d0 secure :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009460 tx=0x7f28dc009a10 comp rx=0 tx=0).stop 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28ec106a80 0x7f28ec06dff0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28ec101420 0x7f28ec06dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 --2- 172.21.15.78:0/3047230170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28ec100a80 0x7f28ec073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:19:57.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.795+0000 7f28f3e3e640 1 -- 172.21.15.78:0/3047230170 >> 172.21.15.78:0/3047230170 conn(0x7f28ec0fc7a0 msgr2=0x7f28ec0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:19:57.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.797+0000 7f28f3e3e640 1 -- 172.21.15.78:0/3047230170 shutdown_connections 2024-09-16T00:19:57.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:19:57.797+0000 7f28f3e3e640 1 -- 172.21.15.78:0/3047230170 wait complete. 2024-09-16T00:19:58.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:57 smithi078 ceph-mon[31157]: pgmap v557: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 585 op/s 2024-09-16T00:19:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:57 smithi193 ceph-mon[35484]: pgmap v557: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 585 op/s 2024-09-16T00:19:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:57 smithi148 ceph-mon[35599]: pgmap v557: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 585 op/s 2024-09-16T00:19:58.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:19:58.914 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:00.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:19:59 smithi078 ceph-mon[31157]: pgmap v558: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-16T00:20:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:19:59 smithi193 ceph-mon[35484]: pgmap v558: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-16T00:20:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:19:59 smithi148 ceph-mon[35599]: pgmap v558: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-16T00:20:00.757 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:00 smithi078 ceph-mon[31157]: overall HEALTH_OK 2024-09-16T00:20:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 360.0 s, 919.8 tps, lat 113.688 ms stddev 144.806 2024-09-16T00:20:00.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 -- 172.21.15.78:0/225873625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0102ef0 msgr2=0x7f20f010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 --2- 172.21.15.78:0/225873625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0102ef0 0x7f20f010da40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009cd0 tx=0x7f20e802a980 comp rx=0 tx=0).stop 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 -- 172.21.15.78:0/225873625 shutdown_connections 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 --2- 172.21.15.78:0/225873625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0102ef0 0x7f20f010da40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 --2- 172.21.15.78:0/225873625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20f0102570 0x7f20f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 --2- 172.21.15.78:0/225873625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20f0100e20 0x7f20f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 -- 172.21.15.78:0/225873625 >> 172.21.15.78:0/225873625 conn(0x7f20f00fc7d0 msgr2=0x7f20f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 -- 172.21.15.78:0/225873625 shutdown_connections 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.887+0000 7f20f65b0640 1 -- 172.21.15.78:0/225873625 wait complete. 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.888+0000 7f20f65b0640 1 Processor -- start 2024-09-16T00:20:00.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.888+0000 7f20f65b0640 1 -- start start 2024-09-16T00:20:00.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f65b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0100e20 0x7f20f01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f65b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20f0102570 0x7f20f0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f65b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20f0102ef0 0x7f20f019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f65b0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01120a0 con 0x7f20f0102570 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f65b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0111f20 con 0x7f20f0102ef0 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f65b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0112220 con 0x7f20f0100e20 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20f0102570 0x7f20f0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20f0102570 0x7f20f0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58630/0 (socket says 172.21.15.78:58630) 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20ef7fe640 1 -- 172.21.15.78:0/3612656911 learned_addr learned my addr 172.21.15.78:0/3612656911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f4b4d640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20f0102ef0 0x7f20f019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20effff640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0100e20 0x7f20f01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f4b4d640 1 -- 172.21.15.78:0/3612656911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0100e20 msgr2=0x7f20f01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f4b4d640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0100e20 0x7f20f01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f4b4d640 1 -- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20f0102570 msgr2=0x7f20f0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f4b4d640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20f0102570 0x7f20f0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20f4b4d640 1 -- 172.21.15.78:0/3612656911 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f019d490 con 0x7f20f0102ef0 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.889+0000 7f20effff640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0100e20 0x7f20f01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:20:00.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.890+0000 7f20f4b4d640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20f0102ef0 0x7f20f019cdb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005cc0 tx=0x7f20e802f620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:00.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.890+0000 7f20ed7fa640 1 -- 172.21.15.78:0/3612656911 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8034040 con 0x7f20f0102ef0 2024-09-16T00:20:00.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.890+0000 7f20f65b0640 1 -- 172.21.15.78:0/3612656911 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f019d720 con 0x7f20f0102ef0 2024-09-16T00:20:00.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.890+0000 7f20ed7fa640 1 -- 172.21.15.78:0/3612656911 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e8003e50 con 0x7f20f0102ef0 2024-09-16T00:20:00.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.890+0000 7f20ed7fa640 1 -- 172.21.15.78:0/3612656911 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80045e0 con 0x7f20f0102ef0 2024-09-16T00:20:00.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.890+0000 7f20f65b0640 1 -- 172.21.15.78:0/3612656911 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f010e400 con 0x7f20f0102ef0 2024-09-16T00:20:00.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.894+0000 7f20ed7fa640 1 -- 172.21.15.78:0/3612656911 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f20e8037020 con 0x7f20f0102ef0 2024-09-16T00:20:00.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20f8989740 1 Processor -- start 2024-09-16T00:20:00.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20f8989740 1 -- start start 2024-09-16T00:20:00.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20f8989740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f3436090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20f8989740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3436a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20f8989740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f3443760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20f8989740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9f339c2f0 con 0x55f9f3437b40 2024-09-16T00:20:00.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20f8989740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f9f3343cf0 con 0x55f9f3441370 2024-09-16T00:20:00.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20f8989740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9f33982b0 con 0x55f9f34365d0 2024-09-16T00:20:00.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f3436090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20effff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3436a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f3436090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58646/0 (socket says 172.21.15.78:58646) 2024-09-16T00:20:00.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20effff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3436a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37392/0 (socket says 172.21.15.78:37392) 2024-09-16T00:20:00.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.895+0000 7f20ef7fe640 1 -- 172.21.15.78:0/1535729259 learned_addr learned my addr 172.21.15.78:0/1535729259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:00.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/1535729259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f3443760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20effff640 1 -- 172.21.15.78:0/1535729259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 msgr2=0x55f9f3443760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:00.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20effff640 1 --2- 172.21.15.78:0/1535729259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f3443760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20effff640 1 -- 172.21.15.78:0/1535729259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 msgr2=0x55f9f3436090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20effff640 1 --2- 172.21.15.78:0/1535729259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f3436090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20effff640 1 -- 172.21.15.78:0/1535729259 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f33283a0 con 0x55f9f34365d0 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20effff640 1 --2- 172.21.15.78:0/1535729259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3436a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009c30 tx=0x7f20e40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcde4bf65923e7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20ed7fa640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20d80a3240 0x7f20d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20d67fc640 1 -- 172.21.15.78:0/1535729259 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4012450 con 0x55f9f34365d0 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20d67fc640 1 -- 172.21.15.78:0/1535729259 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e4012ad0 con 0x55f9f34365d0 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.896+0000 7f20d67fc640 1 -- 172.21.15.78:0/1535729259 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e400fab0 con 0x55f9f34365d0 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20d80a3240 0x7f20d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20ed7fa640 1 -- 172.21.15.78:0/3612656911 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f20e80f0bc0 con 0x7f20f0102ef0 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20f8989740 1 -- 172.21.15.78:0/1535729259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 msgr2=0x55f9f3436a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20f8989740 1 --2- 172.21.15.78:0/1535729259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3436a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009c30 tx=0x7f20e40117b0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20f8989740 1 -- 172.21.15.78:0/1535729259 shutdown_connections 2024-09-16T00:20:00.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20f8989740 1 --2- 172.21.15.78:0/1535729259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f3443760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20f8989740 1 --2- 172.21.15.78:0/1535729259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3436a10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20f8989740 1 --2- 172.21.15.78:0/1535729259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f3436090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20f8989740 1 -- 172.21.15.78:0/1535729259 >> 172.21.15.78:0/1535729259 conn(0x55f9f33ff310 msgr2=0x55f9f3432c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.897+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20d80a3240 0x7f20d80a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005e00 tx=0x7f20e0014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.898+0000 7f20f8989740 1 -- 172.21.15.78:0/1535729259 shutdown_connections 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.898+0000 7f20f8989740 1 -- 172.21.15.78:0/1535729259 wait complete. 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.898+0000 7f20f8989740 1 -- start start 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.898+0000 7f20f8989740 1 Processor -- start 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f8989740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3440ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f8989740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f343df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f8989740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f343e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f8989740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9f3444070 con 0x55f9f3437b40 2024-09-16T00:20:00.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f8989740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f9f3357060 con 0x55f9f3441370 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f8989740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9f34441f0 con 0x55f9f34365d0 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f8989740 10 client.0 _ll_register_callbacks cb 0x55f9f30aa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f343df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f4b4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f343e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20effff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3440ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f343df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58648/0 (socket says 172.21.15.78:58648) 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f4b4d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f343e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60082/0 (socket says 172.21.15.78:60082) 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20ef7fe640 1 -- 172.21.15.78:0/4236040605 learned_addr learned my addr 172.21.15.78:0/4236040605 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20effff640 1 -- 172.21.15.78:0/4236040605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 msgr2=0x55f9f343e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20effff640 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f343e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20effff640 1 -- 172.21.15.78:0/4236040605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 msgr2=0x55f9f343df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20effff640 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f343df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20effff640 1 -- 172.21.15.78:0/4236040605 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f310d3c0 con 0x55f9f34365d0 2024-09-16T00:20:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.899+0000 7f20f4b4d640 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f343e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.900+0000 7f20effff640 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3440ce0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f20e4005830 tx=0x7f20e4012f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31986a8ed496b082 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.900+0000 7f20cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:20:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.900+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4236040605 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4029020 con 0x55f9f34365d0 2024-09-16T00:20:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.900+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4236040605 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f20e4012760 con 0x55f9f34365d0 2024-09-16T00:20:00.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.900+0000 7f20f8989740 1 -- 172.21.15.78:0/4236040605 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e4012ad0 con 0x55f9f34365d0 2024-09-16T00:20:00.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.900+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4236040605 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f20e4034050 con 0x55f9f34365d0 2024-09-16T00:20:00.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.901+0000 7f20f8989740 10 client.25846 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.901+0000 7f20f8989740 1 -- 172.21.15.78:0/4236040605 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f9f344fa80 con 0x55f9f34365d0 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4236040605 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e4012ad0 con 0x55f9f34365d0 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4236040605 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f20e400fd10 con 0x55f9f34365d0 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20f8989740 10 client.25846 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20f8989740 10 client.25846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20f8989740 20 client.25846 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20f8989740 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55f9f3450ee0 0x55f9f34532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20f8989740 4 client.25846 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20f8989740 1 -- 172.21.15.78:0/4236040605 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9f33a9860 con 0x55f9f3450ee0 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.903+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55f9f3450ee0 0x55f9f34532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.904+0000 7f20cf7fe640 10 client.25846 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.904+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55f9f3450ee0 0x55f9f34532d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25d5bdfe50028f87 server_cookie=e22647b3f5c54e7e in_seq=0 out_seq=0 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4236040605 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9f33a9860 con 0x55f9f3450ee0 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20cf7fe640 10 client.25846 handle_command_reply: tid=0 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 client.25846 shutdown 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 -- 172.21.15.78:0/4236040605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 msgr2=0x55f9f3440ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3440ce0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f20e4005830 tx=0x7f20e4012f30 comp rx=0 tx=0).stop 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 -- 172.21.15.78:0/4236040605 shutdown_connections 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55f9f3450ee0 0x55f9f34532d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f343e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3437b40 0x55f9f343df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 --2- 172.21.15.78:0/4236040605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3440ce0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20f8989740 1 -- 172.21.15.78:0/4236040605 >> 172.21.15.78:0/4236040605 conn(0x55f9f33ff310 msgr2=0x55f9f3438960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.905+0000 7f20cf7fe640 0 client.25846 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.906+0000 7f20f8989740 1 -- 172.21.15.78:0/4236040605 shutdown_connections 2024-09-16T00:20:00.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.906+0000 7f20f8989740 1 -- 172.21.15.78:0/4236040605 wait complete. 2024-09-16T00:20:00.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.906+0000 7f20f8989740 20 client.25846 trim_cache size 0 max 16384 2024-09-16T00:20:00.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.921+0000 7f20f8989740 1 Processor -- start 2024-09-16T00:20:00.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20f8989740 1 -- start start 2024-09-16T00:20:00.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20f8989740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f34365d0 0x55f9f344d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20f8989740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f3437b40 0x55f9f3439120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20f8989740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f3439660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20f8989740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b0dd0 con 0x55f9f34365d0 2024-09-16T00:20:00.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20f8989740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b2910 con 0x55f9f3441370 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20f8989740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b14a0 con 0x55f9f3437b40 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f34365d0 0x55f9f344d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f34365d0 0x55f9f344d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58652/0 (socket says 172.21.15.78:58652) 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20effff640 1 -- 172.21.15.78:0/4091928128 learned_addr learned my addr 172.21.15.78:0/4091928128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20f4b4d640 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f3439660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f3437b40 0x55f9f3439120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20effff640 1 -- 172.21.15.78:0/4091928128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f3437b40 msgr2=0x55f9f3439120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20effff640 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f3437b40 0x55f9f3439120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20effff640 1 -- 172.21.15.78:0/4091928128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 msgr2=0x55f9f3439660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20effff640 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f3439660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.922+0000 7f20effff640 1 -- 172.21.15.78:0/4091928128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f3439ba0 con 0x55f9f34365d0 2024-09-16T00:20:00.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.923+0000 7f20effff640 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f34365d0 0x55f9f344d310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f20e4011cb0 tx=0x7f20e4009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940ddfb4e57eacbb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.923+0000 7f20d5ffb640 1 -- 172.21.15.78:0/4091928128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e403d070 con 0x55f9f34365d0 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.923+0000 7f20d5ffb640 1 -- 172.21.15.78:0/4091928128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e4009530 con 0x55f9f34365d0 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.923+0000 7f20d5ffb640 1 -- 172.21.15.78:0/4091928128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4029070 con 0x55f9f34365d0 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 -- 172.21.15.78:0/4091928128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f34365d0 msgr2=0x55f9f344d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f34365d0 0x55f9f344d310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f20e4011cb0 tx=0x7f20e4009a80 comp rx=0 tx=0).stop 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 -- 172.21.15.78:0/4091928128 shutdown_connections 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3441370 0x55f9f3439660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f3437b40 0x55f9f3439120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 --2- 172.21.15.78:0/4091928128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f34365d0 0x55f9f344d310 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 -- 172.21.15.78:0/4091928128 >> 172.21.15.78:0/4091928128 conn(0x55f9f33ff310 msgr2=0x55f9f343d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 -- 172.21.15.78:0/4091928128 shutdown_connections 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 -- 172.21.15.78:0/4091928128 wait complete. 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.924+0000 7f20f8989740 1 -- start start 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f8989740 1 Processor -- start 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f8989740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3449480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f8989740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3437b40 0x55f9f34499c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f8989740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3441370 0x55f9f343a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f8989740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b2240 con 0x55f9f3441370 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f8989740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b1b70 con 0x55f9f3437b40 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f8989740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01aedb0 con 0x55f9f34365d0 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3437b40 0x55f9f34499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20ef7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3437b40 0x55f9f34499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60108/0 (socket says 172.21.15.78:60108) 2024-09-16T00:20:00.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20ef7fe640 1 -- 172.21.15.78:0/4121951773 learned_addr learned my addr 172.21.15.78:0/4121951773 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f8989740 10 client.0 _ll_register_callbacks cb 0x7f20e0004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20effff640 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3449480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f4b4d640 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3441370 0x55f9f343a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20ef7fe640 1 -- 172.21.15.78:0/4121951773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 msgr2=0x55f9f3449480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3449480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20ef7fe640 1 -- 172.21.15.78:0/4121951773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3441370 msgr2=0x55f9f343a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3441370 0x55f9f343a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20ef7fe640 1 -- 172.21.15.78:0/4121951773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f343be20 con 0x55f9f3437b40 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.925+0000 7f20f4b4d640 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3441370 0x55f9f343a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20ef7fe640 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3437b40 0x55f9f34499c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f20e0004650 tx=0x7f20e000b4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193082331aa8662 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4121951773 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0023b60 con 0x55f9f3437b40 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20f8989740 1 -- 172.21.15.78:0/4121951773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e4009530 con 0x55f9f3437b40 2024-09-16T00:20:00.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4121951773 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f20e0023d00 con 0x55f9f3437b40 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4121951773 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f20e003c050 con 0x55f9f3437b40 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20f8989740 10 client.26012 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20f8989740 1 -- 172.21.15.78:0/4121951773 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f9f34bae30 con 0x55f9f3437b40 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.926+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4121951773 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e4009530 con 0x55f9f3437b40 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4121951773 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f20e4009d20 con 0x55f9f3437b40 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20f8989740 10 client.26012 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20f8989740 10 client.26012 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20f8989740 20 client.26012 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20f8989740 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55f9f34bb540 0x55f9f34bd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20f8989740 4 client.26012 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20f8989740 1 -- 172.21.15.78:0/4121951773 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9f33a3d70 con 0x55f9f34bb540 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20effff640 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55f9f34bb540 0x55f9f34bd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20cf7fe640 10 client.26012 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:20:00.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.927+0000 7f20effff640 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55f9f34bb540 0x55f9f34bd960 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=622e343d86183f3b server_cookie=cb9ad7e69740bd77 in_seq=0 out_seq=0 2024-09-16T00:20:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20cf7fe640 1 -- 172.21.15.78:0/4121951773 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9f33a3d70 con 0x55f9f34bb540 2024-09-16T00:20:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20cf7fe640 10 client.26012 handle_command_reply: tid=0 2024-09-16T00:20:00.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 client.26012 shutdown 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 -- 172.21.15.78:0/4121951773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3437b40 msgr2=0x55f9f34499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3437b40 0x55f9f34499c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f20e0004650 tx=0x7f20e000b4a0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 -- 172.21.15.78:0/4121951773 shutdown_connections 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f3441370 0x55f9f343a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55f9f34bb540 0x55f9f34bd960 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9f3437b40 0x55f9f34499c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 --2- 172.21.15.78:0/4121951773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9f34365d0 0x55f9f3449480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20cf7fe640 0 client.26012 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 -- 172.21.15.78:0/4121951773 >> 172.21.15.78:0/4121951773 conn(0x55f9f33ff310 msgr2=0x55f9f34314e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 -- 172.21.15.78:0/4121951773 shutdown_connections 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 1 -- 172.21.15.78:0/4121951773 wait complete. 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.928+0000 7f20f8989740 20 client.26012 trim_cache size 0 max 16384 2024-09-16T00:20:00.934 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:20:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 -- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20d80a3240 msgr2=0x7f20d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20d80a3240 0x7f20d80a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005e00 tx=0x7f20e0014040 comp rx=0 tx=0).stop 2024-09-16T00:20:00.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 -- 172.21.15.78:0/3612656911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20f0102ef0 msgr2=0x7f20f019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20f0102ef0 0x7f20f019cdb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f20e8005cc0 tx=0x7f20e802f620 comp rx=0 tx=0).stop 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 -- 172.21.15.78:0/3612656911 shutdown_connections 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20d80a3240 0x7f20d80a5700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20f0102ef0 0x7f20f019cdb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20f0102570 0x7f20f0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 --2- 172.21.15.78:0/3612656911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0100e20 0x7f20f01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 -- 172.21.15.78:0/3612656911 >> 172.21.15.78:0/3612656911 conn(0x7f20f00fc7d0 msgr2=0x7f20f01048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 -- 172.21.15.78:0/3612656911 shutdown_connections 2024-09-16T00:20:00.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:00.929+0000 7f20f65b0640 1 -- 172.21.15.78:0/3612656911 wait complete. 2024-09-16T00:20:01.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:00 smithi193 ceph-mon[35484]: overall HEALTH_OK 2024-09-16T00:20:01.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:00 smithi148 ceph-mon[35599]: overall HEALTH_OK 2024-09-16T00:20:01.756 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:01 smithi078 ceph-mon[31157]: pgmap v559: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 669 op/s 2024-09-16T00:20:01.757 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:20:02.139 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:02.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:01 smithi193 ceph-mon[35484]: pgmap v559: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 669 op/s 2024-09-16T00:20:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:01 smithi148 ceph-mon[35599]: pgmap v559: 129 pgs: 129 active+clean; 1.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 669 op/s 2024-09-16T00:20:02.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.714+0000 7fd4e3645640 1 -- 172.21.15.78:0/995963287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc109450 msgr2=0x7fd4dc110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.714+0000 7fd4e3645640 1 --2- 172.21.15.78:0/995963287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc109450 0x7fd4dc110250 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009900 tx=0x7fd4d402a440 comp rx=0 tx=0).stop 2024-09-16T00:20:02.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.717+0000 7fd4e3645640 1 -- 172.21.15.78:0/995963287 shutdown_connections 2024-09-16T00:20:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.717+0000 7fd4e3645640 1 --2- 172.21.15.78:0/995963287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4dc11b140 0x7fd4dc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.717+0000 7fd4e3645640 1 --2- 172.21.15.78:0/995963287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4dc109830 0x7fd4dc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.717+0000 7fd4e3645640 1 --2- 172.21.15.78:0/995963287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc109450 0x7fd4dc110250 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.717+0000 7fd4e3645640 1 -- 172.21.15.78:0/995963287 >> 172.21.15.78:0/995963287 conn(0x7fd4dc06e140 msgr2=0x7fd4dc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.725+0000 7fd4e3645640 1 -- 172.21.15.78:0/995963287 shutdown_connections 2024-09-16T00:20:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.725+0000 7fd4e3645640 1 -- 172.21.15.78:0/995963287 wait complete. 2024-09-16T00:20:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.726+0000 7fd4e3645640 1 Processor -- start 2024-09-16T00:20:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.726+0000 7fd4e3645640 1 -- start start 2024-09-16T00:20:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.726+0000 7fd4e3645640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4dc109450 0x7fd4dc1a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.726+0000 7fd4e3645640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4dc109830 0x7fd4dc1a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.726+0000 7fd4e3645640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc11b140 0x7fd4dc1abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.726+0000 7fd4e3645640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc120ed0 con 0x7fd4dc11b140 2024-09-16T00:20:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.726+0000 7fd4e3645640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc120d50 con 0x7fd4dc109450 2024-09-16T00:20:02.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.726+0000 7fd4e3645640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd4dc121050 con 0x7fd4dc109830 2024-09-16T00:20:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.727+0000 7fd4e13e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4dc109450 0x7fd4dc1a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.727+0000 7fd4e13e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4dc109450 0x7fd4dc1a74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60114/0 (socket says 172.21.15.78:60114) 2024-09-16T00:20:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.727+0000 7fd4e13e1640 1 -- 172.21.15.78:0/3546056124 learned_addr learned my addr 172.21.15.78:0/3546056124 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:02.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.729+0000 7fd4e1be2640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc11b140 0x7fd4dc1abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.730+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4dc109830 0x7fd4dc1a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.730+0000 7fd4e1be2640 1 -- 172.21.15.78:0/3546056124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4dc109830 msgr2=0x7fd4dc1a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.730+0000 7fd4e1be2640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4dc109830 0x7fd4dc1a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.730+0000 7fd4e1be2640 1 -- 172.21.15.78:0/3546056124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4dc109450 msgr2=0x7fd4dc1a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.730+0000 7fd4e1be2640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4dc109450 0x7fd4dc1a74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.730+0000 7fd4e1be2640 1 -- 172.21.15.78:0/3546056124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4dc1ac480 con 0x7fd4dc11b140 2024-09-16T00:20:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.731+0000 7fd4e1be2640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc11b140 0x7fd4dc1abd80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800be00 tx=0x7fd4d800bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.731+0000 7fd4ce7fc640 1 -- 172.21.15.78:0/3546056124 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8019070 con 0x7fd4dc11b140 2024-09-16T00:20:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.731+0000 7fd4e3645640 1 -- 172.21.15.78:0/3546056124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4dc1ac770 con 0x7fd4dc11b140 2024-09-16T00:20:02.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.731+0000 7fd4e3645640 1 -- 172.21.15.78:0/3546056124 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4dc110920 con 0x7fd4dc11b140 2024-09-16T00:20:02.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.735+0000 7fd4e5a1e740 1 Processor -- start 2024-09-16T00:20:02.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.735+0000 7fd4e5a1e740 1 -- start start 2024-09-16T00:20:02.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.735+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e1b40 0x5592640e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.735+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e05d0 0x5592640e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592640ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5592640462f0 con 0x5592640eb370 2024-09-16T00:20:02.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559263fedcf0 con 0x5592640e1b40 2024-09-16T00:20:02.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5592640422b0 con 0x5592640e05d0 2024-09-16T00:20:02.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4ce7fc640 1 -- 172.21.15.78:0/3546056124 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d800cd90 con 0x7fd4dc11b140 2024-09-16T00:20:02.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4ce7fc640 1 -- 172.21.15.78:0/3546056124 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80128f0 con 0x7fd4dc11b140 2024-09-16T00:20:02.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4e0be0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e1b40 0x5592640e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4e0be0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e1b40 0x5592640e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60116/0 (socket says 172.21.15.78:60116) 2024-09-16T00:20:02.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4e0be0640 1 -- 172.21.15.78:0/817461237 learned_addr learned my addr 172.21.15.78:0/817461237 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:02.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.736+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e05d0 0x5592640e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.737+0000 7fd4e0be0640 1 -- 172.21.15.78:0/817461237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e05d0 msgr2=0x5592640e0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:02.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.737+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e05d0 0x5592640e0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.737+0000 7fd4e0be0640 1 -- 172.21.15.78:0/817461237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 msgr2=0x5592640ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:02.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.737+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592640ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.737+0000 7fd4e0be0640 1 -- 172.21.15.78:0/817461237 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559263fd23a0 con 0x5592640e1b40 2024-09-16T00:20:02.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.738+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e1b40 0x5592640e0090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000c940 tx=0x7fd4d000ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abc0a4eb3a32dfa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:02.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.738+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e05d0 0x5592640e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:02.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.739+0000 7fd4c77fe640 1 -- 172.21.15.78:0/817461237 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000f040 con 0x5592640e1b40 2024-09-16T00:20:02.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.739+0000 7fd4c77fe640 1 -- 172.21.15.78:0/817461237 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d0018070 con 0x5592640e1b40 2024-09-16T00:20:02.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.739+0000 7fd4c77fe640 1 -- 172.21.15.78:0/817461237 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0012390 con 0x5592640e1b40 2024-09-16T00:20:02.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.740+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/817461237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e1b40 msgr2=0x5592640e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.740+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e1b40 0x5592640e0090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000c940 tx=0x7fd4d000ce00 comp rx=0 tx=0).stop 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.740+0000 7fd4ce7fc640 1 -- 172.21.15.78:0/3546056124 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd4d801c070 con 0x7fd4dc11b140 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.741+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/817461237 shutdown_connections 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.741+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592640ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.741+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e05d0 0x5592640e0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.741+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/817461237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e1b40 0x5592640e0090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.741+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/817461237 >> 172.21.15.78:0/817461237 conn(0x5592640a9310 msgr2=0x5592640dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.741+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/817461237 shutdown_connections 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.741+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/817461237 wait complete. 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.742+0000 7fd4ce7fc640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4c80a32e0 0x7fd4c80a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.742+0000 7fd4ce7fc640 1 -- 172.21.15.78:0/3546056124 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd4d80cf2d0 con 0x7fd4dc11b140 2024-09-16T00:20:02.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.742+0000 7fd4e5a1e740 1 -- start start 2024-09-16T00:20:02.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.743+0000 7fd4e5a1e740 1 Processor -- start 2024-09-16T00:20:02.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.744+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640e05d0 0x5592640ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.744+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.744+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4c80a32e0 0x7fd4c80a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.744+0000 7fd4e13e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640e05d0 0x5592640ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.744+0000 7fd4e13e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640e05d0 0x5592640ead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58696/0 (socket says 172.21.15.78:58696) 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.745+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640eb370 0x5592640e8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.745+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5592640ee070 con 0x5592640e05d0 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.745+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559264001060 con 0x5592640eb370 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.745+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5592640ee1f0 con 0x5592640e1b40 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.745+0000 7fd4e5a1e740 10 client.0 _ll_register_callbacks cb 0x559263d548c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.745+0000 7fd4e13e1640 1 -- 172.21.15.78:0/1737920291 learned_addr learned my addr 172.21.15.78:0/1737920291 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.746+0000 7fd4e1be2640 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640eb370 0x5592640e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.747+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4c80a32e0 0x7fd4c80a57a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd4d402aa30 tx=0x7fd4d4031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.747+0000 7fd4e13e1640 1 -- 172.21.15.78:0/1737920291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 msgr2=0x5592640e7fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.747+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640e7fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.747+0000 7fd4e13e1640 1 -- 172.21.15.78:0/1737920291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640eb370 msgr2=0x5592640e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.747+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640eb370 0x5592640e8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.747+0000 7fd4e13e1640 1 -- 172.21.15.78:0/1737920291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559263db73c0 con 0x5592640e05d0 2024-09-16T00:20:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.748+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640e05d0 0x5592640ead00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd4d403f730 tx=0x7fd4d4046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e0712378c83194 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.748+0000 7fd4c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:20:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.750+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/1737920291 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d404f070 con 0x5592640e05d0 2024-09-16T00:20:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.750+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/1737920291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d0018070 con 0x5592640e05d0 2024-09-16T00:20:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.750+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/1737920291 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd4d403fc20 con 0x5592640e05d0 2024-09-16T00:20:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.751+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/1737920291 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd4d404f070 con 0x5592640e05d0 2024-09-16T00:20:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.751+0000 7fd4e5a1e740 10 client.16629 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.752+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/1737920291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5592640f9b00 con 0x5592640e05d0 2024-09-16T00:20:02.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.754+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/1737920291 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d0018070 con 0x5592640e05d0 2024-09-16T00:20:02.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.755+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/1737920291 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd4d4003ad0 con 0x5592640e05d0 2024-09-16T00:20:02.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.755+0000 7fd4e5a1e740 10 client.16629 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:02.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.755+0000 7fd4e5a1e740 10 client.16629 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:20:02.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.756+0000 7fd4e5a1e740 20 client.16629 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:02.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.756+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5592640fb390 0x5592640fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.756+0000 7fd4e5a1e740 4 client.16629 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:02.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.756+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/1737920291 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559264053860 con 0x5592640fb390 2024-09-16T00:20:02.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.757+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5592640fb390 0x5592640fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.759+0000 7fd4c4ff9640 10 client.16629 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:20:02.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.759+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5592640fb390 0x5592640fd850 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed53275fa73305ad server_cookie=551d56fa9dfeb1d0 in_seq=0 out_seq=0 2024-09-16T00:20:02.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/1737920291 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559264053860 con 0x5592640fb390 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4c4ff9640 10 client.16629 handle_command_reply: tid=0 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 client.16629 shutdown 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/1737920291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640e05d0 msgr2=0x5592640ead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640e05d0 0x5592640ead00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd4d403f730 tx=0x7fd4d4046000 comp rx=0 tx=0).stop 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/1737920291 shutdown_connections 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5592640fb390 0x5592640fd850 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4c4ff9640 0 client.16629 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640eb370 0x5592640e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/1737920291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640e05d0 0x5592640ead00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.760+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/1737920291 >> 172.21.15.78:0/1737920291 conn(0x5592640a9310 msgr2=0x5592640e2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.761+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/1737920291 shutdown_connections 2024-09-16T00:20:02.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.761+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/1737920291 wait complete. 2024-09-16T00:20:02.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.761+0000 7fd4e5a1e740 20 client.16629 trim_cache size 0 max 16384 2024-09-16T00:20:02.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e5a1e740 1 Processor -- start 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e5a1e740 1 -- start start 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x559264103160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592641036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c000fda0 con 0x5592640eb370 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c00118e0 con 0x5592640e05d0 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0010470 con 0x5592640e1b40 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e13e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e13e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60132/0 (socket says 172.21.15.78:60132) 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e13e1640 1 -- 172.21.15.78:0/254868442 learned_addr learned my addr 172.21.15.78:0/254868442 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e1be2640 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592641036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.777+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x559264103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4e13e1640 1 -- 172.21.15.78:0/254868442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 msgr2=0x559264103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x559264103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4e13e1640 1 -- 172.21.15.78:0/254868442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 msgr2=0x5592641036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592641036a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4e13e1640 1 -- 172.21.15.78:0/254868442 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592640dd960 con 0x5592640e05d0 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f73a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4000c00 tx=0x7fd4d4003870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e814698787ea97dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4c6ffd640 1 -- 172.21.15.78:0/254868442 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d404f070 con 0x5592640e05d0 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4c6ffd640 1 -- 172.21.15.78:0/254868442 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d0018070 con 0x5592640e05d0 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.778+0000 7fd4c6ffd640 1 -- 172.21.15.78:0/254868442 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d403ac60 con 0x5592640e05d0 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/254868442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 msgr2=0x5592640f73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f73a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4000c00 tx=0x7fd4d4003870 comp rx=0 tx=0).stop 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/254868442 shutdown_connections 2024-09-16T00:20:02.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592641036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x559264103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/254868442 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f73a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/254868442 >> 172.21.15.78:0/254868442 conn(0x5592640a9310 msgr2=0x5592640e6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/254868442 shutdown_connections 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.779+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/254868442 wait complete. 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.780+0000 7fd4e5a1e740 1 -- start start 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.780+0000 7fd4e5a1e740 1 Processor -- start 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e5a1e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592640ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0011210 con 0x5592640eb370 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0010b40 con 0x5592640e05d0 2024-09-16T00:20:02.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e5a1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c000dd20 con 0x5592640e1b40 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e0be0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e0be0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640eef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37454/0 (socket says 172.21.15.78:37454) 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e0be0640 1 -- 172.21.15.78:0/884520495 learned_addr learned my addr 172.21.15.78:0/884520495 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.781+0000 7fd4e5a1e740 10 client.0 _ll_register_callbacks cb 0x7fd4d4031a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4e0be0640 1 -- 172.21.15.78:0/884520495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 msgr2=0x5592640f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4e0be0640 1 -- 172.21.15.78:0/884520495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 msgr2=0x5592640ef4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:20:02.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592640ef4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4e0be0640 1 -- 172.21.15.78:0/884520495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592640efcb0 con 0x5592640e1b40 2024-09-16T00:20:02.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4e0be0640 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640eef90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0008010 tx=0x7fd4d0006a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64d7af0d84ee3216 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:02.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:20:02.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.782+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/884520495 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0006b10 con 0x5592640e1b40 2024-09-16T00:20:02.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/884520495 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd4d0013640 con 0x5592640e1b40 2024-09-16T00:20:02.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/884520495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d0018070 con 0x5592640e1b40 2024-09-16T00:20:02.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/884520495 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd4d0004030 con 0x5592640e1b40 2024-09-16T00:20:02.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4e5a1e740 10 client.25858 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:02.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/884520495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559264166140 con 0x5592640e1b40 2024-09-16T00:20:02.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/884520495 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d0018070 con 0x5592640e1b40 2024-09-16T00:20:02.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/884520495 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd4d0012070 con 0x5592640e1b40 2024-09-16T00:20:02.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4e5a1e740 10 client.25858 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:02.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4e5a1e740 10 client.25858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:20:02.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4e5a1e740 20 client.25858 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:02.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.783+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5592640fb390 0x5592641687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:02.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.784+0000 7fd4e5a1e740 4 client.25858 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:20:02.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.784+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/884520495 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55926404dd70 con 0x5592640fb390 2024-09-16T00:20:02.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.784+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5592640fb390 0x5592641687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:02.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.784+0000 7fd4c4ff9640 10 client.25858 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:20:02.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.784+0000 7fd4e13e1640 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5592640fb390 0x5592641687a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=335f8a423d5eb6b8 server_cookie=7b200624fa687c3c in_seq=0 out_seq=0 2024-09-16T00:20:02.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.784+0000 7fd4c4ff9640 1 -- 172.21.15.78:0/884520495 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55926404dd70 con 0x5592640fb390 2024-09-16T00:20:02.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.784+0000 7fd4c4ff9640 10 client.25858 handle_command_reply: tid=0 2024-09-16T00:20:02.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.784+0000 7fd4e5a1e740 1 client.25858 shutdown 2024-09-16T00:20:02.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/884520495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 msgr2=0x5592640eef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640eef90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0008010 tx=0x7fd4d0006a20 comp rx=0 tx=0).stop 2024-09-16T00:20:02.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/884520495 shutdown_connections 2024-09-16T00:20:02.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5592640fb390 0x5592641687a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5592640eb370 0x5592640ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4c4ff9640 0 client.25858 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:20:02.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5592640e1b40 0x5592640eef90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 --2- 172.21.15.78:0/884520495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5592640e05d0 0x5592640f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/884520495 >> 172.21.15.78:0/884520495 conn(0x5592640a9310 msgr2=0x5592640db4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:02.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/884520495 shutdown_connections 2024-09-16T00:20:02.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 1 -- 172.21.15.78:0/884520495 wait complete. 2024-09-16T00:20:02.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.785+0000 7fd4e5a1e740 20 client.25858 trim_cache size 0 max 16384 2024-09-16T00:20:02.794 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:20:02.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 -- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4c80a32e0 msgr2=0x7fd4c80a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4c80a32e0 0x7fd4c80a57a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd4d402aa30 tx=0x7fd4d4031640 comp rx=0 tx=0).stop 2024-09-16T00:20:02.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 -- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc11b140 msgr2=0x7fd4dc1abd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:02.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc11b140 0x7fd4dc1abd80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800be00 tx=0x7fd4d800bef0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 -- 172.21.15.78:0/3546056124 shutdown_connections 2024-09-16T00:20:02.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd4c80a32e0 0x7fd4c80a57a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4dc11b140 0x7fd4dc1abd80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4dc109830 0x7fd4dc1a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 --2- 172.21.15.78:0/3546056124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4dc109450 0x7fd4dc1a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:02.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 -- 172.21.15.78:0/3546056124 >> 172.21.15.78:0/3546056124 conn(0x7fd4dc06e140 msgr2=0x7fd4dc11be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:02.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 -- 172.21.15.78:0/3546056124 shutdown_connections 2024-09-16T00:20:02.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:02.786+0000 7fd4e3645640 1 -- 172.21.15.78:0/3546056124 wait complete. 2024-09-16T00:20:03.514 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:20:03.964 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:04.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:03 smithi078 ceph-mon[31157]: pgmap v560: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 712 op/s 2024-09-16T00:20:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:03 smithi193 ceph-mon[35484]: pgmap v560: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 712 op/s 2024-09-16T00:20:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:03 smithi148 ceph-mon[35599]: pgmap v560: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 15 MiB/s wr, 712 op/s 2024-09-16T00:20:05.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:04 smithi078 ceph-mon[31157]: pgmap v561: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 674 op/s 2024-09-16T00:20:05.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:04 smithi193 ceph-mon[35484]: pgmap v561: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 674 op/s 2024-09-16T00:20:05.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:04 smithi148 ceph-mon[35599]: pgmap v561: 129 pgs: 129 active+clean; 1.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 14 MiB/s wr, 674 op/s 2024-09-16T00:20:05.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 -- 172.21.15.78:0/2190866613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4112a20 msgr2=0x7f65c4114e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 --2- 172.21.15.78:0/2190866613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4112a20 0x7f65c4114e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009900 tx=0x7f65b002a410 comp rx=0 tx=0).stop 2024-09-16T00:20:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 -- 172.21.15.78:0/2190866613 shutdown_connections 2024-09-16T00:20:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 --2- 172.21.15.78:0/2190866613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4112a20 0x7f65c4114e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 --2- 172.21.15.78:0/2190866613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c4077040 0x7f65c41124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 --2- 172.21.15.78:0/2190866613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c4076720 0x7f65c4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 -- 172.21.15.78:0/2190866613 >> 172.21.15.78:0/2190866613 conn(0x7f65c40ffa60 msgr2=0x7f65c4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 -- 172.21.15.78:0/2190866613 shutdown_connections 2024-09-16T00:20:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.454+0000 7f65c9716640 1 -- 172.21.15.78:0/2190866613 wait complete. 2024-09-16T00:20:05.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.455+0000 7f65c9716640 1 Processor -- start 2024-09-16T00:20:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.455+0000 7f65c9716640 1 -- start start 2024-09-16T00:20:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c9716640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c4076720 0x7f65c41a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c9716640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4077040 0x7f65c41a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c9716640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c4112a20 0x7f65c419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c9716640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41185b0 con 0x7f65c4076720 2024-09-16T00:20:05.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c9716640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4118430 con 0x7f65c4112a20 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c9716640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4118730 con 0x7f65c4077040 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4077040 0x7f65c41a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4077040 0x7f65c41a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37486/0 (socket says 172.21.15.78:37486) 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c27fc640 1 -- 172.21.15.78:0/958106173 learned_addr learned my addr 172.21.15.78:0/958106173 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c4076720 0x7f65c41a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.456+0000 7f65c37fe640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c4112a20 0x7f65c419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.457+0000 7f65c2ffd640 1 -- 172.21.15.78:0/958106173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4077040 msgr2=0x7f65c41a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.457+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4077040 0x7f65c41a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.457+0000 7f65c2ffd640 1 -- 172.21.15.78:0/958106173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c4112a20 msgr2=0x7f65c419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.457+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c4112a20 0x7f65c419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.457+0000 7f65c2ffd640 1 -- 172.21.15.78:0/958106173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c419eff0 con 0x7f65c4076720 2024-09-16T00:20:05.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.457+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c4076720 0x7f65c41a4700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00eb00 tx=0x7f65bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.458+0000 7f65bbfff640 1 -- 172.21.15.78:0/958106173 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc010430 con 0x7f65c4076720 2024-09-16T00:20:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.458+0000 7f65bbfff640 1 -- 172.21.15.78:0/958106173 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65bc010ab0 con 0x7f65c4076720 2024-09-16T00:20:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.458+0000 7f65bbfff640 1 -- 172.21.15.78:0/958106173 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc018bd0 con 0x7f65c4076720 2024-09-16T00:20:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.458+0000 7f65c9716640 1 -- 172.21.15.78:0/958106173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c419f2e0 con 0x7f65c4076720 2024-09-16T00:20:05.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.458+0000 7f65c9716640 1 -- 172.21.15.78:0/958106173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41ab5d0 con 0x7f65c4076720 2024-09-16T00:20:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.461+0000 7f65bbfff640 1 -- 172.21.15.78:0/958106173 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f65bc01f020 con 0x7f65c4076720 2024-09-16T00:20:05.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.461+0000 7f65cbaef740 1 Processor -- start 2024-09-16T00:20:05.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.461+0000 7f65cbaef740 1 -- start start 2024-09-16T00:20:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e2b40 0x560fd97e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e15d0 0x560fd97e1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97ec370 0x560fd97ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65cbaef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560fd97472f0 con 0x560fd97e2b40 2024-09-16T00:20:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65cbaef740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560fd96eecf0 con 0x560fd97ec370 2024-09-16T00:20:05.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65cbaef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560fd97432b0 con 0x560fd97e15d0 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97ec370 0x560fd97ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97ec370 0x560fd97ee760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:60172/0 (socket says 172.21.15.78:60172) 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c27fc640 1 -- 172.21.15.78:0/2479123411 learned_addr learned my addr 172.21.15.78:0/2479123411 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e15d0 0x560fd97e1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c37fe640 1 --2- 172.21.15.78:0/2479123411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e2b40 0x560fd97e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c27fc640 1 -- 172.21.15.78:0/2479123411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e15d0 msgr2=0x560fd97e1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c27fc640 1 --2- 172.21.15.78:0/2479123411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e15d0 0x560fd97e1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c27fc640 1 -- 172.21.15.78:0/2479123411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e2b40 msgr2=0x560fd97e1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c27fc640 1 --2- 172.21.15.78:0/2479123411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e2b40 0x560fd97e1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65c27fc640 1 -- 172.21.15.78:0/2479123411 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd96d33a0 con 0x560fd97ec370 2024-09-16T00:20:05.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.462+0000 7f65bbfff640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65a80a3390 0x7f65a80a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.463+0000 7f65c27fc640 1 --2- 172.21.15.78:0/2479123411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97ec370 0x560fd97ee760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f65b4004580 tx=0x7f65b400bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ffbcabb978730ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.463+0000 7f65bbfff640 1 -- 172.21.15.78:0/958106173 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f65bc0d0310 con 0x7f65c4076720 2024-09-16T00:20:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.463+0000 7f65c37fe640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65a80a3390 0x7f65a80a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.463+0000 7f65c37fe640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65a80a3390 0x7f65a80a5850 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f65b002a930 tx=0x7f65b00046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.464+0000 7f65b97fa640 1 -- 172.21.15.78:0/2479123411 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4003bd0 con 0x560fd97ec370 2024-09-16T00:20:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.464+0000 7f65b97fa640 1 -- 172.21.15.78:0/2479123411 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4015070 con 0x560fd97ec370 2024-09-16T00:20:05.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.464+0000 7f65b97fa640 1 -- 172.21.15.78:0/2479123411 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4019680 con 0x560fd97ec370 2024-09-16T00:20:05.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 -- 172.21.15.78:0/2479123411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97ec370 msgr2=0x560fd97ee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 --2- 172.21.15.78:0/2479123411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97ec370 0x560fd97ee760 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f65b4004580 tx=0x7f65b400bbb0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 -- 172.21.15.78:0/2479123411 shutdown_connections 2024-09-16T00:20:05.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 --2- 172.21.15.78:0/2479123411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97ec370 0x560fd97ee760 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 --2- 172.21.15.78:0/2479123411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e15d0 0x560fd97e1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 --2- 172.21.15.78:0/2479123411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e2b40 0x560fd97e1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 -- 172.21.15.78:0/2479123411 >> 172.21.15.78:0/2479123411 conn(0x560fd97aa310 msgr2=0x560fd97ddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 -- 172.21.15.78:0/2479123411 shutdown_connections 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.465+0000 7f65cbaef740 1 -- 172.21.15.78:0/2479123411 wait complete. 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 1 -- start start 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 1 Processor -- start 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97ebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 0x560fd97e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560fd97ef070 con 0x560fd97ec370 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560fd9702060 con 0x560fd97e15d0 2024-09-16T00:20:05.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560fd97ef1f0 con 0x560fd97e2b40 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65c27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65c27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37516/0 (socket says 172.21.15.78:37516) 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65c27fc640 1 -- 172.21.15.78:0/4060451797 learned_addr learned my addr 172.21.15.78:0/4060451797 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97ebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.466+0000 7f65cbaef740 10 client.0 _ll_register_callbacks cb 0x560fd94558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65c37fe640 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 0x560fd97e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65c2ffd640 1 -- 172.21.15.78:0/4060451797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 msgr2=0x560fd97e9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65c2ffd640 1 -- 172.21.15.78:0/4060451797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 msgr2=0x560fd97e97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 0x560fd97e97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65c2ffd640 1 -- 172.21.15.78:0/4060451797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd94b83c0 con 0x560fd97e15d0 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97ebde0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0cdba0 tx=0x7f65bc0989d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd6a593e2c9042d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65a67fc640 1 -- 172.21.15.78:0/4060451797 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0d0170 con 0x560fd97e15d0 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65a67fc640 1 -- 172.21.15.78:0/4060451797 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f65bc010c60 con 0x560fd97e15d0 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65cbaef740 1 -- 172.21.15.78:0/4060451797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b4015070 con 0x560fd97e15d0 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.467+0000 7f65a67fc640 1 -- 172.21.15.78:0/4060451797 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f65bc09cd70 con 0x560fd97e15d0 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65cbaef740 10 client.26039 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65cbaef740 1 -- 172.21.15.78:0/4060451797 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560fd97fab80 con 0x560fd97e15d0 2024-09-16T00:20:05.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65a67fc640 1 -- 172.21.15.78:0/4060451797 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65bc09c670 con 0x560fd97e15d0 2024-09-16T00:20:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65a67fc640 1 -- 172.21.15.78:0/4060451797 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f65bc0b0b90 con 0x560fd97e15d0 2024-09-16T00:20:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65cbaef740 10 client.26039 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65cbaef740 10 client.26039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:20:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65cbaef740 20 client.26039 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65cbaef740 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560fd97fbf90 0x560fd97fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65cbaef740 4 client.26039 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65cbaef740 1 -- 172.21.15.78:0/4060451797 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f65b4015070 con 0x560fd97fbf90 2024-09-16T00:20:05.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.468+0000 7f65c27fc640 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560fd97fbf90 0x560fd97fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.469+0000 7f65c27fc640 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560fd97fbf90 0x560fd97fe3e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc6b30edf13d1e0 server_cookie=1948461c4ac53553 in_seq=0 out_seq=0 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.469+0000 7f65a67fc640 10 client.26039 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.469+0000 7f65a67fc640 1 -- 172.21.15.78:0/4060451797 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f65b4015070 con 0x560fd97fbf90 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.469+0000 7f65a67fc640 10 client.26039 handle_command_reply: tid=0 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.469+0000 7f65cbaef740 1 client.26039 shutdown 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.470+0000 7f65cbaef740 1 -- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 msgr2=0x560fd97ebde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.470+0000 7f65cbaef740 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97ebde0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0cdba0 tx=0x7f65bc0989d0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 1 -- 172.21.15.78:0/4060451797 shutdown_connections 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560fd97fbf90 0x560fd97fe3e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65a67fc640 0 client.26039 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 0x560fd97e97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 1 --2- 172.21.15.78:0/4060451797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97ebde0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 1 -- 172.21.15.78:0/4060451797 >> 172.21.15.78:0/4060451797 conn(0x560fd97aa310 msgr2=0x560fd97e3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 1 -- 172.21.15.78:0/4060451797 shutdown_connections 2024-09-16T00:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 1 -- 172.21.15.78:0/4060451797 wait complete. 2024-09-16T00:20:05.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.471+0000 7f65cbaef740 20 client.26039 trim_cache size 0 max 16384 2024-09-16T00:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65cbaef740 1 Processor -- start 2024-09-16T00:20:05.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65cbaef740 1 -- start start 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97f8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 0x560fd97ed760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65cbaef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b52c0 con 0x560fd97ec370 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65cbaef740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b6e00 con 0x560fd97e15d0 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65cbaef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b5990 con 0x560fd97e2b40 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65c27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 0x560fd97ed760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65c27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37530/0 (socket says 172.21.15.78:37530) 2024-09-16T00:20:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.490+0000 7f65c27fc640 1 -- 172.21.15.78:0/2807116252 learned_addr learned my addr 172.21.15.78:0/2807116252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65c27fc640 1 -- 172.21.15.78:0/2807116252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 msgr2=0x560fd97f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65c27fc640 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65c27fc640 1 -- 172.21.15.78:0/2807116252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 msgr2=0x560fd97ed760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65c27fc640 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 0x560fd97ed760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65c27fc640 1 -- 172.21.15.78:0/2807116252 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd97e9b40 con 0x560fd97e2b40 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65c27fc640 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e8070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65b40044a0 tx=0x7f65b4011e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f095e1d83df52a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65b8ff9640 1 -- 172.21.15.78:0/2807116252 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4003dd0 con 0x560fd97e2b40 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65b8ff9640 1 -- 172.21.15.78:0/2807116252 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4015040 con 0x560fd97e2b40 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.491+0000 7f65b8ff9640 1 -- 172.21.15.78:0/2807116252 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4019920 con 0x560fd97e2b40 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 -- 172.21.15.78:0/2807116252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 msgr2=0x560fd97e8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e8070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65b40044a0 tx=0x7f65b4011e60 comp rx=0 tx=0).stop 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 -- 172.21.15.78:0/2807116252 shutdown_connections 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97ec370 0x560fd97ed760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97e2b40 0x560fd97e8070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 --2- 172.21.15.78:0/2807116252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e15d0 0x560fd97f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 -- 172.21.15.78:0/2807116252 >> 172.21.15.78:0/2807116252 conn(0x560fd97aa310 msgr2=0x560fd97e7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 -- 172.21.15.78:0/2807116252 shutdown_connections 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.492+0000 7f65cbaef740 1 -- 172.21.15.78:0/2807116252 wait complete. 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.493+0000 7f65cbaef740 1 -- start start 2024-09-16T00:20:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.493+0000 7f65cbaef740 1 Processor -- start 2024-09-16T00:20:05.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e15d0 0x560fd97e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e2b40 0x560fd97f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65cbaef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97ec370 0x560fd97f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65cbaef740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b6730 con 0x560fd97e15d0 2024-09-16T00:20:05.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65cbaef740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b6060 con 0x560fd97e2b40 2024-09-16T00:20:05.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65cbaef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41b32a0 con 0x560fd97ec370 2024-09-16T00:20:05.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e15d0 0x560fd97e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65c2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e15d0 0x560fd97e6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58738/0 (socket says 172.21.15.78:58738) 2024-09-16T00:20:05.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65c2ffd640 1 -- 172.21.15.78:0/594597914 learned_addr learned my addr 172.21.15.78:0/594597914 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:05.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65c37fe640 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97ec370 0x560fd97f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65c27fc640 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e2b40 0x560fd97f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.494+0000 7f65cbaef740 10 client.0 _ll_register_callbacks cb 0x7f65bc018420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.495+0000 7f65c2ffd640 1 -- 172.21.15.78:0/594597914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97ec370 msgr2=0x560fd97f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.495+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97ec370 0x560fd97f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.495+0000 7f65c2ffd640 1 -- 172.21.15.78:0/594597914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e2b40 msgr2=0x560fd97f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.495+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e2b40 0x560fd97f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.495+0000 7f65c2ffd640 1 -- 172.21.15.78:0/594597914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd97f50b0 con 0x560fd97e15d0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.495+0000 7f65c2ffd640 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e15d0 0x560fd97e6ac0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f65bc09c600 tx=0x7f65bc0a50f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3ef16c5dbf0fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.495+0000 7f65a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.496+0000 7f65a67fc640 1 -- 172.21.15.78:0/594597914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc09bde0 con 0x560fd97e15d0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.496+0000 7f65a67fc640 1 -- 172.21.15.78:0/594597914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f65bc0a4e50 con 0x560fd97e15d0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.496+0000 7f65cbaef740 1 -- 172.21.15.78:0/594597914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b4015040 con 0x560fd97e15d0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.496+0000 7f65a67fc640 1 -- 172.21.15.78:0/594597914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f65bc0d3b70 con 0x560fd97e15d0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.496+0000 7f65cbaef740 10 client.16650 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.496+0000 7f65cbaef740 1 -- 172.21.15.78:0/594597914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560fd9866ce0 con 0x560fd97e15d0 2024-09-16T00:20:05.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.497+0000 7f65a67fc640 1 -- 172.21.15.78:0/594597914 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b4015040 con 0x560fd97e15d0 2024-09-16T00:20:05.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.497+0000 7f65a67fc640 1 -- 172.21.15.78:0/594597914 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f65bc09b530 con 0x560fd97e15d0 2024-09-16T00:20:05.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.497+0000 7f65cbaef740 10 client.16650 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:05.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.497+0000 7f65cbaef740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:20:05.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.497+0000 7f65cbaef740 20 client.16650 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:05.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.497+0000 7f65cbaef740 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560fd98672f0 0x560fd98697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:05.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.497+0000 7f65cbaef740 4 client.16650 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:20:05.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.497+0000 7f65cbaef740 1 -- 172.21.15.78:0/594597914 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fd9754860 con 0x560fd98672f0 2024-09-16T00:20:05.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.498+0000 7f65c27fc640 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560fd98672f0 0x560fd98697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:05.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.498+0000 7f65a67fc640 10 client.16650 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:05.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.498+0000 7f65c27fc640 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560fd98672f0 0x560fd98697b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f8175e25ae3c71 server_cookie=8f34669eb5ab23c0 in_seq=0 out_seq=0 2024-09-16T00:20:05.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.498+0000 7f65a67fc640 1 -- 172.21.15.78:0/594597914 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fd9754860 con 0x560fd98672f0 2024-09-16T00:20:05.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.498+0000 7f65a67fc640 10 client.16650 handle_command_reply: tid=0 2024-09-16T00:20:05.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.498+0000 7f65cbaef740 1 client.16650 shutdown 2024-09-16T00:20:05.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 -- 172.21.15.78:0/594597914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e15d0 msgr2=0x560fd97e6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e15d0 0x560fd97e6ac0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f65bc09c600 tx=0x7f65bc0a50f0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 -- 172.21.15.78:0/594597914 shutdown_connections 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560fd98672f0 0x560fd98697b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560fd97ec370 0x560fd97f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560fd97e2b40 0x560fd97f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65a67fc640 0 client.16650 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 --2- 172.21.15.78:0/594597914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560fd97e15d0 0x560fd97e6ac0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 -- 172.21.15.78:0/594597914 >> 172.21.15.78:0/594597914 conn(0x560fd97aa310 msgr2=0x560fd97dc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 -- 172.21.15.78:0/594597914 shutdown_connections 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.499+0000 7f65cbaef740 1 -- 172.21.15.78:0/594597914 wait complete. 2024-09-16T00:20:05.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.500+0000 7f65cbaef740 20 client.16650 trim_cache size 0 max 16384 2024-09-16T00:20:05.505 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 -- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65a80a3390 msgr2=0x7f65a80a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65a80a3390 0x7f65a80a5850 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f65b002a930 tx=0x7f65b00046c0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 -- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c4076720 msgr2=0x7f65c41a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c4076720 0x7f65c41a4700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f65bc00eb00 tx=0x7f65bc00c2d0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 -- 172.21.15.78:0/958106173 shutdown_connections 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65a80a3390 0x7f65a80a5850 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65c4112a20 0x7f65c419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65c4077040 0x7f65c41a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 --2- 172.21.15.78:0/958106173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c4076720 0x7f65c41a4700 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:05.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.501+0000 7f65c9716640 1 -- 172.21.15.78:0/958106173 >> 172.21.15.78:0/958106173 conn(0x7f65c40ffa60 msgr2=0x7f65c4101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:05.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.502+0000 7f65c9716640 1 -- 172.21.15.78:0/958106173 shutdown_connections 2024-09-16T00:20:05.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:05.502+0000 7f65c9716640 1 -- 172.21.15.78:0/958106173 wait complete. 2024-09-16T00:20:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 365.0 s, 1019.2 tps, lat 98.435 ms stddev 111.845 2024-09-16T00:20:07.379 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:20:07.379 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:20:07.771 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:07 smithi078 ceph-mon[31157]: pgmap v562: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 688 op/s 2024-09-16T00:20:07.852 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:07.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:07 smithi193 ceph-mon[35484]: pgmap v562: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 688 op/s 2024-09-16T00:20:07.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:07 smithi148 ceph-mon[35599]: pgmap v562: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 688 op/s 2024-09-16T00:20:08.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 -- 172.21.15.78:0/3863836271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e8108e00 msgr2=0x7f33e810ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 --2- 172.21.15.78:0/3863836271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e8108e00 0x7f33e810ce00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009900 tx=0x7f33d802a410 comp rx=0 tx=0).stop 2024-09-16T00:20:08.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 -- 172.21.15.78:0/3863836271 shutdown_connections 2024-09-16T00:20:08.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 --2- 172.21.15.78:0/3863836271 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e810d530 0x7f33e81150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 --2- 172.21.15.78:0/3863836271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e8108e00 0x7f33e810ce00 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 --2- 172.21.15.78:0/3863836271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33e8108450 0x7f33e8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 -- 172.21.15.78:0/3863836271 >> 172.21.15.78:0/3863836271 conn(0x7f33e8077ba0 msgr2=0x7f33e8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:08.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 -- 172.21.15.78:0/3863836271 shutdown_connections 2024-09-16T00:20:08.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.463+0000 7f33f05a4640 1 -- 172.21.15.78:0/3863836271 wait complete. 2024-09-16T00:20:08.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.464+0000 7f33f05a4640 1 Processor -- start 2024-09-16T00:20:08.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.464+0000 7f33f05a4640 1 -- start start 2024-09-16T00:20:08.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33f05a4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33e8108450 0x7f33e819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33f05a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e8108e00 0x7f33e819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33f05a4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e810d530 0x7f33e81a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33ee340640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33e8108450 0x7f33e819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33f05a4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33e8118580 con 0x7f33e810d530 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33f05a4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33e8118400 con 0x7f33e8108e00 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33f05a4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33e8118700 con 0x7f33e8108450 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33edb3f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e8108e00 0x7f33e819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33edb3f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e8108e00 0x7f33e819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:49716/0 (socket says 172.21.15.78:49716) 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33ee340640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33e8108450 0x7f33e819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36964/0 (socket says 172.21.15.78:36964) 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33edb3f640 1 -- 172.21.15.78:0/2461343056 learned_addr learned my addr 172.21.15.78:0/2461343056 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33eeb41640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e810d530 0x7f33e81a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33edb3f640 1 -- 172.21.15.78:0/2461343056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33e8108450 msgr2=0x7f33e819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33edb3f640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33e8108450 0x7f33e819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33edb3f640 1 -- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e810d530 msgr2=0x7f33e81a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.465+0000 7f33edb3f640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e810d530 0x7f33e81a3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.466+0000 7f33edb3f640 1 -- 172.21.15.78:0/2461343056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e81a3ab0 con 0x7f33e8108e00 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.466+0000 7f33ee340640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33e8108450 0x7f33e819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.466+0000 7f33edb3f640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e8108e00 0x7f33e819efc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f33d8004770 tx=0x7f33d8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.466+0000 7f33df7fe640 1 -- 172.21.15.78:0/2461343056 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d803f070 con 0x7f33e8108e00 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.466+0000 7f33df7fe640 1 -- 172.21.15.78:0/2461343056 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33d8031ca0 con 0x7f33e8108e00 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.466+0000 7f33df7fe640 1 -- 172.21.15.78:0/2461343056 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8035030 con 0x7f33e8108e00 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.466+0000 7f33f05a4640 1 -- 172.21.15.78:0/2461343056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e81a3d40 con 0x7f33e8108e00 2024-09-16T00:20:08.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.466+0000 7f33f05a4640 1 -- 172.21.15.78:0/2461343056 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e81ab630 con 0x7f33e8108e00 2024-09-16T00:20:08.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.469+0000 7f33f297d740 1 Processor -- start 2024-09-16T00:20:08.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.470+0000 7f33f297d740 1 -- start start 2024-09-16T00:20:08.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.470+0000 7f33f297d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1deb60 0x56253d1def40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.470+0000 7f33f297d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 0x56253d1e9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.470+0000 7f33f297d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1e9d40 0x56253d1ec130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.470+0000 7f33f297d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56253d144280 con 0x56253d1deb60 2024-09-16T00:20:08.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.470+0000 7f33f297d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56253d0ebb10 con 0x56253d1e9d40 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.470+0000 7f33f297d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56253d140240 con 0x56253d1df480 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33eeb41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1deb60 0x56253d1def40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33eeb41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1deb60 0x56253d1def40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42318/0 (socket says 172.21.15.78:42318) 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33eeb41640 1 -- 172.21.15.78:0/4207217741 learned_addr learned my addr 172.21.15.78:0/4207217741 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33eeb41640 1 -- 172.21.15.78:0/4207217741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 msgr2=0x56253d1e9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33eeb41640 1 --2- 172.21.15.78:0/4207217741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 0x56253d1e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33df7fe640 1 -- 172.21.15.78:0/2461343056 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f33d8042070 con 0x7f33e8108e00 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33eeb41640 1 -- 172.21.15.78:0/4207217741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1e9d40 msgr2=0x56253d1ec130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33eeb41640 1 --2- 172.21.15.78:0/4207217741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1e9d40 0x56253d1ec130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.471+0000 7f33eeb41640 1 -- 172.21.15.78:0/4207217741 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253d0f81f0 con 0x56253d1deb60 2024-09-16T00:20:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.472+0000 7f33df7fe640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33d40a32c0 0x7f33d40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.472+0000 7f33df7fe640 1 -- 172.21.15.78:0/2461343056 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f33d80efeb0 con 0x7f33e8108e00 2024-09-16T00:20:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.472+0000 7f33ee340640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33d40a32c0 0x7f33d40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.472+0000 7f33eeb41640 1 --2- 172.21.15.78:0/4207217741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1deb60 0x56253d1def40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009f40 tx=0x7f33e00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a162f06c48b65e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.473+0000 7f33dcff9640 1 -- 172.21.15.78:0/4207217741 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0004b20 con 0x56253d1deb60 2024-09-16T00:20:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.473+0000 7f33ee340640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33d40a32c0 0x7f33d40a5780 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009730 tx=0x7f33e40096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:08.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.473+0000 7f33dcff9640 1 -- 172.21.15.78:0/4207217741 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e0004cc0 con 0x56253d1deb60 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.473+0000 7f33dcff9640 1 -- 172.21.15.78:0/4207217741 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0005020 con 0x56253d1deb60 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.474+0000 7f33f297d740 1 -- 172.21.15.78:0/4207217741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1deb60 msgr2=0x56253d1def40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.474+0000 7f33f297d740 1 --2- 172.21.15.78:0/4207217741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1deb60 0x56253d1def40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009f40 tx=0x7f33e00118d0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.474+0000 7f33f297d740 1 -- 172.21.15.78:0/4207217741 shutdown_connections 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.474+0000 7f33f297d740 1 --2- 172.21.15.78:0/4207217741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1e9d40 0x56253d1ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.474+0000 7f33f297d740 1 --2- 172.21.15.78:0/4207217741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 0x56253d1e9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.474+0000 7f33f297d740 1 --2- 172.21.15.78:0/4207217741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1deb60 0x56253d1def40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.474+0000 7f33f297d740 1 -- 172.21.15.78:0/4207217741 >> 172.21.15.78:0/4207217741 conn(0x56253d1db0f0 msgr2=0x56253d1db4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:08.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.474+0000 7f33f297d740 1 -- 172.21.15.78:0/4207217741 shutdown_connections 2024-09-16T00:20:08.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.478+0000 7f33f297d740 1 -- 172.21.15.78:0/4207217741 wait complete. 2024-09-16T00:20:08.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.478+0000 7f33f297d740 1 -- start start 2024-09-16T00:20:08.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.478+0000 7f33f297d740 1 Processor -- start 2024-09-16T00:20:08.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.479+0000 7f33f297d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.479+0000 7f33f297d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 0x56253d1e7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.479+0000 7f33f297d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.479+0000 7f33f297d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56253d1ec910 con 0x56253d1e9d40 2024-09-16T00:20:08.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.479+0000 7f33f297d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56253d0fef10 con 0x56253d1deb60 2024-09-16T00:20:08.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.479+0000 7f33f297d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56253d1eca90 con 0x56253d1df480 2024-09-16T00:20:08.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.479+0000 7f33f297d740 10 client.0 _ll_register_callbacks cb 0x56253d0c7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:08.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.483+0000 7f33eeb41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.483+0000 7f33eeb41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42328/0 (socket says 172.21.15.78:42328) 2024-09-16T00:20:08.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.483+0000 7f33eeb41640 1 -- 172.21.15.78:0/331765490 learned_addr learned my addr 172.21.15.78:0/331765490 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:08.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.486+0000 7f33edb3f640 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 0x56253d1e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.487+0000 7f33ee340640 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.489+0000 7f33ee340640 1 -- 172.21.15.78:0/331765490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 msgr2=0x56253d1e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:08.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.489+0000 7f33ee340640 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 0x56253d1e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.489+0000 7f33ee340640 1 -- 172.21.15.78:0/331765490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 msgr2=0x56253d1e76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:08.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.489+0000 7f33ee340640 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1e76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.489+0000 7f33ee340640 1 -- 172.21.15.78:0/331765490 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253ceb56a0 con 0x56253d1deb60 2024-09-16T00:20:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.492+0000 7f33ee340640 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e6c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f33e4004700 tx=0x7f33e4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c928c72804fb10 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.495+0000 7f33d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.495+0000 7f33d1ffb640 1 -- 172.21.15.78:0/331765490 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e401cdb0 con 0x56253d1deb60 2024-09-16T00:20:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.496+0000 7f33d1ffb640 1 -- 172.21.15.78:0/331765490 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f33e40263f0 con 0x56253d1deb60 2024-09-16T00:20:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.496+0000 7f33f297d740 1 -- 172.21.15.78:0/331765490 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e0004cc0 con 0x56253d1deb60 2024-09-16T00:20:08.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.497+0000 7f33d1ffb640 1 -- 172.21.15.78:0/331765490 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f33e40265f0 con 0x56253d1deb60 2024-09-16T00:20:08.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.499+0000 7f33f297d740 10 client.26054 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:08.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.499+0000 7f33f297d740 1 -- 172.21.15.78:0/331765490 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56253d1f85c0 con 0x56253d1deb60 2024-09-16T00:20:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.501+0000 7f33d1ffb640 1 -- 172.21.15.78:0/331765490 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e0004cc0 con 0x56253d1deb60 2024-09-16T00:20:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.501+0000 7f33d1ffb640 1 -- 172.21.15.78:0/331765490 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f33e402f940 con 0x56253d1deb60 2024-09-16T00:20:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.501+0000 7f33f297d740 10 client.26054 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.501+0000 7f33f297d740 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:20:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.501+0000 7f33f297d740 20 client.26054 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.501+0000 7f33f297d740 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56253d1f9a20 0x56253d1fbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.501+0000 7f33f297d740 4 client.26054 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:08.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.501+0000 7f33f297d740 1 -- 172.21.15.78:0/331765490 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56253d1517f0 con 0x56253d1f9a20 2024-09-16T00:20:08.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.504+0000 7f33edb3f640 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56253d1f9a20 0x56253d1fbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.516+0000 7f33edb3f640 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56253d1f9a20 0x56253d1fbea0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1afa284cceef6a95 server_cookie=e262a29169b6eb43 in_seq=0 out_seq=0 2024-09-16T00:20:08.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.517+0000 7f33d1ffb640 10 client.26054 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:08.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.525+0000 7f33d1ffb640 1 -- 172.21.15.78:0/331765490 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56253d1517f0 con 0x56253d1f9a20 2024-09-16T00:20:08.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.525+0000 7f33d1ffb640 10 client.26054 handle_command_reply: tid=0 2024-09-16T00:20:08.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.525+0000 7f33f297d740 1 client.26054 shutdown 2024-09-16T00:20:08.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.532+0000 7f33f297d740 1 -- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 msgr2=0x56253d1e6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.532+0000 7f33f297d740 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e6c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f33e4004700 tx=0x7f33e4005f90 comp rx=0 tx=0).stop 2024-09-16T00:20:08.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.533+0000 7f33f297d740 1 -- 172.21.15.78:0/331765490 shutdown_connections 2024-09-16T00:20:08.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.533+0000 7f33f297d740 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56253d1f9a20 0x56253d1fbea0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.533+0000 7f33f297d740 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.533+0000 7f33f297d740 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1df480 0x56253d1e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.533+0000 7f33f297d740 1 --2- 172.21.15.78:0/331765490 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e6c40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.533+0000 7f33f297d740 1 -- 172.21.15.78:0/331765490 >> 172.21.15.78:0/331765490 conn(0x56253d1db0f0 msgr2=0x56253d1e0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:08.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.533+0000 7f33d1ffb640 0 client.26054 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:08.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.535+0000 7f33f297d740 1 -- 172.21.15.78:0/331765490 shutdown_connections 2024-09-16T00:20:08.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.535+0000 7f33f297d740 1 -- 172.21.15.78:0/331765490 wait complete. 2024-09-16T00:20:08.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.535+0000 7f33f297d740 20 client.26054 trim_cache size 0 max 16384 2024-09-16T00:20:08.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.570+0000 7f33f297d740 1 Processor -- start 2024-09-16T00:20:08.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.571+0000 7f33f297d740 1 -- start start 2024-09-16T00:20:08.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33f297d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33f297d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1df480 0x56253d2017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33f297d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1e9d40 0x56253d201d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33f297d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33e81b52d0 con 0x56253d1df480 2024-09-16T00:20:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33f297d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33e81b6e10 con 0x56253d1deb60 2024-09-16T00:20:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33eeb41640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1e9d40 0x56253d201d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33eeb41640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1e9d40 0x56253d201d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36974/0 (socket says 172.21.15.78:36974) 2024-09-16T00:20:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33eeb41640 1 -- 172.21.15.78:0/2596388807 learned_addr learned my addr 172.21.15.78:0/2596388807 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.572+0000 7f33ee340640 1 --2- 172.21.15.78:0/2596388807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.573+0000 7f33f297d740 1 -- 172.21.15.78:0/2596388807 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33e81b59a0 con 0x56253d1e9d40 2024-09-16T00:20:08.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.575+0000 7f33eeb41640 1 -- 172.21.15.78:0/2596388807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 msgr2=0x56253d1e0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:08.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.575+0000 7f33eeb41640 1 --2- 172.21.15.78:0/2596388807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.575+0000 7f33eeb41640 1 -- 172.21.15.78:0/2596388807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1df480 msgr2=0x56253d2017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:08.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.575+0000 7f33eeb41640 1 --2- 172.21.15.78:0/2596388807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1df480 0x56253d2017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.575+0000 7f33eeb41640 1 -- 172.21.15.78:0/2596388807 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253d1e5bf0 con 0x56253d1e9d40 2024-09-16T00:20:08.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.575+0000 7f33eeb41640 1 --2- 172.21.15.78:0/2596388807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1e9d40 0x56253d201d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f33e0005c10 tx=0x7f33e0004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dbc25881f987595 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:08.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.581+0000 7f33d3fff640 1 -- 172.21.15.78:0/2596388807 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0027020 con 0x56253d1e9d40 2024-09-16T00:20:08.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.581+0000 7f33d3fff640 1 -- 172.21.15.78:0/2596388807 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e0004e50 con 0x56253d1e9d40 2024-09-16T00:20:08.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.581+0000 7f33d3fff640 1 -- 172.21.15.78:0/2596388807 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0005040 con 0x56253d1e9d40 2024-09-16T00:20:08.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.583+0000 7f33f297d740 1 -- 172.21.15.78:0/2596388807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1e9d40 msgr2=0x56253d201d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.583+0000 7f33f297d740 1 --2- 172.21.15.78:0/2596388807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1e9d40 0x56253d201d00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f33e0005c10 tx=0x7f33e0004540 comp rx=0 tx=0).stop 2024-09-16T00:20:08.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.583+0000 7f33f297d740 1 -- 172.21.15.78:0/2596388807 shutdown_connections 2024-09-16T00:20:08.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.583+0000 7f33f297d740 1 --2- 172.21.15.78:0/2596388807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1e9d40 0x56253d201d00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.583+0000 7f33f297d740 1 --2- 172.21.15.78:0/2596388807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1df480 0x56253d2017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.583+0000 7f33f297d740 1 --2- 172.21.15.78:0/2596388807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1deb60 0x56253d1e0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.583+0000 7f33f297d740 1 -- 172.21.15.78:0/2596388807 >> 172.21.15.78:0/2596388807 conn(0x56253d1db0f0 msgr2=0x56253d1e05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:08.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.585+0000 7f33f297d740 1 -- 172.21.15.78:0/2596388807 shutdown_connections 2024-09-16T00:20:08.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.585+0000 7f33f297d740 1 -- 172.21.15.78:0/2596388807 wait complete. 2024-09-16T00:20:08.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.585+0000 7f33f297d740 1 -- start start 2024-09-16T00:20:08.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.586+0000 7f33f297d740 1 Processor -- start 2024-09-16T00:20:08.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.586+0000 7f33f297d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1deb60 0x56253d1edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.586+0000 7f33f297d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1df480 0x56253d1ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.587+0000 7f33f297d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.587+0000 7f33f297d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33e81b6740 con 0x56253d1e9d40 2024-09-16T00:20:08.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.587+0000 7f33eeb41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.587+0000 7f33eeb41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1ee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42350/0 (socket says 172.21.15.78:42350) 2024-09-16T00:20:08.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.587+0000 7f33eeb41640 1 -- 172.21.15.78:0/3622020643 learned_addr learned my addr 172.21.15.78:0/3622020643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:08.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.587+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33e81b6070 con 0x56253d1df480 2024-09-16T00:20:08.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.587+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33e81b32b0 con 0x56253d1deb60 2024-09-16T00:20:08.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.588+0000 7f33f297d740 10 client.0 _ll_register_callbacks cb 0x7f33e4009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:08.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.589+0000 7f33edb3f640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1df480 0x56253d1ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.590+0000 7f33ee340640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1deb60 0x56253d1edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.590+0000 7f33edb3f640 1 -- 172.21.15.78:0/3622020643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1deb60 msgr2=0x56253d1edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:08.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.590+0000 7f33edb3f640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1deb60 0x56253d1edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.590+0000 7f33edb3f640 1 -- 172.21.15.78:0/3622020643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 msgr2=0x56253d1ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:08.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.590+0000 7f33edb3f640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.590+0000 7f33edb3f640 1 -- 172.21.15.78:0/3622020643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253d2639e0 con 0x56253d1df480 2024-09-16T00:20:08.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.590+0000 7f33ee340640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1deb60 0x56253d1edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:08.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.590+0000 7f33edb3f640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1df480 0x56253d1ee1d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x56253d1dc470 tx=0x7f33d8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7aa62e94802f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:08.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.591+0000 7f33eeb41640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:08.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.591+0000 7f33d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:08.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.591+0000 7f33d1ffb640 1 -- 172.21.15.78:0/3622020643 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8049040 con 0x56253d1df480 2024-09-16T00:20:08.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.591+0000 7f33d1ffb640 1 -- 172.21.15.78:0/3622020643 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f33d80ed700 con 0x56253d1df480 2024-09-16T00:20:08.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.591+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e0004e50 con 0x56253d1df480 2024-09-16T00:20:08.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.592+0000 7f33d1ffb640 1 -- 172.21.15.78:0/3622020643 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f33d80c4e40 con 0x56253d1df480 2024-09-16T00:20:08.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.595+0000 7f33f297d740 10 client.26060 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:08.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.595+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56253d264d40 con 0x56253d1df480 2024-09-16T00:20:08.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.597+0000 7f33d1ffb640 1 -- 172.21.15.78:0/3622020643 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e0004e50 con 0x56253d1df480 2024-09-16T00:20:08.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.598+0000 7f33d1ffb640 1 -- 172.21.15.78:0/3622020643 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f33d804c060 con 0x56253d1df480 2024-09-16T00:20:08.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.598+0000 7f33f297d740 10 client.26060 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:08.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.598+0000 7f33f297d740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:20:08.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.599+0000 7f33f297d740 20 client.26060 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:08.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.599+0000 7f33f297d740 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56253d265470 0x56253d267890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:08.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.599+0000 7f33f297d740 4 client.26060 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:20:08.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.599+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56253d14bd00 con 0x56253d265470 2024-09-16T00:20:08.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.607+0000 7f33ee340640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56253d265470 0x56253d267890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:08.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.609+0000 7f33ee340640 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56253d265470 0x56253d267890 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e8b7ad9746acfa server_cookie=ff7d4435f42131d0 in_seq=0 out_seq=0 2024-09-16T00:20:08.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.609+0000 7f33d1ffb640 10 client.26060 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:08.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.610+0000 7f33d1ffb640 1 -- 172.21.15.78:0/3622020643 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56253d14bd00 con 0x56253d265470 2024-09-16T00:20:08.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.610+0000 7f33d1ffb640 10 client.26060 handle_command_reply: tid=0 2024-09-16T00:20:08.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.610+0000 7f33f297d740 1 client.26060 shutdown 2024-09-16T00:20:08.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.615+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1df480 msgr2=0x56253d1ee1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.615+0000 7f33f297d740 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1df480 0x56253d1ee1d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x56253d1dc470 tx=0x7f33d8009260 comp rx=0 tx=0).stop 2024-09-16T00:20:08.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.619+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 shutdown_connections 2024-09-16T00:20:08.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.619+0000 7f33f297d740 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56253d265470 0x56253d267890 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.619+0000 7f33f297d740 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56253d1e9d40 0x56253d1ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.619+0000 7f33d1ffb640 0 client.26060 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:08.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.620+0000 7f33f297d740 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56253d1df480 0x56253d1ee1d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.620+0000 7f33f297d740 1 --2- 172.21.15.78:0/3622020643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56253d1deb60 0x56253d1edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.620+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 >> 172.21.15.78:0/3622020643 conn(0x56253d1db0f0 msgr2=0x56253d1da270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:08.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.621+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 shutdown_connections 2024-09-16T00:20:08.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.621+0000 7f33f297d740 1 -- 172.21.15.78:0/3622020643 wait complete. 2024-09-16T00:20:08.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.621+0000 7f33f297d740 20 client.26060 trim_cache size 0 max 16384 2024-09-16T00:20:08.622 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:20:08.622 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:20:08.622 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "c8520dd3-ef7a-4e3e-90c2-dceaae6a5280", 2024-09-16T00:20:08.622 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:20:08.622 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:20:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 -- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33d40a32c0 msgr2=0x7f33d40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33d40a32c0 0x7f33d40a5780 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009730 tx=0x7f33e40096c0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 -- 172.21.15.78:0/2461343056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e8108e00 msgr2=0x7f33e819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e8108e00 0x7f33e819efc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f33d8004770 tx=0x7f33d8003c10 comp rx=0 tx=0).stop 2024-09-16T00:20:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 -- 172.21.15.78:0/2461343056 shutdown_connections 2024-09-16T00:20:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33d40a32c0 0x7f33d40a5780 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33e810d530 0x7f33e81a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33e8108e00 0x7f33e819efc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 --2- 172.21.15.78:0/2461343056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33e8108450 0x7f33e819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:08.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.622+0000 7f33f05a4640 1 -- 172.21.15.78:0/2461343056 >> 172.21.15.78:0/2461343056 conn(0x7f33e8077ba0 msgr2=0x7f33e8078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.624+0000 7f33f05a4640 1 -- 172.21.15.78:0/2461343056 shutdown_connections 2024-09-16T00:20:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:08.624+0000 7f33f05a4640 1 -- 172.21.15.78:0/2461343056 wait complete. 2024-09-16T00:20:09.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:20:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:09.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: pgmap v563: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 654 op/s 2024-09-16T00:20:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:20:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:20:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:20:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:20:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: pgmap v563: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 654 op/s 2024-09-16T00:20:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:20:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:20:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:20:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:20:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: pgmap v563: 129 pgs: 129 active+clean; 1.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 13 MiB/s wr, 654 op/s 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:09.975 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:20:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.434+0000 7f7cc36d7640 1 -- 172.21.15.78:0/1716010214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc109450 msgr2=0x7f7cbc110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.434+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/1716010214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc109450 0x7f7cbc110250 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009ca0 tx=0x7f7cb402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.435+0000 7f7cc36d7640 1 -- 172.21.15.78:0/1716010214 shutdown_connections 2024-09-16T00:20:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.435+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/1716010214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cbc11b140 0x7f7cbc11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.435+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/1716010214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cbc109830 0x7f7cbc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.435+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/1716010214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc109450 0x7f7cbc110250 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.435+0000 7f7cc36d7640 1 -- 172.21.15.78:0/1716010214 >> 172.21.15.78:0/1716010214 conn(0x7f7cbc06e140 msgr2=0x7f7cbc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.436+0000 7f7cc36d7640 1 -- 172.21.15.78:0/1716010214 shutdown_connections 2024-09-16T00:20:10.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.436+0000 7f7cc36d7640 1 -- 172.21.15.78:0/1716010214 wait complete. 2024-09-16T00:20:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.437+0000 7f7cc36d7640 1 Processor -- start 2024-09-16T00:20:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.437+0000 7f7cc36d7640 1 -- start start 2024-09-16T00:20:10.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.437+0000 7f7cc36d7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cbc109830 0x7f7cbc11aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.437+0000 7f7cc36d7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cbc11b140 0x7f7cbc114a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc36d7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc114fc0 0x7f7cbc115470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc36d7640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc111330 con 0x7f7cbc114fc0 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc36d7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1111b0 con 0x7f7cbc11b140 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc36d7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1114b0 con 0x7f7cbc109830 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc0c72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cbc11b140 0x7f7cbc114a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc0c72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cbc11b140 0x7f7cbc114a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:49770/0 (socket says 172.21.15.78:49770) 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc0c72640 1 -- 172.21.15.78:0/3959270460 learned_addr learned my addr 172.21.15.78:0/3959270460 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc1c74640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc114fc0 0x7f7cbc115470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc0c72640 1 -- 172.21.15.78:0/3959270460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cbc109830 msgr2=0x7f7cbc11aa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:20:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cbc109830 0x7f7cbc11aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc0c72640 1 -- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc114fc0 msgr2=0x7f7cbc115470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc114fc0 0x7f7cbc115470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.438+0000 7f7cc0c72640 1 -- 172.21.15.78:0/3959270460 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cbc1bc2f0 con 0x7f7cbc11b140 2024-09-16T00:20:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.439+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cbc11b140 0x7f7cbc114a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800e9f0 tx=0x7f7cb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.439+0000 7f7cb27fc640 1 -- 172.21.15.78:0/3959270460 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800cd80 con 0x7f7cbc11b140 2024-09-16T00:20:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.439+0000 7f7cc36d7640 1 -- 172.21.15.78:0/3959270460 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cbc1bc5e0 con 0x7f7cbc11b140 2024-09-16T00:20:10.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.439+0000 7f7cc36d7640 1 -- 172.21.15.78:0/3959270460 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cbc1bcb70 con 0x7f7cbc11b140 2024-09-16T00:20:10.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.440+0000 7f7cb27fc640 1 -- 172.21.15.78:0/3959270460 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb8010910 con 0x7f7cbc11b140 2024-09-16T00:20:10.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.440+0000 7f7cb27fc640 1 -- 172.21.15.78:0/3959270460 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8018960 con 0x7f7cbc11b140 2024-09-16T00:20:10.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.459+0000 7f7cb27fc640 1 -- 172.21.15.78:0/3959270460 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7cb8018be0 con 0x7f7cbc11b140 2024-09-16T00:20:10.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.462+0000 7f7cc5ab0740 1 Processor -- start 2024-09-16T00:20:10.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.462+0000 7f7cb27fc640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7ca80a33c0 0x7f7ca80a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.463+0000 7f7cb27fc640 1 -- 172.21.15.78:0/3959270460 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7cb80d00f0 con 0x7f7cbc11b140 2024-09-16T00:20:10.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.463+0000 7f7cc5ab0740 1 -- start start 2024-09-16T00:20:10.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.463+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e034a0 0x55f441e03880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.463+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e0e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.463+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.463+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f441d692e0 con 0x55f441e034a0 2024-09-16T00:20:10.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.463+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f441d10ce0 con 0x55f441e03dc0 2024-09-16T00:20:10.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.463+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f441d652a0 con 0x55f441e0e680 2024-09-16T00:20:10.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.464+0000 7f7cc1c74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.464+0000 7f7cc1c74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e10a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36990/0 (socket says 172.21.15.78:36990) 2024-09-16T00:20:10.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.464+0000 7f7cc1c74640 1 -- 172.21.15.78:0/981041651 learned_addr learned my addr 172.21.15.78:0/981041651 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:10.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.465+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/981041651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e0e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.465+0000 7f7cc1c74640 1 -- 172.21.15.78:0/981041651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 msgr2=0x55f441e0e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:10.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.465+0000 7f7cc1c74640 1 --2- 172.21.15.78:0/981041651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e0e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.465+0000 7f7cc1c74640 1 -- 172.21.15.78:0/981041651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e034a0 msgr2=0x55f441e03880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:20:10.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.465+0000 7f7cc1c74640 1 --2- 172.21.15.78:0/981041651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e034a0 0x55f441e03880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.465+0000 7f7cc1c74640 1 -- 172.21.15.78:0/981041651 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f441cf5440 con 0x55f441e0e680 2024-09-16T00:20:10.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.465+0000 7f7cc1c74640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7ca80a33c0 0x7f7ca80a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.465+0000 7f7cc1c74640 1 --2- 172.21.15.78:0/981041651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e10a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009c70 tx=0x7f7cac012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd69d273874b5314 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:10.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.466+0000 7f7cc1c74640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7ca80a33c0 0x7f7ca80a5880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7cac01f000 tx=0x7f7cac019c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:10.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.467+0000 7f7ca77fe640 1 -- 172.21.15.78:0/981041651 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac022070 con 0x55f441e0e680 2024-09-16T00:20:10.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.467+0000 7f7ca77fe640 1 -- 172.21.15.78:0/981041651 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac00e3b0 con 0x55f441e0e680 2024-09-16T00:20:10.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.468+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/981041651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 msgr2=0x55f441e10a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:10.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.468+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/981041651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e10a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009c70 tx=0x7f7cac012820 comp rx=0 tx=0).stop 2024-09-16T00:20:10.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.470+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/981041651 shutdown_connections 2024-09-16T00:20:10.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.470+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/981041651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e10a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.470+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/981041651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e0e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.470+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/981041651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e034a0 0x55f441e03880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.470+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/981041651 >> 172.21.15.78:0/981041651 conn(0x55f441dffa30 msgr2=0x55f441dffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:10.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.471+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/981041651 shutdown_connections 2024-09-16T00:20:10.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.471+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/981041651 wait complete. 2024-09-16T00:20:10.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.471+0000 7f7cc5ab0740 1 -- start start 2024-09-16T00:20:10.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.472+0000 7f7cc5ab0740 1 Processor -- start 2024-09-16T00:20:10.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.472+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e034a0 0x55f441e0dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.472+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e03dc0 0x55f441e0b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.472+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e0b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.472+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f441e112a0 con 0x55f441e03dc0 2024-09-16T00:20:10.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.472+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f441d24050 con 0x55f441e034a0 2024-09-16T00:20:10.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.472+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f441e11420 con 0x55f441e0e680 2024-09-16T00:20:10.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.472+0000 7f7cc5ab0740 10 client.0 _ll_register_callbacks cb 0x55f441d64160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc1473640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e034a0 0x55f441e0dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc0c72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e03dc0 0x55f441e0b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc0c72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e03dc0 0x55f441e0b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42374/0 (socket says 172.21.15.78:42374) 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc0c72640 1 -- 172.21.15.78:0/1124822679 learned_addr learned my addr 172.21.15.78:0/1124822679 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc1473640 1 -- 172.21.15.78:0/1124822679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 msgr2=0x55f441e0b6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc1473640 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e0b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc1473640 1 -- 172.21.15.78:0/1124822679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e03dc0 msgr2=0x55f441e0b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc1473640 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e03dc0 0x55f441e0b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.473+0000 7f7cc1473640 1 -- 172.21.15.78:0/1124822679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f441ada3c0 con 0x55f441e034a0 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.474+0000 7f7cc1473640 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e034a0 0x55f441e0dea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4008000 tx=0x7f7cb4032df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59316f06141b8be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.474+0000 7f7ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.474+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/1124822679 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4030c00 con 0x55f441e034a0 2024-09-16T00:20:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.474+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1124822679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cac00e3b0 con 0x55f441e034a0 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.475+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/1124822679 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7cb4030da0 con 0x55f441e034a0 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.475+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/1124822679 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7cb4003970 con 0x55f441e034a0 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.475+0000 7f7cc5ab0740 10 client.26069 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.475+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1124822679 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f441e1cc50 con 0x55f441e034a0 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.475+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/1124822679 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cac00e3b0 con 0x55f441e034a0 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.476+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/1124822679 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7cb4004040 con 0x55f441e034a0 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.476+0000 7f7cc5ab0740 10 client.26069 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.476+0000 7f7cc5ab0740 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:20:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.476+0000 7f7cc5ab0740 20 client.26069 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:10.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.476+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f441e1e0b0 0x55f441e20510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.476+0000 7f7cc5ab0740 4 client.26069 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:10.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.476+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1124822679 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f441d76850 con 0x55f441e1e0b0 2024-09-16T00:20:10.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.476+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f441e1e0b0 0x55f441e20510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.477+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f441e1e0b0 0x55f441e20510 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d969c279faec3 server_cookie=129be1b6258fb559 in_seq=0 out_seq=0 2024-09-16T00:20:10.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.477+0000 7f7ca4ff9640 10 client.26069 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.477+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/1124822679 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f441d76850 con 0x55f441e1e0b0 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.477+0000 7f7ca4ff9640 10 client.26069 handle_command_reply: tid=0 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 client.26069 shutdown 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1124822679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e034a0 msgr2=0x55f441e0dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e034a0 0x55f441e0dea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4008000 tx=0x7f7cb4032df0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1124822679 shutdown_connections 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f441e1e0b0 0x55f441e20510 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e0b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e03dc0 0x55f441e0b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1124822679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e034a0 0x55f441e0dea0 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4008000 tx=0x7f7cb4032df0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1124822679 >> 172.21.15.78:0/1124822679 conn(0x55f441dffa30 msgr2=0x55f441e05760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7ca4ff9640 0 client.26069 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1124822679 shutdown_connections 2024-09-16T00:20:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.478+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1124822679 wait complete. 2024-09-16T00:20:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.479+0000 7f7cc5ab0740 20 client.26069 trim_cache size 0 max 16384 2024-09-16T00:20:10.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc5ab0740 1 Processor -- start 2024-09-16T00:20:10.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc5ab0740 1 -- start start 2024-09-16T00:20:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e03dc0 0x55f441e1a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e0e680 0x55f441e05650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca000fbe0 con 0x55f441e05b90 2024-09-16T00:20:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0011720 con 0x55f441e0e680 2024-09-16T00:20:10.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca00102b0 con 0x55f441e03dc0 2024-09-16T00:20:10.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc1c74640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc0c72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e0e680 0x55f441e05650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc1473640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e03dc0 0x55f441e1a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc0c72640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e0e680 0x55f441e05650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:49788/0 (socket says 172.21.15.78:49788) 2024-09-16T00:20:10.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.515+0000 7f7cc0c72640 1 -- 172.21.15.78:0/1300406627 learned_addr learned my addr 172.21.15.78:0/1300406627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:10.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7cc0c72640 1 -- 172.21.15.78:0/1300406627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e03dc0 msgr2=0x55f441e1a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:10.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/1300406627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e03dc0 0x55f441e1a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7cc0c72640 1 -- 172.21.15.78:0/1300406627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 msgr2=0x55f441e25de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:10.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/1300406627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e25de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7cc0c72640 1 -- 172.21.15.78:0/1300406627 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f441dff860 con 0x55f441e0e680 2024-09-16T00:20:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7cc1c74640 1 --2- 172.21.15.78:0/1300406627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e25de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/1300406627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e0e680 0x55f441e05650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8018530 tx=0x7f7cb80cd700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e753e49acc3ee3a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7ca6ffd640 1 -- 172.21.15.78:0/1300406627 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80a4fc0 con 0x55f441e0e680 2024-09-16T00:20:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7ca6ffd640 1 -- 172.21.15.78:0/1300406627 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb8016e10 con 0x55f441e0e680 2024-09-16T00:20:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.516+0000 7f7ca6ffd640 1 -- 172.21.15.78:0/1300406627 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80b0580 con 0x55f441e0e680 2024-09-16T00:20:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1300406627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e0e680 msgr2=0x55f441e05650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1300406627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e0e680 0x55f441e05650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8018530 tx=0x7f7cb80cd700 comp rx=0 tx=0).stop 2024-09-16T00:20:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1300406627 shutdown_connections 2024-09-16T00:20:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1300406627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e25de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1300406627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e0e680 0x55f441e05650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/1300406627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e03dc0 0x55f441e1a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1300406627 >> 172.21.15.78:0/1300406627 conn(0x55f441dffa30 msgr2=0x55f441e04f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1300406627 shutdown_connections 2024-09-16T00:20:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.517+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/1300406627 wait complete. 2024-09-16T00:20:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.518+0000 7f7cc5ab0740 1 -- start start 2024-09-16T00:20:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.518+0000 7f7cc5ab0740 1 Processor -- start 2024-09-16T00:20:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.518+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.518+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e08910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.518+0000 7f7cc5ab0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e08e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0011050 con 0x55f441e05b90 2024-09-16T00:20:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc1473640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc0c72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e08910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc1c74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e08e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc1473640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:49790/0 (socket says 172.21.15.78:49790) 2024-09-16T00:20:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc1473640 1 -- 172.21.15.78:0/240680151 learned_addr learned my addr 172.21.15.78:0/240680151 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc0c72640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e08910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42392/0 (socket says 172.21.15.78:42392) 2024-09-16T00:20:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc5ab0740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0010980 con 0x55f441e03dc0 2024-09-16T00:20:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca000dbc0 con 0x55f441e0e680 2024-09-16T00:20:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.519+0000 7f7cc5ab0740 10 client.0 _ll_register_callbacks cb 0x7f7cb8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.520+0000 7f7cc0c72640 1 -- 172.21.15.78:0/240680151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 msgr2=0x55f441e08e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.520+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e08e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.520+0000 7f7cc0c72640 1 -- 172.21.15.78:0/240680151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 msgr2=0x55f441e083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.520+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.520+0000 7f7cc0c72640 1 -- 172.21.15.78:0/240680151 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f441e87fb0 con 0x55f441e05b90 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.520+0000 7f7cc1473640 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.521+0000 7f7cc0c72640 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e08910 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55f441dff5f0 tx=0x7f7cb8018b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338a19a4110b0d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.521+0000 7f7ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.522+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/240680151 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8098b40 con 0x55f441e05b90 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.522+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/240680151 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7cb80cd940 con 0x55f441e05b90 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.522+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb8016e10 con 0x55f441e05b90 2024-09-16T00:20:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/240680151 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7cb80a4fc0 con 0x55f441e05b90 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc5ab0740 10 client.16677 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f441e89280 con 0x55f441e05b90 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/240680151 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cb8016e10 con 0x55f441e05b90 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/240680151 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7cb809b7a0 con 0x55f441e05b90 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc5ab0740 10 client.16677 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc5ab0740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc5ab0740 20 client.16677 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f441e89900 0x55f441e8bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc5ab0740 4 client.16677 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f441d70d60 con 0x55f441e89900 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.523+0000 7f7cc1473640 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f441e89900 0x55f441e8bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7ca4ff9640 10 client.16677 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc1473640 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f441e89900 0x55f441e8bd20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8132749a2cc02 server_cookie=d94e0ef24239d8fd in_seq=0 out_seq=0 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7ca4ff9640 1 -- 172.21.15.78:0/240680151 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f441d70d60 con 0x55f441e89900 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7ca4ff9640 10 client.16677 handle_command_reply: tid=0 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 client.16677 shutdown 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 msgr2=0x55f441e08910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e08910 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55f441dff5f0 tx=0x7f7cb8018b00 comp rx=0 tx=0).stop 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 shutdown_connections 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f441e89900 0x55f441e8bd20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f441e0e680 0x55f441e08e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f441e05b90 0x55f441e08910 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 --2- 172.21.15.78:0/240680151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f441e03dc0 0x55f441e083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.524+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 >> 172.21.15.78:0/240680151 conn(0x55f441dffa30 msgr2=0x55f441dfe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.525+0000 7f7ca4ff9640 0 client.16677 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.525+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 shutdown_connections 2024-09-16T00:20:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.525+0000 7f7cc5ab0740 1 -- 172.21.15.78:0/240680151 wait complete. 2024-09-16T00:20:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.525+0000 7f7cc5ab0740 20 client.16677 trim_cache size 0 max 16384 2024-09-16T00:20:10.537 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:20:10.537 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:20:10.538 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:20:10.538 INFO:teuthology.orchestra.run.smithi078.stdout: "c8520dd3-ef7a-4e3e-90c2-dceaae6a5280": { 2024-09-16T00:20:10.538 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:20:10.538 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "c8520dd3-ef7a-4e3e-90c2-dceaae6a5280", 2024-09-16T00:20:10.538 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:20:10.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:20:10.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:20:10.538 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:20:10.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.525+0000 7f7cc36d7640 1 -- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7ca80a33c0 msgr2=0x7f7ca80a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:10.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.525+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7ca80a33c0 0x7f7ca80a5880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7cac01f000 tx=0x7f7cac019c70 comp rx=0 tx=0).stop 2024-09-16T00:20:10.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.525+0000 7f7cc36d7640 1 -- 172.21.15.78:0/3959270460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cbc11b140 msgr2=0x7f7cbc114a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:10.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.525+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cbc11b140 0x7f7cbc114a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800e9f0 tx=0x7f7cb800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.526+0000 7f7cc36d7640 1 -- 172.21.15.78:0/3959270460 shutdown_connections 2024-09-16T00:20:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.526+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7ca80a33c0 0x7f7ca80a5880 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.526+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cbc114fc0 0x7f7cbc115470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.526+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7cbc11b140 0x7f7cbc114a80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.526+0000 7f7cc36d7640 1 --2- 172.21.15.78:0/3959270460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cbc109830 0x7f7cbc11aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.526+0000 7f7cc36d7640 1 -- 172.21.15.78:0/3959270460 >> 172.21.15.78:0/3959270460 conn(0x7f7cbc06e140 msgr2=0x7f7cbc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.526+0000 7f7cc36d7640 1 -- 172.21.15.78:0/3959270460 shutdown_connections 2024-09-16T00:20:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:10.526+0000 7f7cc36d7640 1 -- 172.21.15.78:0/3959270460 wait complete. 2024-09-16T00:20:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 370.0 s, 1319.0 tps, lat 76.091 ms stddev 89.934 2024-09-16T00:20:11.339 INFO:tasks.cephfs.filesystem:scrub status for tag:c8520dd3-ef7a-4e3e-90c2-dceaae6a5280 - {'path': '/', 'tag': 'c8520dd3-ef7a-4e3e-90c2-dceaae6a5280', 'options': 'recursive,force'} 2024-09-16T00:20:11.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:11 smithi193 ceph-mon[35484]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 675 op/s 2024-09-16T00:20:11.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:11 smithi148 ceph-mon[35599]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 675 op/s 2024-09-16T00:20:12.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:11 smithi078 ceph-mon[31157]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 14 MiB/s wr, 675 op/s 2024-09-16T00:20:13.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:13 smithi193 ceph-mon[35484]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 623 op/s 2024-09-16T00:20:13.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:13 smithi148 ceph-mon[35599]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 623 op/s 2024-09-16T00:20:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:13 smithi078 ceph-mon[31157]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 623 op/s 2024-09-16T00:20:15.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 375.0 s, 1293.6 tps, lat 77.193 ms stddev 92.748 2024-09-16T00:20:15.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:15 smithi193 ceph-mon[35484]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 571 op/s 2024-09-16T00:20:15.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:15 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:15 smithi148 ceph-mon[35599]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 571 op/s 2024-09-16T00:20:15.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:15 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:16.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:15 smithi078 ceph-mon[31157]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 571 op/s 2024-09-16T00:20:16.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:15 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:16.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:16.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:20:16.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:16.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:20:17.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:20:17.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:20:17.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:17 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi148 to 176.0M 2024-09-16T00:20:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:17 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-16T00:20:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:17 smithi193 ceph-mon[35484]: Unable to set osd_memory_target on smithi148 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-16T00:20:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:17 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-16T00:20:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:17 smithi193 ceph-mon[35484]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T00:20:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:17 smithi193 ceph-mon[35484]: Unable to set osd_memory_target on smithi078 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-16T00:20:17.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:17 smithi193 ceph-mon[35484]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 583 op/s 2024-09-16T00:20:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:17 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi148 to 176.0M 2024-09-16T00:20:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:17 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-16T00:20:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:17 smithi148 ceph-mon[35599]: Unable to set osd_memory_target on smithi148 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-16T00:20:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:17 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-16T00:20:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:17 smithi148 ceph-mon[35599]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T00:20:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:17 smithi148 ceph-mon[35599]: Unable to set osd_memory_target on smithi078 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-16T00:20:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:17 smithi148 ceph-mon[35599]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 583 op/s 2024-09-16T00:20:18.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:17 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi148 to 176.0M 2024-09-16T00:20:18.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:17 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-16T00:20:18.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:17 smithi078 ceph-mon[31157]: Unable to set osd_memory_target on smithi148 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-16T00:20:18.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:17 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-16T00:20:18.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:17 smithi078 ceph-mon[31157]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T00:20:18.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:17 smithi078 ceph-mon[31157]: Unable to set osd_memory_target on smithi078 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-16T00:20:18.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:17 smithi078 ceph-mon[31157]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 583 op/s 2024-09-16T00:20:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:19 smithi193 ceph-mon[35484]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 553 op/s 2024-09-16T00:20:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:20:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:20:19.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:20:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:19 smithi148 ceph-mon[35599]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 553 op/s 2024-09-16T00:20:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:20:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:20:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:20:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:19 smithi078 ceph-mon[31157]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 553 op/s 2024-09-16T00:20:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:20:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:20:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:20:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 380.0 s, 594.4 tps, lat 168.322 ms stddev 410.826 2024-09-16T00:20:21.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:21 smithi193 ceph-mon[35484]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-09-16T00:20:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:21 smithi148 ceph-mon[35599]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-09-16T00:20:22.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:21 smithi078 ceph-mon[31157]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 MiB/s wr, 1.14k op/s 2024-09-16T00:20:23.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:23 smithi193 ceph-mon[35484]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-09-16T00:20:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:23 smithi148 ceph-mon[35599]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-09-16T00:20:24.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:23 smithi078 ceph-mon[31157]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-09-16T00:20:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 385.0 s, 1317.2 tps, lat 74.840 ms stddev 86.433 2024-09-16T00:20:25.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:25 smithi193 ceph-mon[35484]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-09-16T00:20:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:25 smithi148 ceph-mon[35599]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-09-16T00:20:26.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:25 smithi078 ceph-mon[31157]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-09-16T00:20:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:27 smithi193 ceph-mon[35484]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.55k op/s 2024-09-16T00:20:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:27 smithi148 ceph-mon[35599]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.55k op/s 2024-09-16T00:20:28.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:27 smithi078 ceph-mon[31157]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 21 MiB/s wr, 1.55k op/s 2024-09-16T00:20:29.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:29 smithi193 ceph-mon[35484]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 1.53k op/s 2024-09-16T00:20:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:29 smithi148 ceph-mon[35599]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 1.53k op/s 2024-09-16T00:20:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:29 smithi078 ceph-mon[31157]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 20 MiB/s wr, 1.53k op/s 2024-09-16T00:20:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 390.0 s, 942.8 tps, lat 107.291 ms stddev 126.559 2024-09-16T00:20:31.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:31 smithi193 ceph-mon[35484]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-09-16T00:20:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:31 smithi148 ceph-mon[35599]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-09-16T00:20:32.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:31 smithi078 ceph-mon[31157]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 21 MiB/s wr, 1.66k op/s 2024-09-16T00:20:33.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:33 smithi148 ceph-mon[35599]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-09-16T00:20:34.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:33 smithi078 ceph-mon[31157]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-09-16T00:20:34.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:33 smithi193 ceph-mon[35484]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-09-16T00:20:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 395.0 s, 1153.2 tps, lat 85.739 ms stddev 101.134 2024-09-16T00:20:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:35 smithi148 ceph-mon[35599]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 663 op/s 2024-09-16T00:20:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:35 smithi078 ceph-mon[31157]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 663 op/s 2024-09-16T00:20:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:35 smithi193 ceph-mon[35484]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 MiB/s wr, 663 op/s 2024-09-16T00:20:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:37 smithi148 ceph-mon[35599]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 667 op/s 2024-09-16T00:20:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:37 smithi078 ceph-mon[31157]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 667 op/s 2024-09-16T00:20:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:37 smithi193 ceph-mon[35484]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 MiB/s wr, 667 op/s 2024-09-16T00:20:39.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:39 smithi148 ceph-mon[35599]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-09-16T00:20:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:39 smithi078 ceph-mon[31157]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-09-16T00:20:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:39 smithi193 ceph-mon[35484]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.5 MiB/s wr, 643 op/s 2024-09-16T00:20:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 400.0 s, 847.4 tps, lat 118.513 ms stddev 127.313 2024-09-16T00:20:41.340 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:20:41.704 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:41 smithi078 ceph-mon[31157]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 714 op/s 2024-09-16T00:20:41.800 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:41.956 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:41 smithi148 ceph-mon[35599]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 714 op/s 2024-09-16T00:20:41.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:41 smithi193 ceph-mon[35484]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 714 op/s 2024-09-16T00:20:42.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 -- 172.21.15.78:0/2328963418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100cc0 msgr2=0x7f545c106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 --2- 172.21.15.78:0/2328963418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100cc0 0x7f545c106800 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f544c009900 tx=0x7f544c02a410 comp rx=0 tx=0).stop 2024-09-16T00:20:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 -- 172.21.15.78:0/2328963418 shutdown_connections 2024-09-16T00:20:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 --2- 172.21.15.78:0/2328963418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c106f30 0x7f545c10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 --2- 172.21.15.78:0/2328963418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100cc0 0x7f545c106800 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 --2- 172.21.15.78:0/2328963418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f545c100310 0x7f545c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 -- 172.21.15.78:0/2328963418 >> 172.21.15.78:0/2328963418 conn(0x7f545c077210 msgr2=0x7f545c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 -- 172.21.15.78:0/2328963418 shutdown_connections 2024-09-16T00:20:42.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 -- 172.21.15.78:0/2328963418 wait complete. 2024-09-16T00:20:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.571+0000 7f5463535640 1 Processor -- start 2024-09-16T00:20:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f5463535640 1 -- start start 2024-09-16T00:20:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f5463535640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100310 0x7f545c101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f5463535640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c100cc0 0x7f545c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f5463535640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f545c106f30 0x7f545c101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f5463535640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f545c111f90 con 0x7f545c106f30 2024-09-16T00:20:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f5463535640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f545c111e10 con 0x7f545c100310 2024-09-16T00:20:42.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f5463535640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f545c112110 con 0x7f545c100cc0 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f54612d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100310 0x7f545c101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f54612d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100310 0x7f545c101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:55912/0 (socket says 172.21.15.78:55912) 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f54612d1640 1 -- 172.21.15.78:0/558047976 learned_addr learned my addr 172.21.15.78:0/558047976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f54612d1640 1 -- 172.21.15.78:0/558047976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c100cc0 msgr2=0x7f545c1017d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f54612d1640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c100cc0 0x7f545c1017d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f54612d1640 1 -- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f545c106f30 msgr2=0x7f545c101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f54612d1640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f545c106f30 0x7f545c101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.572+0000 7f54612d1640 1 -- 172.21.15.78:0/558047976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c19c5c0 con 0x7f545c100310 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.573+0000 7f54612d1640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100310 0x7f545c101290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f545800e9f0 tx=0x7f545800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.573+0000 7f54527fc640 1 -- 172.21.15.78:0/558047976 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545800cd80 con 0x7f545c100310 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.573+0000 7f54527fc640 1 -- 172.21.15.78:0/558047976 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5458010910 con 0x7f545c100310 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.573+0000 7f54527fc640 1 -- 172.21.15.78:0/558047976 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458018950 con 0x7f545c100310 2024-09-16T00:20:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.573+0000 7f5463535640 1 -- 172.21.15.78:0/558047976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c19c8b0 con 0x7f545c100310 2024-09-16T00:20:42.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.573+0000 7f5463535640 1 -- 172.21.15.78:0/558047976 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c19cce0 con 0x7f545c100310 2024-09-16T00:20:42.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.577+0000 7f54527fc640 1 -- 172.21.15.78:0/558047976 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5458018b30 con 0x7f545c100310 2024-09-16T00:20:42.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.577+0000 7f546590e740 1 Processor -- start 2024-09-16T00:20:42.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.577+0000 7f546590e740 1 -- start start 2024-09-16T00:20:42.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f546590e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f546590e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f546590e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f546590e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb4c5162e0 con 0x55fb4c5bb680 2024-09-16T00:20:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f546590e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fb4c4bdce0 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f546590e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fb4c5122a0 con 0x55fb4c5b04a0 2024-09-16T00:20:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f54527fc640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54480a3410 0x7f54480a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f5461ad2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f5461ad2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53380/0 (socket says 172.21.15.78:53380) 2024-09-16T00:20:42.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f5461ad2640 1 -- 172.21.15.78:0/3944920434 learned_addr learned my addr 172.21.15.78:0/3944920434 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:42.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f54527fc640 1 -- 172.21.15.78:0/558047976 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f54580cff90 con 0x7f545c100310 2024-09-16T00:20:42.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f5460ad0640 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.578+0000 7f54612d1640 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f5461ad2640 1 -- 172.21.15.78:0/3944920434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 msgr2=0x55fb4c5bb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:42.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f5461ad2640 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f5461ad2640 1 -- 172.21.15.78:0/3944920434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 msgr2=0x55fb4c5bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f5461ad2640 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f5461ad2640 1 -- 172.21.15.78:0/3944920434 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4c4a2440 con 0x55fb4c5b04a0 2024-09-16T00:20:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f54612d1640 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f5461ad2640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54480a3410 0x7f54480a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f5461ad2640 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5b0880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5454009fc0 tx=0x7f5454010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7174f944275f9647 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f54477fe640 1 -- 172.21.15.78:0/3944920434 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454022070 con 0x55fb4c5b04a0 2024-09-16T00:20:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.579+0000 7f54477fe640 1 -- 172.21.15.78:0/3944920434 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5454017070 con 0x55fb4c5b04a0 2024-09-16T00:20:42.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f54477fe640 1 -- 172.21.15.78:0/3944920434 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454011640 con 0x55fb4c5b04a0 2024-09-16T00:20:42.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 -- 172.21.15.78:0/3944920434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 msgr2=0x55fb4c5b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:42.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f5461ad2640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54480a3410 0x7f54480a58d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f545401f000 tx=0x7f5454022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:42.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5b0880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5454009fc0 tx=0x7f5454010740 comp rx=0 tx=0).stop 2024-09-16T00:20:42.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 -- 172.21.15.78:0/3944920434 shutdown_connections 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 --2- 172.21.15.78:0/3944920434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5b0880 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 -- 172.21.15.78:0/3944920434 >> 172.21.15.78:0/3944920434 conn(0x55fb4c5aca30 msgr2=0x55fb4c5ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 -- 172.21.15.78:0/3944920434 shutdown_connections 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.580+0000 7f546590e740 1 -- 172.21.15.78:0/3944920434 wait complete. 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 1 -- start start 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 1 Processor -- start 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5baf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55fb4c5be2a0 con 0x55fb4c5bb680 2024-09-16T00:20:42.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fb4c4d1050 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fb4c5be420 con 0x55fb4c5b04a0 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.581+0000 7f546590e740 10 client.0 _ll_register_callbacks cb 0x55fb4c511160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5baf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:53394/0 (socket says 172.21.15.78:53394) 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 -- 172.21.15.78:0/233098188 learned_addr learned my addr 172.21.15.78:0/233098188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f5460ad0640 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 -- 172.21.15.78:0/233098188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 msgr2=0x55fb4c5b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 -- 172.21.15.78:0/233098188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 msgr2=0x55fb4c5b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 -- 172.21.15.78:0/233098188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4c2873c0 con 0x55fb4c5b04a0 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f54612d1640 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5baf30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f54580cd670 tx=0x7f545809c1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2727930291ab71af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:42.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f5444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f5444ff9640 1 -- 172.21.15.78:0/233098188 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580a0070 con 0x55fb4c5b04a0 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f5444ff9640 1 -- 172.21.15.78:0/233098188 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f54580cdda0 con 0x55fb4c5b04a0 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f546590e740 1 -- 172.21.15.78:0/233098188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5454017070 con 0x55fb4c5b04a0 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f5444ff9640 1 -- 172.21.15.78:0/233098188 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f54580a0210 con 0x55fb4c5b04a0 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f546590e740 10 client.25909 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.582+0000 7f546590e740 1 -- 172.21.15.78:0/233098188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fb4c5c9bf0 con 0x55fb4c5b04a0 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f5444ff9640 1 -- 172.21.15.78:0/233098188 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5454017070 con 0x55fb4c5b04a0 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f5444ff9640 1 -- 172.21.15.78:0/233098188 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f54580a18c0 con 0x55fb4c5b04a0 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f546590e740 10 client.25909 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f546590e740 10 client.25909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f546590e740 20 client.25909 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f546590e740 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55fb4c5cb0b0 0x55fb4c5cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f546590e740 4 client.25909 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:42.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f546590e740 1 -- 172.21.15.78:0/233098188 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb4c523850 con 0x55fb4c5cb0b0 2024-09-16T00:20:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.583+0000 7f5460ad0640 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55fb4c5cb0b0 0x55fb4c5cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.584+0000 7f5444ff9640 10 client.25909 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:42.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.584+0000 7f5460ad0640 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55fb4c5cb0b0 0x55fb4c5cd570 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b65eb7a0af5b3fcc server_cookie=4c57d80e84f53499 in_seq=0 out_seq=0 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.585+0000 7f5444ff9640 1 -- 172.21.15.78:0/233098188 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb4c523850 con 0x55fb4c5cb0b0 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.585+0000 7f5444ff9640 10 client.25909 handle_command_reply: tid=0 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.585+0000 7f546590e740 1 client.25909 shutdown 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 -- 172.21.15.78:0/233098188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 msgr2=0x55fb4c5baf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5baf30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f54580cd670 tx=0x7f545809c1a0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 -- 172.21.15.78:0/233098188 shutdown_connections 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55fb4c5cb0b0 0x55fb4c5cd570 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 --2- 172.21.15.78:0/233098188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b04a0 0x55fb4c5baf30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 -- 172.21.15.78:0/233098188 >> 172.21.15.78:0/233098188 conn(0x55fb4c5aca30 msgr2=0x55fb4c5b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:42.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f5444ff9640 0 client.25909 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:42.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 -- 172.21.15.78:0/233098188 shutdown_connections 2024-09-16T00:20:42.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 1 -- 172.21.15.78:0/233098188 wait complete. 2024-09-16T00:20:42.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.586+0000 7f546590e740 20 client.25909 trim_cache size 0 max 16384 2024-09-16T00:20:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f546590e740 1 Processor -- start 2024-09-16T00:20:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f546590e740 1 -- start start 2024-09-16T00:20:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f546590e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f546590e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f546590e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 0x55fb4c5d3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f546590e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f544000faa0 con 0x55fb4c5bb680 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f546590e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f54400115e0 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f546590e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5440010170 con 0x55fb4c5b3900 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5460ad0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5460ad0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47922/0 (socket says 172.21.15.78:47922) 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5460ad0640 1 -- 172.21.15.78:0/287280097 learned_addr learned my addr 172.21.15.78:0/287280097 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5461ad2640 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 0x55fb4c5d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f54612d1640 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5460ad0640 1 -- 172.21.15.78:0/287280097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 msgr2=0x55fb4c5d3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5460ad0640 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 0x55fb4c5d3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5460ad0640 1 -- 172.21.15.78:0/287280097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 msgr2=0x55fb4c5c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5460ad0640 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.610+0000 7f5460ad0640 1 -- 172.21.15.78:0/287280097 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4c5ac860 con 0x55fb4c5bb680 2024-09-16T00:20:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.611+0000 7f5460ad0640 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5d2e40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f544c0096c0 tx=0x7f544c005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42e77343c71cd39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.611+0000 7f5446ffd640 1 -- 172.21.15.78:0/287280097 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c04b070 con 0x55fb4c5bb680 2024-09-16T00:20:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.611+0000 7f5446ffd640 1 -- 172.21.15.78:0/287280097 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544c031810 con 0x55fb4c5bb680 2024-09-16T00:20:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.611+0000 7f5446ffd640 1 -- 172.21.15.78:0/287280097 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c035040 con 0x55fb4c5bb680 2024-09-16T00:20:42.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.611+0000 7f546590e740 1 -- 172.21.15.78:0/287280097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 msgr2=0x55fb4c5d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:42.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.611+0000 7f546590e740 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5d2e40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f544c0096c0 tx=0x7f544c005890 comp rx=0 tx=0).stop 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 -- 172.21.15.78:0/287280097 shutdown_connections 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 0x55fb4c5d3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5bb680 0x55fb4c5d2e40 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 --2- 172.21.15.78:0/287280097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 -- 172.21.15.78:0/287280097 >> 172.21.15.78:0/287280097 conn(0x55fb4c5aca30 msgr2=0x55fb4c5b6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 -- 172.21.15.78:0/287280097 shutdown_connections 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 -- 172.21.15.78:0/287280097 wait complete. 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 -- start start 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.612+0000 7f546590e740 1 Processor -- start 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f546590e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f546590e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 0x55fb4c5bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f546590e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5bb680 0x55fb4c5c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f546590e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5440010f10 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f546590e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5440010840 con 0x55fb4c5bb680 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f546590e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f544000da80 con 0x55fb4c5b3900 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f546590e740 10 client.0 _ll_register_callbacks cb 0x7f545809c8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47930/0 (socket says 172.21.15.78:47930) 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 -- 172.21.15.78:0/1546775489 learned_addr learned my addr 172.21.15.78:0/1546775489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f5461ad2640 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5bb680 0x55fb4c5c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f5460ad0640 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 0x55fb4c5bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 -- 172.21.15.78:0/1546775489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 msgr2=0x55fb4c5bfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 0x55fb4c5bfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 -- 172.21.15.78:0/1546775489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5bb680 msgr2=0x55fb4c5c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5bb680 0x55fb4c5c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 -- 172.21.15.78:0/1546775489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4c635100 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f54612d1640 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bf710 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5458009230 tx=0x7f5458099290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9a6dd380d85e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f5444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f5444ff9640 1 -- 172.21.15.78:0/1546775489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580a0040 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f546590e740 1 -- 172.21.15.78:0/1546775489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544c031810 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.613+0000 7f5444ff9640 1 -- 172.21.15.78:0/1546775489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f545809bda0 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.614+0000 7f5444ff9640 1 -- 172.21.15.78:0/1546775489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f54580a05a0 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.614+0000 7f546590e740 10 client.16689 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.614+0000 7f546590e740 1 -- 172.21.15.78:0/1546775489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fb4c636410 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.614+0000 7f5444ff9640 1 -- 172.21.15.78:0/1546775489 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54580cdda0 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.614+0000 7f5444ff9640 1 -- 172.21.15.78:0/1546775489 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f54580b0270 con 0x55fb4c5b0dc0 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.614+0000 7f546590e740 10 client.16689 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:42.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.614+0000 7f546590e740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.614+0000 7f546590e740 20 client.16689 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.615+0000 7f546590e740 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55fb4c6369f0 0x55fb4c638de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.615+0000 7f546590e740 4 client.16689 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.615+0000 7f546590e740 1 -- 172.21.15.78:0/1546775489 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f544c031810 con 0x55fb4c6369f0 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.615+0000 7f5460ad0640 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55fb4c6369f0 0x55fb4c638de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.616+0000 7f5460ad0640 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55fb4c6369f0 0x55fb4c638de0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=686575f318b12492 server_cookie=35e455bc6be0f891 in_seq=0 out_seq=0 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.616+0000 7f5444ff9640 10 client.16689 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.616+0000 7f5444ff9640 1 -- 172.21.15.78:0/1546775489 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f544c031810 con 0x55fb4c6369f0 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.616+0000 7f5444ff9640 10 client.16689 handle_command_reply: tid=0 2024-09-16T00:20:42.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.616+0000 7f546590e740 1 client.16689 shutdown 2024-09-16T00:20:42.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.617+0000 7f546590e740 1 -- 172.21.15.78:0/1546775489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5b0dc0 msgr2=0x55fb4c5bf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:42.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.617+0000 7f546590e740 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bf710 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5458009230 tx=0x7f5458099290 comp rx=0 tx=0).stop 2024-09-16T00:20:42.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 1 -- 172.21.15.78:0/1546775489 shutdown_connections 2024-09-16T00:20:42.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55fb4c6369f0 0x55fb4c638de0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb4c5bb680 0x55fb4c5c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fb4c5b3900 0x55fb4c5bfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 1 --2- 172.21.15.78:0/1546775489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55fb4c5b0dc0 0x55fb4c5bf710 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 1 -- 172.21.15.78:0/1546775489 >> 172.21.15.78:0/1546775489 conn(0x55fb4c5aca30 msgr2=0x55fb4c5abb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f5444ff9640 0 client.16689 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 1 -- 172.21.15.78:0/1546775489 shutdown_connections 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 1 -- 172.21.15.78:0/1546775489 wait complete. 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.618+0000 7f546590e740 20 client.16689 trim_cache size 0 max 16384 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:20:42.622 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:20:42.623 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:20:42.623 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:20:42.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 -- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54480a3410 msgr2=0x7f54480a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:42.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54480a3410 0x7f54480a58d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f545401f000 tx=0x7f5454022000 comp rx=0 tx=0).stop 2024-09-16T00:20:42.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 -- 172.21.15.78:0/558047976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100310 msgr2=0x7f545c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:42.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100310 0x7f545c101290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f545800e9f0 tx=0x7f545800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 -- 172.21.15.78:0/558047976 shutdown_connections 2024-09-16T00:20:42.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f54480a3410 0x7f54480a58d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f545c106f30 0x7f545c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c100cc0 0x7f545c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 --2- 172.21.15.78:0/558047976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f545c100310 0x7f545c101290 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:42.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 -- 172.21.15.78:0/558047976 >> 172.21.15.78:0/558047976 conn(0x7f545c077210 msgr2=0x7f545c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:42.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 -- 172.21.15.78:0/558047976 shutdown_connections 2024-09-16T00:20:42.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:42.619+0000 7f5463535640 1 -- 172.21.15.78:0/558047976 wait complete. 2024-09-16T00:20:43.433 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:20:43.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:20:43.839 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:43.965 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:43 smithi078 ceph-mon[31157]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-09-16T00:20:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:43 smithi193 ceph-mon[35484]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-09-16T00:20:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:43 smithi148 ceph-mon[35599]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-09-16T00:20:44.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.433+0000 7fcaa644b640 1 -- 172.21.15.78:0/2581491545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 msgr2=0x7fcaa010bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:44.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.433+0000 7fcaa644b640 1 --2- 172.21.15.78:0/2581491545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 0x7fcaa010bb40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fca88009860 tx=0x7fca8802a3d0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.433+0000 7fcaa644b640 1 -- 172.21.15.78:0/2581491545 shutdown_connections 2024-09-16T00:20:44.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.433+0000 7fcaa644b640 1 --2- 172.21.15.78:0/2581491545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa010c160 0x7fcaa010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.433+0000 7fcaa644b640 1 --2- 172.21.15.78:0/2581491545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 0x7fcaa010bb40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.433+0000 7fcaa644b640 1 --2- 172.21.15.78:0/2581491545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaa0102d30 0x7fcaa0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.433+0000 7fcaa644b640 1 -- 172.21.15.78:0/2581491545 >> 172.21.15.78:0/2581491545 conn(0x7fcaa00fc7d0 msgr2=0x7fcaa00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:44.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.433+0000 7fcaa644b640 1 -- 172.21.15.78:0/2581491545 shutdown_connections 2024-09-16T00:20:44.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.434+0000 7fcaa644b640 1 -- 172.21.15.78:0/2581491545 wait complete. 2024-09-16T00:20:44.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.435+0000 7fcaa644b640 1 Processor -- start 2024-09-16T00:20:44.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.435+0000 7fcaa644b640 1 -- start start 2024-09-16T00:20:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fcaa644b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaa0102d30 0x7fcaa01a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fcaa644b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 0x7fcaa01a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fcaa644b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa010c160 0x7fcaa010ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fcaa644b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0111f60 con 0x7fcaa010c160 2024-09-16T00:20:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fcaa644b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0111de0 con 0x7fcaa0103650 2024-09-16T00:20:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fcaa644b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01120e0 con 0x7fcaa0102d30 2024-09-16T00:20:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 0x7fcaa01a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 0x7fcaa01a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:55976/0 (socket says 172.21.15.78:55976) 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9f7fe640 1 -- 172.21.15.78:0/578530563 learned_addr learned my addr 172.21.15.78:0/578530563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fcaa49e8640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa010c160 0x7fcaa010ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9ffff640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaa0102d30 0x7fcaa01a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9f7fe640 1 -- 172.21.15.78:0/578530563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaa0102d30 msgr2=0x7fcaa01a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9f7fe640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaa0102d30 0x7fcaa01a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9f7fe640 1 -- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa010c160 msgr2=0x7fcaa010ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9f7fe640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa010c160 0x7fcaa010ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.436+0000 7fca9f7fe640 1 -- 172.21.15.78:0/578530563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa010b5c0 con 0x7fcaa0103650 2024-09-16T00:20:44.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.437+0000 7fca9f7fe640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 0x7fcaa01a4d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca88000c00 tx=0x7fca88004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.437+0000 7fca9d7fa640 1 -- 172.21.15.78:0/578530563 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88040070 con 0x7fcaa0103650 2024-09-16T00:20:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.437+0000 7fca9d7fa640 1 -- 172.21.15.78:0/578530563 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca88030870 con 0x7fcaa0103650 2024-09-16T00:20:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.437+0000 7fca9d7fa640 1 -- 172.21.15.78:0/578530563 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88003d40 con 0x7fcaa0103650 2024-09-16T00:20:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.437+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa010b850 con 0x7fcaa0103650 2024-09-16T00:20:44.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.437+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01ab580 con 0x7fcaa0103650 2024-09-16T00:20:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.440+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca80005290 con 0x7fcaa0103650 2024-09-16T00:20:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.440+0000 7fca9d7fa640 1 -- 172.21.15.78:0/578530563 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fca88039040 con 0x7fcaa0103650 2024-09-16T00:20:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.441+0000 7fca9d7fa640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fca8c0a32c0 0x7fca8c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.441+0000 7fca9ffff640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fca8c0a32c0 0x7fca8c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.441+0000 7fca9d7fa640 1 -- 172.21.15.78:0/578530563 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fca880f09e0 con 0x7fcaa0103650 2024-09-16T00:20:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.442+0000 7fca9ffff640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fca8c0a32c0 0x7fca8c0a5780 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fca98009b60 tx=0x7fca98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.446+0000 7fca9d7fa640 1 -- 172.21.15.78:0/578530563 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca880bd3a0 con 0x7fcaa0103650 2024-09-16T00:20:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.825+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca80005580 con 0x7fcaa0103650 2024-09-16T00:20:44.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.827+0000 7fca9d7fa640 1 -- 172.21.15.78:0/578530563 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fca880bcca0 con 0x7fcaa0103650 2024-09-16T00:20:44.828 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:20:44.828 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:20:44.828 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:20:44.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.830+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fca8c0a32c0 msgr2=0x7fca8c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:44.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.830+0000 7fcaa644b640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fca8c0a32c0 0x7fca8c0a5780 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fca98009b60 tx=0x7fca98005f90 comp rx=0 tx=0).stop 2024-09-16T00:20:44.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.830+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 msgr2=0x7fcaa01a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:44.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.830+0000 7fcaa644b640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 0x7fcaa01a4d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca88000c00 tx=0x7fca88004610 comp rx=0 tx=0).stop 2024-09-16T00:20:44.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.831+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 shutdown_connections 2024-09-16T00:20:44.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.831+0000 7fcaa644b640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fca8c0a32c0 0x7fca8c0a5780 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.831+0000 7fcaa644b640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa010c160 0x7fcaa010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.831+0000 7fcaa644b640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcaa0103650 0x7fcaa01a4d60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.831+0000 7fcaa644b640 1 --2- 172.21.15.78:0/578530563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaa0102d30 0x7fcaa01a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:44.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.831+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 >> 172.21.15.78:0/578530563 conn(0x7fcaa00fc7d0 msgr2=0x7fcaa0101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:44.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.832+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 shutdown_connections 2024-09-16T00:20:44.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:44.832+0000 7fcaa644b640 1 -- 172.21.15.78:0/578530563 wait complete. 2024-09-16T00:20:45.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 405.0 s, 1236.4 tps, lat 81.270 ms stddev 96.614 2024-09-16T00:20:46.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:45 smithi078 ceph-mon[31157]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-16T00:20:46.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:45 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/578530563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:20:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:45 smithi193 ceph-mon[35484]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-16T00:20:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:45 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/578530563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:20:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:45 smithi148 ceph-mon[35599]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-16T00:20:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:45 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/578530563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:20:48.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:47 smithi078 ceph-mon[31157]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 558 op/s 2024-09-16T00:20:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:47 smithi193 ceph-mon[35484]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 558 op/s 2024-09-16T00:20:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:47 smithi148 ceph-mon[35599]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 558 op/s 2024-09-16T00:20:49.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:20:49.517 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:50.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:49 smithi078 ceph-mon[31157]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 MiB/s wr, 550 op/s 2024-09-16T00:20:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:49 smithi193 ceph-mon[35484]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 MiB/s wr, 550 op/s 2024-09-16T00:20:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:49 smithi148 ceph-mon[35599]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.2 MiB/s wr, 550 op/s 2024-09-16T00:20:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 410.0 s, 1053.0 tps, lat 93.184 ms stddev 111.007 2024-09-16T00:20:50.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.913+0000 7fd6ab118640 1 -- 172.21.15.78:0/3822742850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a411b140 msgr2=0x7fd6a411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.913+0000 7fd6ab118640 1 --2- 172.21.15.78:0/3822742850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a411b140 0x7fd6a411d530 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009be0 tx=0x7fd69c02a710 comp rx=0 tx=0).stop 2024-09-16T00:20:50.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.913+0000 7fd6ab118640 1 -- 172.21.15.78:0/3822742850 shutdown_connections 2024-09-16T00:20:50.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.913+0000 7fd6ab118640 1 --2- 172.21.15.78:0/3822742850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a411b140 0x7fd6a411d530 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.913+0000 7fd6ab118640 1 --2- 172.21.15.78:0/3822742850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6a406ea30 0x7fd6a406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.913+0000 7fd6ab118640 1 --2- 172.21.15.78:0/3822742850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6a4110190 0x7fd6a406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.913+0000 7fd6ab118640 1 -- 172.21.15.78:0/3822742850 >> 172.21.15.78:0/3822742850 conn(0x7fd6a4073180 msgr2=0x7fd6a4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:50.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.917+0000 7fd6ab118640 1 -- 172.21.15.78:0/3822742850 shutdown_connections 2024-09-16T00:20:50.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.917+0000 7fd6ab118640 1 -- 172.21.15.78:0/3822742850 wait complete. 2024-09-16T00:20:50.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.918+0000 7fd6ab118640 1 Processor -- start 2024-09-16T00:20:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.924+0000 7fd6ab118640 1 -- start start 2024-09-16T00:20:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6ab118640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6a406ea30 0x7fd6a411a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6ab118640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4110190 0x7fd6a4113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6ab118640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6a411b140 0x7fd6a4113d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6ab118640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4120bf0 con 0x7fd6a406ea30 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6ab118640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4120a70 con 0x7fd6a411b140 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6ab118640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4120d70 con 0x7fd6a4110190 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6aa917640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6a411b140 0x7fd6a4113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6aa116640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6a406ea30 0x7fd6a411a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6aa917640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6a411b140 0x7fd6a4113d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34800/0 (socket says 172.21.15.78:34800) 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6aa116640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6a406ea30 0x7fd6a411a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39952/0 (socket says 172.21.15.78:39952) 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6aa917640 1 -- 172.21.15.78:0/1665901226 learned_addr learned my addr 172.21.15.78:0/1665901226 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.925+0000 7fd6a9915640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4110190 0x7fd6a4113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.926+0000 7fd6aa917640 1 -- 172.21.15.78:0/1665901226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4110190 msgr2=0x7fd6a4113820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.926+0000 7fd6aa917640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4110190 0x7fd6a4113820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.926+0000 7fd6aa917640 1 -- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6a406ea30 msgr2=0x7fd6a411a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.926+0000 7fd6aa917640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6a406ea30 0x7fd6a411a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.926+0000 7fd6aa917640 1 -- 172.21.15.78:0/1665901226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a4114350 con 0x7fd6a411b140 2024-09-16T00:20:50.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.926+0000 7fd6a9915640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4110190 0x7fd6a4113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:50.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.926+0000 7fd6aa917640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6a411b140 0x7fd6a4113d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd69c02ac30 tx=0x7fd69c030620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.928+0000 7fd69b7fe640 1 -- 172.21.15.78:0/1665901226 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c040020 con 0x7fd6a411b140 2024-09-16T00:20:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.928+0000 7fd69b7fe640 1 -- 172.21.15.78:0/1665901226 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd69c003e50 con 0x7fd6a411b140 2024-09-16T00:20:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.928+0000 7fd69b7fe640 1 -- 172.21.15.78:0/1665901226 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c036040 con 0x7fd6a411b140 2024-09-16T00:20:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.929+0000 7fd6ab118640 1 -- 172.21.15.78:0/1665901226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a41145e0 con 0x7fd6a411b140 2024-09-16T00:20:50.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.929+0000 7fd6ab118640 1 -- 172.21.15.78:0/1665901226 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a41184a0 con 0x7fd6a411b140 2024-09-16T00:20:50.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.933+0000 7fd6ae753740 1 Processor -- start 2024-09-16T00:20:50.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.938+0000 7fd6ae753740 1 -- start start 2024-09-16T00:20:50.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.939+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.939+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa58bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.939+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa63550 0x558a2fa65940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.939+0000 7fd6ae753740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558a2f9be2f0 con 0x558a2fa59d20 2024-09-16T00:20:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.939+0000 7fd6ae753740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558a2f965cf0 con 0x558a2fa587b0 2024-09-16T00:20:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.939+0000 7fd6ae753740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a2f9ba2b0 con 0x558a2fa63550 2024-09-16T00:20:50.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.939+0000 7fd69b7fe640 1 -- 172.21.15.78:0/1665901226 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd69c043020 con 0x7fd6a411b140 2024-09-16T00:20:50.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6a9915640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6a9915640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa58270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39958/0 (socket says 172.21.15.78:39958) 2024-09-16T00:20:50.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6a9915640 1 -- 172.21.15.78:0/38459481 learned_addr learned my addr 172.21.15.78:0/38459481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:50.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6aa116640 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa58bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6a9915640 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa63550 0x558a2fa65940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6aa116640 1 -- 172.21.15.78:0/38459481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa63550 msgr2=0x558a2fa65940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:50.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6aa116640 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa63550 0x558a2fa65940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6aa116640 1 -- 172.21.15.78:0/38459481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 msgr2=0x558a2fa58270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6aa116640 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa58270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6aa116640 1 -- 172.21.15.78:0/38459481 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2f94a3a0 con 0x558a2fa587b0 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.940+0000 7fd6aa116640 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa58bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0004600 tx=0x7fd6a000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e829d735053c1499 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.941+0000 7fd69b7fe640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6900a3410 0x7fd6900a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.941+0000 7fd69b7fe640 1 -- 172.21.15.78:0/1665901226 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd69c0eff30 con 0x7fd6a411b140 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.941+0000 7fd698dea640 1 -- 172.21.15.78:0/38459481 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0003da0 con 0x558a2fa587b0 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.941+0000 7fd698dea640 1 -- 172.21.15.78:0/38459481 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a0015070 con 0x558a2fa587b0 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.942+0000 7fd6ae753740 1 -- 172.21.15.78:0/38459481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 msgr2=0x558a2fa58bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.942+0000 7fd6ae753740 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa58bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0004600 tx=0x7fd6a000bb20 comp rx=0 tx=0).stop 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.942+0000 7fd6ae753740 1 -- 172.21.15.78:0/38459481 shutdown_connections 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.942+0000 7fd6ae753740 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa63550 0x558a2fa65940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.942+0000 7fd6ae753740 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa58bf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.942+0000 7fd6ae753740 1 --2- 172.21.15.78:0/38459481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa58270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.942+0000 7fd6ae753740 1 -- 172.21.15.78:0/38459481 >> 172.21.15.78:0/38459481 conn(0x558a2fa21310 msgr2=0x558a2fa54e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.943+0000 7fd6a9915640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6900a3410 0x7fd6900a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.943+0000 7fd6ae753740 1 -- 172.21.15.78:0/38459481 shutdown_connections 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.943+0000 7fd6ae753740 1 -- 172.21.15.78:0/38459481 wait complete. 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.943+0000 7fd6ae753740 1 -- start start 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.943+0000 7fd6ae753740 1 Processor -- start 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa62e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 0x558a2fa60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa63550 0x558a2fa60660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6ae753740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558a2fa662a0 con 0x558a2fa63550 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6ae753740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558a2f979060 con 0x558a2fa587b0 2024-09-16T00:20:50.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6ae753740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a2fa66420 con 0x558a2fa59d20 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa116640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa62e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa116640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa62e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34814/0 (socket says 172.21.15.78:34814) 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa116640 1 -- 172.21.15.78:0/214859553 learned_addr learned my addr 172.21.15.78:0/214859553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa917640 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa63550 0x558a2fa60660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6ae753740 10 client.0 _ll_register_callbacks cb 0x558a2f6cc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa116640 1 -- 172.21.15.78:0/214859553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 msgr2=0x558a2fa60120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa116640 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 0x558a2fa60120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa116640 1 -- 172.21.15.78:0/214859553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa63550 msgr2=0x558a2fa60660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa116640 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa63550 0x558a2fa60660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.944+0000 7fd6aa116640 1 -- 172.21.15.78:0/214859553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2f72f3c0 con 0x558a2fa587b0 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.945+0000 7fd6ae753740 1 -- 172.21.15.78:0/214859553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a0015070 con 0x558a2fa587b0 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.945+0000 7fd6aa116640 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa62e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0003dd0 tx=0x7fd6a0003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ed0a1a56b0e5c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:50.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.946+0000 7fd68dffb640 10 client.26111 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:50.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.946+0000 7fd68dffb640 1 -- 172.21.15.78:0/214859553 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0005240 con 0x558a2fa587b0 2024-09-16T00:20:50.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.946+0000 7fd68dffb640 1 -- 172.21.15.78:0/214859553 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd6a00053e0 con 0x558a2fa587b0 2024-09-16T00:20:50.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.946+0000 7fd68dffb640 1 -- 172.21.15.78:0/214859553 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd6a00118e0 con 0x558a2fa587b0 2024-09-16T00:20:50.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.946+0000 7fd6ae753740 10 client.26111 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:50.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.946+0000 7fd6ae753740 1 -- 172.21.15.78:0/214859553 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=34}) -- 0x558a2fa93b90 con 0x558a2fa587b0 2024-09-16T00:20:50.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.947+0000 7fd6a9915640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6900a3410 0x7fd6900a58d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x558a2fa59250 tx=0x7fd69400a1f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:50.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.950+0000 7fd68dffb640 1 -- 172.21.15.78:0/214859553 <== mon.2 v2:172.21.15.148:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd6a0011a80 con 0x558a2fa587b0 2024-09-16T00:20:50.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.950+0000 7fd6ae753740 10 client.26111 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:50.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.950+0000 7fd6ae753740 10 client.26111 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:20:50.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.950+0000 7fd6ae753740 20 client.26111 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:50.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.950+0000 7fd6ae753740 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558a2fa95260 0x558a2fa97700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.950+0000 7fd6ae753740 4 client.26111 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:50.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.950+0000 7fd6ae753740 1 -- 172.21.15.78:0/214859553 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a2f9cb860 con 0x558a2fa95260 2024-09-16T00:20:50.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.952+0000 7fd6a9915640 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558a2fa95260 0x558a2fa97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.958+0000 7fd6a9915640 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558a2fa95260 0x558a2fa97700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8bdef581861696f3 server_cookie=ca48733c8beca4f8 in_seq=0 out_seq=0 2024-09-16T00:20:50.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.958+0000 7fd68dffb640 10 client.26111 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:20:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.958+0000 7fd68dffb640 1 -- 172.21.15.78:0/214859553 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a2f9cb860 con 0x558a2fa95260 2024-09-16T00:20:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.958+0000 7fd68dffb640 10 client.26111 handle_command_reply: tid=0 2024-09-16T00:20:50.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 client.26111 shutdown 2024-09-16T00:20:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 -- 172.21.15.78:0/214859553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 msgr2=0x558a2fa62e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa62e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0003dd0 tx=0x7fd6a0003e00 comp rx=0 tx=0).stop 2024-09-16T00:20:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 -- 172.21.15.78:0/214859553 shutdown_connections 2024-09-16T00:20:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa63550 0x558a2fa60660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558a2fa95260 0x558a2fa97700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 0x558a2fa60120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 --2- 172.21.15.78:0/214859553 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa587b0 0x558a2fa62e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd6ae753740 1 -- 172.21.15.78:0/214859553 >> 172.21.15.78:0/214859553 conn(0x558a2fa21310 msgr2=0x558a2fa5ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.959+0000 7fd68dffb640 0 client.26111 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:20:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.960+0000 7fd6ae753740 1 -- 172.21.15.78:0/214859553 shutdown_connections 2024-09-16T00:20:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.960+0000 7fd6ae753740 1 -- 172.21.15.78:0/214859553 wait complete. 2024-09-16T00:20:50.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.960+0000 7fd6ae753740 20 client.26111 trim_cache size 0 max 16384 2024-09-16T00:20:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.976+0000 7fd6ae753740 1 Processor -- start 2024-09-16T00:20:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.976+0000 7fd6ae753740 1 -- start start 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa587b0 0x558a2fa97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 0x558a2fa64940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa116640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa587b0 0x558a2fa97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa116640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa587b0 0x558a2fa97270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39978/0 (socket says 172.21.15.78:39978) 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 0x558a2fa64e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6ae753740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a412a4b0 con 0x558a2fa587b0 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6ae753740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a412bff0 con 0x558a2fa63550 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6ae753740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a412ab80 con 0x558a2fa59d20 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa116640 1 -- 172.21.15.78:0/558421294 learned_addr learned my addr 172.21.15.78:0/558421294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6a9915640 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 0x558a2fa64940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa917640 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 0x558a2fa64e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa116640 1 -- 172.21.15.78:0/558421294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 msgr2=0x558a2fa64940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa116640 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 0x558a2fa64940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa116640 1 -- 172.21.15.78:0/558421294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 msgr2=0x558a2fa64e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa116640 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 0x558a2fa64e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.977+0000 7fd6aa116640 1 -- 172.21.15.78:0/558421294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2fa55e50 con 0x558a2fa587b0 2024-09-16T00:20:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.978+0000 7fd6a9915640 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 0x558a2fa64940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.978+0000 7fd6aa116640 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa587b0 0x558a2fa97270 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0005c80 tx=0x7fd6a0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52ae0a1ef5b2f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.978+0000 7fd68ffff640 1 -- 172.21.15.78:0/558421294 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004120 con 0x558a2fa587b0 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.978+0000 7fd68ffff640 1 -- 172.21.15.78:0/558421294 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a0015070 con 0x558a2fa587b0 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.978+0000 7fd68ffff640 1 -- 172.21.15.78:0/558421294 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00045d0 con 0x558a2fa587b0 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 -- 172.21.15.78:0/558421294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa587b0 msgr2=0x558a2fa97270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa587b0 0x558a2fa97270 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0005c80 tx=0x7fd6a0021000 comp rx=0 tx=0).stop 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 -- 172.21.15.78:0/558421294 shutdown_connections 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 0x558a2fa64e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa59d20 0x558a2fa64940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 --2- 172.21.15.78:0/558421294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa587b0 0x558a2fa97270 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 -- 172.21.15.78:0/558421294 >> 172.21.15.78:0/558421294 conn(0x558a2fa21310 msgr2=0x558a2fa5e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 -- 172.21.15.78:0/558421294 shutdown_connections 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 -- 172.21.15.78:0/558421294 wait complete. 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 -- start start 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.979+0000 7fd6ae753740 1 Processor -- start 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa587b0 0x558a2fa9c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa9c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6ae753740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 0x558a2fa9cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6ae753740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a412b920 con 0x558a2fa59d20 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6ae753740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a412b250 con 0x558a2fa63550 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6ae753740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a406ee70 con 0x558a2fa587b0 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6ae753740 10 client.0 _ll_register_callbacks cb 0x7fd6a0004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6a9915640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa9c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6a9915640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa9c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39988/0 (socket says 172.21.15.78:39988) 2024-09-16T00:20:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6a9915640 1 -- 172.21.15.78:0/991662502 learned_addr learned my addr 172.21.15.78:0/991662502 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6aa116640 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa587b0 0x558a2fa9c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6aa917640 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 0x558a2fa9cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6aa116640 1 -- 172.21.15.78:0/991662502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 msgr2=0x558a2fa9cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6aa116640 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 0x558a2fa9cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6aa116640 1 -- 172.21.15.78:0/991662502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 msgr2=0x558a2fa9c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.980+0000 7fd6aa116640 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa9c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.981+0000 7fd6aa116640 1 -- 172.21.15.78:0/991662502 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2fa8ca90 con 0x558a2fa587b0 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.981+0000 7fd6a9915640 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa9c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.981+0000 7fd6aa116640 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa587b0 0x558a2fa9c060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0000c00 tx=0x7fd6a001a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe3a89ef718dfcb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.981+0000 7fd68dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.981+0000 7fd68dffb640 1 -- 172.21.15.78:0/991662502 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004120 con 0x558a2fa587b0 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.981+0000 7fd68dffb640 1 -- 172.21.15.78:0/991662502 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd6a0022070 con 0x558a2fa587b0 2024-09-16T00:20:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.982+0000 7fd6ae753740 1 -- 172.21.15.78:0/991662502 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a0015070 con 0x558a2fa587b0 2024-09-16T00:20:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.982+0000 7fd68dffb640 1 -- 172.21.15.78:0/991662502 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd6a001ac50 con 0x558a2fa587b0 2024-09-16T00:20:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.982+0000 7fd6ae753740 10 client.25930 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.982+0000 7fd6ae753740 1 -- 172.21.15.78:0/991662502 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558a2fa8dd50 con 0x558a2fa587b0 2024-09-16T00:20:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd68dffb640 1 -- 172.21.15.78:0/991662502 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a003e020 con 0x558a2fa587b0 2024-09-16T00:20:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd68dffb640 1 -- 172.21.15.78:0/991662502 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd6a0030420 con 0x558a2fa587b0 2024-09-16T00:20:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd6ae753740 10 client.25930 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd6ae753740 10 client.25930 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:20:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd6ae753740 20 client.25930 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd6ae753740 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558a2fafffe0 0x558a2fb023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd6ae753740 4 client.25930 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd6ae753740 1 -- 172.21.15.78:0/991662502 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd6a0015070 con 0x558a2fafffe0 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.983+0000 7fd6a9915640 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558a2fafffe0 0x558a2fb023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.984+0000 7fd68dffb640 10 client.25930 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.984+0000 7fd6a9915640 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558a2fafffe0 0x558a2fb023d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=24b036af46ec0484 server_cookie=b184cfaf46ad9a83 in_seq=0 out_seq=0 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.984+0000 7fd68dffb640 1 -- 172.21.15.78:0/991662502 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd6a0015070 con 0x558a2fafffe0 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.984+0000 7fd68dffb640 10 client.25930 handle_command_reply: tid=0 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.984+0000 7fd6ae753740 1 client.25930 shutdown 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.984+0000 7fd6ae753740 1 -- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa587b0 msgr2=0x558a2fa9c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.984+0000 7fd6ae753740 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa587b0 0x558a2fa9c060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0000c00 tx=0x7fd6a001a8e0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 1 -- 172.21.15.78:0/991662502 shutdown_connections 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a2fa63550 0x558a2fa9cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558a2fa59d20 0x558a2fa9c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558a2fafffe0 0x558a2fb023d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 1 --2- 172.21.15.78:0/991662502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a2fa587b0 0x558a2fa9c060 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd68dffb640 0 client.25930 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:20:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 1 -- 172.21.15.78:0/991662502 >> 172.21.15.78:0/991662502 conn(0x558a2fa21310 msgr2=0x558a2fa53780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 1 -- 172.21.15.78:0/991662502 shutdown_connections 2024-09-16T00:20:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 1 -- 172.21.15.78:0/991662502 wait complete. 2024-09-16T00:20:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.985+0000 7fd6ae753740 20 client.25930 trim_cache size 0 max 16384 2024-09-16T00:20:50.991 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:20:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 -- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6900a3410 msgr2=0x7fd6900a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6900a3410 0x7fd6900a58d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x558a2fa59250 tx=0x7fd69400a1f0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 -- 172.21.15.78:0/1665901226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6a411b140 msgr2=0x7fd6a4113d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6a411b140 0x7fd6a4113d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd69c02ac30 tx=0x7fd69c030620 comp rx=0 tx=0).stop 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 -- 172.21.15.78:0/1665901226 shutdown_connections 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd6900a3410 0x7fd6900a58d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd6a411b140 0x7fd6a4113d60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4110190 0x7fd6a4113820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 --2- 172.21.15.78:0/1665901226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6a406ea30 0x7fd6a411a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 -- 172.21.15.78:0/1665901226 >> 172.21.15.78:0/1665901226 conn(0x7fd6a4073180 msgr2=0x7fd6a411b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 -- 172.21.15.78:0/1665901226 shutdown_connections 2024-09-16T00:20:50.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:50.986+0000 7fd6ab118640 1 -- 172.21.15.78:0/1665901226 wait complete. 2024-09-16T00:20:51.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:51 smithi078 ceph-mon[31157]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 MiB/s wr, 644 op/s 2024-09-16T00:20:51.869 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:20:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:51 smithi193 ceph-mon[35484]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 MiB/s wr, 644 op/s 2024-09-16T00:20:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:51 smithi148 ceph-mon[35599]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 MiB/s wr, 644 op/s 2024-09-16T00:20:52.309 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:54.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:53 smithi078 ceph-mon[31157]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 591 op/s 2024-09-16T00:20:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:53 smithi193 ceph-mon[35484]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 591 op/s 2024-09-16T00:20:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:53 smithi148 ceph-mon[35599]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.8 MiB/s wr, 591 op/s 2024-09-16T00:20:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.595+0000 7f3c11950640 1 -- 172.21.15.78:0/2939517447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c0c10e090 msgr2=0x7f3c0c10e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.595+0000 7f3c11950640 1 --2- 172.21.15.78:0/2939517447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c0c10e090 0x7f3c0c10e470 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc008570 tx=0x7f3bfc00ac00 comp rx=0 tx=0).stop 2024-09-16T00:20:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.614+0000 7f3c11950640 1 -- 172.21.15.78:0/2939517447 shutdown_connections 2024-09-16T00:20:54.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.614+0000 7f3c11950640 1 --2- 172.21.15.78:0/2939517447 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c0c06ee70 0x7f3c0c07a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.614+0000 7f3c11950640 1 --2- 172.21.15.78:0/2939517447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c0c06e4f0 0x7f3c0c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.614+0000 7f3c11950640 1 --2- 172.21.15.78:0/2939517447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c0c10e090 0x7f3c0c10e470 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.614+0000 7f3c11950640 1 -- 172.21.15.78:0/2939517447 >> 172.21.15.78:0/2939517447 conn(0x7f3c0c073180 msgr2=0x7f3c0c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.620+0000 7f3c11950640 1 -- 172.21.15.78:0/2939517447 shutdown_connections 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.626+0000 7f3c11950640 1 -- 172.21.15.78:0/2939517447 wait complete. 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.626+0000 7f3c11950640 1 Processor -- start 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.628+0000 7f3c11950640 1 -- start start 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.630+0000 7f3c11950640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c0c06e4f0 0x7f3c0c084ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.630+0000 7f3c11950640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c0c06ee70 0x7f3c0c07ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.630+0000 7f3c11950640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c0c10e090 0x7f3c0c07f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.630+0000 7f3c11950640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c07db30 con 0x7f3c0c10e090 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.630+0000 7f3c11950640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c07d9b0 con 0x7f3c0c06e4f0 2024-09-16T00:20:54.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.630+0000 7f3c11950640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c07dcb0 con 0x7f3c0c06ee70 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.632+0000 7f3c0bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c0c06ee70 0x7f3c0c07ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.632+0000 7f3c0bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c0c06ee70 0x7f3c0c07ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:54680/0 (socket says 172.21.15.78:54680) 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.632+0000 7f3c0bfff640 1 -- 172.21.15.78:0/2897211541 learned_addr learned my addr 172.21.15.78:0/2897211541 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.634+0000 7f3c1094e640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c0c06e4f0 0x7f3c0c084ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.636+0000 7f3c0bfff640 1 -- 172.21.15.78:0/2897211541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c0c06e4f0 msgr2=0x7f3c0c084ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.636+0000 7f3c0bfff640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c0c06e4f0 0x7f3c0c084ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.636+0000 7f3c0bfff640 1 -- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c0c10e090 msgr2=0x7f3c0c07f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.636+0000 7f3c0bfff640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c0c10e090 0x7f3c0c07f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.636+0000 7f3c0bfff640 1 -- 172.21.15.78:0/2897211541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c1bbb80 con 0x7f3c0c06ee70 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.636+0000 7f3c0bfff640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c0c06ee70 0x7f3c0c07ef60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000b450 tx=0x7f3c0000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.638+0000 7f3c09ffb640 1 -- 172.21.15.78:0/2897211541 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00015020 con 0x7f3c0c06ee70 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.638+0000 7f3c09ffb640 1 -- 172.21.15.78:0/2897211541 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c00011d70 con 0x7f3c0c06ee70 2024-09-16T00:20:54.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.638+0000 7f3c09ffb640 1 -- 172.21.15.78:0/2897211541 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000f340 con 0x7f3c0c06ee70 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.640+0000 7f3c11950640 1 -- 172.21.15.78:0/2897211541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c1bbe10 con 0x7f3c0c06ee70 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.640+0000 7f3c11950640 1 -- 172.21.15.78:0/2897211541 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c1bc2d0 con 0x7f3c0c06ee70 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.646+0000 7f3c14f8b740 1 Processor -- start 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.647+0000 7f3c14f8b740 1 -- start start 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.647+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59a0d20 0x55e5a599f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.647+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a599fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.647+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59ac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.647+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5a59052f0 con 0x55e5a59aa550 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.647+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e5a58accf0 con 0x55e5a59a0d20 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.647+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e5a59012b0 con 0x55e5a599f7b0 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.648+0000 7f3c1114f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59a0d20 0x55e5a599f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.648+0000 7f3c1114f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59a0d20 0x55e5a599f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34866/0 (socket says 172.21.15.78:34866) 2024-09-16T00:20:54.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.648+0000 7f3c1114f640 1 -- 172.21.15.78:0/3409500287 learned_addr learned my addr 172.21.15.78:0/3409500287 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.649+0000 7f3c0bfff640 1 --2- 172.21.15.78:0/3409500287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.649+0000 7f3c1114f640 1 -- 172.21.15.78:0/3409500287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 msgr2=0x55e5a599fbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.649+0000 7f3c1114f640 1 --2- 172.21.15.78:0/3409500287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a599fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.649+0000 7f3c1114f640 1 -- 172.21.15.78:0/3409500287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 msgr2=0x55e5a59ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.649+0000 7f3c1114f640 1 --2- 172.21.15.78:0/3409500287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.649+0000 7f3c1114f640 1 -- 172.21.15.78:0/3409500287 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a58913a0 con 0x55e5a59a0d20 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.649+0000 7f3c09ffb640 1 -- 172.21.15.78:0/2897211541 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3c00018020 con 0x7f3c0c06ee70 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.650+0000 7f3c09ffb640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3bf40a3450 0x7f3bf40a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.651+0000 7f3c09ffb640 1 -- 172.21.15.78:0/2897211541 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c000cf590 con 0x7f3c0c06ee70 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.652+0000 7f3c1114f640 1 --2- 172.21.15.78:0/3409500287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59a0d20 0x55e5a599f270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400bf90 tx=0x7f3c040095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb600750cdc1ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:54.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.652+0000 7f3bf2ffd640 1 -- 172.21.15.78:0/3409500287 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04018030 con 0x55e5a59a0d20 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.652+0000 7f3bf2ffd640 1 -- 172.21.15.78:0/3409500287 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c04009d30 con 0x55e5a59a0d20 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.652+0000 7f3c1094e640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3bf40a3450 0x7f3bf40a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.653+0000 7f3c14f8b740 1 -- 172.21.15.78:0/3409500287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59a0d20 msgr2=0x55e5a599f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.653+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/3409500287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59a0d20 0x55e5a599f270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400bf90 tx=0x7f3c040095b0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.654+0000 7f3c14f8b740 1 -- 172.21.15.78:0/3409500287 shutdown_connections 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.654+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/3409500287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.654+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/3409500287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a599fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.654+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/3409500287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59a0d20 0x55e5a599f270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.654+0000 7f3c14f8b740 1 -- 172.21.15.78:0/3409500287 >> 172.21.15.78:0/3409500287 conn(0x55e5a5968310 msgr2=0x55e5a599be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:54.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.661+0000 7f3c1094e640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3bf40a3450 0x7f3bf40a5910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55e5a599cca0 tx=0x7f3bfc0082c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:54.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.661+0000 7f3c14f8b740 1 -- 172.21.15.78:0/3409500287 shutdown_connections 2024-09-16T00:20:54.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.661+0000 7f3c14f8b740 1 -- 172.21.15.78:0/3409500287 wait complete. 2024-09-16T00:20:54.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.662+0000 7f3c14f8b740 1 -- start start 2024-09-16T00:20:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.662+0000 7f3c14f8b740 1 Processor -- start 2024-09-16T00:20:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.662+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a59a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e5a59ad2f0 con 0x55e5a59a0d20 2024-09-16T00:20:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e5a58c0060 con 0x55e5a59aa550 2024-09-16T00:20:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e5a59ad470 con 0x55e5a599f7b0 2024-09-16T00:20:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c14f8b740 10 client.0 _ll_register_callbacks cb 0x55e5a56138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c0bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59a70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40020/0 (socket says 172.21.15.78:40020) 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c0bfff640 1 -- 172.21.15.78:0/1322586418 learned_addr learned my addr 172.21.15.78:0/1322586418 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.663+0000 7f3c1114f640 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a59a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.664+0000 7f3c1114f640 1 -- 172.21.15.78:0/1322586418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 msgr2=0x55e5a59a9e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.664+0000 7f3c1114f640 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59a9e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.664+0000 7f3c1114f640 1 -- 172.21.15.78:0/1322586418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 msgr2=0x55e5a59a70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.664+0000 7f3c1114f640 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59a70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.664+0000 7f3c1114f640 1 -- 172.21.15.78:0/1322586418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a56763c0 con 0x55e5a59aa550 2024-09-16T00:20:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.664+0000 7f3c0bfff640 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59a70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:20:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.665+0000 7f3bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.665+0000 7f3c1114f640 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a59a75f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c04000f80 tx=0x7f3c04005290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31789fcb86a4095e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.665+0000 7f3bebfff640 1 -- 172.21.15.78:0/1322586418 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04003b70 con 0x55e5a59aa550 2024-09-16T00:20:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.665+0000 7f3bebfff640 1 -- 172.21.15.78:0/1322586418 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c04003970 con 0x55e5a59aa550 2024-09-16T00:20:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.665+0000 7f3c14f8b740 1 -- 172.21.15.78:0/1322586418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c04009d30 con 0x55e5a59aa550 2024-09-16T00:20:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.667+0000 7f3bebfff640 1 -- 172.21.15.78:0/1322586418 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3c04003b70 con 0x55e5a59aa550 2024-09-16T00:20:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.667+0000 7f3c14f8b740 10 client.26129 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.667+0000 7f3c14f8b740 1 -- 172.21.15.78:0/1322586418 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e5a59b8cd0 con 0x55e5a59aa550 2024-09-16T00:20:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.668+0000 7f3bebfff640 1 -- 172.21.15.78:0/1322586418 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c04009d30 con 0x55e5a59aa550 2024-09-16T00:20:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.668+0000 7f3bebfff640 1 -- 172.21.15.78:0/1322586418 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3c04004050 con 0x55e5a59aa550 2024-09-16T00:20:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.668+0000 7f3c14f8b740 10 client.26129 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.668+0000 7f3c14f8b740 10 client.26129 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:20:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.668+0000 7f3c14f8b740 20 client.26129 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.668+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e5a59ba0d0 0x55e5a59bc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.668+0000 7f3c14f8b740 4 client.26129 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.668+0000 7f3c14f8b740 1 -- 172.21.15.78:0/1322586418 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5a5912860 con 0x55e5a59ba0d0 2024-09-16T00:20:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.669+0000 7f3c1094e640 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e5a59ba0d0 0x55e5a59bc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.670+0000 7f3c1094e640 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e5a59ba0d0 0x55e5a59bc4c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=29c05ab8800750f5 server_cookie=fba96f91f315ca7b in_seq=0 out_seq=0 2024-09-16T00:20:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.670+0000 7f3bebfff640 10 client.26129 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:20:54.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.671+0000 7f3bebfff640 1 -- 172.21.15.78:0/1322586418 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5a5912860 con 0x55e5a59ba0d0 2024-09-16T00:20:54.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.671+0000 7f3bebfff640 10 client.26129 handle_command_reply: tid=0 2024-09-16T00:20:54.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.671+0000 7f3c14f8b740 1 client.26129 shutdown 2024-09-16T00:20:54.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3c14f8b740 1 -- 172.21.15.78:0/1322586418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 msgr2=0x55e5a59a75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a59a75f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c04000f80 tx=0x7f3c04005290 comp rx=0 tx=0).stop 2024-09-16T00:20:54.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3c14f8b740 1 -- 172.21.15.78:0/1322586418 shutdown_connections 2024-09-16T00:20:54.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e5a59ba0d0 0x55e5a59bc4c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a59a75f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59a70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/1322586418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3c14f8b740 1 -- 172.21.15.78:0/1322586418 >> 172.21.15.78:0/1322586418 conn(0x55e5a5968310 msgr2=0x55e5a59a1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:54.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.672+0000 7f3bebfff640 0 client.26129 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:20:54.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.673+0000 7f3c14f8b740 1 -- 172.21.15.78:0/1322586418 shutdown_connections 2024-09-16T00:20:54.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.673+0000 7f3c14f8b740 1 -- 172.21.15.78:0/1322586418 wait complete. 2024-09-16T00:20:54.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.673+0000 7f3c14f8b740 20 client.26129 trim_cache size 0 max 16384 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.697+0000 7f3c14f8b740 1 Processor -- start 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.697+0000 7f3c14f8b740 1 -- start start 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.697+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a599f7b0 0x55e5a59a5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.697+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a59a0d20 0x55e5a59a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.697+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.697+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1d73a0 con 0x55e5a59aa550 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.697+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1d8ee0 con 0x55e5a599f7b0 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.697+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1d7a70 con 0x55e5a59a0d20 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1114f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1114f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59bf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40034/0 (socket says 172.21.15.78:40034) 2024-09-16T00:20:54.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1114f640 1 -- 172.21.15.78:0/4230613569 learned_addr learned my addr 172.21.15.78:0/4230613569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1094e640 1 --2- 172.21.15.78:0/4230613569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a599f7b0 0x55e5a59a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1114f640 1 -- 172.21.15.78:0/4230613569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a59a0d20 msgr2=0x55e5a59a2300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1114f640 1 --2- 172.21.15.78:0/4230613569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a59a0d20 0x55e5a59a2300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1114f640 1 -- 172.21.15.78:0/4230613569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a599f7b0 msgr2=0x55e5a59a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1114f640 1 --2- 172.21.15.78:0/4230613569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a599f7b0 0x55e5a59a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.698+0000 7f3c1114f640 1 -- 172.21.15.78:0/4230613569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a59b6ff0 con 0x55e5a59aa550 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.699+0000 7f3c1114f640 1 --2- 172.21.15.78:0/4230613569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59bf4e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3c040095f0 tx=0x7f3c04004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f105ea8db50a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.699+0000 7f3bf27fc640 1 -- 172.21.15.78:0/4230613569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04018040 con 0x55e5a59aa550 2024-09-16T00:20:54.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.699+0000 7f3bf27fc640 1 -- 172.21.15.78:0/4230613569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c04021070 con 0x55e5a59aa550 2024-09-16T00:20:54.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.700+0000 7f3c14f8b740 1 -- 172.21.15.78:0/4230613569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 msgr2=0x55e5a59bf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.700+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/4230613569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59bf4e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3c040095f0 tx=0x7f3c04004640 comp rx=0 tx=0).stop 2024-09-16T00:20:54.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.700+0000 7f3c14f8b740 1 -- 172.21.15.78:0/4230613569 shutdown_connections 2024-09-16T00:20:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.700+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/4230613569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59aa550 0x55e5a59bf4e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.700+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/4230613569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a59a0d20 0x55e5a59a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.700+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/4230613569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a599f7b0 0x55e5a59a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.700+0000 7f3c14f8b740 1 -- 172.21.15.78:0/4230613569 >> 172.21.15.78:0/4230613569 conn(0x55e5a5968310 msgr2=0x55e5a59a9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.701+0000 7f3c14f8b740 1 -- 172.21.15.78:0/4230613569 shutdown_connections 2024-09-16T00:20:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.701+0000 7f3c14f8b740 1 -- 172.21.15.78:0/4230613569 wait complete. 2024-09-16T00:20:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 1 -- start start 2024-09-16T00:20:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 1 Processor -- start 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a5a22a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1d8810 con 0x55e5a59a0d20 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1d8140 con 0x55e5a59aa550 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c1d5320 con 0x55e5a599f7b0 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.702+0000 7f3c14f8b740 10 client.0 _ll_register_callbacks cb 0x7f3c0400be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1094e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1094e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59bca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:54698/0 (socket says 172.21.15.78:54698) 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1094e640 1 -- 172.21.15.78:0/2397162635 learned_addr learned my addr 172.21.15.78:0/2397162635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1114f640 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a5a22a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c0bfff640 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1094e640 1 -- 172.21.15.78:0/2397162635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 msgr2=0x55e5a5a22a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1094e640 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a5a22a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1094e640 1 -- 172.21.15.78:0/2397162635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 msgr2=0x55e5a59bcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1094e640 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59bcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.703+0000 7f3c1094e640 1 -- 172.21.15.78:0/2397162635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a5a22fc0 con 0x55e5a599f7b0 2024-09-16T00:20:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.704+0000 7f3c1094e640 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59bca80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc02a580 tx=0x7f3bfc00ac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df06281e9e52eb38 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.704+0000 7f3bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:20:54.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.705+0000 7f3bebfff640 1 -- 172.21.15.78:0/2397162635 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc043030 con 0x55e5a599f7b0 2024-09-16T00:20:54.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.705+0000 7f3bebfff640 1 -- 172.21.15.78:0/2397162635 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3bfc00a520 con 0x55e5a599f7b0 2024-09-16T00:20:54.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.705+0000 7f3c14f8b740 1 -- 172.21.15.78:0/2397162635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c04021070 con 0x55e5a599f7b0 2024-09-16T00:20:54.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.705+0000 7f3bebfff640 1 -- 172.21.15.78:0/2397162635 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3bfc00a720 con 0x55e5a599f7b0 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.705+0000 7f3c14f8b740 10 client.25936 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.705+0000 7f3c14f8b740 1 -- 172.21.15.78:0/2397162635 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e5a5a24320 con 0x55e5a599f7b0 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3bebfff640 1 -- 172.21.15.78:0/2397162635 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c04021070 con 0x55e5a599f7b0 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3bebfff640 1 -- 172.21.15.78:0/2397162635 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3bfc051020 con 0x55e5a599f7b0 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3c14f8b740 10 client.25936 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3c14f8b740 10 client.25936 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3c14f8b740 20 client.25936 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e5a5a249f0 0x55e5a5a26e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3c14f8b740 4 client.25936 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3c14f8b740 1 -- 172.21.15.78:0/2397162635 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5a590cd70 con 0x55e5a5a249f0 2024-09-16T00:20:54.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.706+0000 7f3c0bfff640 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e5a5a249f0 0x55e5a5a26e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:54.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.707+0000 7f3c0bfff640 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e5a5a249f0 0x55e5a5a26e10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=35e488fd0629cab6 server_cookie=fd0e152adaeaf523 in_seq=0 out_seq=0 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.707+0000 7f3bebfff640 10 client.25936 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.707+0000 7f3bebfff640 1 -- 172.21.15.78:0/2397162635 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5a590cd70 con 0x55e5a5a249f0 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.707+0000 7f3bebfff640 10 client.25936 handle_command_reply: tid=0 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.707+0000 7f3c14f8b740 1 client.25936 shutdown 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 -- 172.21.15.78:0/2397162635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 msgr2=0x55e5a59bca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59bca80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc02a580 tx=0x7f3bfc00ac00 comp rx=0 tx=0).stop 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 -- 172.21.15.78:0/2397162635 shutdown_connections 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e5a5a249f0 0x55e5a5a26e10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e5a59aa550 0x55e5a5a22a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e5a59a0d20 0x55e5a59bcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 --2- 172.21.15.78:0/2397162635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e5a599f7b0 0x55e5a59bca80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 -- 172.21.15.78:0/2397162635 >> 172.21.15.78:0/2397162635 conn(0x55e5a5968310 msgr2=0x55e5a599a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:54.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3bebfff640 0 client.25936 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:20:54.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 -- 172.21.15.78:0/2397162635 shutdown_connections 2024-09-16T00:20:54.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 1 -- 172.21.15.78:0/2397162635 wait complete. 2024-09-16T00:20:54.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.708+0000 7f3c14f8b740 20 client.25936 trim_cache size 0 max 16384 2024-09-16T00:20:54.712 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:20:54.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 -- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3bf40a3450 msgr2=0x7f3bf40a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3bf40a3450 0x7f3bf40a5910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55e5a599cca0 tx=0x7f3bfc0082c0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 -- 172.21.15.78:0/2897211541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c0c06ee70 msgr2=0x7f3c0c07ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c0c06ee70 0x7f3c0c07ef60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000b450 tx=0x7f3c0000b910 comp rx=0 tx=0).stop 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 -- 172.21.15.78:0/2897211541 shutdown_connections 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3bf40a3450 0x7f3bf40a5910 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c0c10e090 0x7f3c0c07f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c0c06ee70 0x7f3c0c07ef60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 --2- 172.21.15.78:0/2897211541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c0c06e4f0 0x7f3c0c084ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.709+0000 7f3c11950640 1 -- 172.21.15.78:0/2897211541 >> 172.21.15.78:0/2897211541 conn(0x7f3c0c073180 msgr2=0x7f3c0c06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.710+0000 7f3c11950640 1 -- 172.21.15.78:0/2897211541 shutdown_connections 2024-09-16T00:20:54.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:54.710+0000 7f3c11950640 1 -- 172.21.15.78:0/2897211541 wait complete. 2024-09-16T00:20:55.458 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:20:55.861 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 415.0 s, 730.2 tps, lat 139.998 ms stddev 176.982 2024-09-16T00:20:55.906 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:55 smithi078 ceph-mon[31157]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-09-16T00:20:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:55 smithi193 ceph-mon[35484]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-09-16T00:20:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:55 smithi148 ceph-mon[35599]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 MiB/s wr, 552 op/s 2024-09-16T00:20:56.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 -- 172.21.15.78:0/2332579835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 msgr2=0x7f7db01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 --2- 172.21.15.78:0/2332579835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 0x7f7db01021d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009900 tx=0x7f7dac02a410 comp rx=0 tx=0).stop 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 -- 172.21.15.78:0/2332579835 shutdown_connections 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 --2- 172.21.15.78:0/2332579835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7db0106f20 0x7f7db010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 --2- 172.21.15.78:0/2332579835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db01027a0 0x7f7db01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 --2- 172.21.15.78:0/2332579835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 0x7f7db01021d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 -- 172.21.15.78:0/2332579835 >> 172.21.15.78:0/2332579835 conn(0x7f7db00fd8f0 msgr2=0x7f7db00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 -- 172.21.15.78:0/2332579835 shutdown_connections 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.565+0000 7f7db748b640 1 -- 172.21.15.78:0/2332579835 wait complete. 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.566+0000 7f7db748b640 1 Processor -- start 2024-09-16T00:20:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.566+0000 7f7db748b640 1 -- start start 2024-09-16T00:20:56.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.566+0000 7f7db748b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 0x7f7db00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.566+0000 7f7db748b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7db01027a0 0x7f7db0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.566+0000 7f7db748b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db0106f20 0x7f7db0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.566+0000 7f7db748b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0111fd0 con 0x7f7db0101df0 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.566+0000 7f7db748b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0111e50 con 0x7f7db0106f20 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.566+0000 7f7db748b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0112150 con 0x7f7db01027a0 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 0x7f7db00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 0x7f7db00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51340/0 (socket says 172.21.15.78:51340) 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 -- 172.21.15.78:0/3181658494 learned_addr learned my addr 172.21.15.78:0/3181658494 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5a28640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db0106f20 0x7f7db0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db4a26640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7db01027a0 0x7f7db0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 -- 172.21.15.78:0/3181658494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7db01027a0 msgr2=0x7f7db0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7db01027a0 0x7f7db0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 -- 172.21.15.78:0/3181658494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db0106f20 msgr2=0x7f7db0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db0106f20 0x7f7db0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 -- 172.21.15.78:0/3181658494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db01a68f0 con 0x7f7db0101df0 2024-09-16T00:20:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.567+0000 7f7db5227640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 0x7f7db00744f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7dac004770 tx=0x7f7dac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:56.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.569+0000 7f7da67fc640 1 -- 172.21.15.78:0/3181658494 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac03f070 con 0x7f7db0101df0 2024-09-16T00:20:56.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.569+0000 7f7db748b640 1 -- 172.21.15.78:0/3181658494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db01a6b80 con 0x7f7db0101df0 2024-09-16T00:20:56.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.569+0000 7f7da67fc640 1 -- 172.21.15.78:0/3181658494 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dac031ca0 con 0x7f7db0101df0 2024-09-16T00:20:56.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.569+0000 7f7db748b640 1 -- 172.21.15.78:0/3181658494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db01a70b0 con 0x7f7db0101df0 2024-09-16T00:20:56.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.569+0000 7f7da67fc640 1 -- 172.21.15.78:0/3181658494 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac035030 con 0x7f7db0101df0 2024-09-16T00:20:56.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.572+0000 7f7da67fc640 1 -- 172.21.15.78:0/3181658494 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7dac042070 con 0x7f7db0101df0 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.572+0000 7f7db9864740 1 Processor -- start 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db9864740 1 -- start start 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db9864740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420cb40 0x55687420b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db9864740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420b5d0 0x55687420ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7da67fc640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7d9c0a32c0 0x7f7d9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db9864740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874218760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db9864740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5568741712f0 con 0x55687420b5d0 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db9864740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556874118cf0 con 0x55687420cb40 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db9864740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55687416d2b0 con 0x556874216370 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db5a28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420cb40 0x55687420b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db5a28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420cb40 0x55687420b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:47364/0 (socket says 172.21.15.78:47364) 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db5a28640 1 -- 172.21.15.78:0/2606022128 learned_addr learned my addr 172.21.15.78:0/2606022128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db4a26640 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874218760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db5227640 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420b5d0 0x55687420ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db5a28640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7d9c0a32c0 0x7f7d9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db5a28640 1 -- 172.21.15.78:0/2606022128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 msgr2=0x556874218760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.573+0000 7f7db5a28640 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874218760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7db5a28640 1 -- 172.21.15.78:0/2606022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420b5d0 msgr2=0x55687420ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7db5a28640 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420b5d0 0x55687420ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7db5a28640 1 -- 172.21.15.78:0/2606022128 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568740fd3a0 con 0x55687420cb40 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7db4a26640 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874218760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7db5227640 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420b5d0 0x55687420ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:20:56.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7da67fc640 1 -- 172.21.15.78:0/3181658494 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7dac0f01b0 con 0x7f7db0101df0 2024-09-16T00:20:56.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7db5a28640 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420cb40 0x55687420b090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7da80146e0 tx=0x7f7da8014ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67171ce4a0b601e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:56.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7d9b7fe640 1 -- 172.21.15.78:0/2606022128 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8023070 con 0x55687420cb40 2024-09-16T00:20:56.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7d9b7fe640 1 -- 172.21.15.78:0/2606022128 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da801cdc0 con 0x55687420cb40 2024-09-16T00:20:56.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.574+0000 7f7d9b7fe640 1 -- 172.21.15.78:0/2606022128 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da80056b0 con 0x55687420cb40 2024-09-16T00:20:56.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 -- 172.21.15.78:0/2606022128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420cb40 msgr2=0x55687420b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db5a28640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7d9c0a32c0 0x7f7d9c0a5780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7da8020000 tx=0x7f7da801c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:56.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420cb40 0x55687420b090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7da80146e0 tx=0x7f7da8014ba0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 -- 172.21.15.78:0/2606022128 shutdown_connections 2024-09-16T00:20:56.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874218760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420b5d0 0x55687420ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 --2- 172.21.15.78:0/2606022128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420cb40 0x55687420b090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 -- 172.21.15.78:0/2606022128 >> 172.21.15.78:0/2606022128 conn(0x5568741d4310 msgr2=0x556874207c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 -- 172.21.15.78:0/2606022128 shutdown_connections 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.575+0000 7f7db9864740 1 -- 172.21.15.78:0/2606022128 wait complete. 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.576+0000 7f7db9864740 1 -- start start 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.576+0000 7f7db9864740 1 Processor -- start 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db9864740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874215c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db9864740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x556874212f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db5227640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874215c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db5227640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874215c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:47366/0 (socket says 172.21.15.78:47366) 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db4a26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x556874212f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db4a26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x556874212f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51364/0 (socket says 172.21.15.78:51364) 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db5227640 1 -- 172.21.15.78:0/2402546514 learned_addr learned my addr 172.21.15.78:0/2402546514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db9864740 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874213490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556874219070 con 0x55687420cb40 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55687412c060 con 0x55687420b5d0 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.577+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5568742191f0 con 0x556874216370 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db5a28640 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874213490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db5227640 1 -- 172.21.15.78:0/2402546514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 msgr2=0x556874213490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db5227640 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874213490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db5227640 1 -- 172.21.15.78:0/2402546514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 msgr2=0x556874212f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db5227640 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x556874212f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db5227640 1 -- 172.21.15.78:0/2402546514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556873ee23c0 con 0x55687420b5d0 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db5227640 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874215c50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7dac031550 tx=0x7f7dac08c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f58ef71f23de4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7d98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db9864740 10 client.0 _ll_register_callbacks cb 0x556873e7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da801cdc0 con 0x55687420b5d0 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2402546514 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac049040 con 0x55687420b5d0 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2402546514 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7dac0bc780 con 0x55687420b5d0 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2402546514 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7dac0bc2f0 con 0x55687420b5d0 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db9864740 10 client.26144 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.578+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=34}) -- 0x556874224630 con 0x55687420b5d0 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2402546514 <== mon.2 v2:172.21.15.148:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7dac03b460 con 0x55687420b5d0 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7db9864740 10 client.26144 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7db9864740 10 client.26144 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7db9864740 20 client.26144 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7db9864740 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x556874225c90 0x556874228150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7db9864740 4 client.26144 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:56.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55687417e860 con 0x556874225c90 2024-09-16T00:20:56.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7db4a26640 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x556874225c90 0x556874228150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7db4a26640 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x556874225c90 0x556874228150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=affe35c1f50091cb server_cookie=62840ce00cc4060c in_seq=0 out_seq=0 2024-09-16T00:20:56.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.579+0000 7f7d98ff9640 10 client.26144 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:20:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.580+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2402546514 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55687417e860 con 0x556874225c90 2024-09-16T00:20:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.580+0000 7f7d98ff9640 10 client.26144 handle_command_reply: tid=0 2024-09-16T00:20:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.580+0000 7f7db9864740 1 client.26144 shutdown 2024-09-16T00:20:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 msgr2=0x556874215c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874215c50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7dac031550 tx=0x7f7dac08c480 comp rx=0 tx=0).stop 2024-09-16T00:20:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 shutdown_connections 2024-09-16T00:20:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556874216370 0x556874213490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x556874212f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x556874225c90 0x556874228150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7d98ff9640 0 client.26144 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:20:56.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 --2- 172.21.15.78:0/2402546514 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874215c50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 >> 172.21.15.78:0/2402546514 conn(0x5568741d4310 msgr2=0x55687420d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:56.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.582+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 shutdown_connections 2024-09-16T00:20:56.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.583+0000 7f7db9864740 1 -- 172.21.15.78:0/2402546514 wait complete. 2024-09-16T00:20:56.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.583+0000 7f7db9864740 20 client.26144 trim_cache size 0 max 16384 2024-09-16T00:20:56.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db9864740 1 Processor -- start 2024-09-16T00:20:56.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db9864740 1 -- start start 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db9864740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874206bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db9864740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420cb40 0x55687422da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db9864740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556874216370 0x55687422dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db5227640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874206bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db5227640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874206bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:47372/0 (socket says 172.21.15.78:47372) 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db4a26640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420cb40 0x55687422da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db4a26640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420cb40 0x55687422da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:38688/0 (socket says 172.21.15.78:38688) 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db5227640 1 -- 172.21.15.78:0/1095832244 learned_addr learned my addr 172.21.15.78:0/1095832244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db5a28640 1 --2- 172.21.15.78:0/1095832244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556874216370 0x55687422dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.605+0000 7f7db9864740 1 -- 172.21.15.78:0/1095832244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9400f9c0 con 0x556874216370 2024-09-16T00:20:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.606+0000 7f7db9864740 1 -- 172.21.15.78:0/1095832244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94011500 con 0x55687420b5d0 2024-09-16T00:20:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.606+0000 7f7db9864740 1 -- 172.21.15.78:0/1095832244 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94010090 con 0x55687420cb40 2024-09-16T00:20:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.606+0000 7f7db5227640 1 -- 172.21.15.78:0/1095832244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420cb40 msgr2=0x55687422da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.606+0000 7f7db5227640 1 --2- 172.21.15.78:0/1095832244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420cb40 0x55687422da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.606+0000 7f7db5227640 1 -- 172.21.15.78:0/1095832244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556874216370 msgr2=0x55687422dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.606+0000 7f7db5227640 1 --2- 172.21.15.78:0/1095832244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556874216370 0x55687422dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.606+0000 7f7db5227640 1 -- 172.21.15.78:0/1095832244 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556874208960 con 0x55687420b5d0 2024-09-16T00:20:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.606+0000 7f7db5227640 1 --2- 172.21.15.78:0/1095832244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874206bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7dac000c00 tx=0x7f7dac0f2070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77b8ec583c9c31c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.607+0000 7f7d9affd640 1 -- 172.21.15.78:0/1095832244 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac049040 con 0x55687420b5d0 2024-09-16T00:20:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.607+0000 7f7d9affd640 1 -- 172.21.15.78:0/1095832244 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da801cdc0 con 0x55687420b5d0 2024-09-16T00:20:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.607+0000 7f7d9affd640 1 -- 172.21.15.78:0/1095832244 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0bbe80 con 0x55687420b5d0 2024-09-16T00:20:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.608+0000 7f7db9864740 1 -- 172.21.15.78:0/1095832244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 msgr2=0x556874206bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.608+0000 7f7db9864740 1 --2- 172.21.15.78:0/1095832244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874206bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7dac000c00 tx=0x7f7dac0f2070 comp rx=0 tx=0).stop 2024-09-16T00:20:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.612+0000 7f7db9864740 1 -- 172.21.15.78:0/1095832244 shutdown_connections 2024-09-16T00:20:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.612+0000 7f7db9864740 1 --2- 172.21.15.78:0/1095832244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556874216370 0x55687422dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.612+0000 7f7db9864740 1 --2- 172.21.15.78:0/1095832244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420cb40 0x55687422da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.612+0000 7f7db9864740 1 --2- 172.21.15.78:0/1095832244 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55687420b5d0 0x556874206bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.612+0000 7f7db9864740 1 -- 172.21.15.78:0/1095832244 >> 172.21.15.78:0/1095832244 conn(0x5568741d4310 msgr2=0x5568742064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:56.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.616+0000 7f7db9864740 1 -- 172.21.15.78:0/1095832244 shutdown_connections 2024-09-16T00:20:56.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.616+0000 7f7db9864740 1 -- 172.21.15.78:0/1095832244 wait complete. 2024-09-16T00:20:56.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.616+0000 7f7db9864740 1 -- start start 2024-09-16T00:20:56.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.616+0000 7f7db9864740 1 Processor -- start 2024-09-16T00:20:56.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db9864740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420b5d0 0x55687420ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db9864740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x5568742104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db9864740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556874216370 0x5568742109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db5227640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420b5d0 0x55687420ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db5227640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420b5d0 0x55687420ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:38694/0 (socket says 172.21.15.78:38694) 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db4a26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x5568742104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db4a26640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x5568742104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51388/0 (socket says 172.21.15.78:51388) 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db5227640 1 -- 172.21.15.78:0/2766207241 learned_addr learned my addr 172.21.15.78:0/2766207241 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94010e30 con 0x55687420cb40 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94010760 con 0x556874216370 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9400d9a0 con 0x55687420b5d0 2024-09-16T00:20:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.620+0000 7f7db9864740 10 client.0 _ll_register_callbacks cb 0x7f7dac005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.621+0000 7f7db5a28640 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556874216370 0x5568742109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.624+0000 7f7db5a28640 1 -- 172.21.15.78:0/2766207241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420b5d0 msgr2=0x55687420ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.624+0000 7f7db5a28640 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420b5d0 0x55687420ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.624+0000 7f7db5a28640 1 -- 172.21.15.78:0/2766207241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 msgr2=0x5568742104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.624+0000 7f7db5a28640 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x5568742104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.624+0000 7f7db5a28640 1 -- 172.21.15.78:0/2766207241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556874288a20 con 0x556874216370 2024-09-16T00:20:56.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.625+0000 7f7db5227640 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420b5d0 0x55687420ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:56.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.625+0000 7f7db4a26640 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x5568742104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:56.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.628+0000 7f7db5a28640 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556874216370 0x5568742109e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009640 tx=0x7f7da8009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=980fdf2893caa2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:56.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.629+0000 7f7d98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:20:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.633+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2766207241 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8033020 con 0x556874216370 2024-09-16T00:20:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.633+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2766207241 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7da801c8e0 con 0x556874216370 2024-09-16T00:20:56.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.634+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da801cdc0 con 0x556874216370 2024-09-16T00:20:56.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.638+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2766207241 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7da8023040 con 0x556874216370 2024-09-16T00:20:56.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.638+0000 7f7db9864740 10 client.26150 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:56.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.638+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556874289ce0 con 0x556874216370 2024-09-16T00:20:56.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.645+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2766207241 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da801cae0 con 0x556874216370 2024-09-16T00:20:56.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.645+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2766207241 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7da800f040 con 0x556874216370 2024-09-16T00:20:56.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.648+0000 7f7db9864740 10 client.26150 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.648+0000 7f7db9864740 10 client.26150 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:20:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.648+0000 7f7db9864740 20 client.26150 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.648+0000 7f7db9864740 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x556874225c90 0x55687428c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.648+0000 7f7db9864740 4 client.26150 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:20:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.648+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7da801cdc0 con 0x556874225c90 2024-09-16T00:20:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.648+0000 7f7db5227640 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x556874225c90 0x55687428c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.649+0000 7f7db5227640 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x556874225c90 0x55687428c1d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91611a673696b770 server_cookie=aaf10176bd6ccc49 in_seq=0 out_seq=0 2024-09-16T00:20:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.649+0000 7f7d98ff9640 10 client.26150 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:20:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.649+0000 7f7d98ff9640 1 -- 172.21.15.78:0/2766207241 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7da801cdc0 con 0x556874225c90 2024-09-16T00:20:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.649+0000 7f7d98ff9640 10 client.26150 handle_command_reply: tid=0 2024-09-16T00:20:56.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.654+0000 7f7db9864740 1 client.26150 shutdown 2024-09-16T00:20:56.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.655+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556874216370 msgr2=0x5568742109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.655+0000 7f7db9864740 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556874216370 0x5568742109e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009640 tx=0x7f7da8009a60 comp rx=0 tx=0).stop 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 shutdown_connections 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x556874225c90 0x55687428c1d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556874216370 0x5568742109e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55687420cb40 0x5568742104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 1 --2- 172.21.15.78:0/2766207241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55687420b5d0 0x55687420ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 >> 172.21.15.78:0/2766207241 conn(0x5568741d4310 msgr2=0x55687420d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7d98ff9640 0 client.26150 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 shutdown_connections 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 1 -- 172.21.15.78:0/2766207241 wait complete. 2024-09-16T00:20:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.657+0000 7f7db9864740 20 client.26150 trim_cache size 0 max 16384 2024-09-16T00:20:56.660 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:20:56.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.660+0000 7f7db748b640 1 -- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7d9c0a32c0 msgr2=0x7f7d9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.660+0000 7f7db748b640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7d9c0a32c0 0x7f7d9c0a5780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7da8020000 tx=0x7f7da801c000 comp rx=0 tx=0).stop 2024-09-16T00:20:56.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.660+0000 7f7db748b640 1 -- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 msgr2=0x7f7db00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:56.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.660+0000 7f7db748b640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 0x7f7db00744f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7dac004770 tx=0x7f7dac003c10 comp rx=0 tx=0).stop 2024-09-16T00:20:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.664+0000 7f7db748b640 1 -- 172.21.15.78:0/3181658494 shutdown_connections 2024-09-16T00:20:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.664+0000 7f7db748b640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7d9c0a32c0 0x7f7d9c0a5780 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.664+0000 7f7db748b640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7db0106f20 0x7f7db0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.664+0000 7f7db748b640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7db01027a0 0x7f7db0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.664+0000 7f7db748b640 1 --2- 172.21.15.78:0/3181658494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db0101df0 0x7f7db00744f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.664+0000 7f7db748b640 1 -- 172.21.15.78:0/3181658494 >> 172.21.15.78:0/3181658494 conn(0x7f7db00fd8f0 msgr2=0x7f7db00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:56.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.665+0000 7f7db748b640 1 -- 172.21.15.78:0/3181658494 shutdown_connections 2024-09-16T00:20:56.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:56.665+0000 7f7db748b640 1 -- 172.21.15.78:0/3181658494 wait complete. 2024-09-16T00:20:57.526 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:20:58.015 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:57 smithi078 ceph-mon[31157]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 561 op/s 2024-09-16T00:20:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:57 smithi193 ceph-mon[35484]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 561 op/s 2024-09-16T00:20:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:57 smithi148 ceph-mon[35599]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 MiB/s wr, 561 op/s 2024-09-16T00:20:58.940 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.7 2024-09-16T00:20:58.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:20:59.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.453+0000 7f05a559a640 1 -- 172.21.15.78:0/2139597437 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590030ac0 con 0x7f05a810ddc0 2024-09-16T00:20:59.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.454+0000 7f05acae3640 1 -- 172.21.15.78:0/2139597437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 msgr2=0x7f05a8115940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.454+0000 7f05acae3640 1 --2- 172.21.15.78:0/2139597437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 0x7f05a8115940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f059000aa10 tx=0x7f059002b720 comp rx=0 tx=0).stop 2024-09-16T00:20:59.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.454+0000 7f05acae3640 1 -- 172.21.15.78:0/2139597437 shutdown_connections 2024-09-16T00:20:59.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.454+0000 7f05acae3640 1 --2- 172.21.15.78:0/2139597437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 0x7f05a8115940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.454+0000 7f05acae3640 1 --2- 172.21.15.78:0/2139597437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05a8109e00 0x7f05a810d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.454+0000 7f05acae3640 1 --2- 172.21.15.78:0/2139597437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05a8109450 0x7f05a8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.454+0000 7f05acae3640 1 -- 172.21.15.78:0/2139597437 >> 172.21.15.78:0/2139597437 conn(0x7f05a806e140 msgr2=0x7f05a806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:59.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.455+0000 7f05acae3640 1 -- 172.21.15.78:0/2139597437 shutdown_connections 2024-09-16T00:20:59.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.455+0000 7f05acae3640 1 -- 172.21.15.78:0/2139597437 wait complete. 2024-09-16T00:20:59.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.455+0000 7f05acae3640 1 Processor -- start 2024-09-16T00:20:59.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05acae3640 1 -- start start 2024-09-16T00:20:59.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05acae3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05a8109450 0x7f05a8119960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05acae3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05a8109e00 0x7f05a8119ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05acae3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 0x7f05a811a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05acae3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81180a0 con 0x7f05a8109e00 2024-09-16T00:20:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05acae3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8117f20 con 0x7f05a810ddc0 2024-09-16T00:20:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05acae3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05a8118220 con 0x7f05a8109450 2024-09-16T00:20:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05a6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 0x7f05a811a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05a6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 0x7f05a811a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:47404/0 (socket says 172.21.15.78:47404) 2024-09-16T00:20:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05a6d9d640 1 -- 172.21.15.78:0/420904836 learned_addr learned my addr 172.21.15.78:0/420904836 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:59.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.456+0000 7f05a5d9b640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05a8109e00 0x7f05a8119ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.457+0000 7f05a659c640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05a8109450 0x7f05a8119960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.457+0000 7f05a5d9b640 1 -- 172.21.15.78:0/420904836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05a8109450 msgr2=0x7f05a8119960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.457+0000 7f05a5d9b640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05a8109450 0x7f05a8119960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.457+0000 7f05a5d9b640 1 -- 172.21.15.78:0/420904836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 msgr2=0x7f05a811a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.457+0000 7f05a5d9b640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 0x7f05a811a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.457+0000 7f05a5d9b640 1 -- 172.21.15.78:0/420904836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a81b34a0 con 0x7f05a8109e00 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.457+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 0x7f05a811a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.457+0000 7f05a5d9b640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05a8109e00 0x7f05a8119ea0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f059800eb00 tx=0x7f059800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.458+0000 7f059f7fe640 1 -- 172.21.15.78:0/420904836 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598010430 con 0x7f05a8109e00 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.458+0000 7f059f7fe640 1 -- 172.21.15.78:0/420904836 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0598010ab0 con 0x7f05a8109e00 2024-09-16T00:20:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.458+0000 7f059f7fe640 1 -- 172.21.15.78:0/420904836 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598017bb0 con 0x7f05a8109e00 2024-09-16T00:20:59.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.459+0000 7f05acae3640 1 -- 172.21.15.78:0/420904836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a81b3790 con 0x7f05a8109e00 2024-09-16T00:20:59.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.459+0000 7f05acae3640 1 -- 172.21.15.78:0/420904836 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a81b3c50 con 0x7f05a8109e00 2024-09-16T00:20:59.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.463+0000 7f059f7fe640 1 -- 172.21.15.78:0/420904836 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0598010c60 con 0x7f05a8109e00 2024-09-16T00:20:59.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.463+0000 7f05aeebc740 1 Processor -- start 2024-09-16T00:20:59.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05aeebc740 1 -- start start 2024-09-16T00:20:59.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f56b40 0x565269f55090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f55a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f60370 0x565269f62760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05aeebc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565269ebb2f0 con 0x565269f555d0 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05aeebc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x565269e62cf0 con 0x565269f60370 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05aeebc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565269eb72b0 con 0x565269f56b40 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05a6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f56b40 0x565269f55090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05a6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f56b40 0x565269f55090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:38716/0 (socket says 172.21.15.78:38716) 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05a6d9d640 1 -- 172.21.15.78:0/1871266644 learned_addr learned my addr 172.21.15.78:0/1871266644 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.464+0000 7f05a659c640 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f55a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f05a5d9b640 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f60370 0x565269f62760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f05a6d9d640 1 -- 172.21.15.78:0/1871266644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f60370 msgr2=0x565269f62760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f60370 0x565269f62760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f05a6d9d640 1 -- 172.21.15.78:0/1871266644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 msgr2=0x565269f55a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f55a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f05a6d9d640 1 -- 172.21.15.78:0/1871266644 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565269e473a0 con 0x565269f56b40 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f059f7fe640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05940a3410 0x7f05940a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f56b40 0x565269f55090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0590005e30 tx=0x7f0590005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39098e19607addad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f059cff9640 1 -- 172.21.15.78:0/1871266644 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590041030 con 0x565269f56b40 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f059cff9640 1 -- 172.21.15.78:0/1871266644 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0590030ba0 con 0x565269f56b40 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f059cff9640 1 -- 172.21.15.78:0/1871266644 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590036040 con 0x565269f56b40 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f059f7fe640 1 -- 172.21.15.78:0/420904836 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f05980d04b0 con 0x7f05a8109e00 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.465+0000 7f05a659c640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05940a3410 0x7f05940a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.466+0000 7f05a659c640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05940a3410 0x7f05940a58d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x565269f52ac0 tx=0x7f05a000d500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.466+0000 7f05aeebc740 1 -- 172.21.15.78:0/1871266644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f56b40 msgr2=0x565269f55090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.467+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f56b40 0x565269f55090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0590005e30 tx=0x7f0590005e60 comp rx=0 tx=0).stop 2024-09-16T00:20:59.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.469+0000 7f05aeebc740 1 -- 172.21.15.78:0/1871266644 shutdown_connections 2024-09-16T00:20:59.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.469+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f60370 0x565269f62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.469+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f55a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.469+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1871266644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f56b40 0x565269f55090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.469+0000 7f05aeebc740 1 -- 172.21.15.78:0/1871266644 >> 172.21.15.78:0/1871266644 conn(0x565269f1e310 msgr2=0x565269f51c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:59.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.469+0000 7f05aeebc740 1 -- 172.21.15.78:0/1871266644 shutdown_connections 2024-09-16T00:20:59.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.469+0000 7f05aeebc740 1 -- 172.21.15.78:0/1871266644 wait complete. 2024-09-16T00:20:59.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.470+0000 7f05aeebc740 1 -- start start 2024-09-16T00:20:59.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.470+0000 7f05aeebc740 1 Processor -- start 2024-09-16T00:20:59.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.470+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f5fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f5d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05a659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f5fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05a659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f5fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51428/0 (socket says 172.21.15.78:51428) 2024-09-16T00:20:59.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05a659c640 1 -- 172.21.15.78:0/2546105091 learned_addr learned my addr 172.21.15.78:0/2546105091 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:59.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05aeebc740 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x565269f63010 con 0x565269f555d0 2024-09-16T00:20:59.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05a5d9b640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f5d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x565269e76060 con 0x565269f56b40 2024-09-16T00:20:59.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565269f63190 con 0x565269f60370 2024-09-16T00:20:59.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05aeebc740 10 client.0 _ll_register_callbacks cb 0x565269bc98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:59.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05a6d9d640 1 -- 172.21.15.78:0/2546105091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 msgr2=0x565269f5d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:59.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.471+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f5d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.472+0000 7f05a6d9d640 1 -- 172.21.15.78:0/2546105091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 msgr2=0x565269f5fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:59.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.472+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f5fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.472+0000 7f05a6d9d640 1 -- 172.21.15.78:0/2546105091 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565269c2c3c0 con 0x565269f60370 2024-09-16T00:20:59.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.472+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5d5d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0590005780 tx=0x7f0590005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f995bab243d5213f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:59.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.472+0000 7f058dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:20:59.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.472+0000 7f05a659c640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f5fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:59.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.472+0000 7f05a5d9b640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f5d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:20:59.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.473+0000 7f058dffb640 1 -- 172.21.15.78:0/2546105091 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590036030 con 0x565269f60370 2024-09-16T00:20:59.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.473+0000 7f058dffb640 1 -- 172.21.15.78:0/2546105091 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0590003d60 con 0x565269f60370 2024-09-16T00:20:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.473+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0590030ba0 con 0x565269f60370 2024-09-16T00:20:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.473+0000 7f058dffb640 1 -- 172.21.15.78:0/2546105091 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0590041040 con 0x565269f60370 2024-09-16T00:20:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.473+0000 7f05aeebc740 10 client.25963 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.473+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565269f63060 con 0x565269f60370 2024-09-16T00:20:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.473+0000 7f058dffb640 1 -- 172.21.15.78:0/2546105091 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0590030ba0 con 0x565269f60370 2024-09-16T00:20:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.473+0000 7f058dffb640 1 -- 172.21.15.78:0/2546105091 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f059003b040 con 0x565269f60370 2024-09-16T00:20:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.474+0000 7f05aeebc740 10 client.25963 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:59.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.474+0000 7f05aeebc740 10 client.25963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:20:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.474+0000 7f05aeebc740 20 client.25963 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.474+0000 7f05aeebc740 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x565269f6fbf0 0x565269f71fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.474+0000 7f05aeebc740 4 client.25963 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:20:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.474+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565269ec8860 con 0x565269f6fbf0 2024-09-16T00:20:59.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.474+0000 7f05a659c640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x565269f6fbf0 0x565269f71fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.477+0000 7f058dffb640 10 client.25963 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:20:59.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.477+0000 7f05a659c640 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x565269f6fbf0 0x565269f71fe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a27b354803fc90f server_cookie=758cf4231bae3bf0 in_seq=0 out_seq=0 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f058dffb640 1 -- 172.21.15.78:0/2546105091 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565269ec8860 con 0x565269f6fbf0 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f058dffb640 10 client.25963 handle_command_reply: tid=0 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 client.25963 shutdown 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 msgr2=0x565269f5d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5d5d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0590005780 tx=0x7f0590005a20 comp rx=0 tx=0).stop 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 shutdown_connections 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x565269f6fbf0 0x565269f71fe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f058dffb640 0 client.25963 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5d5d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f5d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 --2- 172.21.15.78:0/2546105091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f555d0 0x565269f5fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 >> 172.21.15.78:0/2546105091 conn(0x565269f1e310 msgr2=0x565269f57960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:59.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 shutdown_connections 2024-09-16T00:20:59.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 1 -- 172.21.15.78:0/2546105091 wait complete. 2024-09-16T00:20:59.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.478+0000 7f05aeebc740 20 client.25963 trim_cache size 0 max 16384 2024-09-16T00:20:59.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.502+0000 7f05aeebc740 1 Processor -- start 2024-09-16T00:20:59.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.502+0000 7f05aeebc740 1 -- start start 2024-09-16T00:20:59.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.503+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f555d0 0x565269f51d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.503+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f6c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.503+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f60370 0x565269f5bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.503+0000 7f05aeebc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81bda00 con 0x565269f60370 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.503+0000 7f05aeebc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81bf540 con 0x565269f56b40 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.503+0000 7f05aeebc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81be0d0 con 0x565269f555d0 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.504+0000 7f05a659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f555d0 0x565269f51d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.505+0000 7f05a659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f555d0 0x565269f51d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:38724/0 (socket says 172.21.15.78:38724) 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.505+0000 7f05a659c640 1 -- 172.21.15.78:0/249663776 learned_addr learned my addr 172.21.15.78:0/249663776 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.505+0000 7f05a5d9b640 1 --2- 172.21.15.78:0/249663776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f6c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.505+0000 7f05a659c640 1 -- 172.21.15.78:0/249663776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 msgr2=0x565269f6c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.505+0000 7f05a659c640 1 --2- 172.21.15.78:0/249663776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f6c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.505+0000 7f05a659c640 1 -- 172.21.15.78:0/249663776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f60370 msgr2=0x565269f5bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.505+0000 7f05a659c640 1 --2- 172.21.15.78:0/249663776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f60370 0x565269f5bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.505+0000 7f05a659c640 1 -- 172.21.15.78:0/249663776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565269f60080 con 0x565269f555d0 2024-09-16T00:20:59.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.515+0000 7f05a659c640 1 --2- 172.21.15.78:0/249663776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f555d0 0x565269f51d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05a0005fd0 tx=0x7f05a0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd04907a9943d5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:59.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.519+0000 7f058ffff640 1 -- 172.21.15.78:0/249663776 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0005020 con 0x565269f555d0 2024-09-16T00:20:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.519+0000 7f058ffff640 1 -- 172.21.15.78:0/249663776 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a00047a0 con 0x565269f555d0 2024-09-16T00:20:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.519+0000 7f058ffff640 1 -- 172.21.15.78:0/249663776 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a001c6b0 con 0x565269f555d0 2024-09-16T00:20:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.520+0000 7f05aeebc740 1 -- 172.21.15.78:0/249663776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f555d0 msgr2=0x565269f51d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.520+0000 7f05aeebc740 1 --2- 172.21.15.78:0/249663776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f555d0 0x565269f51d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05a0005fd0 tx=0x7f05a0015000 comp rx=0 tx=0).stop 2024-09-16T00:20:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.530+0000 7f05aeebc740 1 -- 172.21.15.78:0/249663776 shutdown_connections 2024-09-16T00:20:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.530+0000 7f05aeebc740 1 --2- 172.21.15.78:0/249663776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f60370 0x565269f5bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.530+0000 7f05aeebc740 1 --2- 172.21.15.78:0/249663776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f56b40 0x565269f6c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.530+0000 7f05aeebc740 1 --2- 172.21.15.78:0/249663776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f555d0 0x565269f51d70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.530+0000 7f05aeebc740 1 -- 172.21.15.78:0/249663776 >> 172.21.15.78:0/249663776 conn(0x565269f1e310 msgr2=0x565269f5b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:59.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.536+0000 7f05aeebc740 1 -- 172.21.15.78:0/249663776 shutdown_connections 2024-09-16T00:20:59.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.539+0000 7f05aeebc740 1 -- 172.21.15.78:0/249663776 wait complete. 2024-09-16T00:20:59.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.539+0000 7f05aeebc740 1 -- start start 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.540+0000 7f05aeebc740 1 Processor -- start 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.540+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f555d0 0x565269f68bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.540+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f56b40 0x565269f59e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.540+0000 7f05aeebc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.540+0000 7f05aeebc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81bee70 con 0x565269f56b40 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.541+0000 7f05aeebc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81be7a0 con 0x565269f555d0 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.541+0000 7f05a6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.541+0000 7f05a6d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:38734/0 (socket says 172.21.15.78:38734) 2024-09-16T00:20:59.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.541+0000 7f05a6d9d640 1 -- 172.21.15.78:0/1273994804 learned_addr learned my addr 172.21.15.78:0/1273994804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:20:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.541+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05a81bb9e0 con 0x565269f60370 2024-09-16T00:20:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.541+0000 7f05aeebc740 10 client.0 _ll_register_callbacks cb 0x7f05a000d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:20:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.545+0000 7f05a6d9d640 1 -- 172.21.15.78:0/1273994804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f555d0 msgr2=0x565269f68bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.545+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f555d0 0x565269f68bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.545+0000 7f05a6d9d640 1 -- 172.21.15.78:0/1273994804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f56b40 msgr2=0x565269f59e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:20:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.545+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f56b40 0x565269f59e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.545+0000 7f05a6d9d640 1 -- 172.21.15.78:0/1273994804 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565269f5abd0 con 0x565269f60370 2024-09-16T00:20:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.552+0000 7f05a6d9d640 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5a3a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f059002bce0 tx=0x7f059003d6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f0ae2ef6b833f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:20:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.552+0000 7f058dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:20:59.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.556+0000 7f058dffb640 1 -- 172.21.15.78:0/1273994804 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590036040 con 0x565269f60370 2024-09-16T00:20:59.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.556+0000 7f058dffb640 1 -- 172.21.15.78:0/1273994804 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0590003a50 con 0x565269f60370 2024-09-16T00:20:59.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.556+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05a00047a0 con 0x565269f60370 2024-09-16T00:20:59.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.559+0000 7f058dffb640 1 -- 172.21.15.78:0/1273994804 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0590005d50 con 0x565269f60370 2024-09-16T00:20:59.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.563+0000 7f05aeebc740 10 client.25969 fetch_fsmap learned FSMap version 34 2024-09-16T00:20:59.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.563+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565269fda7c0 con 0x565269f60370 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f058dffb640 1 -- 172.21.15.78:0/1273994804 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05900043f0 con 0x565269f60370 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f058dffb640 1 -- 172.21.15.78:0/1273994804 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f059000b040 con 0x565269f60370 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f05aeebc740 10 client.25969 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f05aeebc740 10 client.25969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f05aeebc740 20 client.25969 populate_metadata read hostname 'smithi078' 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x565269fdaf10 0x565269fdd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f05aeebc740 4 client.25969 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f05a00047a0 con 0x565269fdaf10 2024-09-16T00:20:59.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.567+0000 7f05a659c640 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x565269fdaf10 0x565269fdd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:20:59.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.570+0000 7f05a659c640 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x565269fdaf10 0x565269fdd350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d00243cdc95d6967 server_cookie=48250b25cf6ab9f0 in_seq=0 out_seq=0 2024-09-16T00:20:59.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.571+0000 7f058dffb640 10 client.25969 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:20:59.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.571+0000 7f058dffb640 1 -- 172.21.15.78:0/1273994804 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f05a00047a0 con 0x565269fdaf10 2024-09-16T00:20:59.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.571+0000 7f058dffb640 10 client.25969 handle_command_reply: tid=0 2024-09-16T00:20:59.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.571+0000 7f05aeebc740 1 client.25969 shutdown 2024-09-16T00:20:59.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.581+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 msgr2=0x565269f5a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.581+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5a3a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f059002bce0 tx=0x7f059003d6a0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.582+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 shutdown_connections 2024-09-16T00:20:59.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.582+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x565269fdaf10 0x565269fdd350 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.582+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565269f60370 0x565269f5a3a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.582+0000 7f058dffb640 0 client.25969 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:20:59.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.582+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x565269f56b40 0x565269f59e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.583+0000 7f05aeebc740 1 --2- 172.21.15.78:0/1273994804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x565269f555d0 0x565269f68bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.583+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 >> 172.21.15.78:0/1273994804 conn(0x565269f1e310 msgr2=0x565269f5e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:59.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.583+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 shutdown_connections 2024-09-16T00:20:59.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.583+0000 7f05aeebc740 1 -- 172.21.15.78:0/1273994804 wait complete. 2024-09-16T00:20:59.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.583+0000 7f05aeebc740 20 client.25969 trim_cache size 0 max 16384 2024-09-16T00:20:59.585 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:20:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 -- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05940a3410 msgr2=0x7f05940a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05940a3410 0x7f05940a58d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x565269f52ac0 tx=0x7f05a000d500 comp rx=0 tx=0).stop 2024-09-16T00:20:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 -- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05a8109e00 msgr2=0x7f05a8119ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:20:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05a8109e00 0x7f05a8119ea0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f059800eb00 tx=0x7f059800c2d0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 -- 172.21.15.78:0/420904836 shutdown_connections 2024-09-16T00:20:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05940a3410 0x7f05940a58d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a810ddc0 0x7f05a811a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05a8109e00 0x7f05a8119ea0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 --2- 172.21.15.78:0/420904836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05a8109450 0x7f05a8119960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:20:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.584+0000 7f05acae3640 1 -- 172.21.15.78:0/420904836 >> 172.21.15.78:0/420904836 conn(0x7f05a806e140 msgr2=0x7f05a8070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:20:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.585+0000 7f05acae3640 1 -- 172.21.15.78:0/420904836 shutdown_connections 2024-09-16T00:20:59.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:20:59.585+0000 7f05acae3640 1 -- 172.21.15.78:0/420904836 wait complete. 2024-09-16T00:20:59.841 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:20:59.968 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:20:59 smithi078 ceph-mon[31157]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2024-09-16T00:21:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:20:59 smithi193 ceph-mon[35484]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2024-09-16T00:21:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:20:59 smithi148 ceph-mon[35599]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2024-09-16T00:21:00.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:21:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 420.0 s, 870.6 tps, lat 113.106 ms stddev 130.176 2024-09-16T00:21:01.920 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:01 smithi078 ceph-mon[31157]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 632 op/s 2024-09-16T00:21:01.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:01 smithi148 ceph-mon[35599]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 632 op/s 2024-09-16T00:21:02.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 -- 172.21.15.78:0/882753456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80101440 msgr2=0x7f9e8010df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 --2- 172.21.15.78:0/882753456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80101440 0x7f9e8010df30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009980 tx=0x7f9e6803b420 comp rx=0 tx=0).stop 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 -- 172.21.15.78:0/882753456 shutdown_connections 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 --2- 172.21.15.78:0/882753456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8010e500 0x7f9e80110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 --2- 172.21.15.78:0/882753456 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80101440 0x7f9e8010df30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 --2- 172.21.15.78:0/882753456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80100b20 0x7f9e80100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 -- 172.21.15.78:0/882753456 >> 172.21.15.78:0/882753456 conn(0x7f9e800fc7d0 msgr2=0x7f9e800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 -- 172.21.15.78:0/882753456 shutdown_connections 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.047+0000 7f9e87174640 1 -- 172.21.15.78:0/882753456 wait complete. 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e87174640 1 Processor -- start 2024-09-16T00:21:02.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e87174640 1 -- start start 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e87174640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80100b20 0x7f9e8019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e87174640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80101440 0x7f9e8019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e84f10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80100b20 0x7f9e8019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e84f10640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80100b20 0x7f9e8019c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51460/0 (socket says 172.21.15.78:51460) 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e84f10640 1 -- 172.21.15.78:0/4211611823 learned_addr learned my addr 172.21.15.78:0/4211611823 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e87174640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8010e500 0x7f9e801965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e87174640 1 -- 172.21.15.78:0/4211611823 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80105460 con 0x7f9e80100b20 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e87174640 1 -- 172.21.15.78:0/4211611823 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801052e0 con 0x7f9e8010e500 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e87174640 1 -- 172.21.15.78:0/4211611823 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801055e0 con 0x7f9e80101440 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.048+0000 7f9e7ffff640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80101440 0x7f9e8019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e85711640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8010e500 0x7f9e801965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7ffff640 1 -- 172.21.15.78:0/4211611823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8010e500 msgr2=0x7f9e801965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7ffff640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8010e500 0x7f9e801965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7ffff640 1 -- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80100b20 msgr2=0x7f9e8019c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7ffff640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80100b20 0x7f9e8019c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7ffff640 1 -- 172.21.15.78:0/4211611823 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e80196d70 con 0x7f9e80101440 2024-09-16T00:21:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e84f10640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80100b20 0x7f9e8019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:21:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7ffff640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80101440 0x7f9e8019c560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006010 tx=0x7f9e680040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7dffb640 1 -- 172.21.15.78:0/4211611823 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68050030 con 0x7f9e80101440 2024-09-16T00:21:02.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e87174640 1 -- 172.21.15.78:0/4211611823 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e80197000 con 0x7f9e80101440 2024-09-16T00:21:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7dffb640 1 -- 172.21.15.78:0/4211611823 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e680044c0 con 0x7f9e80101440 2024-09-16T00:21:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.049+0000 7f9e7dffb640 1 -- 172.21.15.78:0/4211611823 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68046070 con 0x7f9e80101440 2024-09-16T00:21:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.050+0000 7f9e87174640 1 -- 172.21.15.78:0/4211611823 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e801a2e80 con 0x7f9e80101440 2024-09-16T00:21:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.051+0000 7f9e87174640 1 -- 172.21.15.78:0/4211611823 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e80100f00 con 0x7f9e80101440 2024-09-16T00:21:02.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.053+0000 7f9e7dffb640 1 -- 172.21.15.78:0/4211611823 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9e68053030 con 0x7f9e80101440 2024-09-16T00:21:02.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.053+0000 7f9e7dffb640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e6c0a3340 0x7f9e6c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.054+0000 7f9e84f10640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e6c0a3340 0x7f9e6c0a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.054+0000 7f9e7dffb640 1 -- 172.21.15.78:0/4211611823 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9e68100f40 con 0x7f9e80101440 2024-09-16T00:21:02.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.054+0000 7f9e84f10640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e6c0a3340 0x7f9e6c0a5800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e70011e00 tx=0x7f9e70013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.057+0000 7f9e7dffb640 1 -- 172.21.15.78:0/4211611823 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e680cd250 con 0x7f9e80101440 2024-09-16T00:21:02.147 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:02.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:01 smithi193 ceph-mon[35484]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 632 op/s 2024-09-16T00:21:02.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.434+0000 7f9e87174640 1 -- 172.21.15.78:0/4211611823 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e80108320 con 0x7f9e80101440 2024-09-16T00:21:02.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.447+0000 7f9e7dffb640 1 -- 172.21.15.78:0/4211611823 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f9e680d1550 con 0x7f9e80101440 2024-09-16T00:21:02.449 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:21:02.449 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:21:02.449 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:21:02.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.452+0000 7f9e777fe640 1 -- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e6c0a3340 msgr2=0x7f9e6c0a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.452+0000 7f9e777fe640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e6c0a3340 0x7f9e6c0a5800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e70011e00 tx=0x7f9e70013040 comp rx=0 tx=0).stop 2024-09-16T00:21:02.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.452+0000 7f9e777fe640 1 -- 172.21.15.78:0/4211611823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80101440 msgr2=0x7f9e8019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.452+0000 7f9e777fe640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80101440 0x7f9e8019c560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e68006010 tx=0x7f9e680040c0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.455+0000 7f9e777fe640 1 -- 172.21.15.78:0/4211611823 shutdown_connections 2024-09-16T00:21:02.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.457+0000 7f9e777fe640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9e6c0a3340 0x7f9e6c0a5800 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.457+0000 7f9e777fe640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e8010e500 0x7f9e801965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.457+0000 7f9e777fe640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e80101440 0x7f9e8019c560 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.457+0000 7f9e777fe640 1 --2- 172.21.15.78:0/4211611823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9e80100b20 0x7f9e8019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.457+0000 7f9e777fe640 1 -- 172.21.15.78:0/4211611823 >> 172.21.15.78:0/4211611823 conn(0x7f9e800fc7d0 msgr2=0x7f9e8010f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:02.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.458+0000 7f9e777fe640 1 -- 172.21.15.78:0/4211611823 shutdown_connections 2024-09-16T00:21:02.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.461+0000 7f9e777fe640 1 -- 172.21.15.78:0/4211611823 wait complete. 2024-09-16T00:21:02.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a4bfff640 1 -- 172.21.15.78:0/3278648461 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440319b0 con 0x7f9a4c1008d0 2024-09-16T00:21:02.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 -- 172.21.15.78:0/3278648461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1008d0 msgr2=0x7f9a4c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3278648461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1008d0 0x7f9a4c100d30 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009930 tx=0x7f9a4402a420 comp rx=0 tx=0).stop 2024-09-16T00:21:02.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 -- 172.21.15.78:0/3278648461 shutdown_connections 2024-09-16T00:21:02.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3278648461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c101270 0x7f9a4c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3278648461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1008d0 0x7f9a4c100d30 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3278648461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a4c1068d0 0x7f9a4c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 -- 172.21.15.78:0/3278648461 >> 172.21.15.78:0/3278648461 conn(0x7f9a4c0fc610 msgr2=0x7f9a4c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 -- 172.21.15.78:0/3278648461 shutdown_connections 2024-09-16T00:21:02.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.965+0000 7f9a526c6640 1 -- 172.21.15.78:0/3278648461 wait complete. 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a526c6640 1 Processor -- start 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a526c6640 1 -- start start 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a526c6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c1008d0 0x7f9a4c10e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a526c6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a4c101270 0x7f9a4c10e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a526c6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1068d0 0x7f9a4c10ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a526c6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111e30 con 0x7f9a4c1068d0 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a526c6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111cb0 con 0x7f9a4c1008d0 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a526c6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c111fb0 con 0x7f9a4c101270 2024-09-16T00:21:02.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a51ec5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1068d0 0x7f9a4c10ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a51ec5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1068d0 0x7f9a4c10ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51478/0 (socket says 172.21.15.78:51478) 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a51ec5640 1 -- 172.21.15.78:0/3159676168 learned_addr learned my addr 172.21.15.78:0/3159676168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.966+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c1008d0 0x7f9a4c10e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a4c101270 0x7f9a4c10e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a516c4640 1 -- 172.21.15.78:0/3159676168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a4c101270 msgr2=0x7f9a4c10e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a4c101270 0x7f9a4c10e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a516c4640 1 -- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1068d0 msgr2=0x7f9a4c10ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1068d0 0x7f9a4c10ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a516c4640 1 -- 172.21.15.78:0/3159676168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c1aac60 con 0x7f9a4c1008d0 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c1008d0 0x7f9a4c10e230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c970 tx=0x7f9a4000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a4a7fc640 1 -- 172.21.15.78:0/3159676168 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40007bb0 con 0x7f9a4c1008d0 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a526c6640 1 -- 172.21.15.78:0/3159676168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c1aae90 con 0x7f9a4c1008d0 2024-09-16T00:21:02.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a4a7fc640 1 -- 172.21.15.78:0/3159676168 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a40007d50 con 0x7f9a4c1008d0 2024-09-16T00:21:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.967+0000 7f9a526c6640 1 -- 172.21.15.78:0/3159676168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c1ab2f0 con 0x7f9a4c1008d0 2024-09-16T00:21:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.968+0000 7f9a4a7fc640 1 -- 172.21.15.78:0/3159676168 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000f450 con 0x7f9a4c1008d0 2024-09-16T00:21:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.970+0000 7f9a4a7fc640 1 -- 172.21.15.78:0/3159676168 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9a40004f50 con 0x7f9a4c1008d0 2024-09-16T00:21:02.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.971+0000 7f9a4a7fc640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9a340a32c0 0x7f9a340a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.972+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9a340a32c0 0x7f9a340a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.972+0000 7f9a4a7fc640 1 -- 172.21.15.78:0/3159676168 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9a400cf870 con 0x7f9a4c1008d0 2024-09-16T00:21:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a55d01740 1 Processor -- start 2024-09-16T00:21:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a55d01740 1 -- start start 2024-09-16T00:21:02.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9a340a32c0 0x7f9a340a5780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c19c3f0 tx=0x7f9a440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c91d20 0x55ce14c90270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c907b0 0x55ce14c90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c9d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a55d01740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce14bf62f0 con 0x55ce14c907b0 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a55d01740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ce14b9dcf0 con 0x55ce14c91d20 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.973+0000 7f9a55d01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ce14bf22b0 con 0x55ce14c9b550 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a51ec5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c9d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a516c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c91d20 0x55ce14c90270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a516c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c91d20 0x55ce14c90270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:47466/0 (socket says 172.21.15.78:47466) 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a516c4640 1 -- 172.21.15.78:0/2962391507 learned_addr learned my addr 172.21.15.78:0/2962391507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:02.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c907b0 0x55ce14c90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a51ec5640 1 -- 172.21.15.78:0/2962391507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c91d20 msgr2=0x55ce14c90270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a51ec5640 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c91d20 0x55ce14c90270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a51ec5640 1 -- 172.21.15.78:0/2962391507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c907b0 msgr2=0x55ce14c90bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a51ec5640 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c907b0 0x55ce14c90bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a51ec5640 1 -- 172.21.15.78:0/2962391507 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce14b823a0 con 0x55ce14c9b550 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.974+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c907b0 0x55ce14c90bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.975+0000 7f9a51ec5640 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c9d940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c007f00 tx=0x7f9a3c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2366c5e7d330685 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.975+0000 7f9a337fe640 1 -- 172.21.15.78:0/2962391507 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c004d20 con 0x55ce14c9b550 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.975+0000 7f9a337fe640 1 -- 172.21.15.78:0/2962391507 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a3c010b20 con 0x55ce14c9b550 2024-09-16T00:21:02.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.975+0000 7f9a337fe640 1 -- 172.21.15.78:0/2962391507 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c005780 con 0x55ce14c9b550 2024-09-16T00:21:02.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.977+0000 7f9a55d01740 1 -- 172.21.15.78:0/2962391507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 msgr2=0x55ce14c9d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:02.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.977+0000 7f9a55d01740 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c9d940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c007f00 tx=0x7f9a3c00f790 comp rx=0 tx=0).stop 2024-09-16T00:21:02.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.981+0000 7f9a55d01740 1 -- 172.21.15.78:0/2962391507 shutdown_connections 2024-09-16T00:21:02.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.981+0000 7f9a55d01740 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c9d940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.981+0000 7f9a55d01740 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c907b0 0x55ce14c90bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.981+0000 7f9a55d01740 1 --2- 172.21.15.78:0/2962391507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c91d20 0x55ce14c90270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:02.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.981+0000 7f9a55d01740 1 -- 172.21.15.78:0/2962391507 >> 172.21.15.78:0/2962391507 conn(0x55ce14c59310 msgr2=0x55ce14c8ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:02.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.989+0000 7f9a55d01740 1 -- 172.21.15.78:0/2962391507 shutdown_connections 2024-09-16T00:21:02.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.989+0000 7f9a55d01740 1 -- 172.21.15.78:0/2962391507 wait complete. 2024-09-16T00:21:03.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.989+0000 7f9a55d01740 1 -- start start 2024-09-16T00:21:03.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.989+0000 7f9a55d01740 1 Processor -- start 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.990+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c907b0 0x55ce14c9af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.990+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 0x55ce14c98200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.990+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c9b550 0x55ce14c98810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.990+0000 7f9a55d01740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ce14c9e250 con 0x55ce14c91d20 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.990+0000 7f9a55d01740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ce14bb1060 con 0x55ce14c9b550 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.990+0000 7f9a55d01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ce14c9e3d0 con 0x55ce14c907b0 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.990+0000 7f9a516c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c907b0 0x55ce14c9af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.991+0000 7f9a51ec5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c9b550 0x55ce14c98810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.991+0000 7f9a51ec5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c9b550 0x55ce14c98810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:47470/0 (socket says 172.21.15.78:47470) 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.991+0000 7f9a51ec5640 1 -- 172.21.15.78:0/3346878629 learned_addr learned my addr 172.21.15.78:0/3346878629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.993+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 0x55ce14c98200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.993+0000 7f9a55d01740 10 client.0 _ll_register_callbacks cb 0x55ce149048c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:03.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.993+0000 7f9a50ec3640 1 -- 172.21.15.78:0/3346878629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c907b0 msgr2=0x55ce14c9af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.993+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c907b0 0x55ce14c9af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.993+0000 7f9a50ec3640 1 -- 172.21.15.78:0/3346878629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c9b550 msgr2=0x55ce14c98810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.993+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c9b550 0x55ce14c98810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.993+0000 7f9a50ec3640 1 -- 172.21.15.78:0/3346878629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce149673c0 con 0x55ce14c91d20 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.994+0000 7f9a50ec3640 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 0x55ce14c98200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9a44005940 tx=0x7f9a44009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef71d23e75a471 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.995+0000 7f9a30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.995+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3346878629 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4404e070 con 0x55ce14c91d20 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.995+0000 7f9a55d01740 1 -- 172.21.15.78:0/3346878629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3c010b20 con 0x55ce14c91d20 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.995+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3346878629 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9a44003a00 con 0x55ce14c91d20 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.998+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3346878629 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9a44058050 con 0x55ce14c91d20 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.998+0000 7f9a55d01740 10 client.16767 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:03.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:02.998+0000 7f9a55d01740 1 -- 172.21.15.78:0/3346878629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ce14ca0280 con 0x55ce14c91d20 2024-09-16T00:21:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.000+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3346878629 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a3c010b20 con 0x55ce14c91d20 2024-09-16T00:21:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.000+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3346878629 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9a44053440 con 0x55ce14c91d20 2024-09-16T00:21:03.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.000+0000 7f9a55d01740 10 client.16767 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.000+0000 7f9a55d01740 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.000+0000 7f9a55d01740 20 client.16767 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.000+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ce14ca1b50 0x55ce14ca3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.000+0000 7f9a55d01740 4 client.16767 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:03.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.000+0000 7f9a55d01740 1 -- 172.21.15.78:0/3346878629 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce14c03860 con 0x55ce14ca1b50 2024-09-16T00:21:03.012 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:02 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4211611823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.010+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ce14ca1b50 0x55ce14ca3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.028+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ce14ca1b50 0x55ce14ca3f90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e230a269c52ea925 server_cookie=1b571527f6fff342 in_seq=0 out_seq=0 2024-09-16T00:21:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.028+0000 7f9a30ff9640 10 client.16767 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:03.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.031+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3346878629 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce14c03860 con 0x55ce14ca1b50 2024-09-16T00:21:03.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.031+0000 7f9a30ff9640 10 client.16767 handle_command_reply: tid=0 2024-09-16T00:21:03.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.032+0000 7f9a55d01740 1 client.16767 shutdown 2024-09-16T00:21:03.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a55d01740 1 -- 172.21.15.78:0/3346878629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 msgr2=0x55ce14c98200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:03.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 0x55ce14c98200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9a44005940 tx=0x7f9a44009260 comp rx=0 tx=0).stop 2024-09-16T00:21:03.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a55d01740 1 -- 172.21.15.78:0/3346878629 shutdown_connections 2024-09-16T00:21:03.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ce14ca1b50 0x55ce14ca3f90 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c9b550 0x55ce14c98810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 0x55ce14c98200 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3346878629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c907b0 0x55ce14c9af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a55d01740 1 -- 172.21.15.78:0/3346878629 >> 172.21.15.78:0/3346878629 conn(0x55ce14c59310 msgr2=0x55ce14c92b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:03.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.037+0000 7f9a30ff9640 0 client.16767 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:03.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.039+0000 7f9a55d01740 1 -- 172.21.15.78:0/3346878629 shutdown_connections 2024-09-16T00:21:03.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.039+0000 7f9a55d01740 1 -- 172.21.15.78:0/3346878629 wait complete. 2024-09-16T00:21:03.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.040+0000 7f9a55d01740 20 client.16767 trim_cache size 0 max 16384 2024-09-16T00:21:03.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a55d01740 1 Processor -- start 2024-09-16T00:21:03.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a55d01740 1 -- start start 2024-09-16T00:21:03.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14c97ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 0x55ce14c93300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c93840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a55d01740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1ac880 con 0x55ce14c91d20 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a516c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14c97ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a516c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14c97ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:47478/0 (socket says 172.21.15.78:47478) 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.075+0000 7f9a516c4640 1 -- 172.21.15.78:0/1650069529 learned_addr learned my addr 172.21.15.78:0/1650069529 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.076+0000 7f9a55d01740 1 -- 172.21.15.78:0/1650069529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1ae3c0 con 0x55ce14c907b0 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.076+0000 7f9a55d01740 1 -- 172.21.15.78:0/1650069529 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1acf50 con 0x55ce14c9b550 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.076+0000 7f9a51ec5640 1 --2- 172.21.15.78:0/1650069529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c93840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.076+0000 7f9a516c4640 1 -- 172.21.15.78:0/1650069529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 msgr2=0x55ce14c93840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.076+0000 7f9a516c4640 1 --2- 172.21.15.78:0/1650069529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c93840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.076+0000 7f9a516c4640 1 -- 172.21.15.78:0/1650069529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 msgr2=0x55ce14c93300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.076+0000 7f9a516c4640 1 --2- 172.21.15.78:0/1650069529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 0x55ce14c93300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.076+0000 7f9a516c4640 1 -- 172.21.15.78:0/1650069529 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce14c9b270 con 0x55ce14c907b0 2024-09-16T00:21:03.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.077+0000 7f9a516c4640 1 --2- 172.21.15.78:0/1650069529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14c97ac0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a400ccfb0 tx=0x7f9a40086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4722d1e361abeaa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:03.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.077+0000 7f9a32ffd640 1 -- 172.21.15.78:0/1650069529 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4001d040 con 0x55ce14c907b0 2024-09-16T00:21:03.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.077+0000 7f9a32ffd640 1 -- 172.21.15.78:0/1650069529 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a400a4740 con 0x55ce14c907b0 2024-09-16T00:21:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.078+0000 7f9a55d01740 1 -- 172.21.15.78:0/1650069529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 msgr2=0x55ce14c97ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.078+0000 7f9a55d01740 1 --2- 172.21.15.78:0/1650069529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14c97ac0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a400ccfb0 tx=0x7f9a40086040 comp rx=0 tx=0).stop 2024-09-16T00:21:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.078+0000 7f9a55d01740 1 -- 172.21.15.78:0/1650069529 shutdown_connections 2024-09-16T00:21:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.078+0000 7f9a55d01740 1 --2- 172.21.15.78:0/1650069529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c9b550 0x55ce14c93840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.078+0000 7f9a55d01740 1 --2- 172.21.15.78:0/1650069529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c91d20 0x55ce14c93300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.078+0000 7f9a55d01740 1 --2- 172.21.15.78:0/1650069529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14c97ac0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.078+0000 7f9a55d01740 1 -- 172.21.15.78:0/1650069529 >> 172.21.15.78:0/1650069529 conn(0x55ce14c59310 msgr2=0x55ce14c8c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.079+0000 7f9a55d01740 1 -- 172.21.15.78:0/1650069529 shutdown_connections 2024-09-16T00:21:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.079+0000 7f9a55d01740 1 -- 172.21.15.78:0/1650069529 wait complete. 2024-09-16T00:21:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.079+0000 7f9a55d01740 1 -- start start 2024-09-16T00:21:03.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.080+0000 7f9a55d01740 1 Processor -- start 2024-09-16T00:21:03.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14ca99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c91d20 0x55ce14ca9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a55d01740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c9b550 0x55ce14ca7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a55d01740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1adcf0 con 0x55ce14c9b550 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a55d01740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c1ad620 con 0x55ce14c907b0 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a55d01740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c100d30 con 0x55ce14c91d20 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a55d01740 10 client.0 _ll_register_callbacks cb 0x7f9a44040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a51ec5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c9b550 0x55ce14ca7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a51ec5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c9b550 0x55ce14ca7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51514/0 (socket says 172.21.15.78:51514) 2024-09-16T00:21:03.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a51ec5640 1 -- 172.21.15.78:0/3421029673 learned_addr learned my addr 172.21.15.78:0/3421029673 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:03.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.081+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14ca99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a51ec5640 1 -- 172.21.15.78:0/3421029673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c91d20 msgr2=0x55ce14ca9f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:21:03.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a51ec5640 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c91d20 0x55ce14ca9f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a51ec5640 1 -- 172.21.15.78:0/3421029673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 msgr2=0x55ce14ca99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:03.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a51ec5640 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14ca99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a51ec5640 1 -- 172.21.15.78:0/3421029673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce14ca8670 con 0x55ce14c9b550 2024-09-16T00:21:03.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a51ec5640 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c9b550 0x55ce14ca7e40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c007f00 tx=0x7f9a3c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2584ae64bd988055 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:03.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3421029673 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c03e040 con 0x55ce14c9b550 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3421029673 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9a3c010640 con 0x55ce14c9b550 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.082+0000 7f9a55d01740 1 -- 172.21.15.78:0/3421029673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a400a4740 con 0x55ce14c9b550 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3421029673 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9a3c056050 con 0x55ce14c9b550 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a55d01740 10 client.16770 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a55d01740 1 -- 172.21.15.78:0/3421029673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ce14d15ec0 con 0x55ce14c9b550 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3421029673 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a3c03f040 con 0x55ce14c9b550 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3421029673 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9a3c045770 con 0x55ce14c9b550 2024-09-16T00:21:03.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a55d01740 10 client.16770 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a55d01740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a55d01740 20 client.16770 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ce14d16650 0x55ce14d18a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a55d01740 4 client.16770 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.083+0000 7f9a55d01740 1 -- 172.21.15.78:0/3421029673 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9a400a4740 con 0x55ce14d16650 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.084+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ce14d16650 0x55ce14d18a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.084+0000 7f9a30ff9640 10 client.16770 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.084+0000 7f9a516c4640 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ce14d16650 0x55ce14d18a70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548a3da6ad8362f1 server_cookie=3faa13d56f50c0d1 in_seq=0 out_seq=0 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a30ff9640 1 -- 172.21.15.78:0/3421029673 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9a400a4740 con 0x55ce14d16650 2024-09-16T00:21:03.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a30ff9640 10 client.16770 handle_command_reply: tid=0 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 client.16770 shutdown 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 -- 172.21.15.78:0/3421029673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c9b550 msgr2=0x55ce14ca7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c9b550 0x55ce14ca7e40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c007f00 tx=0x7f9a3c007a90 comp rx=0 tx=0).stop 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 -- 172.21.15.78:0/3421029673 shutdown_connections 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ce14d16650 0x55ce14d18a70 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ce14c9b550 0x55ce14ca7e40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce14c91d20 0x55ce14ca9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 --2- 172.21.15.78:0/3421029673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ce14c907b0 0x55ce14ca99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 -- 172.21.15.78:0/3421029673 >> 172.21.15.78:0/3421029673 conn(0x55ce14c59310 msgr2=0x55ce14c8b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a30ff9640 0 client.16770 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 -- 172.21.15.78:0/3421029673 shutdown_connections 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 1 -- 172.21.15.78:0/3421029673 wait complete. 2024-09-16T00:21:03.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.085+0000 7f9a55d01740 20 client.16770 trim_cache size 0 max 16384 2024-09-16T00:21:03.093 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 -- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9a340a32c0 msgr2=0x7f9a340a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9a340a32c0 0x7f9a340a5780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c19c3f0 tx=0x7f9a440058d0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 -- 172.21.15.78:0/3159676168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c1008d0 msgr2=0x7f9a4c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c1008d0 0x7f9a4c10e230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c970 tx=0x7f9a4000ce30 comp rx=0 tx=0).stop 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 -- 172.21.15.78:0/3159676168 shutdown_connections 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9a340a32c0 0x7f9a340a5780 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a4c1068d0 0x7f9a4c10ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a4c101270 0x7f9a4c10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 --2- 172.21.15.78:0/3159676168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9a4c1008d0 0x7f9a4c10e230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 -- 172.21.15.78:0/3159676168 >> 172.21.15.78:0/3159676168 conn(0x7f9a4c0fc610 msgr2=0x7f9a4c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:03.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 -- 172.21.15.78:0/3159676168 shutdown_connections 2024-09-16T00:21:03.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:03.086+0000 7f9a526c6640 1 -- 172.21.15.78:0/3159676168 wait complete. 2024-09-16T00:21:03.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:02 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4211611823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:03.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:02 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4211611823' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:03.713 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0' 2024-09-16T00:21:03.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0 2024-09-16T00:21:04.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:03 smithi078 ceph-mon[31157]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-09-16T00:21:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:03 smithi193 ceph-mon[35484]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-09-16T00:21:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:03 smithi148 ceph-mon[35599]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-09-16T00:21:05.671 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:05.848 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:05 smithi078 ceph-mon[31157]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-09-16T00:21:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 425.0 s, 684.2 tps, lat 150.232 ms stddev 208.404 2024-09-16T00:21:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:05 smithi193 ceph-mon[35484]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-09-16T00:21:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:05 smithi148 ceph-mon[35599]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-09-16T00:21:06.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.336+0000 7fecfa880640 1 -- 172.21.15.78:0/2064301549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecec0aa5e0 msgr2=0x7fecec0aa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.336+0000 7fecfa880640 1 --2- 172.21.15.78:0/2064301549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecec0aa5e0 0x7fecec0aa9c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009900 tx=0x7fecf002a420 comp rx=0 tx=0).stop 2024-09-16T00:21:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.341+0000 7fecfa880640 1 -- 172.21.15.78:0/2064301549 shutdown_connections 2024-09-16T00:21:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.341+0000 7fecfa880640 1 --2- 172.21.15.78:0/2064301549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecec0a4d30 0x7fecec0a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.341+0000 7fecfa880640 1 --2- 172.21.15.78:0/2064301549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0aaf00 0x7fecec0a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.341+0000 7fecfa880640 1 --2- 172.21.15.78:0/2064301549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecec0aa5e0 0x7fecec0aa9c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.341+0000 7fecfa880640 1 -- 172.21.15.78:0/2064301549 >> 172.21.15.78:0/2064301549 conn(0x7fecec01a500 msgr2=0x7fecec01a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:06.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.345+0000 7fecfa880640 1 -- 172.21.15.78:0/2064301549 shutdown_connections 2024-09-16T00:21:06.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.347+0000 7fecfa880640 1 -- 172.21.15.78:0/2064301549 wait complete. 2024-09-16T00:21:06.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.347+0000 7fecfa880640 1 Processor -- start 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.348+0000 7fecfa880640 1 -- start start 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecfa880640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecec0a4d30 0x7fecec13f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecfa880640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0aa5e0 0x7fecec13f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecfa880640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecec0aaf00 0x7fecec143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecfa880640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fecec0ae7e0 con 0x7fecec0aa5e0 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecfa880640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fecec0ae660 con 0x7fecec0a4d30 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecfa880640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fecec0b8dd0 con 0x7fecec0aaf00 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf907d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0aa5e0 0x7fecec13f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf907d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0aa5e0 0x7fecec13f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35412/0 (socket says 172.21.15.78:35412) 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf907d640 1 -- 172.21.15.78:0/3968742974 learned_addr learned my addr 172.21.15.78:0/3968742974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:06.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecfa07f640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecec0aaf00 0x7fecec143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf987e640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecec0a4d30 0x7fecec13f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf907d640 1 -- 172.21.15.78:0/3968742974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecec0aaf00 msgr2=0x7fecec143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf907d640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecec0aaf00 0x7fecec143990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf907d640 1 -- 172.21.15.78:0/3968742974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecec0a4d30 msgr2=0x7fecec13f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf907d640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecec0a4d30 0x7fecec13f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.349+0000 7fecf907d640 1 -- 172.21.15.78:0/3968742974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec144110 con 0x7fecec0aa5e0 2024-09-16T00:21:06.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.350+0000 7fecf907d640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0aa5e0 0x7fecec13f600 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fece400b4a0 tx=0x7fece400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:06.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.350+0000 7fecf987e640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecec0a4d30 0x7fecec13f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.352+0000 7feceaffd640 1 -- 172.21.15.78:0/3968742974 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4014070 con 0x7fecec0aa5e0 2024-09-16T00:21:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.352+0000 7fecfa880640 1 -- 172.21.15.78:0/3968742974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec144400 con 0x7fecec0aa5e0 2024-09-16T00:21:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.352+0000 7fecfa880640 1 -- 172.21.15.78:0/3968742974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec14bc40 con 0x7fecec0aa5e0 2024-09-16T00:21:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.352+0000 7feceaffd640 1 -- 172.21.15.78:0/3968742974 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fece4010af0 con 0x7fecec0aa5e0 2024-09-16T00:21:06.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.352+0000 7feceaffd640 1 -- 172.21.15.78:0/3968742974 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4017d20 con 0x7fecec0aa5e0 2024-09-16T00:21:06.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.356+0000 7fecfdebb740 1 Processor -- start 2024-09-16T00:21:06.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.361+0000 7feceaffd640 1 -- 172.21.15.78:0/3968742974 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fece4017070 con 0x7fecec0aa5e0 2024-09-16T00:21:06.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.362+0000 7feceaffd640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fece00a3420 0x7fece00a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.362+0000 7feceaffd640 1 -- 172.21.15.78:0/3968742974 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fece40cf000 con 0x7fecec0aa5e0 2024-09-16T00:21:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.362+0000 7fecfdebb740 1 -- start start 2024-09-16T00:21:06.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013261110 0x5600132614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfa07f640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fece00a3420 0x7fece00a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261a30 0x56001326bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x56001326e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfdebb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5600131be2a0 con 0x56001326c2f0 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfdebb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560013165b10 con 0x560013261110 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfdebb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5600131ba260 con 0x560013261a30 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfa07f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x56001326e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfa07f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x56001326e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35428/0 (socket says 172.21.15.78:35428) 2024-09-16T00:21:06.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.363+0000 7fecfa07f640 1 -- 172.21.15.78:0/3010966831 learned_addr learned my addr 172.21.15.78:0/3010966831 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:06.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.364+0000 7fecfa07f640 1 -- 172.21.15.78:0/3010966831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261a30 msgr2=0x56001326bdb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:21:06.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.364+0000 7fecfa07f640 1 --2- 172.21.15.78:0/3010966831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261a30 0x56001326bdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.364+0000 7fecfa07f640 1 -- 172.21.15.78:0/3010966831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013261110 msgr2=0x5600132614f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:21:06.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.364+0000 7fecfa07f640 1 --2- 172.21.15.78:0/3010966831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013261110 0x5600132614f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.364+0000 7fecfa07f640 1 -- 172.21.15.78:0/3010966831 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560012f2fb90 con 0x56001326c2f0 2024-09-16T00:21:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.365+0000 7fecfa07f640 1 --2- 172.21.15.78:0/3010966831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x56001326e6e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fecf4051500 tx=0x7fecf4080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf62933ec40e6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.365+0000 7fecfa07f640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fece00a3420 0x7fece00a58e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fecf402dd80 tx=0x7fecf4068ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.365+0000 7fecdffff640 1 -- 172.21.15.78:0/3010966831 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4086070 con 0x56001326c2f0 2024-09-16T00:21:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.365+0000 7fecdffff640 1 -- 172.21.15.78:0/3010966831 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4079040 con 0x56001326c2f0 2024-09-16T00:21:06.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.365+0000 7fecdffff640 1 -- 172.21.15.78:0/3010966831 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4078070 con 0x56001326c2f0 2024-09-16T00:21:06.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.366+0000 7fecfdebb740 1 -- 172.21.15.78:0/3010966831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 msgr2=0x56001326e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.366+0000 7fecfdebb740 1 --2- 172.21.15.78:0/3010966831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x56001326e6e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fecf4051500 tx=0x7fecf4080000 comp rx=0 tx=0).stop 2024-09-16T00:21:06.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.367+0000 7fecfdebb740 1 -- 172.21.15.78:0/3010966831 shutdown_connections 2024-09-16T00:21:06.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.367+0000 7fecfdebb740 1 --2- 172.21.15.78:0/3010966831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x56001326e6e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.367+0000 7fecfdebb740 1 --2- 172.21.15.78:0/3010966831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261a30 0x56001326bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.367+0000 7fecfdebb740 1 --2- 172.21.15.78:0/3010966831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013261110 0x5600132614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.367+0000 7fecfdebb740 1 -- 172.21.15.78:0/3010966831 >> 172.21.15.78:0/3010966831 conn(0x56001325d6a0 msgr2=0x56001325da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:06.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.368+0000 7fecfdebb740 1 -- 172.21.15.78:0/3010966831 shutdown_connections 2024-09-16T00:21:06.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.368+0000 7fecfdebb740 1 -- 172.21.15.78:0/3010966831 wait complete. 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.369+0000 7fecfdebb740 1 -- start start 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.369+0000 7fecfdebb740 1 Processor -- start 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261110 0x5600132693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x560013269910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x5600132783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecfdebb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56001326eec0 con 0x56001326c2f0 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecfdebb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5600131466e0 con 0x560013269e50 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecfdebb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56001326f040 con 0x560013261110 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecfdebb740 10 client.0 _ll_register_callbacks cb 0x56001316aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecf907d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x560013269910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecf907d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x560013269910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35438/0 (socket says 172.21.15.78:35438) 2024-09-16T00:21:06.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecf907d640 1 -- 172.21.15.78:0/1933293506 learned_addr learned my addr 172.21.15.78:0/1933293506 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:06.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.370+0000 7fecf987e640 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261110 0x5600132693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.371+0000 7fecfa07f640 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x5600132783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.371+0000 7fecfa07f640 1 -- 172.21.15.78:0/1933293506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261110 msgr2=0x5600132693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:06.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.371+0000 7fecfa07f640 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261110 0x5600132693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.371+0000 7fecfa07f640 1 -- 172.21.15.78:0/1933293506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 msgr2=0x560013269910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:06.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.371+0000 7fecfa07f640 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x560013269910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.371+0000 7fecfa07f640 1 -- 172.21.15.78:0/1933293506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560013278a40 con 0x560013269e50 2024-09-16T00:21:06.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.372+0000 7fecf987e640 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261110 0x5600132693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:21:06.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.373+0000 7fecfa07f640 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x5600132783d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fecf4067960 tx=0x7fecf4068300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5623419070bf4439 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:06.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.374+0000 7fecdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:21:06.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.375+0000 7fecdd7fa640 1 -- 172.21.15.78:0/1933293506 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4078070 con 0x560013269e50 2024-09-16T00:21:06.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.375+0000 7fecdd7fa640 1 -- 172.21.15.78:0/1933293506 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fecf406b980 con 0x560013269e50 2024-09-16T00:21:06.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.375+0000 7fecfdebb740 1 -- 172.21.15.78:0/1933293506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf4079040 con 0x560013269e50 2024-09-16T00:21:06.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.375+0000 7fecdd7fa640 1 -- 172.21.15.78:0/1933293506 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fecf4086050 con 0x560013269e50 2024-09-16T00:21:06.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.375+0000 7fecfdebb740 10 client.26186 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:06.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.375+0000 7fecfdebb740 1 -- 172.21.15.78:0/1933293506 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56001329d3c0 con 0x560013269e50 2024-09-16T00:21:06.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.377+0000 7fecdd7fa640 1 -- 172.21.15.78:0/1933293506 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4079040 con 0x560013269e50 2024-09-16T00:21:06.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.378+0000 7fecdd7fa640 1 -- 172.21.15.78:0/1933293506 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fecf4092070 con 0x560013269e50 2024-09-16T00:21:06.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.380+0000 7fecfdebb740 10 client.26186 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:06.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.380+0000 7fecfdebb740 10 client.26186 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:21:06.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.380+0000 7fecfdebb740 20 client.26186 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:06.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.380+0000 7fecfdebb740 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56001329e770 0x5600132a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.380+0000 7fecfdebb740 4 client.26186 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:06.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.380+0000 7fecfdebb740 1 -- 172.21.15.78:0/1933293506 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600131cb9d0 con 0x56001329e770 2024-09-16T00:21:06.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.380+0000 7fecf987e640 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56001329e770 0x5600132a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.381+0000 7fecf987e640 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56001329e770 0x5600132a0b60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b63b97832bd068d server_cookie=d085fd51d75f9616 in_seq=0 out_seq=0 2024-09-16T00:21:06.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.381+0000 7fecdd7fa640 10 client.26186 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:06.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.382+0000 7fecdd7fa640 1 -- 172.21.15.78:0/1933293506 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600131cb9d0 con 0x56001329e770 2024-09-16T00:21:06.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.382+0000 7fecdd7fa640 10 client.26186 handle_command_reply: tid=0 2024-09-16T00:21:06.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.382+0000 7fecfdebb740 1 client.26186 shutdown 2024-09-16T00:21:06.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.382+0000 7fecfdebb740 1 -- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 msgr2=0x5600132783d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.382+0000 7fecfdebb740 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x5600132783d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fecf4067960 tx=0x7fecf4068300 comp rx=0 tx=0).stop 2024-09-16T00:21:06.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.383+0000 7fecfdebb740 1 -- 172.21.15.78:0/1933293506 shutdown_connections 2024-09-16T00:21:06.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.383+0000 7fecfdebb740 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56001329e770 0x5600132a0b60 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.383+0000 7fecfdebb740 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x5600132783d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.383+0000 7fecfdebb740 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56001326c2f0 0x560013269910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.383+0000 7fecfdebb740 1 --2- 172.21.15.78:0/1933293506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013261110 0x5600132693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.383+0000 7fecfdebb740 1 -- 172.21.15.78:0/1933293506 >> 172.21.15.78:0/1933293506 conn(0x56001325d6a0 msgr2=0x5600132633d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.383+0000 7fecdd7fa640 0 client.26186 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.385+0000 7fecfdebb740 1 -- 172.21.15.78:0/1933293506 shutdown_connections 2024-09-16T00:21:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.385+0000 7fecfdebb740 1 -- 172.21.15.78:0/1933293506 wait complete. 2024-09-16T00:21:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.385+0000 7fecfdebb740 20 client.26186 trim_cache size 0 max 16384 2024-09-16T00:21:06.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.412+0000 7fecfdebb740 1 Processor -- start 2024-09-16T00:21:06.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.412+0000 7fecfdebb740 1 -- start start 2024-09-16T00:21:06.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.412+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132781c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.412+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x560013253ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.412+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56001326c2f0 0x5600132a6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.413+0000 7fecf907d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x560013253ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.413+0000 7fecf907d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x560013253ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39066/0 (socket says 172.21.15.78:39066) 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.413+0000 7fecf987e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.413+0000 7fecf987e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132781c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35454/0 (socket says 172.21.15.78:35454) 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.413+0000 7fecf907d640 1 -- 172.21.15.78:0/401658611 learned_addr learned my addr 172.21.15.78:0/401658611 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.413+0000 7fecfdebb740 1 -- 172.21.15.78:0/401658611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fecd800fbf0 con 0x560013261110 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.413+0000 7fecfdebb740 1 -- 172.21.15.78:0/401658611 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fecd8011730 con 0x560013269e50 2024-09-16T00:21:06.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.413+0000 7fecfdebb740 1 -- 172.21.15.78:0/401658611 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fecd80102c0 con 0x56001326c2f0 2024-09-16T00:21:06.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.414+0000 7fecf987e640 1 -- 172.21.15.78:0/401658611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56001326c2f0 msgr2=0x5600132a6210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:21:06.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.414+0000 7fecf987e640 1 --2- 172.21.15.78:0/401658611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56001326c2f0 0x5600132a6210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.414+0000 7fecf987e640 1 -- 172.21.15.78:0/401658611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 msgr2=0x560013253ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.414+0000 7fecf987e640 1 --2- 172.21.15.78:0/401658611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x560013253ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.414+0000 7fecf987e640 1 -- 172.21.15.78:0/401658611 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560013260e90 con 0x560013261110 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.415+0000 7fecf987e640 1 --2- 172.21.15.78:0/401658611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132781c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fecf002a940 tx=0x7fecf0031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b45cabc98fd611 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.415+0000 7fecdf7fe640 1 -- 172.21.15.78:0/401658611 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0048070 con 0x560013261110 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.415+0000 7fecdf7fe640 1 -- 172.21.15.78:0/401658611 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf0005520 con 0x560013261110 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.415+0000 7fecdf7fe640 1 -- 172.21.15.78:0/401658611 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0035040 con 0x560013261110 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 -- 172.21.15.78:0/401658611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 msgr2=0x5600132781c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 --2- 172.21.15.78:0/401658611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132781c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fecf002a940 tx=0x7fecf0031460 comp rx=0 tx=0).stop 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 -- 172.21.15.78:0/401658611 shutdown_connections 2024-09-16T00:21:06.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 --2- 172.21.15.78:0/401658611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56001326c2f0 0x5600132a6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 --2- 172.21.15.78:0/401658611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560013269e50 0x560013253ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 --2- 172.21.15.78:0/401658611 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132781c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 -- 172.21.15.78:0/401658611 >> 172.21.15.78:0/401658611 conn(0x56001325d6a0 msgr2=0x5600132537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 -- 172.21.15.78:0/401658611 shutdown_connections 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 -- 172.21.15.78:0/401658611 wait complete. 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.416+0000 7fecfdebb740 1 -- start start 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfdebb740 1 Processor -- start 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013269e50 0x5600132705e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfdebb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56001326c2f0 0x560013270b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfdebb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fecd8011060 con 0x560013261110 2024-09-16T00:21:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfdebb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fecd8010990 con 0x56001326c2f0 2024-09-16T00:21:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfdebb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fecd800dbd0 con 0x560013269e50 2024-09-16T00:21:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecf907d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013269e50 0x5600132705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecf987e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecf907d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013269e50 0x5600132705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:38896/0 (socket says 172.21.15.78:38896) 2024-09-16T00:21:06.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecf907d640 1 -- 172.21.15.78:0/2376271844 learned_addr learned my addr 172.21.15.78:0/2376271844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfdebb740 10 client.0 _ll_register_callbacks cb 0x7fecf4080a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.417+0000 7fecfa07f640 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56001326c2f0 0x560013270b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecf907d640 1 -- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56001326c2f0 msgr2=0x560013270b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecf907d640 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56001326c2f0 0x560013270b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecf907d640 1 -- 172.21.15.78:0/2376271844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 msgr2=0x5600132700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecf907d640 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecf907d640 1 -- 172.21.15.78:0/2376271844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560013308210 con 0x560013269e50 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecf987e640 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:06.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecfa07f640 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56001326c2f0 0x560013270b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecf907d640 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013269e50 0x5600132705e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fece400be60 tx=0x7fece40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0e1a69e39805a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecdd7fa640 1 -- 172.21.15.78:0/2376271844 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece40b4020 con 0x560013269e50 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecfdebb740 1 -- 172.21.15.78:0/2376271844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf0005520 con 0x560013269e50 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecdd7fa640 1 -- 172.21.15.78:0/2376271844 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fece40cc850 con 0x560013269e50 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecdd7fa640 1 -- 172.21.15.78:0/2376271844 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fece401e040 con 0x560013269e50 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecfdebb740 10 client.25996 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecfdebb740 1 -- 172.21.15.78:0/2376271844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560013309510 con 0x560013269e50 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.418+0000 7fecdd7fa640 1 -- 172.21.15.78:0/2376271844 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf0005520 con 0x560013269e50 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecdd7fa640 1 -- 172.21.15.78:0/2376271844 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fece40a0170 con 0x560013269e50 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecfdebb740 10 client.25996 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:06.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecfdebb740 10 client.25996 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:21:06.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecfdebb740 20 client.25996 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:06.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecfdebb740 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560013309d30 0x56001330c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:06.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecfdebb740 4 client.25996 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}] 2024-09-16T00:21:06.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecfdebb740 1 -- 172.21.15.78:0/2376271844 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}) -- 0x5600131c5d20 con 0x560013309d30 2024-09-16T00:21:06.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecf987e640 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560013309d30 0x56001330c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:06.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecdd7fa640 10 client.25996 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:06.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.419+0000 7fecf987e640 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560013309d30 0x56001330c150 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d32f27582e42dc server_cookie=f4e303fbf4ace82c in_seq=0 out_seq=0 2024-09-16T00:21:06.644 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:21:06.644 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:21:06.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecdd7fa640 1 -- 172.21.15.78:0/2376271844 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5600131c5d20 con 0x560013309d30 2024-09-16T00:21:06.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecdd7fa640 10 client.25996 handle_command_reply: tid=0 2024-09-16T00:21:06.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecfdebb740 1 client.25996 shutdown 2024-09-16T00:21:06.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecfdebb740 1 -- 172.21.15.78:0/2376271844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013269e50 msgr2=0x5600132705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecfdebb740 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013269e50 0x5600132705e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fece400be60 tx=0x7fece40046c0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecfdebb740 1 -- 172.21.15.78:0/2376271844 shutdown_connections 2024-09-16T00:21:06.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecfdebb740 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560013309d30 0x56001330c150 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecfdebb740 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56001326c2f0 0x560013270b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecfdebb740 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560013269e50 0x5600132705e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecdd7fa640 0 client.25996 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:06.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.696+0000 7fecfdebb740 1 --2- 172.21.15.78:0/2376271844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560013261110 0x5600132700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.697+0000 7fecfdebb740 1 -- 172.21.15.78:0/2376271844 >> 172.21.15.78:0/2376271844 conn(0x56001325d6a0 msgr2=0x56001326b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:06.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.697+0000 7fecfdebb740 1 -- 172.21.15.78:0/2376271844 shutdown_connections 2024-09-16T00:21:06.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.697+0000 7fecfdebb740 1 -- 172.21.15.78:0/2376271844 wait complete. 2024-09-16T00:21:06.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.697+0000 7fecfdebb740 20 client.25996 trim_cache size 0 max 16384 2024-09-16T00:21:06.702 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:21:06.703 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:21:06.703 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:21:06.703 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 26, 2024-09-16T00:21:06.704 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:21:06.704 INFO:teuthology.orchestra.run.smithi078.stdout: "3d8b79c8": { 2024-09-16T00:21:06.704 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 26, 2024-09-16T00:21:06.704 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:21:06.705 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:21:06.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:21:06.705 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:21:06.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:21:06.706 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:21:06.706 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 143.7, 2024-09-16T00:21:06.706 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:21:06.706 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:21:06.707 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:21:06.707 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:21:06.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:21:06.707 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:21:06.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:06.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:06.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:06.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:06.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:06.709 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:21:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.697+0000 7fecfa880640 1 -- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fece00a3420 msgr2=0x7fece00a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.697+0000 7fecfa880640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fece00a3420 0x7fece00a58e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fecf402dd80 tx=0x7fecf4068ea0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.697+0000 7fecfa880640 1 -- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0aa5e0 msgr2=0x7fecec13f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.697+0000 7fecfa880640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0aa5e0 0x7fecec13f600 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fece400b4a0 tx=0x7fece400b960 comp rx=0 tx=0).stop 2024-09-16T00:21:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.698+0000 7fecfa880640 1 -- 172.21.15.78:0/3968742974 shutdown_connections 2024-09-16T00:21:06.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.698+0000 7fecfa880640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fece00a3420 0x7fece00a58e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.698+0000 7fecfa880640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecec0aaf00 0x7fecec143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.698+0000 7fecfa880640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0aa5e0 0x7fecec13f600 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.698+0000 7fecfa880640 1 --2- 172.21.15.78:0/3968742974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fecec0a4d30 0x7fecec13f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.698+0000 7fecfa880640 1 -- 172.21.15.78:0/3968742974 >> 172.21.15.78:0/3968742974 conn(0x7fecec01a500 msgr2=0x7fecec0a6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:06.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.698+0000 7fecfa880640 1 -- 172.21.15.78:0/3968742974 shutdown_connections 2024-09-16T00:21:06.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:06.698+0000 7fecfa880640 1 -- 172.21.15.78:0/3968742974 wait complete. 2024-09-16T00:21:07.544 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3d8b79c8', quiesce took 8.6 seconds. Will release after: 15.1 2024-09-16T00:21:07.617 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:07.768 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:07 smithi078 ceph-mon[31157]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 581 op/s 2024-09-16T00:21:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.079+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/730167232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e8101440 msgr2=0x7fa8e810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.079+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/730167232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e8101440 0x7fa8e810df30 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009900 tx=0x7fa8dc02a410 comp rx=0 tx=0).stop 2024-09-16T00:21:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.079+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/730167232 shutdown_connections 2024-09-16T00:21:08.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.079+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/730167232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e810e500 0x7fa8e8110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.079+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/730167232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e8101440 0x7fa8e810df30 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.079+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/730167232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8e8100b20 0x7fa8e8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.079+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/730167232 >> 172.21.15.78:0/730167232 conn(0x7fa8e80fc7d0 msgr2=0x7fa8e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/730167232 shutdown_connections 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/730167232 wait complete. 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 Processor -- start 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 -- start start 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e8100b20 0x7fa8e81a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8e8101440 0x7fa8e81a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e810e500 0x7fa8e819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81142d0 con 0x7fa8e810e500 2024-09-16T00:21:08.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8114150 con 0x7fa8e8101440 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ef8f6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8114450 con 0x7fa8e8100b20 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ede93640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e810e500 0x7fa8e819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ede93640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e810e500 0x7fa8e819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35480/0 (socket says 172.21.15.78:35480) 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.080+0000 7fa8ede93640 1 -- 172.21.15.78:0/3846503297 learned_addr learned my addr 172.21.15.78:0/3846503297 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ece91640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8e8101440 0x7fa8e81a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ed692640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e8100b20 0x7fa8e81a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ede93640 1 -- 172.21.15.78:0/3846503297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e8100b20 msgr2=0x7fa8e81a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ede93640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e8100b20 0x7fa8e81a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ede93640 1 -- 172.21.15.78:0/3846503297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8e8101440 msgr2=0x7fa8e81a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ede93640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8e8101440 0x7fa8e81a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ede93640 1 -- 172.21.15.78:0/3846503297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e819aed0 con 0x7fa8e810e500 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ede93640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e810e500 0x7fa8e819a6a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000b3f0 tx=0x7fa8e000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:08.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8e67fc640 1 -- 172.21.15.78:0/3846503297 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0015020 con 0x7fa8e810e500 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8e67fc640 1 -- 172.21.15.78:0/3846503297 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e0011d10 con 0x7fa8e810e500 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/3846503297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e819b1c0 con 0x7fa8e810e500 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8e67fc640 1 -- 172.21.15.78:0/3846503297 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000f2e0 con 0x7fa8e810e500 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.081+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/3846503297 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e81a7530 con 0x7fa8e810e500 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.083+0000 7fa8e67fc640 1 -- 172.21.15.78:0/3846503297 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa8e0018020 con 0x7fa8e810e500 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.084+0000 7fa8f1ccf740 1 Processor -- start 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.084+0000 7fa8f1ccf740 1 -- start start 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.084+0000 7fa8e67fc640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8d00a3240 0x7fa8d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.084+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e92f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.084+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 0x55f0c5e9d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.084+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f0c5df8280 con 0x55f0c5e92b60 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f0c5d9fb10 con 0x55f0c5e93480 2024-09-16T00:21:08.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f0c5df4240 con 0x55f0c5e9dd40 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e92f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8e67fc640 1 -- 172.21.15.78:0/3846503297 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa8e00cf530 con 0x7fa8e810e500 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e92f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35488/0 (socket says 172.21.15.78:35488) 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ece91640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8d00a3240 0x7fa8d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 -- 172.21.15.78:0/1327907630 learned_addr learned my addr 172.21.15.78:0/1327907630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ede93640 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ece91640 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 0x55f0c5e9d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 -- 172.21.15.78:0/1327907630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 msgr2=0x55f0c5ea0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 -- 172.21.15.78:0/1327907630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 msgr2=0x55f0c5e9d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 0x55f0c5e9d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 -- 172.21.15.78:0/1327907630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c5dac1f0 con 0x55f0c5e92b60 2024-09-16T00:21:08.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ede93640 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ece91640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8d00a3240 0x7fa8d00a5700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc004770 tx=0x7fa8dc0318e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8ed692640 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e92f40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009f60 tx=0x7fa8d8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a631ab2375126f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8cf7fe640 1 -- 172.21.15.78:0/1327907630 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8016070 con 0x55f0c5e92b60 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8cf7fe640 1 -- 172.21.15.78:0/1327907630 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d801bb40 con 0x55f0c5e92b60 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.085+0000 7fa8cf7fe640 1 -- 172.21.15.78:0/1327907630 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8012cc0 con 0x55f0c5e92b60 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/1327907630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 msgr2=0x55f0c5e92f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e92f40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009f60 tx=0x7fa8d8009f90 comp rx=0 tx=0).stop 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/1327907630 shutdown_connections 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 0x55f0c5e9d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/1327907630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e92f40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/1327907630 >> 172.21.15.78:0/1327907630 conn(0x55f0c5e8f0f0 msgr2=0x55f0c5e8f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:08.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/1327907630 shutdown_connections 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.086+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/1327907630 wait complete. 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 1 -- start start 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 1 Processor -- start 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e9ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e93480 0x55f0c5e9b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 0x55f0c5e9b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f0c5ea0910 con 0x55f0c5e92b60 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f0c5db2f10 con 0x55f0c5e9dd40 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f0c5ea0a90 con 0x55f0c5e93480 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8f1ccf740 10 client.0 _ll_register_callbacks cb 0x55f0c5d7ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8ed692640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e9ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8ed692640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e9ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35502/0 (socket says 172.21.15.78:35502) 2024-09-16T00:21:08.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8ed692640 1 -- 172.21.15.78:0/3397603467 learned_addr learned my addr 172.21.15.78:0/3397603467 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8ece91640 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e93480 0x55f0c5e9b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8ed692640 1 -- 172.21.15.78:0/3397603467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e93480 msgr2=0x55f0c5e9b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.087+0000 7fa8ede93640 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 0x55f0c5e9b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ed692640 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e93480 0x55f0c5e9b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ed692640 1 -- 172.21.15.78:0/3397603467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 msgr2=0x55f0c5e9b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ed692640 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 0x55f0c5e9b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ed692640 1 -- 172.21.15.78:0/3397603467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c5b696a0 con 0x55f0c5e92b60 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ed692640 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e9ac50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005f40 tx=0x7fa8d8013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59297a29ff843d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ccff9640 1 -- 172.21.15.78:0/3397603467 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d802b060 con 0x55f0c5e92b60 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ccff9640 1 -- 172.21.15.78:0/3397603467 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa8d8003970 con 0x55f0c5e92b60 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/3397603467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d801bb40 con 0x55f0c5e92b60 2024-09-16T00:21:08.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ccff9640 1 -- 172.21.15.78:0/3397603467 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa8d8032050 con 0x55f0c5e92b60 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8f1ccf740 10 client.16794 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/3397603467 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f0c5eac540 con 0x55f0c5e92b60 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ccff9640 1 -- 172.21.15.78:0/3397603467 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d801bb40 con 0x55f0c5e92b60 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8ccff9640 1 -- 172.21.15.78:0/3397603467 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa8d800f040 con 0x55f0c5e92b60 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8f1ccf740 10 client.16794 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8f1ccf740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8f1ccf740 20 client.16794 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.088+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f0c5eada00 0x55f0c5eafdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.089+0000 7fa8f1ccf740 4 client.16794 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.089+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/3397603467 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c5e057f0 con 0x55f0c5eada00 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.089+0000 7fa8ece91640 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f0c5eada00 0x55f0c5eafdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.089+0000 7fa8ece91640 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f0c5eada00 0x55f0c5eafdf0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b630c0319c7a142 server_cookie=13b9187d6b0aa95 in_seq=0 out_seq=0 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.089+0000 7fa8ccff9640 10 client.16794 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:08.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8ccff9640 1 -- 172.21.15.78:0/3397603467 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0c5e057f0 con 0x55f0c5eada00 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8ccff9640 10 client.16794 handle_command_reply: tid=0 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 client.16794 shutdown 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/3397603467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 msgr2=0x55f0c5e9ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e9ac50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8005f40 tx=0x7fa8d8013c00 comp rx=0 tx=0).stop 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/3397603467 shutdown_connections 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f0c5eada00 0x55f0c5eafdf0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 0x55f0c5e9b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e93480 0x55f0c5e9b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8ccff9640 0 client.16794 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/3397603467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e9ac50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.090+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/3397603467 >> 172.21.15.78:0/3397603467 conn(0x55f0c5e8f0f0 msgr2=0x55f0c5e8e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.091+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/3397603467 shutdown_connections 2024-09-16T00:21:08.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.091+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/3397603467 wait complete. 2024-09-16T00:21:08.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.091+0000 7fa8f1ccf740 20 client.16794 trim_cache size 0 max 16384 2024-09-16T00:21:08.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8f1ccf740 1 Processor -- start 2024-09-16T00:21:08.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8f1ccf740 1 -- start start 2024-09-16T00:21:08.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 0x55f0c5eb5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5eb5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81b0f50 con 0x55f0c5e92b60 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81b2a90 con 0x55f0c5e93480 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81b1620 con 0x55f0c5e9dd40 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8ed692640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8ed692640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35504/0 (socket says 172.21.15.78:35504) 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8ed692640 1 -- 172.21.15.78:0/180946307 learned_addr learned my addr 172.21.15.78:0/180946307 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8ede93640 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5eb5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8ece91640 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 0x55f0c5eb5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.108+0000 7fa8ed692640 1 -- 172.21.15.78:0/180946307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 msgr2=0x55f0c5eb5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:08.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.109+0000 7fa8ed692640 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5eb5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.109+0000 7fa8ed692640 1 -- 172.21.15.78:0/180946307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 msgr2=0x55f0c5eb5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:08.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.109+0000 7fa8ed692640 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 0x55f0c5eb5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.109+0000 7fa8ed692640 1 -- 172.21.15.78:0/180946307 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c5e99c00 con 0x55f0c5e92b60 2024-09-16T00:21:08.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.109+0000 7fa8ed692640 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e955c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8002230 tx=0x7fa8d80097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195ea54d43f2cc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:08.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.109+0000 7fa8ceffd640 1 -- 172.21.15.78:0/180946307 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8016040 con 0x55f0c5e92b60 2024-09-16T00:21:08.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.109+0000 7fa8ceffd640 1 -- 172.21.15.78:0/180946307 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d8009b60 con 0x55f0c5e92b60 2024-09-16T00:21:08.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.109+0000 7fa8ceffd640 1 -- 172.21.15.78:0/180946307 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800eb70 con 0x55f0c5e92b60 2024-09-16T00:21:08.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/180946307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 msgr2=0x55f0c5e955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e955c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8002230 tx=0x7fa8d80097e0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/180946307 shutdown_connections 2024-09-16T00:21:08.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e9dd40 0x55f0c5eb5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e93480 0x55f0c5eb5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/180946307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e92b60 0x55f0c5e955c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/180946307 >> 172.21.15.78:0/180946307 conn(0x55f0c5e8f0f0 msgr2=0x55f0c5e9cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:08.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/180946307 shutdown_connections 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/180946307 wait complete. 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 -- start start 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.110+0000 7fa8f1ccf740 1 Processor -- start 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e92b60 0x55f0c5ea1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e93480 0x55f0c5ea2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8f1ccf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81b23c0 con 0x55f0c5e93480 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81b1cf0 con 0x55f0c5e9dd40 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8f1ccf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81aef30 con 0x55f0c5e92b60 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8f1ccf740 10 client.0 _ll_register_callbacks cb 0x7fa8d8009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e93480 0x55f0c5ea2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e93480 0x55f0c5ea2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35520/0 (socket says 172.21.15.78:35520) 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 -- 172.21.15.78:0/144105751 learned_addr learned my addr 172.21.15.78:0/144105751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:08.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ed692640 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e92b60 0x55f0c5ea1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ede93640 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 -- 172.21.15.78:0/144105751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e92b60 msgr2=0x55f0c5ea1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e92b60 0x55f0c5ea1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 -- 172.21.15.78:0/144105751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 msgr2=0x55f0c5ea2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 -- 172.21.15.78:0/144105751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c5f17780 con 0x55f0c5e93480 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ece91640 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e93480 0x55f0c5ea2230 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa8e806a140 tx=0x7fa8dc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb3ed2f5464abf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ccff9640 1 -- 172.21.15.78:0/144105751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc035040 con 0x55f0c5e93480 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8ccff9640 1 -- 172.21.15.78:0/144105751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa8dc0035f0 con 0x55f0c5e93480 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.111+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/144105751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8d8009b60 con 0x55f0c5e93480 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8ccff9640 1 -- 172.21.15.78:0/144105751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa8dc060050 con 0x55f0c5e93480 2024-09-16T00:21:08.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8f1ccf740 10 client.16800 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/144105751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f0c5f18b20 con 0x55f0c5e93480 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8ccff9640 1 -- 172.21.15.78:0/144105751 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8d8009b60 con 0x55f0c5e93480 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8ccff9640 1 -- 172.21.15.78:0/144105751 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa8dc04b730 con 0x55f0c5e93480 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8f1ccf740 10 client.16800 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8f1ccf740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8f1ccf740 20 client.16800 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f0c5f192a0 0x55f0c5f1b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8f1ccf740 4 client.16800 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/144105751 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f0c5dffd00 con 0x55f0c5f192a0 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.112+0000 7fa8ed692640 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f0c5f192a0 0x55f0c5f1b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.113+0000 7fa8ccff9640 10 client.16800 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.113+0000 7fa8ed692640 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f0c5f192a0 0x55f0c5f1b6c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbbb703beea09739 server_cookie=5a6c2e3f7ef14034 in_seq=0 out_seq=0 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8ccff9640 1 -- 172.21.15.78:0/144105751 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f0c5dffd00 con 0x55f0c5f192a0 2024-09-16T00:21:08.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8ccff9640 10 client.16800 handle_command_reply: tid=0 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 client.16800 shutdown 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/144105751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e93480 msgr2=0x55f0c5ea2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e93480 0x55f0c5ea2230 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa8e806a140 tx=0x7fa8dc02a410 comp rx=0 tx=0).stop 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/144105751 shutdown_connections 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f0c5f192a0 0x55f0c5f1b6c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c5e9dd40 0x55f0c5ea2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8ccff9640 0 client.16800 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f0c5e93480 0x55f0c5ea2230 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 --2- 172.21.15.78:0/144105751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f0c5e92b60 0x55f0c5ea1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/144105751 >> 172.21.15.78:0/144105751 conn(0x55f0c5e8f0f0 msgr2=0x55f0c5e8da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/144105751 shutdown_connections 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 1 -- 172.21.15.78:0/144105751 wait complete. 2024-09-16T00:21:08.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.114+0000 7fa8f1ccf740 20 client.16800 trim_cache size 0 max 16384 2024-09-16T00:21:08.120 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:21:08.120 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:21:08.120 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "0bfa4937-5556-45b1-9ab7-34a3d5c2adf3", 2024-09-16T00:21:08.120 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:21:08.120 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8d00a3240 msgr2=0x7fa8d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8d00a3240 0x7fa8d00a5700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc004770 tx=0x7fa8dc0318e0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e810e500 msgr2=0x7fa8e819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e810e500 0x7fa8e819a6a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000b3f0 tx=0x7fa8e000b8b0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/3846503297 shutdown_connections 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8d00a3240 0x7fa8d00a5700 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8e810e500 0x7fa8e819a6a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa8e8101440 0x7fa8e81a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 --2- 172.21.15.78:0/3846503297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e8100b20 0x7fa8e81a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:08.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/3846503297 >> 172.21.15.78:0/3846503297 conn(0x7fa8e80fc7d0 msgr2=0x7fa8e810f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:08.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/3846503297 shutdown_connections 2024-09-16T00:21:08.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:08.115+0000 7fa8ef8f6640 1 -- 172.21.15.78:0/3846503297 wait complete. 2024-09-16T00:21:08.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:07 smithi193 ceph-mon[35484]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 581 op/s 2024-09-16T00:21:08.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:07 smithi148 ceph-mon[35599]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.8 MiB/s wr, 581 op/s 2024-09-16T00:21:09.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:08 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:21:09.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:08 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:21:09.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:08 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:09.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:08 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:21:09.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:08 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:21:09.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:08 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:09.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:08 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:21:09.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:08 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:21:09.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:08 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:09.327 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:21:09.575 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:09.988 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:09 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:21:09.989 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:09.989 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:09 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:21:09.989 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:09.989 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:09.989 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:09 smithi078 ceph-mon[31157]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-16T00:21:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:09 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:21:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:09 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:21:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:09 smithi193 ceph-mon[35484]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-16T00:21:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:09 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:21:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:09 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:21:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:10.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:21:10.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:09 smithi148 ceph-mon[35599]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-09-16T00:21:10.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 -- 172.21.15.78:0/3316590471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb4102db0 msgr2=0x7fcdb4103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 --2- 172.21.15.78:0/3316590471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb4102db0 0x7fcdb4103190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009900 tx=0x7fcda402a480 comp rx=0 tx=0).stop 2024-09-16T00:21:10.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 -- 172.21.15.78:0/3316590471 shutdown_connections 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 --2- 172.21.15.78:0/3316590471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb410c190 0x7fcdb410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 --2- 172.21.15.78:0/3316590471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb41036d0 0x7fcdb410bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 --2- 172.21.15.78:0/3316590471 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb4102db0 0x7fcdb4103190 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 -- 172.21.15.78:0/3316590471 >> 172.21.15.78:0/3316590471 conn(0x7fcdb40fc7d0 msgr2=0x7fcdb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 -- 172.21.15.78:0/3316590471 shutdown_connections 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.215+0000 7fcdba923640 1 -- 172.21.15.78:0/3316590471 wait complete. 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdba923640 1 Processor -- start 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdba923640 1 -- start start 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdba923640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb4102db0 0x7fcdb4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdba923640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb41036d0 0x7fcdb4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdba923640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb410c190 0x7fcdb419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdba923640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4111f70 con 0x7fcdb410c190 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdba923640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4111df0 con 0x7fcdb41036d0 2024-09-16T00:21:10.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdba923640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41120f0 con 0x7fcdb4102db0 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb8ec0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb410c190 0x7fcdb419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb8ec0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb410c190 0x7fcdb419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35544/0 (socket says 172.21.15.78:35544) 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb8ec0640 1 -- 172.21.15.78:0/2495625196 learned_addr learned my addr 172.21.15.78:0/2495625196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb41036d0 0x7fcdb4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb4102db0 0x7fcdb4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb8ec0640 1 -- 172.21.15.78:0/2495625196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb4102db0 msgr2=0x7fcdb4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb8ec0640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb4102db0 0x7fcdb4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb8ec0640 1 -- 172.21.15.78:0/2495625196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb41036d0 msgr2=0x7fcdb4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb8ec0640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb41036d0 0x7fcdb4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.216+0000 7fcdb8ec0640 1 -- 172.21.15.78:0/2495625196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb419d3e0 con 0x7fcdb410c190 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.217+0000 7fcdb8ec0640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb410c190 0x7fcdb419cd00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00cad0 tx=0x7fcdac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:10.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.217+0000 7fcdb17fa640 1 -- 172.21.15.78:0/2495625196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ae20 con 0x7fcdb410c190 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.217+0000 7fcdb17fa640 1 -- 172.21.15.78:0/2495625196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdac005020 con 0x7fcdb410c190 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.217+0000 7fcdba923640 1 -- 172.21.15.78:0/2495625196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb419d6d0 con 0x7fcdb410c190 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.217+0000 7fcdb17fa640 1 -- 172.21.15.78:0/2495625196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00f6f0 con 0x7fcdb410c190 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.217+0000 7fcdba923640 1 -- 172.21.15.78:0/2495625196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb41a4fb0 con 0x7fcdb410c190 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdb17fa640 1 -- 172.21.15.78:0/2495625196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcdac0051d0 con 0x7fcdb410c190 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdbccfc740 1 Processor -- start 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdbccfc740 1 -- start start 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c234740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c234c80 0x559c6c243550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c243a90 0x559c6c245e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c6c19a2e0 con 0x559c6c243a90 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559c6c141ce0 con 0x559c6c234360 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.219+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c6c1962a0 con 0x559c6c234c80 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c243a90 0x559c6c245e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c243a90 0x559c6c245e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35554/0 (socket says 172.21.15.78:35554) 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 -- 172.21.15.78:0/3106114676 learned_addr learned my addr 172.21.15.78:0/3106114676 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c234c80 0x559c6c243550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb17fa640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd9c0a3340 0x7fcd9c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c234740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 -- 172.21.15.78:0/3106114676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c234c80 msgr2=0x559c6c243550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c234c80 0x559c6c243550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb8ec0640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd9c0a3340 0x7fcd9c0a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 -- 172.21.15.78:0/3106114676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 msgr2=0x559c6c234740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c234740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 -- 172.21.15.78:0/3106114676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6c126440 con 0x559c6c243a90 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb17fa640 1 -- 172.21.15.78:0/2495625196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcdac0cfc50 con 0x7fcdb410c190 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c243a90 0x559c6c245e80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcda80125c0 tx=0x7fcda800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932c6eb44780c02f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcd9a7fc640 1 -- 172.21.15.78:0/3106114676 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda80106b0 con 0x559c6c243a90 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcd9a7fc640 1 -- 172.21.15.78:0/3106114676 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8010d30 con 0x559c6c243a90 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.220+0000 7fcd9a7fc640 1 -- 172.21.15.78:0/3106114676 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda801dd30 con 0x559c6c243a90 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3106114676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c243a90 msgr2=0x559c6c245e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdb8ec0640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd9c0a3340 0x7fcd9c0a5800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0a45f0 tx=0x7fcdac06ed30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c243a90 0x559c6c245e80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcda80125c0 tx=0x7fcda800b870 comp rx=0 tx=0).stop 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3106114676 shutdown_connections 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c243a90 0x559c6c245e80 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c234c80 0x559c6c243550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3106114676 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c234740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3106114676 >> 172.21.15.78:0/3106114676 conn(0x559c6c230a30 msgr2=0x559c6c230e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3106114676 shutdown_connections 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3106114676 wait complete. 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.221+0000 7fcdbccfc740 1 -- start start 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdbccfc740 1 Processor -- start 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c243450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c240700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c240cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c6c246660 con 0x559c6c234c80 2024-09-16T00:21:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559c6c155050 con 0x559c6c234360 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c6c2467e0 con 0x559c6c243a90 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdbccfc740 10 client.0 _ll_register_callbacks cb 0x559c6c195160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c240700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c240700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35560/0 (socket says 172.21.15.78:35560) 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb3fff640 1 -- 172.21.15.78:0/3197299636 learned_addr learned my addr 172.21.15.78:0/3197299636 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c240cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c243450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb3fff640 1 -- 172.21.15.78:0/3197299636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 msgr2=0x559c6c240cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c240cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb3fff640 1 -- 172.21.15.78:0/3197299636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 msgr2=0x559c6c243450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c243450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb3fff640 1 -- 172.21.15.78:0/3197299636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6bf0b3c0 con 0x559c6c234c80 2024-09-16T00:21:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.222+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c240cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c240700 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009900 tx=0x7fcda4031990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f05761e030ced8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcd937fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcd937fe640 1 -- 172.21.15.78:0/3197299636 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4035040 con 0x559c6c234c80 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcd937fe640 1 -- 172.21.15.78:0/3197299636 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcda4004240 con 0x559c6c234c80 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3197299636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda8010d30 con 0x559c6c234c80 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcd937fe640 1 -- 172.21.15.78:0/3197299636 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcda4004440 con 0x559c6c234c80 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 10 client.16812 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3197299636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559c6c2520c0 con 0x559c6c234c80 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcd937fe640 1 -- 172.21.15.78:0/3197299636 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8010d30 con 0x559c6c234c80 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcd937fe640 1 -- 172.21.15.78:0/3197299636 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcda4004bb0 con 0x559c6c234c80 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 10 client.16812 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 20 client.16812 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559c6c2535d0 0x559c6c2559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 4 client.16812 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.223+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3197299636 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c6c1a7850 con 0x559c6c2535d0 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.224+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559c6c2535d0 0x559c6c2559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.224+0000 7fcd937fe640 10 client.16812 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.224+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559c6c2535d0 0x559c6c2559e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36bf463db473a43 server_cookie=f9ec9af451b3baac in_seq=0 out_seq=0 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcd937fe640 1 -- 172.21.15.78:0/3197299636 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c6c1a7850 con 0x559c6c2535d0 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcd937fe640 10 client.16812 handle_command_reply: tid=0 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 client.16812 shutdown 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3197299636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 msgr2=0x559c6c240700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c240700 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009900 tx=0x7fcda4031990 comp rx=0 tx=0).stop 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3197299636 shutdown_connections 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559c6c2535d0 0x559c6c2559e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c240cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcd937fe640 0 client.16812 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c240700 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/3197299636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c243450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3197299636 >> 172.21.15.78:0/3197299636 conn(0x559c6c230a30 msgr2=0x559c6c22fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3197299636 shutdown_connections 2024-09-16T00:21:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 1 -- 172.21.15.78:0/3197299636 wait complete. 2024-09-16T00:21:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.225+0000 7fcdbccfc740 20 client.16812 trim_cache size 0 max 16384 2024-09-16T00:21:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.242+0000 7fcdbccfc740 1 Processor -- start 2024-09-16T00:21:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.242+0000 7fcdbccfc740 1 -- start start 2024-09-16T00:21:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c24f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c25b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c25b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41aca70 con 0x559c6c234c80 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41ae5b0 con 0x559c6c234360 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41ad140 con 0x559c6c243a90 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c25b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c25b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35564/0 (socket says 172.21.15.78:35564) 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 -- 172.21.15.78:0/2892717370 learned_addr learned my addr 172.21.15.78:0/2892717370 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c24f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 -- 172.21.15.78:0/2892717370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 msgr2=0x559c6c25b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c25b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 -- 172.21.15.78:0/2892717370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 msgr2=0x559c6c24f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c24f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 -- 172.21.15.78:0/2892717370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6c23f6b0 con 0x559c6c234c80 2024-09-16T00:21:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c24f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.243+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c25b2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcda402a9a0 tx=0x7fcda402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185ba5c27050cc5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcd99ffb640 1 -- 172.21.15.78:0/2892717370 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4035070 con 0x559c6c234c80 2024-09-16T00:21:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcd99ffb640 1 -- 172.21.15.78:0/2892717370 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda40039b0 con 0x559c6c234c80 2024-09-16T00:21:10.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcd99ffb640 1 -- 172.21.15.78:0/2892717370 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4003eb0 con 0x559c6c234c80 2024-09-16T00:21:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcdbccfc740 1 -- 172.21.15.78:0/2892717370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 msgr2=0x559c6c25b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c25b2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcda402a9a0 tx=0x7fcda402a480 comp rx=0 tx=0).stop 2024-09-16T00:21:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcdbccfc740 1 -- 172.21.15.78:0/2892717370 shutdown_connections 2024-09-16T00:21:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c25b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c25b2f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/2892717370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c24f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.244+0000 7fcdbccfc740 1 -- 172.21.15.78:0/2892717370 >> 172.21.15.78:0/2892717370 conn(0x559c6c230a30 msgr2=0x559c6c23bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 -- 172.21.15.78:0/2892717370 shutdown_connections 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 -- 172.21.15.78:0/2892717370 wait complete. 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 -- start start 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 Processor -- start 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c247980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c247ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c248400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41adee0 con 0x559c6c234c80 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41ad810 con 0x559c6c234360 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41aaa50 con 0x559c6c243a90 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.245+0000 7fcdbccfc740 10 client.0 _ll_register_callbacks cb 0x7fcda4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c247ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c247ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35572/0 (socket says 172.21.15.78:35572) 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 -- 172.21.15.78:0/190657669 learned_addr learned my addr 172.21.15.78:0/190657669 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c247980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c248400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 -- 172.21.15.78:0/190657669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 msgr2=0x559c6c248400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c248400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 -- 172.21.15.78:0/190657669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 msgr2=0x559c6c247980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c247980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 -- 172.21.15.78:0/190657669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6c2bd420 con 0x559c6c234c80 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdb3fff640 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c247ec0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcda4004f80 tx=0x7fcda40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed851fb3c7de417 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcd937fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcd937fe640 1 -- 172.21.15.78:0/190657669 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4035040 con 0x559c6c234c80 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcd937fe640 1 -- 172.21.15.78:0/190657669 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcda40037b0 con 0x559c6c234c80 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdbccfc740 1 -- 172.21.15.78:0/190657669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda40039b0 con 0x559c6c234c80 2024-09-16T00:21:10.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcd937fe640 1 -- 172.21.15.78:0/190657669 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcda4063050 con 0x559c6c234c80 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdbccfc740 10 client.16818 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.246+0000 7fcdbccfc740 1 -- 172.21.15.78:0/190657669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559c6c2be620 con 0x559c6c234c80 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcd937fe640 1 -- 172.21.15.78:0/190657669 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda40039b0 con 0x559c6c234c80 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcd937fe640 1 -- 172.21.15.78:0/190657669 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcda4004100 con 0x559c6c234c80 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcdbccfc740 10 client.16818 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcdbccfc740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcdbccfc740 20 client.16818 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559c6c2bed00 0x559c6c2c1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcdbccfc740 4 client.16818 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcdbccfc740 1 -- 172.21.15.78:0/190657669 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c6c1a1d60 con 0x559c6c2bed00 2024-09-16T00:21:10.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.247+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559c6c2bed00 0x559c6c2c1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcd937fe640 10 client.16818 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdb37fe640 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559c6c2bed00 0x559c6c2c1120 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22cda11ec8e3d550 server_cookie=d73d6be325fa0789 in_seq=0 out_seq=0 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcd937fe640 1 -- 172.21.15.78:0/190657669 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559c6c1a1d60 con 0x559c6c2bed00 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcd937fe640 10 client.16818 handle_command_reply: tid=0 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 client.16818 shutdown 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 -- 172.21.15.78:0/190657669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 msgr2=0x559c6c247ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c247ec0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcda4004f80 tx=0x7fcda40045e0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 -- 172.21.15.78:0/190657669 shutdown_connections 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559c6c2bed00 0x559c6c2c1120 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c6c243a90 0x559c6c248400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c6c234c80 0x559c6c247ec0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 --2- 172.21.15.78:0/190657669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c6c234360 0x559c6c247980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcd937fe640 0 client.16818 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 -- 172.21.15.78:0/190657669 >> 172.21.15.78:0/190657669 conn(0x559c6c230a30 msgr2=0x559c6c241f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 -- 172.21.15.78:0/190657669 shutdown_connections 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 1 -- 172.21.15.78:0/190657669 wait complete. 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.248+0000 7fcdbccfc740 20 client.16818 trim_cache size 0 max 16384 2024-09-16T00:21:10.257 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:21:10.257 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi078.stdout: "0bfa4937-5556-45b1-9ab7-34a3d5c2adf3": { 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "0bfa4937-5556-45b1-9ab7-34a3d5c2adf3", 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:10.259 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:10.259 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:21:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 -- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd9c0a3340 msgr2=0x7fcd9c0a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd9c0a3340 0x7fcd9c0a5800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0a45f0 tx=0x7fcdac06ed30 comp rx=0 tx=0).stop 2024-09-16T00:21:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 -- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb410c190 msgr2=0x7fcdb419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb410c190 0x7fcdb419cd00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00cad0 tx=0x7fcdac00cf90 comp rx=0 tx=0).stop 2024-09-16T00:21:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 -- 172.21.15.78:0/2495625196 shutdown_connections 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd9c0a3340 0x7fcd9c0a5800 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcdb410c190 0x7fcdb419cd00 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcdb41036d0 0x7fcdb4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 --2- 172.21.15.78:0/2495625196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdb4102db0 0x7fcdb4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 -- 172.21.15.78:0/2495625196 >> 172.21.15.78:0/2495625196 conn(0x7fcdb40fc7d0 msgr2=0x7fcdb40fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 -- 172.21.15.78:0/2495625196 shutdown_connections 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:10.249+0000 7fcdba923640 1 -- 172.21.15.78:0/2495625196 wait complete. 2024-09-16T00:21:10.862 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 430.0 s, 117.7 tps, lat 121.635 ms stddev 125.401 2024-09-16T00:21:11.075 INFO:tasks.cephfs.filesystem:scrub status for tag:0bfa4937-5556-45b1-9ab7-34a3d5c2adf3 - {'path': '/', 'tag': '0bfa4937-5556-45b1-9ab7-34a3d5c2adf3', 'options': 'recursive,force'} 2024-09-16T00:21:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:11 smithi193 ceph-mon[35484]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-09-16T00:21:12.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:11 smithi148 ceph-mon[35599]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-09-16T00:21:12.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:11 smithi078 ceph-mon[31157]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 5.8 MiB/s wr, 563 op/s 2024-09-16T00:21:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:13 smithi193 ceph-mon[35484]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 410 op/s 2024-09-16T00:21:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:13 smithi148 ceph-mon[35599]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 410 op/s 2024-09-16T00:21:14.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:13 smithi078 ceph-mon[31157]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 410 op/s 2024-09-16T00:21:15.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 435.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:21:16.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:15 smithi193 ceph-mon[35484]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-09-16T00:21:16.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:15 smithi148 ceph-mon[35599]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-09-16T00:21:16.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:15 smithi078 ceph-mon[31157]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.4 MiB/s wr, 293 op/s 2024-09-16T00:21:17.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:17.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:17.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:17.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:16 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:21:17.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:17.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:17.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:17.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:16 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:21:17.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:21:17.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:21:17.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:21:17.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:16 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:21:18.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:17 smithi193 ceph-mon[35484]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-09-16T00:21:18.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:17 smithi148 ceph-mon[35599]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-09-16T00:21:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:17 smithi078 ceph-mon[31157]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.8 MiB/s wr, 221 op/s 2024-09-16T00:21:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:19 smithi193 ceph-mon[35484]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-09-16T00:21:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:21:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:21:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:21:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:19 smithi148 ceph-mon[35599]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-09-16T00:21:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:21:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:21:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:21:20.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:19 smithi078 ceph-mon[31157]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-09-16T00:21:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:21:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:21:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:21:20.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 440.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:21:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:21 smithi193 ceph-mon[35484]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-16T00:21:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:21 smithi148 ceph-mon[35599]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-16T00:21:22.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:21 smithi078 ceph-mon[31157]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-16T00:21:22.645 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:21:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:23.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:22 smithi078 ceph-mon[31157]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 510 KiB/s wr, 52 op/s 2024-09-16T00:21:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:22 smithi193 ceph-mon[35484]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 510 KiB/s wr, 52 op/s 2024-09-16T00:21:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:22 smithi148 ceph-mon[35599]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 510 KiB/s wr, 52 op/s 2024-09-16T00:21:23.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.403+0000 7f0221eea640 1 -- 172.21.15.78:0/2474870142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c102780 msgr2=0x7f021c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.403+0000 7f0221eea640 1 --2- 172.21.15.78:0/2474870142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c102780 0x7f021c1067d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0204009900 tx=0x7f020402a410 comp rx=0 tx=0).stop 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 -- 172.21.15.78:0/2474870142 shutdown_connections 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 --2- 172.21.15.78:0/2474870142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c106f00 0x7f021c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 --2- 172.21.15.78:0/2474870142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c102780 0x7f021c1067d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 --2- 172.21.15.78:0/2474870142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f021c101dd0 0x7f021c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 -- 172.21.15.78:0/2474870142 >> 172.21.15.78:0/2474870142 conn(0x7f021c0fd8b0 msgr2=0x7f021c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 -- 172.21.15.78:0/2474870142 shutdown_connections 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 -- 172.21.15.78:0/2474870142 wait complete. 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 Processor -- start 2024-09-16T00:21:23.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 -- start start 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c101dd0 0x7f021c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f021c102780 0x7f021c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c106f00 0x7f021c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f021c111f60 con 0x7f021c101dd0 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f021c111de0 con 0x7f021c106f00 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f0221eea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1120e0 con 0x7f021c102780 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.404+0000 7f021b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c101dd0 0x7f021c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c101dd0 0x7f021c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51528/0 (socket says 172.21.15.78:51528) 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021b7fe640 1 -- 172.21.15.78:0/3752814398 learned_addr learned my addr 172.21.15.78:0/3752814398 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f0213fff640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f021c102780 0x7f021c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:23.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021b7fe640 1 -- 172.21.15.78:0/3752814398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f021c102780 msgr2=0x7f021c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021b7fe640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f021c102780 0x7f021c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021b7fe640 1 -- 172.21.15.78:0/3752814398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c106f00 msgr2=0x7f021c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021bfff640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c106f00 0x7f021c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021b7fe640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c106f00 0x7f021c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021b7fe640 1 -- 172.21.15.78:0/3752814398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c1a6860 con 0x7f021c101dd0 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f0213fff640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f021c102780 0x7f021c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021bfff640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c106f00 0x7f021c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f021b7fe640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c101dd0 0x7f021c0744f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f020c00d770 tx=0x7f020c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f02197fa640 1 -- 172.21.15.78:0/3752814398 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c017070 con 0x7f021c101dd0 2024-09-16T00:21:23.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c1a6b50 con 0x7f021c101dd0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f02197fa640 1 -- 172.21.15.78:0/3752814398 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f020c005500 con 0x7f021c101dd0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f02197fa640 1 -- 172.21.15.78:0/3752814398 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c004e00 con 0x7f021c101dd0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.405+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c1a7090 con 0x7f021c101dd0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.406+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021c1021b0 con 0x7f021c101dd0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.408+0000 7f02197fa640 1 -- 172.21.15.78:0/3752814398 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f020c004fa0 con 0x7f021c101dd0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.409+0000 7f02197fa640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02080a3240 0x7f02080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.409+0000 7f0213fff640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02080a3240 0x7f02080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.409+0000 7f02197fa640 1 -- 172.21.15.78:0/3752814398 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f020c0cf7a0 con 0x7f021c101dd0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.410+0000 7f0213fff640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02080a3240 0x7f02080a5700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f021c075f80 tx=0x7f0204005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:23.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.413+0000 7f02197fa640 1 -- 172.21.15.78:0/3752814398 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f020c09c160 con 0x7f021c101dd0 2024-09-16T00:21:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.623+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f021c10a140 con 0x7f021c101dd0 2024-09-16T00:21:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.623+0000 7f02197fa640 1 -- 172.21.15.78:0/3752814398 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f020c09ba60 con 0x7f021c101dd0 2024-09-16T00:21:23.624 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:21:23.625 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:21:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:21:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.625+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02080a3240 msgr2=0x7f02080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.625+0000 7f0221eea640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02080a3240 0x7f02080a5700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f021c075f80 tx=0x7f0204005a20 comp rx=0 tx=0).stop 2024-09-16T00:21:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.625+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c101dd0 msgr2=0x7f021c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:23.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.625+0000 7f0221eea640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c101dd0 0x7f021c0744f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f020c00d770 tx=0x7f020c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:21:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.626+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 shutdown_connections 2024-09-16T00:21:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.626+0000 7f0221eea640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f02080a3240 0x7f02080a5700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.626+0000 7f0221eea640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f021c106f00 0x7f021c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.626+0000 7f0221eea640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f021c102780 0x7f021c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.626+0000 7f0221eea640 1 --2- 172.21.15.78:0/3752814398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f021c101dd0 0x7f021c0744f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.626+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 >> 172.21.15.78:0/3752814398 conn(0x7f021c0fd8b0 msgr2=0x7f021c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.626+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 shutdown_connections 2024-09-16T00:21:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:23.626+0000 7f0221eea640 1 -- 172.21.15.78:0/3752814398 wait complete. 2024-09-16T00:21:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:23 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3752814398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:23 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3752814398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:24.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:23 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3752814398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:24.415 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 3d8b79c8 --release --await' 2024-09-16T00:21:24.415 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 3d8b79c8 --release --await 2024-09-16T00:21:24.674 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:25.108 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:24 smithi078 ceph-mon[31157]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 398 KiB/s wr, 48 op/s 2024-09-16T00:21:25.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:24 smithi193 ceph-mon[35484]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 398 KiB/s wr, 48 op/s 2024-09-16T00:21:25.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:24 smithi148 ceph-mon[35599]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 398 KiB/s wr, 48 op/s 2024-09-16T00:21:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 -- 172.21.15.78:0/3853940705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 msgr2=0x7f3b44106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 --2- 172.21.15.78:0/3853940705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 0x7f3b44106e60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009930 tx=0x7f3b3802a420 comp rx=0 tx=0).stop 2024-09-16T00:21:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 -- 172.21.15.78:0/3853940705 shutdown_connections 2024-09-16T00:21:25.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 --2- 172.21.15.78:0/3853940705 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b44101420 0x7f3b4410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 --2- 172.21.15.78:0/3853940705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b44100a80 0x7f3b44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 --2- 172.21.15.78:0/3853940705 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 0x7f3b44106e60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 -- 172.21.15.78:0/3853940705 >> 172.21.15.78:0/3853940705 conn(0x7f3b440fc7a0 msgr2=0x7f3b440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 -- 172.21.15.78:0/3853940705 shutdown_connections 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.195+0000 7f3b4af99640 1 -- 172.21.15.78:0/3853940705 wait complete. 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b4af99640 1 Processor -- start 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b4af99640 1 -- start start 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b4af99640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b44100a80 0x7f3b441985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b4af99640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b44101420 0x7f3b44198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b4af99640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 0x7f3b4419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b4af99640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44110f20 con 0x7f3b44106a80 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b4af99640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44110da0 con 0x7f3b44100a80 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b4af99640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441110a0 con 0x7f3b44101420 2024-09-16T00:21:25.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b49536640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 0x7f3b4419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b49536640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 0x7f3b4419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51544/0 (socket says 172.21.15.78:51544) 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b49536640 1 -- 172.21.15.78:0/1788512971 learned_addr learned my addr 172.21.15.78:0/1788512971 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b44100a80 0x7f3b441985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b44101420 0x7f3b44198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b49536640 1 -- 172.21.15.78:0/1788512971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b44101420 msgr2=0x7f3b44198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b49536640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b44101420 0x7f3b44198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b49536640 1 -- 172.21.15.78:0/1788512971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b44100a80 msgr2=0x7f3b441985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b49536640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b44100a80 0x7f3b441985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b49536640 1 -- 172.21.15.78:0/1788512971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4419d550 con 0x7f3b44106a80 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.196+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b44101420 0x7f3b44198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.197+0000 7f3b49536640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 0x7f3b4419ce70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00b970 tx=0x7f3b3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.197+0000 7f3b41ffb640 1 -- 172.21.15.78:0/1788512971 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c019070 con 0x7f3b44106a80 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.197+0000 7f3b41ffb640 1 -- 172.21.15.78:0/1788512971 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3c00cb40 con 0x7f3b44106a80 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.197+0000 7f3b4af99640 1 -- 172.21.15.78:0/1788512971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4419d840 con 0x7f3b44106a80 2024-09-16T00:21:25.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.197+0000 7f3b41ffb640 1 -- 172.21.15.78:0/1788512971 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c007650 con 0x7f3b44106a80 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.197+0000 7f3b4af99640 1 -- 172.21.15.78:0/1788512971 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441a5120 con 0x7f3b44106a80 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.199+0000 7f3b41ffb640 1 -- 172.21.15.78:0/1788512971 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3b3c01c070 con 0x7f3b44106a80 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b4d372740 1 Processor -- start 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b4d372740 1 -- start start 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b41ffb640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3b340a32c0 0x7f3b340a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1eef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1ef9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1efc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b4d372740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5557e1e54620 con 0x5557e1eef060 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b4d372740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5557e1dfbe50 con 0x5557e1efa240 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b4d372740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557e1e505e0 con 0x5557e1eef980 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3b340a32c0 0x7f3b340a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b41ffb640 1 -- 172.21.15.78:0/1788512971 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3b3c0cf1f0 con 0x7f3b44106a80 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b48d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1eef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b48d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1eef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51558/0 (socket says 172.21.15.78:51558) 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b48d35640 1 -- 172.21.15.78:0/3230004569 learned_addr learned my addr 172.21.15.78:0/3230004569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b43fff640 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1ef9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b49536640 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1efc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b48d35640 1 -- 172.21.15.78:0/3230004569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 msgr2=0x5557e1ef9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b48d35640 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1ef9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b48d35640 1 -- 172.21.15.78:0/3230004569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 msgr2=0x5557e1efc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b48d35640 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1efc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b48d35640 1 -- 172.21.15.78:0/3230004569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e1decd40 con 0x5557e1eef060 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b43fff640 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1ef9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b49536640 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1efc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.200+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3b340a32c0 0x7f3b340a5780 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3b44105210 tx=0x7f3b30009d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b48d35640 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1eef440 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3b3802a940 tx=0x7f3b380316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80380a16bd5ebd11 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:25.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b2effd640 1 -- 172.21.15.78:0/3230004569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38040070 con 0x5557e1eef060 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b2effd640 1 -- 172.21.15.78:0/3230004569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b38005020 con 0x5557e1eef060 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b2effd640 1 -- 172.21.15.78:0/3230004569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38035040 con 0x5557e1eef060 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b4d372740 1 -- 172.21.15.78:0/3230004569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 msgr2=0x5557e1eef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b4d372740 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1eef440 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3b3802a940 tx=0x7f3b380316e0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b4d372740 1 -- 172.21.15.78:0/3230004569 shutdown_connections 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b4d372740 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1efc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b4d372740 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1ef9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b4d372740 1 --2- 172.21.15.78:0/3230004569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1eef440 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.201+0000 7f3b4d372740 1 -- 172.21.15.78:0/3230004569 >> 172.21.15.78:0/3230004569 conn(0x5557e1eeb5f0 msgr2=0x5557e1eeb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 -- 172.21.15.78:0/3230004569 shutdown_connections 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 -- 172.21.15.78:0/3230004569 wait complete. 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 -- start start 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 Processor -- start 2024-09-16T00:21:25.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 0x5557e1ef7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1ef7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5557e1f01220 con 0x5557e1eef060 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5557e1ddcf10 con 0x5557e1eef980 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557e1f013a0 con 0x5557e1efa240 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b4d372740 10 client.0 _ll_register_callbacks cb 0x5557e1e00db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b48d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.202+0000 7f3b48d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51570/0 (socket says 172.21.15.78:51570) 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b48d35640 1 -- 172.21.15.78:0/1635173893 learned_addr learned my addr 172.21.15.78:0/1635173893 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b49536640 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1ef7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b48d35640 1 -- 172.21.15.78:0/1635173893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 msgr2=0x5557e1ef7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1ef7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b48d35640 1 -- 172.21.15.78:0/1635173893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 msgr2=0x5557e1ef7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 0x5557e1ef7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b48d35640 1 -- 172.21.15.78:0/1635173893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e1b63580 con 0x5557e1eef060 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef6fd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009900 tx=0x7f3b380314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1511284d5e95a62a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b23fff640 1 -- 172.21.15.78:0/1635173893 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38040070 con 0x5557e1eef060 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b23fff640 1 -- 172.21.15.78:0/1635173893 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3b38004250 con 0x5557e1eef060 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b4d372740 1 -- 172.21.15.78:0/1635173893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b38005020 con 0x5557e1eef060 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b23fff640 1 -- 172.21.15.78:0/1635173893 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3b38056050 con 0x5557e1eef060 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b4d372740 10 client.16836 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b4d372740 1 -- 172.21.15.78:0/1635173893 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5557e1c2d960 con 0x5557e1eef060 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.203+0000 7f3b23fff640 1 -- 172.21.15.78:0/1635173893 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b38005020 con 0x5557e1eef060 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b23fff640 1 -- 172.21.15.78:0/1635173893 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3b380039e0 con 0x5557e1eef060 2024-09-16T00:21:25.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b4d372740 10 client.16836 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b4d372740 10 client.16836 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b4d372740 20 client.16836 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5557e1f0de60 0x5557e1f10250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b4d372740 4 client.16836 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b4d372740 1 -- 172.21.15.78:0/1635173893 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557e1e61b90 con 0x5557e1f0de60 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5557e1f0de60 0x5557e1f10250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5557e1f0de60 0x5557e1f10250 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422293fc4fba0547 server_cookie=ad671b7662de52d0 in_seq=0 out_seq=0 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.204+0000 7f3b23fff640 10 client.16836 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b23fff640 1 -- 172.21.15.78:0/1635173893 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557e1e61b90 con 0x5557e1f0de60 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b23fff640 10 client.16836 handle_command_reply: tid=0 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 client.16836 shutdown 2024-09-16T00:21:25.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 -- 172.21.15.78:0/1635173893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 msgr2=0x5557e1ef6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef6fd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009900 tx=0x7f3b380314b0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 -- 172.21.15.78:0/1635173893 shutdown_connections 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5557e1f0de60 0x5557e1f10250 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1ef7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 0x5557e1ef7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b23fff640 0 client.16836 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1635173893 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef6fd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 -- 172.21.15.78:0/1635173893 >> 172.21.15.78:0/1635173893 conn(0x5557e1eeb5f0 msgr2=0x5557e1ef12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 -- 172.21.15.78:0/1635173893 shutdown_connections 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 1 -- 172.21.15.78:0/1635173893 wait complete. 2024-09-16T00:21:25.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.205+0000 7f3b4d372740 20 client.16836 trim_cache size 0 max 16384 2024-09-16T00:21:25.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.222+0000 7f3b4d372740 1 Processor -- start 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.222+0000 7f3b4d372740 1 -- start start 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.222+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.222+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 0x5557e1f15880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.222+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1f15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b4d372740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441aca90 con 0x5557e1eef060 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b4d372740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441ae5d0 con 0x5557e1eef980 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b4d372740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441ad160 con 0x5557e1efa240 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51572/0 (socket says 172.21.15.78:51572) 2024-09-16T00:21:25.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 -- 172.21.15.78:0/1570969888 learned_addr learned my addr 172.21.15.78:0/1570969888 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 0x5557e1f15880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b49536640 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1f15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 -- 172.21.15.78:0/1570969888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 msgr2=0x5557e1f15dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1f15dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 -- 172.21.15.78:0/1570969888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 msgr2=0x5557e1f15880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 0x5557e1f15880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 -- 172.21.15.78:0/1570969888 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e1eeb420 con 0x5557e1eef060 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 0x5557e1f15880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b49536640 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1f15dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef8ea0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3b38031460 tx=0x7f3b3802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02aca8a9206d18d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b2e7fc640 1 -- 172.21.15.78:0/1570969888 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3805f070 con 0x5557e1eef060 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b2e7fc640 1 -- 172.21.15.78:0/1570969888 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3804d070 con 0x5557e1eef060 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.223+0000 7f3b2e7fc640 1 -- 172.21.15.78:0/1570969888 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38040040 con 0x5557e1eef060 2024-09-16T00:21:25.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 -- 172.21.15.78:0/1570969888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 msgr2=0x5557e1ef8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef8ea0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3b38031460 tx=0x7f3b3802a420 comp rx=0 tx=0).stop 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 -- 172.21.15.78:0/1570969888 shutdown_connections 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1efa240 0x5557e1f15dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1eef980 0x5557e1f15880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1570969888 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1ef8ea0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 -- 172.21.15.78:0/1570969888 >> 172.21.15.78:0/1570969888 conn(0x5557e1eeb5f0 msgr2=0x5557e1eea750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 -- 172.21.15.78:0/1570969888 shutdown_connections 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 -- 172.21.15.78:0/1570969888 wait complete. 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 -- start start 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.224+0000 7f3b4d372740 1 Processor -- start 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1f015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1f01af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b4d372740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1f02030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b4d372740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441adf00 con 0x5557e1eef060 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b4d372740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441ad830 con 0x5557e1efa240 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b4d372740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441aaa70 con 0x5557e1eef980 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b4d372740 10 client.0 _ll_register_callbacks cb 0x7f3b380095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1f015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1f015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51584/0 (socket says 172.21.15.78:51584) 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 -- 172.21.15.78:0/1723271354 learned_addr learned my addr 172.21.15.78:0/1723271354 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b49536640 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1f02030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1f01af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 -- 172.21.15.78:0/1723271354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 msgr2=0x5557e1f01af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1f01af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 -- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 msgr2=0x5557e1f02030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1f02030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 -- 172.21.15.78:0/1723271354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e1f78850 con 0x5557e1eef060 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b49536640 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1f02030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b48d35640 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1f015b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009850 tx=0x7f3b38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ddf53634a64638 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.225+0000 7f3b23fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b23fff640 1 -- 172.21.15.78:0/1723271354 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3805f070 con 0x5557e1eef060 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b23fff640 1 -- 172.21.15.78:0/1723271354 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3b380057f0 con 0x5557e1eef060 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 1 -- 172.21.15.78:0/1723271354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b3804d070 con 0x5557e1eef060 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b23fff640 1 -- 172.21.15.78:0/1723271354 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3b3806d050 con 0x5557e1eef060 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 10 client.16842 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 1 -- 172.21.15.78:0/1723271354 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5557e1f79a10 con 0x5557e1eef060 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b23fff640 1 -- 172.21.15.78:0/1723271354 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3804d070 con 0x5557e1eef060 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b23fff640 1 -- 172.21.15.78:0/1723271354 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3b380042d0 con 0x5557e1eef060 2024-09-16T00:21:25.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 10 client.16842 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 10 client.16842 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:21:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 20 client.16842 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5557e1f7a070 0x5557e1f7c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 4 client.16842 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3d8b79c8", "release": true, "await": true}] 2024-09-16T00:21:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.226+0000 7f3b4d372740 1 -- 172.21.15.78:0/1723271354 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3d8b79c8", "release": true, "await": true}) -- 0x5557e1e5c0a0 con 0x5557e1f7a070 2024-09-16T00:21:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.227+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5557e1f7a070 0x5557e1f7c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.227+0000 7f3b23fff640 10 client.16842 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:25.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.227+0000 7f3b43fff640 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5557e1f7a070 0x5557e1f7c460 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c33c7a6e622ec53 server_cookie=5a74410222e0759b in_seq=0 out_seq=0 2024-09-16T00:21:25.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.266+0000 7f3b23fff640 1 -- 172.21.15.78:0/1723271354 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5557e1e5c0a0 con 0x5557e1f7a070 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.266+0000 7f3b23fff640 10 client.16842 handle_command_reply: tid=0 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 client.16842 shutdown 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 -- 172.21.15.78:0/1723271354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 msgr2=0x5557e1f015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1f015b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009850 tx=0x7f3b38000bc0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 -- 172.21.15.78:0/1723271354 shutdown_connections 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5557e1f7a070 0x5557e1f7c460 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b23fff640 0 client.16842 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5557e1efa240 0x5557e1f02030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557e1eef980 0x5557e1f01af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 --2- 172.21.15.78:0/1723271354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5557e1eef060 0x5557e1f015b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.267+0000 7f3b4d372740 1 -- 172.21.15.78:0/1723271354 >> 172.21.15.78:0/1723271354 conn(0x5557e1eeb5f0 msgr2=0x5557e1ee9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.271+0000 7f3b4d372740 1 -- 172.21.15.78:0/1723271354 shutdown_connections 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.271+0000 7f3b4d372740 1 -- 172.21.15.78:0/1723271354 wait complete. 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.272+0000 7f3b4d372740 20 client.16842 trim_cache size 0 max 16384 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 28, 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:21:25.273 INFO:teuthology.orchestra.run.smithi078.stdout: "3d8b79c8": { 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 28, 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 143.7, 2024-09-16T00:21:25.274 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:21:25.275 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:21:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 -- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3b340a32c0 msgr2=0x7f3b340a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3b340a32c0 0x7f3b340a5780 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3b44105210 tx=0x7f3b30009d40 comp rx=0 tx=0).stop 2024-09-16T00:21:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 -- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 msgr2=0x7f3b4419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 0x7f3b4419ce70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00b970 tx=0x7f3b3c00be30 comp rx=0 tx=0).stop 2024-09-16T00:21:25.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 -- 172.21.15.78:0/1788512971 shutdown_connections 2024-09-16T00:21:25.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3b340a32c0 0x7f3b340a5780 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b44106a80 0x7f3b4419ce70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b44101420 0x7f3b44198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 --2- 172.21.15.78:0/1788512971 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3b44100a80 0x7f3b441985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:25.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.274+0000 7f3b4af99640 1 -- 172.21.15.78:0/1788512971 >> 172.21.15.78:0/1788512971 conn(0x7f3b440fc7a0 msgr2=0x7f3b4406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:25.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.275+0000 7f3b4af99640 1 -- 172.21.15.78:0/1788512971 shutdown_connections 2024-09-16T00:21:25.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:25.279+0000 7f3b4af99640 1 -- 172.21.15.78:0/1788512971 wait complete. 2024-09-16T00:21:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 445.0 s, 129.8 tps, lat 2904.423 ms stddev 6744.394 2024-09-16T00:21:26.116 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3d8b79c8', total seconds elapsed: 27.2 2024-09-16T00:21:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:27 smithi193 ceph-mon[35484]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 416 KiB/s wr, 52 op/s 2024-09-16T00:21:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:27 smithi148 ceph-mon[35599]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 416 KiB/s wr, 52 op/s 2024-09-16T00:21:28.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:27 smithi078 ceph-mon[31157]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 416 KiB/s wr, 52 op/s 2024-09-16T00:21:29.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:29 smithi193 ceph-mon[35484]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 627 KiB/s wr, 79 op/s 2024-09-16T00:21:29.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:29 smithi148 ceph-mon[35599]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 627 KiB/s wr, 79 op/s 2024-09-16T00:21:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:29 smithi078 ceph-mon[31157]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 627 KiB/s wr, 79 op/s 2024-09-16T00:21:30.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 450.0 s, 1297.6 tps, lat 83.544 ms stddev 348.540 2024-09-16T00:21:31.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:31 smithi193 ceph-mon[35484]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-09-16T00:21:31.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:31 smithi148 ceph-mon[35599]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-09-16T00:21:32.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:31 smithi078 ceph-mon[31157]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-09-16T00:21:33.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:33 smithi193 ceph-mon[35484]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-16T00:21:33.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:33 smithi148 ceph-mon[35599]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-16T00:21:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:33 smithi078 ceph-mon[31157]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-16T00:21:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 455.0 s, 993.6 tps, lat 98.635 ms stddev 115.836 2024-09-16T00:21:35.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:35 smithi193 ceph-mon[35484]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-09-16T00:21:35.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:35 smithi148 ceph-mon[35599]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-09-16T00:21:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:35 smithi078 ceph-mon[31157]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.5 MiB/s wr, 356 op/s 2024-09-16T00:21:37.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:37 smithi148 ceph-mon[35599]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 429 op/s 2024-09-16T00:21:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:37 smithi078 ceph-mon[31157]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 429 op/s 2024-09-16T00:21:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:37 smithi193 ceph-mon[35484]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 MiB/s wr, 429 op/s 2024-09-16T00:21:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:39 smithi078 ceph-mon[31157]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 528 op/s 2024-09-16T00:21:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:39 smithi193 ceph-mon[35484]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 528 op/s 2024-09-16T00:21:40.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:39 smithi148 ceph-mon[35599]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 528 op/s 2024-09-16T00:21:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 460.0 s, 786.0 tps, lat 127.681 ms stddev 136.219 2024-09-16T00:21:41.077 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:21:41.551 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:41.941 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:41 smithi078 ceph-mon[31157]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 MiB/s wr, 611 op/s 2024-09-16T00:21:41.981 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:41 smithi193 ceph-mon[35484]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 MiB/s wr, 611 op/s 2024-09-16T00:21:42.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:41 smithi148 ceph-mon[35599]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 MiB/s wr, 611 op/s 2024-09-16T00:21:42.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.408+0000 7fcd74353640 1 -- 172.21.15.78:0/1810650661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c103ff0 msgr2=0x7fcd6c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.408+0000 7fcd74353640 1 --2- 172.21.15.78:0/1810650661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c103ff0 0x7fcd6c1043d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009900 tx=0x7fcd6802a410 comp rx=0 tx=0).stop 2024-09-16T00:21:42.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.420+0000 7fcd74353640 1 -- 172.21.15.78:0/1810650661 shutdown_connections 2024-09-16T00:21:42.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.420+0000 7fcd74353640 1 --2- 172.21.15.78:0/1810650661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd6c109120 0x7fcd6c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.420+0000 7fcd74353640 1 --2- 172.21.15.78:0/1810650661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd6c1049a0 0x7fcd6c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.420+0000 7fcd74353640 1 --2- 172.21.15.78:0/1810650661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c103ff0 0x7fcd6c1043d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.420+0000 7fcd74353640 1 -- 172.21.15.78:0/1810650661 >> 172.21.15.78:0/1810650661 conn(0x7fcd6c0ffab0 msgr2=0x7fcd6c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:42.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.427+0000 7fcd74353640 1 -- 172.21.15.78:0/1810650661 shutdown_connections 2024-09-16T00:21:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.428+0000 7fcd74353640 1 -- 172.21.15.78:0/1810650661 wait complete. 2024-09-16T00:21:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.429+0000 7fcd74353640 1 Processor -- start 2024-09-16T00:21:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.429+0000 7fcd74353640 1 -- start start 2024-09-16T00:21:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.429+0000 7fcd74353640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd6c103ff0 0x7fcd6c1109e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.429+0000 7fcd74353640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd6c1049a0 0x7fcd6c10dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.430+0000 7fcd74353640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c109120 0x7fcd6c10e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.430+0000 7fcd74353640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c114170 con 0x7fcd6c109120 2024-09-16T00:21:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.430+0000 7fcd74353640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c113ff0 con 0x7fcd6c1049a0 2024-09-16T00:21:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.430+0000 7fcd74353640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1142f0 con 0x7fcd6c103ff0 2024-09-16T00:21:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.431+0000 7fcd720ef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd6c103ff0 0x7fcd6c1109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.431+0000 7fcd720ef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd6c103ff0 0x7fcd6c1109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:52226/0 (socket says 172.21.15.78:52226) 2024-09-16T00:21:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.431+0000 7fcd720ef640 1 -- 172.21.15.78:0/1103707070 learned_addr learned my addr 172.21.15.78:0/1103707070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.432+0000 7fcd718ee640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd6c1049a0 0x7fcd6c10dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.435+0000 7fcd728f0640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c109120 0x7fcd6c10e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.440+0000 7fcd720ef640 1 -- 172.21.15.78:0/1103707070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd6c1049a0 msgr2=0x7fcd6c10dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.440+0000 7fcd720ef640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd6c1049a0 0x7fcd6c10dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.440+0000 7fcd720ef640 1 -- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c109120 msgr2=0x7fcd6c10e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.440+0000 7fcd720ef640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c109120 0x7fcd6c10e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.440+0000 7fcd720ef640 1 -- 172.21.15.78:0/1103707070 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6c10e7f0 con 0x7fcd6c103ff0 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.440+0000 7fcd720ef640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd6c103ff0 0x7fcd6c1109e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009850 tx=0x7fcd68030940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.441+0000 7fcd637fe640 1 -- 172.21.15.78:0/1103707070 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6803f070 con 0x7fcd6c103ff0 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.441+0000 7fcd637fe640 1 -- 172.21.15.78:0/1103707070 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd680035f0 con 0x7fcd6c103ff0 2024-09-16T00:21:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.441+0000 7fcd718ee640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd6c1049a0 0x7fcd6c10dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:42.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.442+0000 7fcd728f0640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c109120 0x7fcd6c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:42.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.442+0000 7fcd637fe640 1 -- 172.21.15.78:0/1103707070 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68035030 con 0x7fcd6c103ff0 2024-09-16T00:21:42.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.442+0000 7fcd74353640 1 -- 172.21.15.78:0/1103707070 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6c1a0690 con 0x7fcd6c103ff0 2024-09-16T00:21:42.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.442+0000 7fcd74353640 1 -- 172.21.15.78:0/1103707070 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6c1a0b20 con 0x7fcd6c103ff0 2024-09-16T00:21:42.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.446+0000 7fcd7672c740 1 Processor -- start 2024-09-16T00:21:42.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.449+0000 7fcd637fe640 1 -- 172.21.15.78:0/1103707070 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcd68042070 con 0x7fcd6c103ff0 2024-09-16T00:21:42.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.451+0000 7fcd637fe640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd580a3450 0x7fcd580a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.451+0000 7fcd637fe640 1 -- 172.21.15.78:0/1103707070 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcd680f00c0 con 0x7fcd6c103ff0 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.451+0000 7fcd7672c740 1 -- start start 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a974a0 0x5564d4a97880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4aa2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4aa2680 0x5564d4aa4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd7672c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5564d49fd2e0 con 0x5564d4a974a0 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd7672c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5564d49a4ce0 con 0x5564d4a97dc0 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd7672c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564d49f92a0 con 0x5564d4aa2680 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd718ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4aa2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd718ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4aa2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:45726/0 (socket says 172.21.15.78:45726) 2024-09-16T00:21:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.452+0000 7fcd718ee640 1 -- 172.21.15.78:0/3016933955 learned_addr learned my addr 172.21.15.78:0/3016933955 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.453+0000 7fcd718ee640 1 -- 172.21.15.78:0/3016933955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4aa2680 msgr2=0x5564d4aa4a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:21:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.453+0000 7fcd718ee640 1 --2- 172.21.15.78:0/3016933955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4aa2680 0x5564d4aa4a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.453+0000 7fcd718ee640 1 -- 172.21.15.78:0/3016933955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a974a0 msgr2=0x5564d4a97880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.453+0000 7fcd718ee640 1 --2- 172.21.15.78:0/3016933955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a974a0 0x5564d4a97880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.453+0000 7fcd718ee640 1 -- 172.21.15.78:0/3016933955 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d4989440 con 0x5564d4a97dc0 2024-09-16T00:21:42.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.455+0000 7fcd718ee640 1 --2- 172.21.15.78:0/3016933955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4aa2140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009c20 tx=0x7fcd5c00e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18ff834dfa4562e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:42.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.455+0000 7fcd60dea640 1 -- 172.21.15.78:0/3016933955 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c016070 con 0x5564d4a97dc0 2024-09-16T00:21:42.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.455+0000 7fcd60dea640 1 -- 172.21.15.78:0/3016933955 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5c011950 con 0x5564d4a97dc0 2024-09-16T00:21:42.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.456+0000 7fcd7672c740 1 -- 172.21.15.78:0/3016933955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 msgr2=0x5564d4aa2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.456+0000 7fcd7672c740 1 --2- 172.21.15.78:0/3016933955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4aa2140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009c20 tx=0x7fcd5c00e6c0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.465+0000 7fcd728f0640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd580a3450 0x7fcd580a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.465+0000 7fcd7672c740 1 -- 172.21.15.78:0/3016933955 shutdown_connections 2024-09-16T00:21:42.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.465+0000 7fcd7672c740 1 --2- 172.21.15.78:0/3016933955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4aa2680 0x5564d4aa4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.465+0000 7fcd7672c740 1 --2- 172.21.15.78:0/3016933955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4aa2140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.465+0000 7fcd7672c740 1 --2- 172.21.15.78:0/3016933955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a974a0 0x5564d4a97880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.465+0000 7fcd7672c740 1 -- 172.21.15.78:0/3016933955 >> 172.21.15.78:0/3016933955 conn(0x5564d4a93a30 msgr2=0x5564d4a93e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.469+0000 7fcd7672c740 1 -- 172.21.15.78:0/3016933955 shutdown_connections 2024-09-16T00:21:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.469+0000 7fcd7672c740 1 -- 172.21.15.78:0/3016933955 wait complete. 2024-09-16T00:21:42.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 1 -- start start 2024-09-16T00:21:42.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 1 Processor -- start 2024-09-16T00:21:42.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd728f0640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd580a3450 0x7fcd580a5910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcd640092f0 tx=0x7fcd64009240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4aa1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4aa2680 0x5564d4a9f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5564d4aa52f0 con 0x5564d4aa2680 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5564d49b8050 con 0x5564d4a97dc0 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564d4aa5470 con 0x5564d4a974a0 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.470+0000 7fcd7672c740 10 client.0 _ll_register_callbacks cb 0x5564d49f8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd720ef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4aa1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd720ef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4aa1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:52234/0 (socket says 172.21.15.78:52234) 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd720ef640 1 -- 172.21.15.78:0/2236919932 learned_addr learned my addr 172.21.15.78:0/2236919932 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd718ee640 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd718ee640 1 -- 172.21.15.78:0/2236919932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 msgr2=0x5564d4aa1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:42.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd718ee640 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4aa1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd718ee640 1 -- 172.21.15.78:0/2236919932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4aa2680 msgr2=0x5564d4a9f750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:21:42.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd718ee640 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4aa2680 0x5564d4a9f750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.471+0000 7fcd718ee640 1 -- 172.21.15.78:0/2236919932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d476e3c0 con 0x5564d4a97dc0 2024-09-16T00:21:42.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.474+0000 7fcd718ee640 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9f210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006010 tx=0x7fcd5c009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd94ab7a60bd1728 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:42.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.474+0000 7fcd55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:21:42.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.478+0000 7fcd55ffb640 1 -- 172.21.15.78:0/2236919932 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c016050 con 0x5564d4a97dc0 2024-09-16T00:21:42.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.478+0000 7fcd7672c740 1 -- 172.21.15.78:0/2236919932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5564d4a0a850 con 0x5564d4a97dc0 2024-09-16T00:21:42.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.478+0000 7fcd55ffb640 1 -- 172.21.15.78:0/2236919932 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcd5c004220 con 0x5564d4a97dc0 2024-09-16T00:21:42.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.483+0000 7fcd55ffb640 1 -- 172.21.15.78:0/2236919932 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcd5c004420 con 0x5564d4a97dc0 2024-09-16T00:21:42.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.485+0000 7fcd7672c740 10 client.26237 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:42.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.485+0000 7fcd7672c740 1 -- 172.21.15.78:0/2236919932 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5564d4ab0dd0 con 0x5564d4a97dc0 2024-09-16T00:21:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.489+0000 7fcd55ffb640 1 -- 172.21.15.78:0/2236919932 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5564d4a0a850 con 0x5564d4a97dc0 2024-09-16T00:21:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.489+0000 7fcd55ffb640 1 -- 172.21.15.78:0/2236919932 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcd5c00f040 con 0x5564d4a97dc0 2024-09-16T00:21:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.489+0000 7fcd7672c740 10 client.26237 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.489+0000 7fcd7672c740 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.489+0000 7fcd7672c740 20 client.26237 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.489+0000 7fcd7672c740 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5564d4ab26d0 0x5564d4ab4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.489+0000 7fcd7672c740 4 client.26237 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.489+0000 7fcd7672c740 1 -- 172.21.15.78:0/2236919932 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564d4a04d60 con 0x5564d4ab26d0 2024-09-16T00:21:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.490+0000 7fcd720ef640 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5564d4ab26d0 0x5564d4ab4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.495+0000 7fcd720ef640 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5564d4ab26d0 0x5564d4ab4af0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b396c3e10fd38771 server_cookie=f6ca9fe786697eb3 in_seq=0 out_seq=0 2024-09-16T00:21:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.495+0000 7fcd55ffb640 10 client.26237 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.496+0000 7fcd55ffb640 1 -- 172.21.15.78:0/2236919932 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564d4a04d60 con 0x5564d4ab26d0 2024-09-16T00:21:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.496+0000 7fcd55ffb640 10 client.26237 handle_command_reply: tid=0 2024-09-16T00:21:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.496+0000 7fcd7672c740 1 client.26237 shutdown 2024-09-16T00:21:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.501+0000 7fcd7672c740 1 -- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 msgr2=0x5564d4a9f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.501+0000 7fcd7672c740 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9f210 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006010 tx=0x7fcd5c009990 comp rx=0 tx=0).stop 2024-09-16T00:21:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.505+0000 7fcd7672c740 1 -- 172.21.15.78:0/2236919932 shutdown_connections 2024-09-16T00:21:42.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.505+0000 7fcd7672c740 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5564d4ab26d0 0x5564d4ab4af0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.505+0000 7fcd7672c740 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4aa2680 0x5564d4a9f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.505+0000 7fcd7672c740 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9f210 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.505+0000 7fcd7672c740 1 --2- 172.21.15.78:0/2236919932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4aa1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.505+0000 7fcd7672c740 1 -- 172.21.15.78:0/2236919932 >> 172.21.15.78:0/2236919932 conn(0x5564d4a93a30 msgr2=0x5564d4a99790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:42.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.505+0000 7fcd55ffb640 0 client.26237 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.506+0000 7fcd7672c740 1 -- 172.21.15.78:0/2236919932 shutdown_connections 2024-09-16T00:21:42.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.506+0000 7fcd7672c740 1 -- 172.21.15.78:0/2236919932 wait complete. 2024-09-16T00:21:42.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.509+0000 7fcd7672c740 20 client.26237 trim_cache size 0 max 16384 2024-09-16T00:21:42.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.549+0000 7fcd7672c740 1 Processor -- start 2024-09-16T00:21:42.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.550+0000 7fcd7672c740 1 -- start start 2024-09-16T00:21:42.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.550+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a93a30 0x5564d4aae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.550+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4a9dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.550+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.550+0000 7fcd7672c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1bdb50 con 0x5564d4a93a30 2024-09-16T00:21:42.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.550+0000 7fcd7672c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1bf690 con 0x5564d4a97dc0 2024-09-16T00:21:42.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.550+0000 7fcd7672c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1be220 con 0x5564d4a974a0 2024-09-16T00:21:42.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.551+0000 7fcd728f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.551+0000 7fcd728f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:45756/0 (socket says 172.21.15.78:45756) 2024-09-16T00:21:42.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.551+0000 7fcd728f0640 1 -- 172.21.15.78:0/53936435 learned_addr learned my addr 172.21.15.78:0/53936435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:42.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.551+0000 7fcd718ee640 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4a9dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.551+0000 7fcd720ef640 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a93a30 0x5564d4aae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.551+0000 7fcd728f0640 1 -- 172.21.15.78:0/53936435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 msgr2=0x5564d4a9dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:42.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.551+0000 7fcd728f0640 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4a9dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.551+0000 7fcd728f0640 1 -- 172.21.15.78:0/53936435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a93a30 msgr2=0x5564d4aae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:42.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.552+0000 7fcd728f0640 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a93a30 0x5564d4aae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.552+0000 7fcd728f0640 1 -- 172.21.15.78:0/53936435 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d4a9e1c0 con 0x5564d4a97dc0 2024-09-16T00:21:42.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.552+0000 7fcd728f0640 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9a7f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009fa0 tx=0x7fcd64012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fdb81038879c015 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:42.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.552+0000 7fcd57fff640 1 -- 172.21.15.78:0/53936435 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64022070 con 0x5564d4a97dc0 2024-09-16T00:21:42.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.552+0000 7fcd57fff640 1 -- 172.21.15.78:0/53936435 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd6401eb30 con 0x5564d4a97dc0 2024-09-16T00:21:42.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.552+0000 7fcd57fff640 1 -- 172.21.15.78:0/53936435 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64010ba0 con 0x5564d4a97dc0 2024-09-16T00:21:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.558+0000 7fcd7672c740 1 -- 172.21.15.78:0/53936435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 msgr2=0x5564d4a9a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.558+0000 7fcd7672c740 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9a7f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009fa0 tx=0x7fcd64012000 comp rx=0 tx=0).stop 2024-09-16T00:21:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.558+0000 7fcd7672c740 1 -- 172.21.15.78:0/53936435 shutdown_connections 2024-09-16T00:21:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.558+0000 7fcd7672c740 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a97dc0 0x5564d4a9a7f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.558+0000 7fcd7672c740 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a974a0 0x5564d4a9dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.558+0000 7fcd7672c740 1 --2- 172.21.15.78:0/53936435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a93a30 0x5564d4aae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.558+0000 7fcd7672c740 1 -- 172.21.15.78:0/53936435 >> 172.21.15.78:0/53936435 conn(0x5564d4a9f8b0 msgr2=0x5564d4a9a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.559+0000 7fcd7672c740 1 -- 172.21.15.78:0/53936435 shutdown_connections 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.559+0000 7fcd7672c740 1 -- 172.21.15.78:0/53936435 wait complete. 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.560+0000 7fcd7672c740 1 -- start start 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.560+0000 7fcd7672c740 1 Processor -- start 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.560+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a93a30 0x5564d4b18da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.560+0000 7fcd7672c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a974a0 0x5564d4b15390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.560+0000 7fcd720ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a93a30 0x5564d4b18da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.560+0000 7fcd720ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a93a30 0x5564d4b18da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:45770/0 (socket says 172.21.15.78:45770) 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.560+0000 7fcd720ef640 1 -- 172.21.15.78:0/1696884136 learned_addr learned my addr 172.21.15.78:0/1696884136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.560+0000 7fcd7672c740 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a97dc0 0x5564d4b15980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1befc0 con 0x5564d4a974a0 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd728f0640 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a97dc0 0x5564d4b15980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1be8f0 con 0x5564d4a93a30 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c1bbb30 con 0x5564d4a97dc0 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd7672c740 10 client.0 _ll_register_callbacks cb 0x7fcd5c009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd720ef640 1 -- 172.21.15.78:0/1696884136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a97dc0 msgr2=0x5564d4b15980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd720ef640 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a97dc0 0x5564d4b15980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd720ef640 1 -- 172.21.15.78:0/1696884136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a974a0 msgr2=0x5564d4b15390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd720ef640 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a974a0 0x5564d4b15390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd720ef640 1 -- 172.21.15.78:0/1696884136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d4a97bf0 con 0x5564d4a93a30 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.561+0000 7fcd720ef640 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a93a30 0x5564d4b18da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd680bc2b0 tx=0x7fcd680dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4aa5c15669c255 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:42.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.562+0000 7fcd55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:21:42.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.563+0000 7fcd55ffb640 1 -- 172.21.15.78:0/1696884136 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd680c06d0 con 0x5564d4a93a30 2024-09-16T00:21:42.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.563+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd6401eb30 con 0x5564d4a93a30 2024-09-16T00:21:42.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.563+0000 7fcd55ffb640 1 -- 172.21.15.78:0/1696884136 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcd680ed910 con 0x5564d4a93a30 2024-09-16T00:21:42.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.563+0000 7fcd55ffb640 1 -- 172.21.15.78:0/1696884136 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcd680c4f90 con 0x5564d4a93a30 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd7672c740 10 client.26243 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5564d4aaa6b0 con 0x5564d4a93a30 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd55ffb640 1 -- 172.21.15.78:0/1696884136 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd680d5070 con 0x5564d4a93a30 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd55ffb640 1 -- 172.21.15.78:0/1696884136 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcd680037a0 con 0x5564d4a93a30 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd7672c740 10 client.26243 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd7672c740 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd7672c740 20 client.26243 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd7672c740 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5564d4aa2680 0x5564d4aaae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd7672c740 4 client.26243 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.564+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fcd6401eb30 con 0x5564d4aa2680 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.565+0000 7fcd718ee640 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5564d4aa2680 0x5564d4aaae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.565+0000 7fcd55ffb640 10 client.26243 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:42.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.566+0000 7fcd718ee640 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5564d4aa2680 0x5564d4aaae40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b715e259230613 server_cookie=8219e6c75c96db84 in_seq=0 out_seq=0 2024-09-16T00:21:42.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.566+0000 7fcd55ffb640 1 -- 172.21.15.78:0/1696884136 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fcd6401eb30 con 0x5564d4aa2680 2024-09-16T00:21:42.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.566+0000 7fcd55ffb640 10 client.26243 handle_command_reply: tid=0 2024-09-16T00:21:42.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.566+0000 7fcd7672c740 1 client.26243 shutdown 2024-09-16T00:21:42.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.568+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a93a30 msgr2=0x5564d4b18da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.568+0000 7fcd7672c740 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a93a30 0x5564d4b18da0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd680bc2b0 tx=0x7fcd680dc040 comp rx=0 tx=0).stop 2024-09-16T00:21:42.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.568+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 shutdown_connections 2024-09-16T00:21:42.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.568+0000 7fcd7672c740 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5564d4aa2680 0x5564d4aaae40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.569+0000 7fcd7672c740 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564d4a97dc0 0x5564d4b15980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.569+0000 7fcd7672c740 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5564d4a974a0 0x5564d4b15390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.569+0000 7fcd7672c740 1 --2- 172.21.15.78:0/1696884136 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5564d4a93a30 0x5564d4b18da0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.569+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 >> 172.21.15.78:0/1696884136 conn(0x5564d4a9f8b0 msgr2=0x5564d4aa0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.569+0000 7fcd55ffb640 0 client.26243 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.569+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 shutdown_connections 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.569+0000 7fcd7672c740 1 -- 172.21.15.78:0/1696884136 wait complete. 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.569+0000 7fcd7672c740 20 client.26243 trim_cache size 0 max 16384 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:21:42.572 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:21:42.573 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.571+0000 7fcd74353640 1 -- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd580a3450 msgr2=0x7fcd580a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.571+0000 7fcd74353640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd580a3450 0x7fcd580a5910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcd640092f0 tx=0x7fcd64009240 comp rx=0 tx=0).stop 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.571+0000 7fcd74353640 1 -- 172.21.15.78:0/1103707070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd6c103ff0 msgr2=0x7fcd6c1109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.571+0000 7fcd74353640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd6c103ff0 0x7fcd6c1109e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009850 tx=0x7fcd68030940 comp rx=0 tx=0).stop 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.571+0000 7fcd74353640 1 -- 172.21.15.78:0/1103707070 shutdown_connections 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.571+0000 7fcd74353640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcd580a3450 0x7fcd580a5910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.571+0000 7fcd74353640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd6c109120 0x7fcd6c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.572+0000 7fcd74353640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd6c1049a0 0x7fcd6c10dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.572+0000 7fcd74353640 1 --2- 172.21.15.78:0/1103707070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd6c103ff0 0x7fcd6c1109e0 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009850 tx=0x7fcd68030940 comp rx=0 tx=0).stop 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.572+0000 7fcd74353640 1 -- 172.21.15.78:0/1103707070 >> 172.21.15.78:0/1103707070 conn(0x7fcd6c0ffab0 msgr2=0x7fcd6c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.575+0000 7fcd74353640 1 -- 172.21.15.78:0/1103707070 shutdown_connections 2024-09-16T00:21:42.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:42.575+0000 7fcd74353640 1 -- 172.21.15.78:0/1103707070 wait complete. 2024-09-16T00:21:43.378 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:21:43.378 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:21:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:44.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:43 smithi078 ceph-mon[31157]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-09-16T00:21:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:43 smithi193 ceph-mon[35484]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-09-16T00:21:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:43 smithi148 ceph-mon[35599]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-09-16T00:21:44.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.412+0000 7f60f2851640 1 -- 172.21.15.78:0/2739235472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec1008c0 msgr2=0x7f60ec100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:44.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.412+0000 7f60f2851640 1 --2- 172.21.15.78:0/2739235472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec1008c0 0x7f60ec100ca0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f60dc01a910 tx=0x7f60dc03b470 comp rx=0 tx=0).stop 2024-09-16T00:21:44.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.414+0000 7f60f2851640 1 -- 172.21.15.78:0/2739235472 shutdown_connections 2024-09-16T00:21:44.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.414+0000 7f60f2851640 1 --2- 172.21.15.78:0/2739235472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec105ec0 0x7f60ec106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.414+0000 7f60f2851640 1 --2- 172.21.15.78:0/2739235472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ec1011e0 0x7f60ec105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.414+0000 7f60f2851640 1 --2- 172.21.15.78:0/2739235472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec1008c0 0x7f60ec100ca0 secure :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f60dc01a910 tx=0x7f60dc03b470 comp rx=0 tx=0).stop 2024-09-16T00:21:44.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.414+0000 7f60f2851640 1 -- 172.21.15.78:0/2739235472 >> 172.21.15.78:0/2739235472 conn(0x7f60ec0fc7d0 msgr2=0x7f60ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.415+0000 7f60f2851640 1 -- 172.21.15.78:0/2739235472 shutdown_connections 2024-09-16T00:21:44.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.418+0000 7f60f2851640 1 -- 172.21.15.78:0/2739235472 wait complete. 2024-09-16T00:21:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.418+0000 7f60f2851640 1 Processor -- start 2024-09-16T00:21:44.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.418+0000 7f60f2851640 1 -- start start 2024-09-16T00:21:44.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.419+0000 7f60f2851640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec1011e0 0x7f60ec115050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.419+0000 7f60f2851640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ec105ec0 0x7f60ec1110f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.419+0000 7f60f2851640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec111630 0x7f60ec111ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.419+0000 7f60f2851640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec109980 con 0x7f60ec1011e0 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.419+0000 7f60f2851640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec109800 con 0x7f60ec111630 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.419+0000 7f60f2851640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec109b00 con 0x7f60ec105ec0 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60eb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ec105ec0 0x7f60ec1110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60eb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ec105ec0 0x7f60ec1110f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:52246/0 (socket says 172.21.15.78:52246) 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60eb7fe640 1 -- 172.21.15.78:0/2264616186 learned_addr learned my addr 172.21.15.78:0/2264616186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60f0dee640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec111630 0x7f60ec111ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:44.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60ebfff640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec1011e0 0x7f60ec115050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:44.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60eb7fe640 1 -- 172.21.15.78:0/2264616186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec111630 msgr2=0x7f60ec111ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:44.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60eb7fe640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec111630 0x7f60ec111ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60eb7fe640 1 -- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec1011e0 msgr2=0x7f60ec115050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60eb7fe640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec1011e0 0x7f60ec115050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.422+0000 7f60eb7fe640 1 -- 172.21.15.78:0/2264616186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec06bd40 con 0x7f60ec105ec0 2024-09-16T00:21:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.423+0000 7f60eb7fe640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ec105ec0 0x7f60ec1110f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60d400b450 tx=0x7f60d400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.423+0000 7f60e97fa640 1 -- 172.21.15.78:0/2264616186 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4015020 con 0x7f60ec105ec0 2024-09-16T00:21:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.423+0000 7f60e97fa640 1 -- 172.21.15.78:0/2264616186 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4011d70 con 0x7f60ec105ec0 2024-09-16T00:21:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.423+0000 7f60e97fa640 1 -- 172.21.15.78:0/2264616186 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d400f340 con 0x7f60ec105ec0 2024-09-16T00:21:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.424+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec06c030 con 0x7f60ec105ec0 2024-09-16T00:21:44.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.424+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec06c520 con 0x7f60ec105ec0 2024-09-16T00:21:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.427+0000 7f60e97fa640 1 -- 172.21.15.78:0/2264616186 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f60d4018020 con 0x7f60ec105ec0 2024-09-16T00:21:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.428+0000 7f60e97fa640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f60d80a3450 0x7f60d80a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:44.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.429+0000 7f60e97fa640 1 -- 172.21.15.78:0/2264616186 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f60d40cf6e0 con 0x7f60ec105ec0 2024-09-16T00:21:44.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.430+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60cc005290 con 0x7f60ec105ec0 2024-09-16T00:21:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.435+0000 7f60ebfff640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f60d80a3450 0x7f60d80a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:44.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.436+0000 7f60ebfff640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f60d80a3450 0x7f60d80a5910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f60dc01a860 tx=0x7f60dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.441+0000 7f60e97fa640 1 -- 172.21.15.78:0/2264616186 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60d401f020 con 0x7f60ec105ec0 2024-09-16T00:21:44.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.808+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60cc005580 con 0x7f60ec105ec0 2024-09-16T00:21:44.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.812+0000 7f60e97fa640 1 -- 172.21.15.78:0/2264616186 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f60d400d040 con 0x7f60ec105ec0 2024-09-16T00:21:44.815 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:21:44.815 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:21:44.816 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:21:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.819+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f60d80a3450 msgr2=0x7f60d80a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.819+0000 7f60f2851640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f60d80a3450 0x7f60d80a5910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f60dc01a860 tx=0x7f60dc000bc0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.819+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ec105ec0 msgr2=0x7f60ec1110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.819+0000 7f60f2851640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ec105ec0 0x7f60ec1110f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60d400b450 tx=0x7f60d400b910 comp rx=0 tx=0).stop 2024-09-16T00:21:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.820+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 shutdown_connections 2024-09-16T00:21:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.820+0000 7f60f2851640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f60d80a3450 0x7f60d80a5910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.820+0000 7f60f2851640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f60ec111630 0x7f60ec111ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.820+0000 7f60f2851640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60ec105ec0 0x7f60ec1110f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.820+0000 7f60f2851640 1 --2- 172.21.15.78:0/2264616186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec1011e0 0x7f60ec115050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.820+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 >> 172.21.15.78:0/2264616186 conn(0x7f60ec0fc7d0 msgr2=0x7f60ec103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.821+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 shutdown_connections 2024-09-16T00:21:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:44.821+0000 7f60f2851640 1 -- 172.21.15.78:0/2264616186 wait complete. 2024-09-16T00:21:45.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 465.0 s, 814.7 tps, lat 121.607 ms stddev 132.245 2024-09-16T00:21:46.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:21:46.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:45 smithi078 ceph-mon[31157]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 559 op/s 2024-09-16T00:21:46.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:45 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2264616186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:45 smithi193 ceph-mon[35484]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 559 op/s 2024-09-16T00:21:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:45 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2264616186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:45 smithi148 ceph-mon[35599]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 MiB/s wr, 559 op/s 2024-09-16T00:21:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:45 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2264616186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:21:46.496 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.766+0000 7fc759141640 1 -- 172.21.15.78:0/1520209720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754101280 msgr2=0x7fc754105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.766+0000 7fc759141640 1 --2- 172.21.15.78:0/1520209720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754101280 0x7fc754105750 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009900 tx=0x7fc73c02a490 comp rx=0 tx=0).stop 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.770+0000 7fc759141640 1 -- 172.21.15.78:0/1520209720 shutdown_connections 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.770+0000 7fc759141640 1 --2- 172.21.15.78:0/1520209720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc754105d70 0x7fc754106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.770+0000 7fc759141640 1 --2- 172.21.15.78:0/1520209720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754101280 0x7fc754105750 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.770+0000 7fc759141640 1 --2- 172.21.15.78:0/1520209720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc754100960 0x7fc754100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.770+0000 7fc759141640 1 -- 172.21.15.78:0/1520209720 >> 172.21.15.78:0/1520209720 conn(0x7fc7540fc5d0 msgr2=0x7fc7540fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.772+0000 7fc759141640 1 -- 172.21.15.78:0/1520209720 shutdown_connections 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.773+0000 7fc759141640 1 -- 172.21.15.78:0/1520209720 wait complete. 2024-09-16T00:21:47.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.774+0000 7fc759141640 1 Processor -- start 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.774+0000 7fc759141640 1 -- start start 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.774+0000 7fc759141640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754100960 0x7fc754073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.774+0000 7fc759141640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc754101280 0x7fc75406cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc759141640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc754105d70 0x7fc75406d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc753fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754100960 0x7fc754073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc753fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754100960 0x7fc754073c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49326/0 (socket says 172.21.15.78:49326) 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc753fff640 1 -- 172.21.15.78:0/1320538960 learned_addr learned my addr 172.21.15.78:0/1320538960 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc754118460 con 0x7fc754105d70 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541182e0 con 0x7fc754101280 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541185e0 con 0x7fc754100960 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc74bfff640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc754101280 0x7fc75406cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc758940640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc754105d70 0x7fc75406d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc753fff640 1 -- 172.21.15.78:0/1320538960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc754101280 msgr2=0x7fc75406cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc753fff640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc754101280 0x7fc75406cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc753fff640 1 -- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc754105d70 msgr2=0x7fc75406d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc753fff640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc754105d70 0x7fc75406d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.775+0000 7fc753fff640 1 -- 172.21.15.78:0/1320538960 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75406d840 con 0x7fc754100960 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.776+0000 7fc758940640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc754105d70 0x7fc75406d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.776+0000 7fc753fff640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754100960 0x7fc754073c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc74400d950 tx=0x7fc74400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.776+0000 7fc751ffb640 1 -- 172.21.15.78:0/1320538960 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744017070 con 0x7fc754100960 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.776+0000 7fc751ffb640 1 -- 172.21.15.78:0/1320538960 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc744009c30 con 0x7fc754100960 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.776+0000 7fc751ffb640 1 -- 172.21.15.78:0/1320538960 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74400f690 con 0x7fc754100960 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.779+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75406db30 con 0x7fc754100960 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.779+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc754071a80 con 0x7fc754100960 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.783+0000 7fc751ffb640 1 -- 172.21.15.78:0/1320538960 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc7440109d0 con 0x7fc754100960 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.784+0000 7fc75c77c740 1 Processor -- start 2024-09-16T00:21:47.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc75c77c740 1 -- start start 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41d0cf0 0x5624b41cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41da520 0x5624b41dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc75c77c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624b41352f0 con 0x5624b41da520 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc75c77c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5624b40dccf0 con 0x5624b41d0cf0 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc75c77c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5624b41312b0 con 0x5624b41cf780 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41da520 0x5624b41dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.785+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41da520 0x5624b41dc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48154/0 (socket says 172.21.15.78:48154) 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.786+0000 7fc74bfff640 1 -- 172.21.15.78:0/2418247276 learned_addr learned my addr 172.21.15.78:0/2418247276 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.786+0000 7fc758940640 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41d0cf0 0x5624b41cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.786+0000 7fc753fff640 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.786+0000 7fc751ffb640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7400a3380 0x7fc7400a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.786+0000 7fc751ffb640 1 -- 172.21.15.78:0/1320538960 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc7440cf600 con 0x7fc754100960 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.787+0000 7fc758940640 1 -- 172.21.15.78:0/2418247276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 msgr2=0x5624b41cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.787+0000 7fc758940640 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.787+0000 7fc758940640 1 -- 172.21.15.78:0/2418247276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41da520 msgr2=0x5624b41dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.787+0000 7fc758940640 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41da520 0x5624b41dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.787+0000 7fc758940640 1 -- 172.21.15.78:0/2418247276 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b40c13a0 con 0x5624b41d0cf0 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.787+0000 7fc758940640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7400a3380 0x7fc7400a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.787+0000 7fc758940640 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41d0cf0 0x5624b41cf240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc74c007e50 tx=0x7fc74c00c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd59e1d85a907536 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.791+0000 7fc74a7fc640 1 -- 172.21.15.78:0/2418247276 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c00f7e0 con 0x5624b41d0cf0 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.791+0000 7fc74a7fc640 1 -- 172.21.15.78:0/2418247276 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74c01c470 con 0x5624b41d0cf0 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.791+0000 7fc74a7fc640 1 -- 172.21.15.78:0/2418247276 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c01b630 con 0x5624b41d0cf0 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.792+0000 7fc758940640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7400a3380 0x7fc7400a5840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc74c01b000 tx=0x7fc74c01b870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.793+0000 7fc75c77c740 1 -- 172.21.15.78:0/2418247276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41d0cf0 msgr2=0x5624b41cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.793+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41d0cf0 0x5624b41cf240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc74c007e50 tx=0x7fc74c00c670 comp rx=0 tx=0).stop 2024-09-16T00:21:47.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.793+0000 7fc75c77c740 1 -- 172.21.15.78:0/2418247276 shutdown_connections 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.793+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41da520 0x5624b41dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.793+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.793+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2418247276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41d0cf0 0x5624b41cf240 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.793+0000 7fc75c77c740 1 -- 172.21.15.78:0/2418247276 >> 172.21.15.78:0/2418247276 conn(0x5624b4198310 msgr2=0x5624b41cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.793+0000 7fc75c77c740 1 -- 172.21.15.78:0/2418247276 shutdown_connections 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.794+0000 7fc75c77c740 1 -- 172.21.15.78:0/2418247276 wait complete. 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.794+0000 7fc75c77c740 1 -- start start 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.794+0000 7fc75c77c740 1 Processor -- start 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41d7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc75c77c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5624b41dd220 con 0x5624b41d0cf0 2024-09-16T00:21:47.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc75c77c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5624b40f0060 con 0x5624b41da520 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc75c77c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5624b41dd3a0 con 0x5624b41cf780 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48160/0 (socket says 172.21.15.78:48160) 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc74bfff640 1 -- 172.21.15.78:0/3765792746 learned_addr learned my addr 172.21.15.78:0/3765792746 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc75c77c740 10 client.0 _ll_register_callbacks cb 0x5624b3e438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.795+0000 7fc753fff640 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc758940640 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41d7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc74bfff640 1 -- 172.21.15.78:0/3765792746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 msgr2=0x5624b41d9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:47.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc74bfff640 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41d9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc74bfff640 1 -- 172.21.15.78:0/3765792746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 msgr2=0x5624b41d7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:47.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc74bfff640 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41d7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc74bfff640 1 -- 172.21.15.78:0/3765792746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b3ea63c0 con 0x5624b41d0cf0 2024-09-16T00:21:47.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc74bfff640 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d71f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009900 tx=0x7fc73c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9383246ebfc3c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:47.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc7337fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:47.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc7337fe640 1 -- 172.21.15.78:0/3765792746 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c035040 con 0x5624b41d0cf0 2024-09-16T00:21:47.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc7337fe640 1 -- 172.21.15.78:0/3765792746 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc73c005220 con 0x5624b41d0cf0 2024-09-16T00:21:47.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.796+0000 7fc75c77c740 1 -- 172.21.15.78:0/3765792746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc74c01c470 con 0x5624b41d0cf0 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc7337fe640 1 -- 172.21.15.78:0/3765792746 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc73c005420 con 0x5624b41d0cf0 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc75c77c740 10 client.16863 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc75c77c740 1 -- 172.21.15.78:0/3765792746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5624b41e8d20 con 0x5624b41d0cf0 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc7337fe640 1 -- 172.21.15.78:0/3765792746 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc74c01c470 con 0x5624b41d0cf0 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc7337fe640 1 -- 172.21.15.78:0/3765792746 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc73c003a00 con 0x5624b41d0cf0 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc75c77c740 10 client.16863 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc75c77c740 10 client.16863 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc75c77c740 20 client.16863 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc75c77c740 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5624b41ea1e0 0x5624b41ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc75c77c740 4 client.16863 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc75c77c740 1 -- 172.21.15.78:0/3765792746 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624b4142860 con 0x5624b41ea1e0 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.797+0000 7fc753fff640 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5624b41ea1e0 0x5624b41ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.798+0000 7fc753fff640 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5624b41ea1e0 0x5624b41ec630 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad7c9f806ba79bca server_cookie=849380834d688266 in_seq=0 out_seq=0 2024-09-16T00:21:47.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.798+0000 7fc7337fe640 10 client.16863 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.798+0000 7fc7337fe640 1 -- 172.21.15.78:0/3765792746 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624b4142860 con 0x5624b41ea1e0 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.798+0000 7fc7337fe640 10 client.16863 handle_command_reply: tid=0 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.798+0000 7fc75c77c740 1 client.16863 shutdown 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc75c77c740 1 -- 172.21.15.78:0/3765792746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 msgr2=0x5624b41d71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc75c77c740 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d71f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009900 tx=0x7fc73c004440 comp rx=0 tx=0).stop 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc75c77c740 1 -- 172.21.15.78:0/3765792746 shutdown_connections 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc75c77c740 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41d7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc75c77c740 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d71f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc75c77c740 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5624b41ea1e0 0x5624b41ec630 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc7337fe640 0 client.16863 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc75c77c740 1 --2- 172.21.15.78:0/3765792746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.799+0000 7fc75c77c740 1 -- 172.21.15.78:0/3765792746 >> 172.21.15.78:0/3765792746 conn(0x5624b4198310 msgr2=0x5624b41d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.800+0000 7fc75c77c740 1 -- 172.21.15.78:0/3765792746 shutdown_connections 2024-09-16T00:21:47.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.801+0000 7fc75c77c740 1 -- 172.21.15.78:0/3765792746 wait complete. 2024-09-16T00:21:47.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.801+0000 7fc75c77c740 20 client.16863 trim_cache size 0 max 16384 2024-09-16T00:21:47.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.825+0000 7fc75c77c740 1 Processor -- start 2024-09-16T00:21:47.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.829+0000 7fc75c77c740 1 -- start start 2024-09-16T00:21:47.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.829+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.829+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.829+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.829+0000 7fc75c77c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541bd8a0 con 0x5624b41d0cf0 2024-09-16T00:21:47.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.829+0000 7fc75c77c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541bf3e0 con 0x5624b41da520 2024-09-16T00:21:47.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.829+0000 7fc75c77c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541bdf70 con 0x5624b41cf780 2024-09-16T00:21:47.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.830+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.831+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48172/0 (socket says 172.21.15.78:48172) 2024-09-16T00:21:47.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.831+0000 7fc74bfff640 1 -- 172.21.15.78:0/4019684957 learned_addr learned my addr 172.21.15.78:0/4019684957 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:47.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.831+0000 7fc753fff640 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.833+0000 7fc758940640 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.833+0000 7fc74bfff640 1 -- 172.21.15.78:0/4019684957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 msgr2=0x5624b41e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.833+0000 7fc74bfff640 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41e64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.833+0000 7fc74bfff640 1 -- 172.21.15.78:0/4019684957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 msgr2=0x5624b41f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.833+0000 7fc74bfff640 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.833+0000 7fc74bfff640 1 -- 172.21.15.78:0/4019684957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624b41d22d0 con 0x5624b41d0cf0 2024-09-16T00:21:47.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.837+0000 7fc74bfff640 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d9160 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc73c003df0 tx=0x7fc73c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789314ed9e3b9ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:47.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.838+0000 7fc749ffb640 1 -- 172.21.15.78:0/4019684957 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c058070 con 0x5624b41d0cf0 2024-09-16T00:21:47.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.838+0000 7fc749ffb640 1 -- 172.21.15.78:0/4019684957 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc73c031600 con 0x5624b41d0cf0 2024-09-16T00:21:47.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.838+0000 7fc749ffb640 1 -- 172.21.15.78:0/4019684957 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c035030 con 0x5624b41d0cf0 2024-09-16T00:21:47.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.840+0000 7fc75c77c740 1 -- 172.21.15.78:0/4019684957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 msgr2=0x5624b41d9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.840+0000 7fc75c77c740 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d9160 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc73c003df0 tx=0x7fc73c009290 comp rx=0 tx=0).stop 2024-09-16T00:21:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.842+0000 7fc75c77c740 1 -- 172.21.15.78:0/4019684957 shutdown_connections 2024-09-16T00:21:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.843+0000 7fc75c77c740 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.843+0000 7fc75c77c740 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41d0cf0 0x5624b41d9160 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.843+0000 7fc75c77c740 1 --2- 172.21.15.78:0/4019684957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41cf780 0x5624b41e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.843+0000 7fc75c77c740 1 -- 172.21.15.78:0/4019684957 >> 172.21.15.78:0/4019684957 conn(0x5624b4198310 msgr2=0x5624b41d8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:47.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.848+0000 7fc75c77c740 1 -- 172.21.15.78:0/4019684957 shutdown_connections 2024-09-16T00:21:47.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.848+0000 7fc75c77c740 1 -- 172.21.15.78:0/4019684957 wait complete. 2024-09-16T00:21:47.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.849+0000 7fc75c77c740 1 -- start start 2024-09-16T00:21:47.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.849+0000 7fc75c77c740 1 Processor -- start 2024-09-16T00:21:47.853 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:47 smithi078 ceph-mon[31157]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41cf780 0x5624b41e2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41d0cf0 0x5624b41ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc75c77c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41de500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc75c77c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541bed10 con 0x5624b41cf780 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc75c77c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541be640 con 0x5624b41da520 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc75c77c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7541bb880 con 0x5624b41d0cf0 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc75c77c740 10 client.0 _ll_register_callbacks cb 0x7fc73c0040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41d0cf0 0x5624b41ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41d0cf0 0x5624b41ddfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49356/0 (socket says 172.21.15.78:49356) 2024-09-16T00:21:47.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc74bfff640 1 -- 172.21.15.78:0/2172546299 learned_addr learned my addr 172.21.15.78:0/2172546299 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:47.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.852+0000 7fc758940640 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.853+0000 7fc758940640 1 -- 172.21.15.78:0/2172546299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41d0cf0 msgr2=0x5624b41ddfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:47.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.853+0000 7fc758940640 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41d0cf0 0x5624b41ddfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.853+0000 7fc758940640 1 -- 172.21.15.78:0/2172546299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41cf780 msgr2=0x5624b41e2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:47.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.853+0000 7fc758940640 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41cf780 0x5624b41e2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.853+0000 7fc758940640 1 -- 172.21.15.78:0/2172546299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624b41ded30 con 0x5624b41da520 2024-09-16T00:21:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.861+0000 7fc758940640 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41de500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00cb70 tx=0x7fc74c012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab6bca7db5d2fad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.862+0000 7fc7337fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:21:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.864+0000 7fc7337fe640 1 -- 172.21.15.78:0/2172546299 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74c01bd00 con 0x5624b41da520 2024-09-16T00:21:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.864+0000 7fc7337fe640 1 -- 172.21.15.78:0/2172546299 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc74c01b3d0 con 0x5624b41da520 2024-09-16T00:21:47.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.864+0000 7fc75c77c740 1 -- 172.21.15.78:0/2172546299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc73c031600 con 0x5624b41da520 2024-09-16T00:21:47.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.874+0000 7fc7337fe640 1 -- 172.21.15.78:0/2172546299 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc74c039050 con 0x5624b41da520 2024-09-16T00:21:47.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.874+0000 7fc75c77c740 10 client.26258 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:47.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.874+0000 7fc75c77c740 1 -- 172.21.15.78:0/2172546299 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5624b4254de0 con 0x5624b41da520 2024-09-16T00:21:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.876+0000 7fc7337fe640 1 -- 172.21.15.78:0/2172546299 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc73c031600 con 0x5624b41da520 2024-09-16T00:21:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.876+0000 7fc7337fe640 1 -- 172.21.15.78:0/2172546299 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc74c00f840 con 0x5624b41da520 2024-09-16T00:21:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.876+0000 7fc75c77c740 10 client.26258 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.876+0000 7fc75c77c740 10 client.26258 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:21:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.876+0000 7fc75c77c740 20 client.26258 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.876+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5624b41ea1e0 0x5624b42573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.876+0000 7fc75c77c740 4 client.26258 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:21:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.876+0000 7fc75c77c740 1 -- 172.21.15.78:0/2172546299 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624b413cd70 con 0x5624b41ea1e0 2024-09-16T00:21:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.877+0000 7fc753fff640 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5624b41ea1e0 0x5624b42573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:47.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.882+0000 7fc7337fe640 10 client.26258 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:21:47.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.882+0000 7fc753fff640 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5624b41ea1e0 0x5624b42573f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a4b01c75b97e661 server_cookie=afb4ae4bbf6dc5be in_seq=0 out_seq=0 2024-09-16T00:21:47.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.883+0000 7fc7337fe640 1 -- 172.21.15.78:0/2172546299 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624b413cd70 con 0x5624b41ea1e0 2024-09-16T00:21:47.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.883+0000 7fc7337fe640 10 client.26258 handle_command_reply: tid=0 2024-09-16T00:21:47.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.883+0000 7fc75c77c740 1 client.26258 shutdown 2024-09-16T00:21:47.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc75c77c740 1 -- 172.21.15.78:0/2172546299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 msgr2=0x5624b41de500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41de500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00cb70 tx=0x7fc74c012000 comp rx=0 tx=0).stop 2024-09-16T00:21:47.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc75c77c740 1 -- 172.21.15.78:0/2172546299 shutdown_connections 2024-09-16T00:21:47.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5624b41da520 0x5624b41de500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5624b41ea1e0 0x5624b42573f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5624b41d0cf0 0x5624b41ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc75c77c740 1 --2- 172.21.15.78:0/2172546299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5624b41cf780 0x5624b41e2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc75c77c740 1 -- 172.21.15.78:0/2172546299 >> 172.21.15.78:0/2172546299 conn(0x5624b4198310 msgr2=0x5624b41d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:47.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.884+0000 7fc7337fe640 0 client.26258 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:21:47.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.887+0000 7fc75c77c740 1 -- 172.21.15.78:0/2172546299 shutdown_connections 2024-09-16T00:21:47.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.887+0000 7fc75c77c740 1 -- 172.21.15.78:0/2172546299 wait complete. 2024-09-16T00:21:47.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.887+0000 7fc75c77c740 20 client.26258 trim_cache size 0 max 16384 2024-09-16T00:21:47.888 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:21:47.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.891+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7400a3380 msgr2=0x7fc7400a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7400a3380 0x7fc7400a5840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc74c01b000 tx=0x7fc74c01b870 comp rx=0 tx=0).stop 2024-09-16T00:21:47.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754100960 msgr2=0x7fc754073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:47.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754100960 0x7fc754073c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc74400d950 tx=0x7fc74400de10 comp rx=0 tx=0).stop 2024-09-16T00:21:47.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 shutdown_connections 2024-09-16T00:21:47.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc7400a3380 0x7fc7400a5840 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc754105d70 0x7fc75406d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc754101280 0x7fc75406cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 --2- 172.21.15.78:0/1320538960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc754100960 0x7fc754073c60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:47.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.892+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 >> 172.21.15.78:0/1320538960 conn(0x7fc7540fc5d0 msgr2=0x7fc7540fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:47.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.894+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 shutdown_connections 2024-09-16T00:21:47.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:47.894+0000 7fc759141640 1 -- 172.21.15.78:0/1320538960 wait complete. 2024-09-16T00:21:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:47 smithi193 ceph-mon[35484]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-09-16T00:21:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:47 smithi148 ceph-mon[35599]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.4 MiB/s wr, 552 op/s 2024-09-16T00:21:48.841 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:21:49.287 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:49.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.928+0000 7f24570d2640 1 -- 172.21.15.78:0/718724436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245011b140 msgr2=0x7f245011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:49.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.928+0000 7f24570d2640 1 --2- 172.21.15.78:0/718724436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245011b140 0x7f245011d530 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2438009900 tx=0x7f243802a410 comp rx=0 tx=0).stop 2024-09-16T00:21:49.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.933+0000 7f24570d2640 1 -- 172.21.15.78:0/718724436 shutdown_connections 2024-09-16T00:21:49.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.933+0000 7f24570d2640 1 --2- 172.21.15.78:0/718724436 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f245011b140 0x7f245011d530 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.933+0000 7f24570d2640 1 --2- 172.21.15.78:0/718724436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2450109830 0x7f2450109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.933+0000 7f24570d2640 1 --2- 172.21.15.78:0/718724436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2450109450 0x7f2450110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.933+0000 7f24570d2640 1 -- 172.21.15.78:0/718724436 >> 172.21.15.78:0/718724436 conn(0x7f245006e140 msgr2=0x7f245006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:49.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.934+0000 7f24570d2640 1 -- 172.21.15.78:0/718724436 shutdown_connections 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.934+0000 7f24570d2640 1 -- 172.21.15.78:0/718724436 wait complete. 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.935+0000 7f24570d2640 1 Processor -- start 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.935+0000 7f24570d2640 1 -- start start 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.935+0000 7f24570d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2450109450 0x7f2450118650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.935+0000 7f24570d2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2450109830 0x7f2450118b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.936+0000 7f24570d2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f245011b140 0x7f24501118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.936+0000 7f24570d2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2450120da0 con 0x7f245011b140 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.936+0000 7f24570d2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2450120c20 con 0x7f2450109450 2024-09-16T00:21:49.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.936+0000 7f24570d2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2450120f20 con 0x7f2450109830 2024-09-16T00:21:49.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.936+0000 7f244ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2450109830 0x7f2450118b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.936+0000 7f244ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2450109830 0x7f2450118b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49376/0 (socket says 172.21.15.78:49376) 2024-09-16T00:21:49.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.936+0000 7f244ffff640 1 -- 172.21.15.78:0/1645847737 learned_addr learned my addr 172.21.15.78:0/1645847737 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:49.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.936+0000 7f2454e6e640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2450109450 0x7f2450118650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.937+0000 7f245566f640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f245011b140 0x7f24501118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.937+0000 7f244ffff640 1 -- 172.21.15.78:0/1645847737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2450109450 msgr2=0x7f2450118650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.937+0000 7f244ffff640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2450109450 0x7f2450118650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.937+0000 7f244ffff640 1 -- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f245011b140 msgr2=0x7f24501118c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.937+0000 7f244ffff640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f245011b140 0x7f24501118c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.937+0000 7f244ffff640 1 -- 172.21.15.78:0/1645847737 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2450112180 con 0x7f2450109830 2024-09-16T00:21:49.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.939+0000 7f244ffff640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2450109830 0x7f2450118b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f244000e9f0 tx=0x7f244000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:49.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.939+0000 7f244dffb640 1 -- 172.21.15.78:0/1645847737 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244000cd80 con 0x7f2450109830 2024-09-16T00:21:49.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.939+0000 7f244dffb640 1 -- 172.21.15.78:0/1645847737 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2440010910 con 0x7f2450109830 2024-09-16T00:21:49.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.939+0000 7f244dffb640 1 -- 172.21.15.78:0/1645847737 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440018990 con 0x7f2450109830 2024-09-16T00:21:49.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.939+0000 7f24570d2640 1 -- 172.21.15.78:0/1645847737 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2450112410 con 0x7f2450109830 2024-09-16T00:21:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.940+0000 7f24570d2640 1 -- 172.21.15.78:0/1645847737 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24501163b0 con 0x7f2450109830 2024-09-16T00:21:49.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.943+0000 7f24594ab740 1 Processor -- start 2024-09-16T00:21:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.945+0000 7f24594ab740 1 -- start start 2024-09-16T00:21:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.946+0000 7f24594ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x559835953090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.946+0000 7f24594ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x559835953a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.946+0000 7f24594ab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 0x559835960760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.946+0000 7f24594ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5598358b92f0 con 0x5598359535d0 2024-09-16T00:21:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.946+0000 7f24594ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559835860cf0 con 0x559835954b40 2024-09-16T00:21:49.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.946+0000 7f24594ab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5598358b52b0 con 0x55983595e370 2024-09-16T00:21:49.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.946+0000 7f244dffb640 1 -- 172.21.15.78:0/1645847737 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2440010ac0 con 0x7f2450109830 2024-09-16T00:21:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.947+0000 7f244dffb640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f243c0a3450 0x7f243c0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.947+0000 7f244dffb640 1 -- 172.21.15.78:0/1645847737 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f24400cfee0 con 0x7f2450109830 2024-09-16T00:21:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.948+0000 7f2454e6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x559835953a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.948+0000 7f2454e6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x559835953a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48210/0 (socket says 172.21.15.78:48210) 2024-09-16T00:21:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.948+0000 7f2454e6e640 1 -- 172.21.15.78:0/853787347 learned_addr learned my addr 172.21.15.78:0/853787347 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:49.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f245566f640 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x559835953090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f244ffff640 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 0x559835960760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f245566f640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f243c0a3450 0x7f243c0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f245566f640 1 -- 172.21.15.78:0/853787347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 msgr2=0x559835960760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f245566f640 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 0x559835960760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f245566f640 1 -- 172.21.15.78:0/853787347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 msgr2=0x559835953a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f245566f640 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x559835953a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f245566f640 1 -- 172.21.15.78:0/853787347 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598358453a0 con 0x559835954b40 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.949+0000 7f245566f640 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x559835953090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2438009850 tx=0x7f24380045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d7aad026bfefc2c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.950+0000 7f245566f640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f243c0a3450 0x7f243c0a5910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2438009230 tx=0x7f2438046000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:49.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.950+0000 7f2446ffd640 1 -- 172.21.15.78:0/853787347 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438035030 con 0x559835954b40 2024-09-16T00:21:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.950+0000 7f2446ffd640 1 -- 172.21.15.78:0/853787347 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243804d070 con 0x559835954b40 2024-09-16T00:21:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 -- 172.21.15.78:0/853787347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 msgr2=0x559835953090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x559835953090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2438009850 tx=0x7f24380045f0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 -- 172.21.15.78:0/853787347 shutdown_connections 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 0x559835960760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x559835953a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 --2- 172.21.15.78:0/853787347 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x559835953090 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 -- 172.21.15.78:0/853787347 >> 172.21.15.78:0/853787347 conn(0x55983591c310 msgr2=0x55983594fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 -- 172.21.15.78:0/853787347 shutdown_connections 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.951+0000 7f24594ab740 1 -- 172.21.15.78:0/853787347 wait complete. 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.952+0000 7f24594ab740 1 -- start start 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.952+0000 7f24594ab740 1 Processor -- start 2024-09-16T00:21:49.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.952+0000 7f24594ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x55983595dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.952+0000 7f24594ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x55983595b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f24594ab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 0x55983595b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f24594ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559835961060 con 0x5598359535d0 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f24594ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559835874060 con 0x559835954b40 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f24594ab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5598359611e0 con 0x55983595e370 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f24594ab740 10 client.0 _ll_register_callbacks cb 0x5598355c78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f2454e6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x55983595dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f2454e6e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x55983595dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48212/0 (socket says 172.21.15.78:48212) 2024-09-16T00:21:49.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f2454e6e640 1 -- 172.21.15.78:0/2323357849 learned_addr learned my addr 172.21.15.78:0/2323357849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.953+0000 7f245566f640 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 0x55983595b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.954+0000 7f244ffff640 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x55983595b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f2454e6e640 1 -- 172.21.15.78:0/2323357849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 msgr2=0x55983595b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f2454e6e640 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 0x55983595b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f2454e6e640 1 -- 172.21.15.78:0/2323357849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 msgr2=0x55983595b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f2454e6e640 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x55983595b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f2454e6e640 1 -- 172.21.15.78:0/2323357849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55983562a3c0 con 0x5598359535d0 2024-09-16T00:21:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f2454e6e640 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x55983595dde0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2448011d10 tx=0x7f244800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928e96f354789ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f242ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:21:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f242ffff640 1 -- 172.21.15.78:0/2323357849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448003970 con 0x5598359535d0 2024-09-16T00:21:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.956+0000 7f242ffff640 1 -- 172.21.15.78:0/2323357849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2448017050 con 0x5598359535d0 2024-09-16T00:21:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.957+0000 7f24594ab740 1 -- 172.21.15.78:0/2323357849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5598358c6860 con 0x5598359535d0 2024-09-16T00:21:49.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.957+0000 7f242ffff640 1 -- 172.21.15.78:0/2323357849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2448003b10 con 0x5598359535d0 2024-09-16T00:21:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.960+0000 7f24594ab740 10 client.16878 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:49.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.960+0000 7f24594ab740 1 -- 172.21.15.78:0/2323357849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55983598ed30 con 0x5598359535d0 2024-09-16T00:21:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.962+0000 7f242ffff640 1 -- 172.21.15.78:0/2323357849 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5598358c6860 con 0x5598359535d0 2024-09-16T00:21:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.963+0000 7f242ffff640 1 -- 172.21.15.78:0/2323357849 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2448019070 con 0x5598359535d0 2024-09-16T00:21:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.963+0000 7f24594ab740 10 client.16878 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.963+0000 7f24594ab740 10 client.16878 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:21:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.963+0000 7f24594ab740 20 client.16878 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.963+0000 7f24594ab740 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x559835990140 0x559835992530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.963+0000 7f24594ab740 4 client.16878 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:49.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.963+0000 7f24594ab740 1 -- 172.21.15.78:0/2323357849 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598358c0d70 con 0x559835990140 2024-09-16T00:21:49.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.964+0000 7f244ffff640 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x559835990140 0x559835992530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.967+0000 7f244ffff640 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x559835990140 0x559835992530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9269d3ec7a64e8 server_cookie=c0dbbdc031a2f9c in_seq=0 out_seq=0 2024-09-16T00:21:49.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.967+0000 7f242ffff640 10 client.16878 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:21:49.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.970+0000 7f242ffff640 1 -- 172.21.15.78:0/2323357849 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598358c0d70 con 0x559835990140 2024-09-16T00:21:49.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.970+0000 7f242ffff640 10 client.16878 handle_command_reply: tid=0 2024-09-16T00:21:49.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.971+0000 7f24594ab740 1 client.16878 shutdown 2024-09-16T00:21:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 -- 172.21.15.78:0/2323357849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 msgr2=0x55983595dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x55983595dde0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2448011d10 tx=0x7f244800f790 comp rx=0 tx=0).stop 2024-09-16T00:21:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 -- 172.21.15.78:0/2323357849 shutdown_connections 2024-09-16T00:21:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55983595e370 0x55983595b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x559835990140 0x559835992530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559835954b40 0x55983595b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 --2- 172.21.15.78:0/2323357849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5598359535d0 0x55983595dde0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 -- 172.21.15.78:0/2323357849 >> 172.21.15.78:0/2323357849 conn(0x55983591c310 msgr2=0x5598359559c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f242ffff640 0 client.16878 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:21:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 -- 172.21.15.78:0/2323357849 shutdown_connections 2024-09-16T00:21:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 1 -- 172.21.15.78:0/2323357849 wait complete. 2024-09-16T00:21:49.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.972+0000 7f24594ab740 20 client.16878 trim_cache size 0 max 16384 2024-09-16T00:21:49.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.988+0000 7f24594ab740 1 Processor -- start 2024-09-16T00:21:49.991 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:49 smithi078 ceph-mon[31157]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 MiB/s wr, 547 op/s 2024-09-16T00:21:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.989+0000 7f24594ab740 1 -- start start 2024-09-16T00:21:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.990+0000 7f24594ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598359535d0 0x55983595a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.990+0000 7f24594ab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559835954b40 0x55983596a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.990+0000 7f24594ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55983595e370 0x559835997e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.990+0000 7f2454e6e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598359535d0 0x55983595a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.990+0000 7f244ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559835954b40 0x55983596a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.990+0000 7f2454e6e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598359535d0 0x55983595a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:44816/0 (socket says 172.21.15.78:44816) 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.990+0000 7f244ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559835954b40 0x55983596a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49396/0 (socket says 172.21.15.78:49396) 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f2454e6e640 1 -- 172.21.15.78:0/3246890353 learned_addr learned my addr 172.21.15.78:0/3246890353 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f24594ab740 1 -- 172.21.15.78:0/3246890353 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f245007d3e0 con 0x55983595e370 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f24594ab740 1 -- 172.21.15.78:0/3246890353 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f245007ef20 con 0x5598359535d0 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f245566f640 1 --2- 172.21.15.78:0/3246890353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55983595e370 0x559835997e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f24594ab740 1 -- 172.21.15.78:0/3246890353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f245007dab0 con 0x559835954b40 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f2454e6e640 1 -- 172.21.15.78:0/3246890353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559835954b40 msgr2=0x55983596a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f2454e6e640 1 --2- 172.21.15.78:0/3246890353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559835954b40 0x55983596a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f2454e6e640 1 -- 172.21.15.78:0/3246890353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55983595e370 msgr2=0x559835997e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f2454e6e640 1 --2- 172.21.15.78:0/3246890353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55983595e370 0x559835997e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f2454e6e640 1 -- 172.21.15.78:0/3246890353 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55983594fde0 con 0x5598359535d0 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.991+0000 7f2454e6e640 1 --2- 172.21.15.78:0/3246890353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598359535d0 0x55983595a950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2448000f80 tx=0x7f244800fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1539840b1132239a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.992+0000 7f24467fc640 1 -- 172.21.15.78:0/3246890353 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448025470 con 0x5598359535d0 2024-09-16T00:21:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.992+0000 7f24467fc640 1 -- 172.21.15.78:0/3246890353 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2448025610 con 0x5598359535d0 2024-09-16T00:21:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.992+0000 7f24467fc640 1 -- 172.21.15.78:0/3246890353 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480059a0 con 0x5598359535d0 2024-09-16T00:21:49.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 -- 172.21.15.78:0/3246890353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598359535d0 msgr2=0x55983595a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:49.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 --2- 172.21.15.78:0/3246890353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598359535d0 0x55983595a950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2448000f80 tx=0x7f244800fd10 comp rx=0 tx=0).stop 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 -- 172.21.15.78:0/3246890353 shutdown_connections 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 --2- 172.21.15.78:0/3246890353 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55983595e370 0x559835997e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 --2- 172.21.15.78:0/3246890353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559835954b40 0x55983596a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 --2- 172.21.15.78:0/3246890353 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5598359535d0 0x55983595a950 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 -- 172.21.15.78:0/3246890353 >> 172.21.15.78:0/3246890353 conn(0x55983591c310 msgr2=0x55983595a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 -- 172.21.15.78:0/3246890353 shutdown_connections 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.993+0000 7f24594ab740 1 -- 172.21.15.78:0/3246890353 wait complete. 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.994+0000 7f24594ab740 1 -- start start 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.994+0000 7f24594ab740 1 Processor -- start 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.994+0000 7f24594ab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598359535d0 0x559835966810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.994+0000 7f24594ab740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559835954b40 0x559835966d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.994+0000 7f24594ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55983595e370 0x559835961c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.994+0000 7f24594ab740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f245007e850 con 0x559835954b40 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f24594ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f245007e180 con 0x55983595e370 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f24594ab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f245007b3c0 con 0x5598359535d0 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f24594ab740 10 client.0 _ll_register_callbacks cb 0x7f2448009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f2454e6e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598359535d0 0x559835966810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f2454e6e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598359535d0 0x559835966810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49412/0 (socket says 172.21.15.78:49412) 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f2454e6e640 1 -- 172.21.15.78:0/2096868184 learned_addr learned my addr 172.21.15.78:0/2096868184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f245566f640 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55983595e370 0x559835961c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f244ffff640 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559835954b40 0x559835966d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f245566f640 1 -- 172.21.15.78:0/2096868184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598359535d0 msgr2=0x559835966810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f245566f640 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598359535d0 0x559835966810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f245566f640 1 -- 172.21.15.78:0/2096868184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559835954b40 msgr2=0x559835966d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:49.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f245566f640 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559835954b40 0x559835966d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f245566f640 1 -- 172.21.15.78:0/2096868184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559835954790 con 0x55983595e370 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.995+0000 7f2454e6e640 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598359535d0 0x559835966810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f245566f640 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55983595e370 0x559835961c10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f243804a040 tx=0x7f2438005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25105e2817265aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f242ffff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f242ffff640 1 -- 172.21.15.78:0/2096868184 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438035040 con 0x55983595e370 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f242ffff640 1 -- 172.21.15.78:0/2096868184 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f24380035f0 con 0x55983595e370 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f24594ab740 1 -- 172.21.15.78:0/2096868184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55983583d170 con 0x55983595e370 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f242ffff640 1 -- 172.21.15.78:0/2096868184 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24380037f0 con 0x55983595e370 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f24594ab740 10 client.26276 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f24594ab740 1 -- 172.21.15.78:0/2096868184 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5598359fa810 con 0x55983595e370 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.996+0000 7f242ffff640 1 -- 172.21.15.78:0/2096868184 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55983583d170 con 0x55983595e370 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f242ffff640 1 -- 172.21.15.78:0/2096868184 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2438047070 con 0x55983595e370 2024-09-16T00:21:50.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f24594ab740 10 client.26276 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f24594ab740 10 client.26276 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:21:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f24594ab740 20 client.26276 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f24594ab740 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5598359fab50 0x5598359fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f24594ab740 4 client.26276 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:21:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f24594ab740 1 -- 172.21.15.78:0/2096868184 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55983588a9b0 con 0x5598359fab50 2024-09-16T00:21:50.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f2454e6e640 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5598359fab50 0x5598359fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f242ffff640 10 client.26276 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:21:50.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.997+0000 7f2454e6e640 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5598359fab50 0x5598359fcf40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6babfa74210937c server_cookie=35977cb6029be8e1 in_seq=0 out_seq=0 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f242ffff640 1 -- 172.21.15.78:0/2096868184 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55983588a9b0 con 0x5598359fab50 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f242ffff640 10 client.26276 handle_command_reply: tid=0 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 client.26276 shutdown 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 -- 172.21.15.78:0/2096868184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55983595e370 msgr2=0x559835961c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55983595e370 0x559835961c10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f243804a040 tx=0x7f2438005720 comp rx=0 tx=0).stop 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 -- 172.21.15.78:0/2096868184 shutdown_connections 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5598359fab50 0x5598359fcf40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55983595e370 0x559835961c10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559835954b40 0x559835966d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 --2- 172.21.15.78:0/2096868184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598359535d0 0x559835966810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 -- 172.21.15.78:0/2096868184 >> 172.21.15.78:0/2096868184 conn(0x55983591c310 msgr2=0x55983595c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f242ffff640 0 client.26276 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 -- 172.21.15.78:0/2096868184 shutdown_connections 2024-09-16T00:21:50.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 1 -- 172.21.15.78:0/2096868184 wait complete. 2024-09-16T00:21:50.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:49.998+0000 7f24594ab740 20 client.26276 trim_cache size 0 max 16384 2024-09-16T00:21:50.004 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.000+0000 7f24570d2640 1 -- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f243c0a3450 msgr2=0x7f243c0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.000+0000 7f24570d2640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f243c0a3450 0x7f243c0a5910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2438009230 tx=0x7f2438046000 comp rx=0 tx=0).stop 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.000+0000 7f24570d2640 1 -- 172.21.15.78:0/1645847737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2450109830 msgr2=0x7f2450118b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.000+0000 7f24570d2640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2450109830 0x7f2450118b90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f244000e9f0 tx=0x7f244000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.001+0000 7f24570d2640 1 -- 172.21.15.78:0/1645847737 shutdown_connections 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.001+0000 7f24570d2640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f243c0a3450 0x7f243c0a5910 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.001+0000 7f24570d2640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f245011b140 0x7f24501118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.001+0000 7f24570d2640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2450109830 0x7f2450118b90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.001+0000 7f24570d2640 1 --2- 172.21.15.78:0/1645847737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2450109450 0x7f2450118650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.001+0000 7f24570d2640 1 -- 172.21.15.78:0/1645847737 >> 172.21.15.78:0/1645847737 conn(0x7f245006e140 msgr2=0x7f245011c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.001+0000 7f24570d2640 1 -- 172.21.15.78:0/1645847737 shutdown_connections 2024-09-16T00:21:50.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:50.001+0000 7f24570d2640 1 -- 172.21.15.78:0/1645847737 wait complete. 2024-09-16T00:21:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:49 smithi193 ceph-mon[35484]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 MiB/s wr, 547 op/s 2024-09-16T00:21:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:49 smithi148 ceph-mon[35599]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 MiB/s wr, 547 op/s 2024-09-16T00:21:50.749 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:21:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 470.0 s, 653.7 tps, lat 154.284 ms stddev 199.495 2024-09-16T00:21:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:51.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.809+0000 7f3136c44640 1 -- 172.21.15.78:0/302358686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f313006ee10 msgr2=0x7f3130114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:51.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.809+0000 7f3136c44640 1 --2- 172.21.15.78:0/302358686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f313006ee10 0x7f3130114570 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3124009900 tx=0x7f312402a440 comp rx=0 tx=0).stop 2024-09-16T00:21:51.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.813+0000 7f3136c44640 1 -- 172.21.15.78:0/302358686 shutdown_connections 2024-09-16T00:21:51.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.813+0000 7f3136c44640 1 --2- 172.21.15.78:0/302358686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3130114ab0 0x7f3130116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.813+0000 7f3136c44640 1 --2- 172.21.15.78:0/302358686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f313006ee10 0x7f3130114570 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.813+0000 7f3136c44640 1 --2- 172.21.15.78:0/302358686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f313006e4f0 0x7f313006e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.813+0000 7f3136c44640 1 -- 172.21.15.78:0/302358686 >> 172.21.15.78:0/302358686 conn(0x7f3130073180 msgr2=0x7f3130073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:51.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.816+0000 7f3136c44640 1 -- 172.21.15.78:0/302358686 shutdown_connections 2024-09-16T00:21:51.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.817+0000 7f3136c44640 1 -- 172.21.15.78:0/302358686 wait complete. 2024-09-16T00:21:51.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.818+0000 7f3136c44640 1 Processor -- start 2024-09-16T00:21:51.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.820+0000 7f3136c44640 1 -- start start 2024-09-16T00:21:51.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.820+0000 7f3136c44640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313006e4f0 0x7f313011e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.820+0000 7f3136c44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f313006ee10 0x7f313011ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.820+0000 7f3136c44640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3130114ab0 0x7f3130119a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.820+0000 7f3136c44640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3130118420 con 0x7f3130114ab0 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.820+0000 7f3136c44640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f31301182a0 con 0x7f313006e4f0 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.820+0000 7f3136c44640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31301185a0 con 0x7f313006ee10 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.820+0000 7f3136443640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3130114ab0 0x7f3130119a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.821+0000 7f3136443640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3130114ab0 0x7f3130119a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48238/0 (socket says 172.21.15.78:48238) 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.821+0000 7f3135441640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f313006ee10 0x7f313011ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.821+0000 7f3136443640 1 -- 172.21.15.78:0/3956279580 learned_addr learned my addr 172.21.15.78:0/3956279580 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.821+0000 7f3136443640 1 -- 172.21.15.78:0/3956279580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f313006ee10 msgr2=0x7f313011ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.821+0000 7f3136443640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f313006ee10 0x7f313011ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.821+0000 7f3136443640 1 -- 172.21.15.78:0/3956279580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313006e4f0 msgr2=0x7f313011e940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.822+0000 7f3136443640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313006e4f0 0x7f313011e940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.822+0000 7f3136443640 1 -- 172.21.15.78:0/3956279580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313011a160 con 0x7f3130114ab0 2024-09-16T00:21:51.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.822+0000 7f3136443640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3130114ab0 0x7f3130119a60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f312002d9e0 tx=0x7f312002dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:51.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.823+0000 7f312effd640 1 -- 172.21.15.78:0/3956279580 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312003b070 con 0x7f3130114ab0 2024-09-16T00:21:51.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.823+0000 7f3136c44640 1 -- 172.21.15.78:0/3956279580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313011a450 con 0x7f3130114ab0 2024-09-16T00:21:51.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.823+0000 7f312effd640 1 -- 172.21.15.78:0/3956279580 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3120036440 con 0x7f3130114ab0 2024-09-16T00:21:51.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.823+0000 7f3136c44640 1 -- 172.21.15.78:0/3956279580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3130124810 con 0x7f3130114ab0 2024-09-16T00:21:51.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.823+0000 7f312effd640 1 -- 172.21.15.78:0/3956279580 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31200295c0 con 0x7f3130114ab0 2024-09-16T00:21:51.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.827+0000 7f313a27f740 1 Processor -- start 2024-09-16T00:21:51.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.828+0000 7f312effd640 1 -- 172.21.15.78:0/3956279580 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f312003e070 con 0x7f3130114ab0 2024-09-16T00:21:51.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.828+0000 7f313a27f740 1 -- start start 2024-09-16T00:21:51.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.829+0000 7f313a27f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf36054270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.829+0000 7f313a27f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 0x55bf36054bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.829+0000 7f313a27f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf3605f550 0x55bf36061940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.829+0000 7f313a27f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf35fba2f0 con 0x55bf36055d20 2024-09-16T00:21:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.829+0000 7f313a27f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bf35f61cf0 con 0x55bf360547b0 2024-09-16T00:21:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.829+0000 7f313a27f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bf35fb62b0 con 0x55bf3605f550 2024-09-16T00:21:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.830+0000 7f3135441640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf3605f550 0x55bf36061940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.830+0000 7f3135441640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf36054270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.830+0000 7f3135441640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf36054270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48252/0 (socket says 172.21.15.78:48252) 2024-09-16T00:21:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.830+0000 7f3135441640 1 -- 172.21.15.78:0/324933497 learned_addr learned my addr 172.21.15.78:0/324933497 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.830+0000 7f3135c42640 1 --2- 172.21.15.78:0/324933497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 0x55bf36054bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.831+0000 7f312effd640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31180a3450 0x7f31180a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.831+0000 7f3135441640 1 -- 172.21.15.78:0/324933497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf3605f550 msgr2=0x55bf36061940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.831+0000 7f3135441640 1 --2- 172.21.15.78:0/324933497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf3605f550 0x55bf36061940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.831+0000 7f3135441640 1 -- 172.21.15.78:0/324933497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 msgr2=0x55bf36054bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.831+0000 7f3135441640 1 --2- 172.21.15.78:0/324933497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 0x55bf36054bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.831+0000 7f3135441640 1 -- 172.21.15.78:0/324933497 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf35f463a0 con 0x55bf36055d20 2024-09-16T00:21:51.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.831+0000 7f312effd640 1 -- 172.21.15.78:0/3956279580 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f31200f10e0 con 0x7f3130114ab0 2024-09-16T00:21:51.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.832+0000 7f3135441640 1 --2- 172.21.15.78:0/324933497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf36054270 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55bf36055250 tx=0x7f3124000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f229957fc7898189 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:51.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.832+0000 7f3136443640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31180a3450 0x7f31180a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.833+0000 7f3136443640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31180a3450 0x7f31180a5910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f312002e5a0 tx=0x7f312002e510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:51.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.833+0000 7f3117fff640 1 -- 172.21.15.78:0/324933497 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124036040 con 0x55bf36055d20 2024-09-16T00:21:51.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.833+0000 7f3117fff640 1 -- 172.21.15.78:0/324933497 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124037040 con 0x55bf36055d20 2024-09-16T00:21:51.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.834+0000 7f313a27f740 1 -- 172.21.15.78:0/324933497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 msgr2=0x55bf36054270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:51.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.834+0000 7f313a27f740 1 --2- 172.21.15.78:0/324933497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf36054270 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55bf36055250 tx=0x7f3124000bc0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.841+0000 7f313a27f740 1 -- 172.21.15.78:0/324933497 shutdown_connections 2024-09-16T00:21:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.841+0000 7f313a27f740 1 --2- 172.21.15.78:0/324933497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf3605f550 0x55bf36061940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.841+0000 7f313a27f740 1 --2- 172.21.15.78:0/324933497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 0x55bf36054bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.841+0000 7f313a27f740 1 --2- 172.21.15.78:0/324933497 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf36054270 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.841+0000 7f313a27f740 1 -- 172.21.15.78:0/324933497 >> 172.21.15.78:0/324933497 conn(0x55bf3601d310 msgr2=0x55bf36050e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:51.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.843+0000 7f313a27f740 1 -- 172.21.15.78:0/324933497 shutdown_connections 2024-09-16T00:21:51.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.845+0000 7f313a27f740 1 -- 172.21.15.78:0/324933497 wait complete. 2024-09-16T00:21:51.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 1 -- start start 2024-09-16T00:21:51.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 1 Processor -- start 2024-09-16T00:21:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3605ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf3605c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf3605f550 0x55bf3605c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf36062240 con 0x55bf36055d20 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bf35f75060 con 0x55bf3605f550 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bf360623c0 con 0x55bf360547b0 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f313a27f740 10 client.0 _ll_register_callbacks cb 0x55bf35cc88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.846+0000 7f3135441640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3605ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3605ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49450/0 (socket says 172.21.15.78:49450) 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 -- 172.21.15.78:0/2488959167 learned_addr learned my addr 172.21.15.78:0/2488959167 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:51.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf3605f550 0x55bf3605c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135c42640 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf3605c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 -- 172.21.15.78:0/2488959167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf3605f550 msgr2=0x55bf3605c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf3605f550 0x55bf3605c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 -- 172.21.15.78:0/2488959167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 msgr2=0x55bf3605c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf3605c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 -- 172.21.15.78:0/2488959167 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf35d2b3c0 con 0x55bf360547b0 2024-09-16T00:21:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f3135441640 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3605ee50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3124004fb0 tx=0x7f31240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f6e198facfcd8a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f31157fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:21:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.847+0000 7f31157fa640 1 -- 172.21.15.78:0/2488959167 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124036040 con 0x55bf360547b0 2024-09-16T00:21:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.848+0000 7f31157fa640 1 -- 172.21.15.78:0/2488959167 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3124005390 con 0x55bf360547b0 2024-09-16T00:21:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.848+0000 7f313a27f740 1 -- 172.21.15.78:0/2488959167 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3124037040 con 0x55bf360547b0 2024-09-16T00:21:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.848+0000 7f31157fa640 1 -- 172.21.15.78:0/2488959167 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f312405c050 con 0x55bf360547b0 2024-09-16T00:21:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.848+0000 7f313a27f740 10 client.26095 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.848+0000 7f313a27f740 1 -- 172.21.15.78:0/2488959167 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bf36062240 con 0x55bf360547b0 2024-09-16T00:21:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.848+0000 7f31157fa640 1 -- 172.21.15.78:0/2488959167 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124037040 con 0x55bf360547b0 2024-09-16T00:21:51.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.848+0000 7f31157fa640 1 -- 172.21.15.78:0/2488959167 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f312404d4b0 con 0x55bf360547b0 2024-09-16T00:21:51.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.848+0000 7f313a27f740 10 client.26095 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:51.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.849+0000 7f313a27f740 10 client.26095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:21:51.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.849+0000 7f313a27f740 20 client.26095 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.849+0000 7f313a27f740 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55bf3606f0b0 0x55bf360714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.849+0000 7f313a27f740 4 client.26095 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.849+0000 7f313a27f740 1 -- 172.21.15.78:0/2488959167 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf35fc7860 con 0x55bf3606f0b0 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.849+0000 7f3135c42640 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55bf3606f0b0 0x55bf360714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.849+0000 7f3135c42640 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55bf3606f0b0 0x55bf360714f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41c17ad103ca4e0 server_cookie=13041b58992e863d in_seq=0 out_seq=0 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.850+0000 7f31157fa640 10 client.26095 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.851+0000 7f31157fa640 1 -- 172.21.15.78:0/2488959167 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf35fc7860 con 0x55bf3606f0b0 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.851+0000 7f31157fa640 10 client.26095 handle_command_reply: tid=0 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.851+0000 7f313a27f740 1 client.26095 shutdown 2024-09-16T00:21:51.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.851+0000 7f313a27f740 1 -- 172.21.15.78:0/2488959167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 msgr2=0x55bf3605ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.851+0000 7f313a27f740 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3605ee50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3124004fb0 tx=0x7f31240046c0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 1 -- 172.21.15.78:0/2488959167 shutdown_connections 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55bf3606f0b0 0x55bf360714f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf3605f550 0x55bf3605c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf36055d20 0x55bf3605c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 1 --2- 172.21.15.78:0/2488959167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3605ee50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 1 -- 172.21.15.78:0/2488959167 >> 172.21.15.78:0/2488959167 conn(0x55bf3601d310 msgr2=0x55bf36056b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f31157fa640 0 client.26095 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 1 -- 172.21.15.78:0/2488959167 shutdown_connections 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 1 -- 172.21.15.78:0/2488959167 wait complete. 2024-09-16T00:21:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.853+0000 7f313a27f740 20 client.26095 trim_cache size 0 max 16384 2024-09-16T00:21:51.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.867+0000 7f313a27f740 1 Processor -- start 2024-09-16T00:21:51.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.867+0000 7f313a27f740 1 -- start start 2024-09-16T00:21:51.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.867+0000 7f313a27f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3606b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.867+0000 7f313a27f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf36055d20 0x55bf3605a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.867+0000 7f313a27f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 0x55bf3605af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.867+0000 7f313a27f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f311c00fbb0 con 0x55bf3605f550 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.867+0000 7f313a27f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f311c0116f0 con 0x55bf36055d20 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.867+0000 7f313a27f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f311c010280 con 0x55bf360547b0 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135c42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf36055d20 0x55bf3605a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135441640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 0x55bf3605af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135c42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf36055d20 0x55bf3605a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:44856/0 (socket says 172.21.15.78:44856) 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135c42640 1 -- 172.21.15.78:0/1718214270 learned_addr learned my addr 172.21.15.78:0/1718214270 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135441640 1 --2- 172.21.15.78:0/1718214270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3606b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135441640 1 -- 172.21.15.78:0/1718214270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 msgr2=0x55bf3606b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135441640 1 --2- 172.21.15.78:0/1718214270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3606b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135441640 1 -- 172.21.15.78:0/1718214270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf36055d20 msgr2=0x55bf3605a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:51.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135441640 1 --2- 172.21.15.78:0/1718214270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf36055d20 0x55bf3605a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135441640 1 -- 172.21.15.78:0/1718214270 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf3605a650 con 0x55bf3605f550 2024-09-16T00:21:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.868+0000 7f3135441640 1 --2- 172.21.15.78:0/1718214270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 0x55bf3605af10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3124005ab0 tx=0x7f3124000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a924ce92bbfca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.869+0000 7f31177fe640 1 -- 172.21.15.78:0/1718214270 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124036070 con 0x55bf3605f550 2024-09-16T00:21:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.869+0000 7f31177fe640 1 -- 172.21.15.78:0/1718214270 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124003ad0 con 0x55bf3605f550 2024-09-16T00:21:51.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.869+0000 7f31177fe640 1 -- 172.21.15.78:0/1718214270 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124003ea0 con 0x55bf3605f550 2024-09-16T00:21:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.869+0000 7f313a27f740 1 -- 172.21.15.78:0/1718214270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 msgr2=0x55bf3605af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.869+0000 7f313a27f740 1 --2- 172.21.15.78:0/1718214270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 0x55bf3605af10 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3124005ab0 tx=0x7f3124000bc0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 -- 172.21.15.78:0/1718214270 shutdown_connections 2024-09-16T00:21:51.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 --2- 172.21.15.78:0/1718214270 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 0x55bf3605af10 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 --2- 172.21.15.78:0/1718214270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf36055d20 0x55bf3605a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 --2- 172.21.15.78:0/1718214270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf360547b0 0x55bf3606b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 -- 172.21.15.78:0/1718214270 >> 172.21.15.78:0/1718214270 conn(0x55bf3601d310 msgr2=0x55bf3605c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 -- 172.21.15.78:0/1718214270 shutdown_connections 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 -- 172.21.15.78:0/1718214270 wait complete. 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 -- start start 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.870+0000 7f313a27f740 1 Processor -- start 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.871+0000 7f313a27f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 0x55bf36067730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.871+0000 7f313a27f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf36055d20 0x55bf36067c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.871+0000 7f313a27f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 0x55bf360583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.871+0000 7f313a27f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f311c011020 con 0x55bf3605f550 2024-09-16T00:21:51.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.871+0000 7f313a27f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f311c010950 con 0x55bf360547b0 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.871+0000 7f313a27f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f311c00db90 con 0x55bf36055d20 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.871+0000 7f313a27f740 10 client.0 _ll_register_callbacks cb 0x7f3128009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf36055d20 0x55bf36067c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf36055d20 0x55bf36067c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49464/0 (socket says 172.21.15.78:49464) 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 -- 172.21.15.78:0/1228882854 learned_addr learned my addr 172.21.15.78:0/1228882854 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 -- 172.21.15.78:0/1228882854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 msgr2=0x55bf36067730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 0x55bf36067730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 -- 172.21.15.78:0/1228882854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 msgr2=0x55bf360583b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 0x55bf360583b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 -- 172.21.15.78:0/1228882854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf360682e0 con 0x55bf36055d20 2024-09-16T00:21:51.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f3135c42640 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf36055d20 0x55bf36067c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55bf3605b990 tx=0x7f3128009b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12e3ba9ffe58001 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.872+0000 7f31157fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.873+0000 7f31157fa640 1 -- 172.21.15.78:0/1228882854 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128003ca0 con 0x55bf36055d20 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.873+0000 7f31157fa640 1 -- 172.21.15.78:0/1228882854 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f312801d380 con 0x55bf36055d20 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.873+0000 7f313a27f740 1 -- 172.21.15.78:0/1228882854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf35fc1d70 con 0x55bf36055d20 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.873+0000 7f31157fa640 1 -- 172.21.15.78:0/1228882854 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f312802e050 con 0x55bf36055d20 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.873+0000 7f313a27f740 10 client.26101 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.873+0000 7f313a27f740 1 -- 172.21.15.78:0/1228882854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bf360d9390 con 0x55bf36055d20 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.873+0000 7f31157fa640 1 -- 172.21.15.78:0/1228882854 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3128003e40 con 0x55bf36055d20 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f31157fa640 1 -- 172.21.15.78:0/1228882854 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3128026920 con 0x55bf36055d20 2024-09-16T00:21:51.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f313a27f740 10 client.26101 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f313a27f740 10 client.26101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f313a27f740 20 client.26101 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f313a27f740 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55bf360d9b40 0x55bf360dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f313a27f740 4 client.26101 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f313a27f740 1 -- 172.21.15.78:0/1228882854 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf35fc1d70 con 0x55bf360d9b40 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f3135441640 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55bf360d9b40 0x55bf360dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f31157fa640 10 client.26101 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.874+0000 7f3135441640 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55bf360d9b40 0x55bf360dbf60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42100fd150bdcfc6 server_cookie=1bd8587fc1d37337 in_seq=0 out_seq=0 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f31157fa640 1 -- 172.21.15.78:0/1228882854 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf35fc1d70 con 0x55bf360d9b40 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f31157fa640 10 client.26101 handle_command_reply: tid=0 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 client.26101 shutdown 2024-09-16T00:21:51.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 -- 172.21.15.78:0/1228882854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf36055d20 msgr2=0x55bf36067c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf36055d20 0x55bf36067c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55bf3605b990 tx=0x7f3128009b10 comp rx=0 tx=0).stop 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 -- 172.21.15.78:0/1228882854 shutdown_connections 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf3605f550 0x55bf360583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf36055d20 0x55bf36067c70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55bf360d9b40 0x55bf360dbf60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 --2- 172.21.15.78:0/1228882854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf360547b0 0x55bf36067730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 -- 172.21.15.78:0/1228882854 >> 172.21.15.78:0/1228882854 conn(0x55bf3601d310 msgr2=0x55bf3605a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f31157fa640 0 client.26101 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 -- 172.21.15.78:0/1228882854 shutdown_connections 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 1 -- 172.21.15.78:0/1228882854 wait complete. 2024-09-16T00:21:51.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.875+0000 7f313a27f740 20 client.26101 trim_cache size 0 max 16384 2024-09-16T00:21:51.883 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:21:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 -- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31180a3450 msgr2=0x7f31180a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31180a3450 0x7f31180a5910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f312002e5a0 tx=0x7f312002e510 comp rx=0 tx=0).stop 2024-09-16T00:21:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 -- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3130114ab0 msgr2=0x7f3130119a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3130114ab0 0x7f3130119a60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f312002d9e0 tx=0x7f312002dea0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 -- 172.21.15.78:0/3956279580 shutdown_connections 2024-09-16T00:21:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31180a3450 0x7f31180a5910 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3130114ab0 0x7f3130119a60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f313006ee10 0x7f313011ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 --2- 172.21.15.78:0/3956279580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f313006e4f0 0x7f313011e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.876+0000 7f3136c44640 1 -- 172.21.15.78:0/3956279580 >> 172.21.15.78:0/3956279580 conn(0x7f3130073180 msgr2=0x7f313006b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.877+0000 7f3136c44640 1 -- 172.21.15.78:0/3956279580 shutdown_connections 2024-09-16T00:21:51.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:51.877+0000 7f3136c44640 1 -- 172.21.15.78:0/3956279580 wait complete. 2024-09-16T00:21:52.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:51 smithi078 ceph-mon[31157]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.1 MiB/s wr, 563 op/s 2024-09-16T00:21:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:51 smithi193 ceph-mon[35484]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.1 MiB/s wr, 563 op/s 2024-09-16T00:21:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:51 smithi148 ceph-mon[35599]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.1 MiB/s wr, 563 op/s 2024-09-16T00:21:54.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:53 smithi078 ceph-mon[31157]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 MiB/s wr, 575 op/s 2024-09-16T00:21:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:53 smithi193 ceph-mon[35484]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 MiB/s wr, 575 op/s 2024-09-16T00:21:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:53 smithi148 ceph-mon[35599]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 MiB/s wr, 575 op/s 2024-09-16T00:21:54.727 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:21:55.181 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.806+0000 7f31d5de6640 1 -- 172.21.15.78:0/2814689354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c809a550 msgr2=0x7f31c809a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:55.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.806+0000 7f31d5de6640 1 --2- 172.21.15.78:0/2814689354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c809a550 0x7f31c809a930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31c4014d10 tx=0x7f31c4032700 comp rx=0 tx=0).stop 2024-09-16T00:21:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.813+0000 7f31d5de6640 1 -- 172.21.15.78:0/2814689354 shutdown_connections 2024-09-16T00:21:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.813+0000 7f31d5de6640 1 --2- 172.21.15.78:0/2814689354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31c80952d0 0x7f31c80a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.813+0000 7f31d5de6640 1 --2- 172.21.15.78:0/2814689354 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8094930 0x7f31c8094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.813+0000 7f31d5de6640 1 --2- 172.21.15.78:0/2814689354 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c809a550 0x7f31c809a930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.813+0000 7f31d5de6640 1 -- 172.21.15.78:0/2814689354 >> 172.21.15.78:0/2814689354 conn(0x7f31c800a1f0 msgr2=0x7f31c800a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:55.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.816+0000 7f31d5de6640 1 -- 172.21.15.78:0/2814689354 shutdown_connections 2024-09-16T00:21:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.817+0000 7f31d5de6640 1 -- 172.21.15.78:0/2814689354 wait complete. 2024-09-16T00:21:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.817+0000 7f31d5de6640 1 Processor -- start 2024-09-16T00:21:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d5de6640 1 -- start start 2024-09-16T00:21:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d5de6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8094930 0x7f31c812f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d5de6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31c80952d0 0x7f31c812fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d5de6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c809a550 0x7f31c8129a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d5de6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31c80a37c0 con 0x7f31c8094930 2024-09-16T00:21:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d5de6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f31c80a3640 con 0x7f31c809a550 2024-09-16T00:21:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d5de6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31c80a3940 con 0x7f31c80952d0 2024-09-16T00:21:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d4de4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8094930 0x7f31c812f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d4de4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8094930 0x7f31c812f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48276/0 (socket says 172.21.15.78:48276) 2024-09-16T00:21:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.818+0000 7f31d4de4640 1 -- 172.21.15.78:0/1448409021 learned_addr learned my addr 172.21.15.78:0/1448409021 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.819+0000 7f31cffff640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31c80952d0 0x7f31c812fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.819+0000 7f31cffff640 1 -- 172.21.15.78:0/1448409021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c809a550 msgr2=0x7f31c8129a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:21:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.819+0000 7f31cffff640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c809a550 0x7f31c8129a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.819+0000 7f31cffff640 1 -- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8094930 msgr2=0x7f31c812f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.819+0000 7f31cffff640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8094930 0x7f31c812f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.819+0000 7f31cffff640 1 -- 172.21.15.78:0/1448409021 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c812a140 con 0x7f31c80952d0 2024-09-16T00:21:55.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.820+0000 7f31cffff640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31c80952d0 0x7f31c812fd90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e790 tx=0x7f31bc00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.822+0000 7f31cdffb640 1 -- 172.21.15.78:0/1448409021 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc00eea0 con 0x7f31c80952d0 2024-09-16T00:21:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.822+0000 7f31d5de6640 1 -- 172.21.15.78:0/1448409021 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c812a430 con 0x7f31c80952d0 2024-09-16T00:21:55.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.822+0000 7f31d5de6640 1 -- 172.21.15.78:0/1448409021 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c8136720 con 0x7f31c80952d0 2024-09-16T00:21:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.823+0000 7f31cdffb640 1 -- 172.21.15.78:0/1448409021 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31bc00ccc0 con 0x7f31c80952d0 2024-09-16T00:21:55.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.823+0000 7f31cdffb640 1 -- 172.21.15.78:0/1448409021 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc018820 con 0x7f31c80952d0 2024-09-16T00:21:55.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.827+0000 7f31cdffb640 1 -- 172.21.15.78:0/1448409021 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f31bc0189c0 con 0x7f31c80952d0 2024-09-16T00:21:55.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.829+0000 7f31d9421740 1 Processor -- start 2024-09-16T00:21:55.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31d9421740 1 -- start start 2024-09-16T00:21:55.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31d9421740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d2e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31d9421740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d2ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31d9421740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d39520 0x55e1a8d3b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31d9421740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8c942f0 con 0x55e1a8d2e780 2024-09-16T00:21:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31d9421740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8c3bcf0 con 0x55e1a8d39520 2024-09-16T00:21:55.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31d9421740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8c902b0 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d39520 0x55e1a8d3b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d39520 0x55e1a8d3b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:44880/0 (socket says 172.21.15.78:44880) 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31cffff640 1 -- 172.21.15.78:0/1305212440 learned_addr learned my addr 172.21.15.78:0/1305212440 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.830+0000 7f31d55e5640 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d2e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.831+0000 7f31d4de4640 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d2ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.831+0000 7f31cffff640 1 -- 172.21.15.78:0/1305212440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2fcf0 msgr2=0x55e1a8d2e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.831+0000 7f31cffff640 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d2e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.831+0000 7f31cffff640 1 -- 172.21.15.78:0/1305212440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 msgr2=0x55e1a8d2ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.831+0000 7f31cffff640 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d2ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.831+0000 7f31cffff640 1 -- 172.21.15.78:0/1305212440 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a8c203a0 con 0x55e1a8d39520 2024-09-16T00:21:55.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.831+0000 7f31cffff640 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d39520 0x55e1a8d3b910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31bc010c90 tx=0x7f31bc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d773495845e1397 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.832+0000 7f31cdffb640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31c00a3450 0x7f31c00a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.832+0000 7f31cdffb640 1 -- 172.21.15.78:0/1448409021 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f31bc0d0e60 con 0x7f31c80952d0 2024-09-16T00:21:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.833+0000 7f31baffd640 1 -- 172.21.15.78:0/1305212440 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0d2050 con 0x55e1a8d39520 2024-09-16T00:21:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.833+0000 7f31baffd640 1 -- 172.21.15.78:0/1305212440 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31bc0f2070 con 0x55e1a8d39520 2024-09-16T00:21:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.834+0000 7f31baffd640 1 -- 172.21.15.78:0/1305212440 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc018cb0 con 0x55e1a8d39520 2024-09-16T00:21:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.834+0000 7f31d9421740 1 -- 172.21.15.78:0/1305212440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d39520 msgr2=0x55e1a8d3b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:55.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.834+0000 7f31d9421740 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d39520 0x55e1a8d3b910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31bc010c90 tx=0x7f31bc013000 comp rx=0 tx=0).stop 2024-09-16T00:21:55.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d55e5640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31c00a3450 0x7f31c00a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d9421740 1 -- 172.21.15.78:0/1305212440 shutdown_connections 2024-09-16T00:21:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d9421740 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d39520 0x55e1a8d3b910 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d9421740 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d2ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d9421740 1 --2- 172.21.15.78:0/1305212440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d2e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d9421740 1 -- 172.21.15.78:0/1305212440 >> 172.21.15.78:0/1305212440 conn(0x55e1a8cf7310 msgr2=0x55e1a8d2ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d9421740 1 -- 172.21.15.78:0/1305212440 shutdown_connections 2024-09-16T00:21:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d9421740 1 -- 172.21.15.78:0/1305212440 wait complete. 2024-09-16T00:21:55.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.837+0000 7f31d9421740 1 -- start start 2024-09-16T00:21:55.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.838+0000 7f31d9421740 1 Processor -- start 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d9421740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d38e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d9421740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d360c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d9421740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d36600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d9421740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8d4d280 con 0x55e1a8d2e780 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d9421740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8c4f060 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d9421740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1a8d4d400 con 0x55e1a8d39520 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d55e5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d36600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d55e5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d36600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49512/0 (socket says 172.21.15.78:49512) 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d55e5640 1 -- 172.21.15.78:0/3228870854 learned_addr learned my addr 172.21.15.78:0/3228870854 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:55.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d360c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:44884/0 (socket says 172.21.15.78:44884) 2024-09-16T00:21:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d4de4640 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d38e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.839+0000 7f31d9421740 10 client.0 _ll_register_callbacks cb 0x55e1a89a28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:55.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.840+0000 7f31d55e5640 1 -- 172.21.15.78:0/3228870854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 msgr2=0x55e1a8d360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.840+0000 7f31d55e5640 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.840+0000 7f31d55e5640 1 -- 172.21.15.78:0/3228870854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 msgr2=0x55e1a8d38e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.840+0000 7f31d55e5640 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d38e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.840+0000 7f31d55e5640 1 -- 172.21.15.78:0/3228870854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a8a053c0 con 0x55e1a8d39520 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.840+0000 7f31d55e5640 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d36600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f31d0051790 tx=0x7f31d006f780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae472030c639ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.840+0000 7f31affff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.841+0000 7f31affff640 1 -- 172.21.15.78:0/3228870854 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0068ee0 con 0x55e1a8d39520 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.841+0000 7f31affff640 1 -- 172.21.15.78:0/3228870854 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f31d00720d0 con 0x55e1a8d39520 2024-09-16T00:21:55.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.841+0000 7f31d9421740 1 -- 172.21.15.78:0/3228870854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31bc0f2070 con 0x55e1a8d39520 2024-09-16T00:21:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.842+0000 7f31affff640 1 -- 172.21.15.78:0/3228870854 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31d008e050 con 0x55e1a8d39520 2024-09-16T00:21:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.843+0000 7f31d9421740 10 client.26113 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:55.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.843+0000 7f31d9421740 1 -- 172.21.15.78:0/3228870854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e1a8d58d70 con 0x55e1a8d39520 2024-09-16T00:21:55.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31d55e5640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31c00a3450 0x7f31c00a5910 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f31d00828e0 tx=0x7f31d0082ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:55.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31affff640 1 -- 172.21.15.78:0/3228870854 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d0087950 con 0x55e1a8d39520 2024-09-16T00:21:55.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31affff640 1 -- 172.21.15.78:0/3228870854 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f31d0085070 con 0x55e1a8d39520 2024-09-16T00:21:55.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31d9421740 10 client.26113 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:55.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31d9421740 10 client.26113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:21:55.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31d9421740 20 client.26113 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:55.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31d9421740 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55e1a8d5a0c0 0x55e1a8d5c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31d9421740 4 client.26113 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.847+0000 7f31d9421740 1 -- 172.21.15.78:0/3228870854 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f31bc0f2070 con 0x55e1a8d5a0c0 2024-09-16T00:21:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.849+0000 7f31d4de4640 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55e1a8d5a0c0 0x55e1a8d5c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.850+0000 7f31d4de4640 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55e1a8d5a0c0 0x55e1a8d5c560 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4165f362133813c server_cookie=8aa4cd36e8a8d0a8 in_seq=0 out_seq=0 2024-09-16T00:21:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.850+0000 7f31affff640 10 client.26113 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:21:55.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.851+0000 7f31affff640 1 -- 172.21.15.78:0/3228870854 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f31bc0f2070 con 0x55e1a8d5a0c0 2024-09-16T00:21:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.851+0000 7f31affff640 10 client.26113 handle_command_reply: tid=0 2024-09-16T00:21:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.851+0000 7f31d9421740 1 client.26113 shutdown 2024-09-16T00:21:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31d9421740 1 -- 172.21.15.78:0/3228870854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 msgr2=0x55e1a8d36600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31d9421740 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d36600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f31d0051790 tx=0x7f31d006f780 comp rx=0 tx=0).stop 2024-09-16T00:21:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31d9421740 1 -- 172.21.15.78:0/3228870854 shutdown_connections 2024-09-16T00:21:55.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31d9421740 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55e1a8d5a0c0 0x55e1a8d5c560 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31d9421740 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d36600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31d9421740 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31affff640 0 client.26113 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:21:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31d9421740 1 --2- 172.21.15.78:0/3228870854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d38e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.852+0000 7f31d9421740 1 -- 172.21.15.78:0/3228870854 >> 172.21.15.78:0/3228870854 conn(0x55e1a8cf7310 msgr2=0x55e1a8d30ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.853+0000 7f31d9421740 1 -- 172.21.15.78:0/3228870854 shutdown_connections 2024-09-16T00:21:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.853+0000 7f31d9421740 1 -- 172.21.15.78:0/3228870854 wait complete. 2024-09-16T00:21:55.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.853+0000 7f31d9421740 20 client.26113 trim_cache size 0 max 16384 2024-09-16T00:21:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 475.0 s, 676.0 tps, lat 147.040 ms stddev 167.634 2024-09-16T00:21:55.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.872+0000 7f31d9421740 1 Processor -- start 2024-09-16T00:21:55.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.872+0000 7f31d9421740 1 -- start start 2024-09-16T00:21:55.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.872+0000 7f31d9421740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2e780 0x55e1a8d56600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.872+0000 7f31d9421740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d61e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.872+0000 7f31d9421740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d39520 0x55e1a8d62370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.872+0000 7f31d9421740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31b400fbc0 con 0x55e1a8d39520 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d9421740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4011700 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d9421740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4010290 con 0x55e1a8d2e780 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d4de4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2e780 0x55e1a8d56600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d61e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d4de4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2e780 0x55e1a8d56600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:49520/0 (socket says 172.21.15.78:49520) 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d61e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:44890/0 (socket says 172.21.15.78:44890) 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d4de4640 1 -- 172.21.15.78:0/1350213724 learned_addr learned my addr 172.21.15.78:0/1350213724 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:55.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d55e5640 1 --2- 172.21.15.78:0/1350213724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d39520 0x55e1a8d62370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d4de4640 1 -- 172.21.15.78:0/1350213724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 msgr2=0x55e1a8d61e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d4de4640 1 --2- 172.21.15.78:0/1350213724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d61e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d4de4640 1 -- 172.21.15.78:0/1350213724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d39520 msgr2=0x55e1a8d62370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d4de4640 1 --2- 172.21.15.78:0/1350213724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d39520 0x55e1a8d62370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.873+0000 7f31d4de4640 1 -- 172.21.15.78:0/1350213724 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a8d2f550 con 0x55e1a8d2e780 2024-09-16T00:21:55.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.874+0000 7f31d4de4640 1 --2- 172.21.15.78:0/1350213724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2e780 0x55e1a8d56600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31c812ab50 tx=0x7f31c40041d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e4091d62972366 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.874+0000 7f31ba7fc640 1 -- 172.21.15.78:0/1350213724 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4052070 con 0x55e1a8d2e780 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.874+0000 7f31ba7fc640 1 -- 172.21.15.78:0/1350213724 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c4005040 con 0x55e1a8d2e780 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.874+0000 7f31ba7fc640 1 -- 172.21.15.78:0/1350213724 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c4039820 con 0x55e1a8d2e780 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 -- 172.21.15.78:0/1350213724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2e780 msgr2=0x55e1a8d56600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 --2- 172.21.15.78:0/1350213724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2e780 0x55e1a8d56600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31c812ab50 tx=0x7f31c40041d0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 -- 172.21.15.78:0/1350213724 shutdown_connections 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 --2- 172.21.15.78:0/1350213724 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d39520 0x55e1a8d62370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 --2- 172.21.15.78:0/1350213724 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d61e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 --2- 172.21.15.78:0/1350213724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d2e780 0x55e1a8d56600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 -- 172.21.15.78:0/1350213724 >> 172.21.15.78:0/1350213724 conn(0x55e1a8cf7310 msgr2=0x55e1a8d351b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 -- 172.21.15.78:0/1350213724 shutdown_connections 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.875+0000 7f31d9421740 1 -- 172.21.15.78:0/1350213724 wait complete. 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.876+0000 7f31d9421740 1 -- start start 2024-09-16T00:21:55.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.876+0000 7f31d9421740 1 Processor -- start 2024-09-16T00:21:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.876+0000 7f31d9421740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d52c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.876+0000 7f31d9421740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d4df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31d9421740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d4e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31d9421740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4011030 con 0x55e1a8d2e780 2024-09-16T00:21:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d4df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31d9421740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f31b4010960 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d4df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:44894/0 (socket says 172.21.15.78:44894) 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 -- 172.21.15.78:0/3577575505 learned_addr learned my addr 172.21.15.78:0/3577575505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31b400dba0 con 0x55e1a8d39520 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31d55e5640 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d4e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31d9421740 10 client.0 _ll_register_callbacks cb 0x7f31d00687a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31d4de4640 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d52c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 -- 172.21.15.78:0/3577575505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 msgr2=0x55e1a8d4e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d4e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 -- 172.21.15.78:0/3577575505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 msgr2=0x55e1a8d52c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d52c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 -- 172.21.15.78:0/3577575505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a8d312d0 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31d4de4640 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d52c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31cffff640 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d4df60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31bc099a90 tx=0x7f31bc0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cddae4868442bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31affff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:21:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.877+0000 7f31affff640 1 -- 172.21.15.78:0/3577575505 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc099360 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31affff640 1 -- 172.21.15.78:0/3577575505 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f31bc0bd0b0 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31c4005040 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31affff640 1 -- 172.21.15.78:0/3577575505 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31bc09cc00 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 10 client.26300 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e1a8dc4980 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31affff640 1 -- 172.21.15.78:0/3577575505 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31bc0f2040 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31affff640 1 -- 172.21.15.78:0/3577575505 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f31bc09d320 con 0x55e1a8d2fcf0 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 10 client.26300 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 10 client.26300 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 20 client.26300 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55e1a8d5a0c0 0x55e1a8dc6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 4 client.26300 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:21:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f31c4005040 con 0x55e1a8d5a0c0 2024-09-16T00:21:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.878+0000 7f31d4de4640 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55e1a8d5a0c0 0x55e1a8dc6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.879+0000 7f31affff640 10 client.26300 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.881+0000 7f31d4de4640 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55e1a8d5a0c0 0x55e1a8dc6bc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a98b54484e69efe9 server_cookie=8d29515472921c44 in_seq=0 out_seq=0 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31affff640 1 -- 172.21.15.78:0/3577575505 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f31c4005040 con 0x55e1a8d5a0c0 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31affff640 10 client.26300 handle_command_reply: tid=0 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 client.26300 shutdown 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 msgr2=0x55e1a8d4df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d4df60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31bc099a90 tx=0x7f31bc0092d0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 shutdown_connections 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55e1a8d5a0c0 0x55e1a8dc6bc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31affff640 0 client.26300 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:21:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1a8d39520 0x55e1a8d4e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e1a8d2fcf0 0x55e1a8d4df60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 --2- 172.21.15.78:0/3577575505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e1a8d2e780 0x55e1a8d52c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 >> 172.21.15.78:0/3577575505 conn(0x55e1a8cf7310 msgr2=0x55e1a8d37920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 shutdown_connections 2024-09-16T00:21:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.882+0000 7f31d9421740 1 -- 172.21.15.78:0/3577575505 wait complete. 2024-09-16T00:21:55.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.883+0000 7f31d9421740 20 client.26300 trim_cache size 0 max 16384 2024-09-16T00:21:55.885 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 -- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31c00a3450 msgr2=0x7f31c00a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31c00a3450 0x7f31c00a5910 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f31d00828e0 tx=0x7f31d0082ce0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 -- 172.21.15.78:0/1448409021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31c80952d0 msgr2=0x7f31c812fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31c80952d0 0x7f31c812fd90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e790 tx=0x7f31bc00ec50 comp rx=0 tx=0).stop 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 -- 172.21.15.78:0/1448409021 shutdown_connections 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f31c00a3450 0x7f31c00a5910 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31c809a550 0x7f31c8129a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31c80952d0 0x7f31c812fd90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 --2- 172.21.15.78:0/1448409021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f31c8094930 0x7f31c812f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 -- 172.21.15.78:0/1448409021 >> 172.21.15.78:0/1448409021 conn(0x7f31c800a1f0 msgr2=0x7f31c80935d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 -- 172.21.15.78:0/1448409021 shutdown_connections 2024-09-16T00:21:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:55.885+0000 7f31d5de6640 1 -- 172.21.15.78:0/1448409021 wait complete. 2024-09-16T00:21:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:55 smithi078 ceph-mon[31157]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-09-16T00:21:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:55 smithi193 ceph-mon[35484]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-09-16T00:21:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:55 smithi148 ceph-mon[35599]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 5.0 MiB/s wr, 538 op/s 2024-09-16T00:21:56.978 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:21:57.436 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:21:57.892 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:57 smithi078 ceph-mon[31157]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-09-16T00:21:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:57 smithi193 ceph-mon[35484]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-09-16T00:21:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:57 smithi148 ceph-mon[35599]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 518 op/s 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.025+0000 7f152e02b640 1 -- 172.21.15.78:0/2331290619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15200aa230 msgr2=0x7f15200aa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.025+0000 7f152e02b640 1 --2- 172.21.15.78:0/2331290619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15200aa230 0x7f15200aa610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f151c009860 tx=0x7f151c02a380 comp rx=0 tx=0).stop 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.025+0000 7f152e02b640 1 -- 172.21.15.78:0/2331290619 shutdown_connections 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.025+0000 7f152e02b640 1 --2- 172.21.15.78:0/2331290619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15200a4d30 0x7f15200a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.025+0000 7f152e02b640 1 --2- 172.21.15.78:0/2331290619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15200aab50 0x7f15200a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.025+0000 7f152e02b640 1 --2- 172.21.15.78:0/2331290619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15200aa230 0x7f15200aa610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.025+0000 7f152e02b640 1 -- 172.21.15.78:0/2331290619 >> 172.21.15.78:0/2331290619 conn(0x7f152001a0a0 msgr2=0x7f152001a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.027+0000 7f152e02b640 1 -- 172.21.15.78:0/2331290619 shutdown_connections 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.027+0000 7f152e02b640 1 -- 172.21.15.78:0/2331290619 wait complete. 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.030+0000 7f152e02b640 1 Processor -- start 2024-09-16T00:21:58.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.030+0000 7f152e02b640 1 -- start start 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.031+0000 7f152e02b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15200a4d30 0x7f15201408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.031+0000 7f152e02b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15200aa230 0x7f1520140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.031+0000 7f152e02b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15200aab50 0x7f152013aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.031+0000 7f152e02b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15200ae7e0 con 0x7f15200aa230 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.031+0000 7f152e02b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f15200ae660 con 0x7f15200a4d30 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.031+0000 7f152e02b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15200ae960 con 0x7f15200aab50 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.034+0000 7f15277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15200a4d30 0x7f15201408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.034+0000 7f15277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15200a4d30 0x7f15201408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:45532/0 (socket says 172.21.15.78:45532) 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.034+0000 7f15277fe640 1 -- 172.21.15.78:0/3696418383 learned_addr learned my addr 172.21.15.78:0/3696418383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.034+0000 7f1527fff640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15200aab50 0x7f152013aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.036+0000 7f1526ffd640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15200aa230 0x7f1520140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.036+0000 7f1526ffd640 1 -- 172.21.15.78:0/3696418383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15200aab50 msgr2=0x7f152013aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.036+0000 7f1526ffd640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15200aab50 0x7f152013aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.036+0000 7f1526ffd640 1 -- 172.21.15.78:0/3696418383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15200a4d30 msgr2=0x7f15201408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.036+0000 7f1526ffd640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15200a4d30 0x7f15201408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.036+0000 7f1526ffd640 1 -- 172.21.15.78:0/3696418383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152013b1e0 con 0x7f15200aa230 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.037+0000 7f1526ffd640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15200aa230 0x7f1520140e30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1528065a40 tx=0x7f152806ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.037+0000 7f1524ff9640 1 -- 172.21.15.78:0/3696418383 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152807e070 con 0x7f15200aa230 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.037+0000 7f152e02b640 1 -- 172.21.15.78:0/3696418383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152013b4d0 con 0x7f15200aa230 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.037+0000 7f152e02b640 1 -- 172.21.15.78:0/3696418383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15201697d0 con 0x7f15200aa230 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.041+0000 7f1530404740 1 Processor -- start 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.041+0000 7f1524ff9640 1 -- 172.21.15.78:0/3696418383 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f152806bd70 con 0x7f15200aa230 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.041+0000 7f1524ff9640 1 -- 172.21.15.78:0/3696418383 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528076560 con 0x7f15200aa230 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.043+0000 7f1530404740 1 -- start start 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.043+0000 7f1530404740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61152240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.043+0000 7f1530404740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e61152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.043+0000 7f1530404740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 0x556e6115f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.043+0000 7f1530404740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e610b82f0 con 0x556e6115d520 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.043+0000 7f1530404740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e6105fcf0 con 0x556e61152780 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.043+0000 7f1530404740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e610b42b0 con 0x556e61153cf0 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.047+0000 7f15277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e61152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.047+0000 7f15277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e61152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:45540/0 (socket says 172.21.15.78:45540) 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.047+0000 7f15277fe640 1 -- 172.21.15.78:0/358773798 learned_addr learned my addr 172.21.15.78:0/358773798 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.048+0000 7f1527fff640 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.052+0000 7f1526ffd640 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 0x556e6115f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.053+0000 7f1524ff9640 1 -- 172.21.15.78:0/3696418383 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f15280767c0 con 0x7f15200aa230 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.054+0000 7f1524ff9640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f15180a3450 0x7f15180a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.054+0000 7f1524ff9640 1 -- 172.21.15.78:0/3696418383 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1528134f40 con 0x7f15200aa230 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.056+0000 7f15277fe640 1 -- 172.21.15.78:0/358773798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 msgr2=0x556e61152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:58.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.056+0000 7f15277fe640 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.056+0000 7f15277fe640 1 -- 172.21.15.78:0/358773798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 msgr2=0x556e6115f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.056+0000 7f15277fe640 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 0x556e6115f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.056+0000 7f15277fe640 1 -- 172.21.15.78:0/358773798 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e610443a0 con 0x556e61152780 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.056+0000 7f1527fff640 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.056+0000 7f1526ffd640 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 0x556e6115f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.058+0000 7f15277fe640 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e61152bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f151c035040 tx=0x7f151c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1305f2e78b311e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.059+0000 7f1527fff640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f15180a3450 0x7f15180a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.063+0000 7f1511ffb640 1 -- 172.21.15.78:0/358773798 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c041070 con 0x556e61152780 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.063+0000 7f1511ffb640 1 -- 172.21.15.78:0/358773798 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f151c0305b0 con 0x556e61152780 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 -- 172.21.15.78:0/358773798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 msgr2=0x556e61152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e61152bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f151c035040 tx=0x7f151c009580 comp rx=0 tx=0).stop 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 -- 172.21.15.78:0/358773798 shutdown_connections 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 0x556e6115f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e61152bc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 --2- 172.21.15.78:0/358773798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 -- 172.21.15.78:0/358773798 >> 172.21.15.78:0/358773798 conn(0x556e6111b310 msgr2=0x556e6114ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 -- 172.21.15.78:0/358773798 shutdown_connections 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1530404740 1 -- 172.21.15.78:0/358773798 wait complete. 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.064+0000 7f1527fff640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f15180a3450 0x7f15180a5910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1514004650 tx=0x7f15140046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.065+0000 7f1530404740 1 -- start start 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.065+0000 7f1530404740 1 Processor -- start 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.068+0000 7f1530404740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e6115cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.068+0000 7f1530404740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61159f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.068+0000 7f1530404740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 0x556e6115a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.068+0000 7f1530404740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556e61160210 con 0x556e6115d520 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f1530404740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e61073060 con 0x556e61152780 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f1530404740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e61160390 con 0x556e61153cf0 2024-09-16T00:21:58.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f1530404740 10 client.0 _ll_register_callbacks cb 0x556e60dc68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f15277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e6115cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f15277fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e6115cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:45554/0 (socket says 172.21.15.78:45554) 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f15277fe640 1 -- 172.21.15.78:0/880589812 learned_addr learned my addr 172.21.15.78:0/880589812 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f1526ffd640 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f15277fe640 1 -- 172.21.15.78:0/880589812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 msgr2=0x556e61159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f15277fe640 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f15277fe640 1 -- 172.21.15.78:0/880589812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 msgr2=0x556e6115a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f15277fe640 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 0x556e6115a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.069+0000 7f15277fe640 1 -- 172.21.15.78:0/880589812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e60e293c0 con 0x556e61152780 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.070+0000 7f15277fe640 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e6115cce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f151c009980 tx=0x7f151c030760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d67264b687cdcf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.070+0000 7f1506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.072+0000 7f1506ffd640 1 -- 172.21.15.78:0/880589812 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c049020 con 0x556e61152780 2024-09-16T00:21:58.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.073+0000 7f1506ffd640 1 -- 172.21.15.78:0/880589812 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f151c004240 con 0x556e61152780 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.073+0000 7f1530404740 1 -- 172.21.15.78:0/880589812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151c0305b0 con 0x556e61152780 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.073+0000 7f1506ffd640 1 -- 172.21.15.78:0/880589812 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f151c041040 con 0x556e61152780 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.074+0000 7f1530404740 10 client.26312 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.074+0000 7f1530404740 1 -- 172.21.15.78:0/880589812 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556e6116bad0 con 0x556e61152780 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1506ffd640 1 -- 172.21.15.78:0/880589812 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f151c0305b0 con 0x556e61152780 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1506ffd640 1 -- 172.21.15.78:0/880589812 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f151c003750 con 0x556e61152780 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1530404740 10 client.26312 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1530404740 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1530404740 20 client.26312 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1530404740 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556e6116ce20 0x556e6116f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1530404740 4 client.26312 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1530404740 1 -- 172.21.15.78:0/880589812 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e610c5860 con 0x556e6116ce20 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.079+0000 7f1526ffd640 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556e6116ce20 0x556e6116f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.081+0000 7f1526ffd640 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556e6116ce20 0x556e6116f210 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad26ea20f71900d server_cookie=5134bf5df72dc511 in_seq=0 out_seq=0 2024-09-16T00:21:58.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.081+0000 7f1506ffd640 10 client.26312 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.082+0000 7f1506ffd640 1 -- 172.21.15.78:0/880589812 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e610c5860 con 0x556e6116ce20 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.082+0000 7f1506ffd640 10 client.26312 handle_command_reply: tid=0 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.082+0000 7f1530404740 1 client.26312 shutdown 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 -- 172.21.15.78:0/880589812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 msgr2=0x556e6115cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e6115cce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f151c009980 tx=0x7f151c030760 comp rx=0 tx=0).stop 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 -- 172.21.15.78:0/880589812 shutdown_connections 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556e6116ce20 0x556e6116f210 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115d520 0x556e6115a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e61159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 --2- 172.21.15.78:0/880589812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e61152780 0x556e6115cce0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f151c009980 tx=0x7f151c030760 comp rx=0 tx=0).stop 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 -- 172.21.15.78:0/880589812 >> 172.21.15.78:0/880589812 conn(0x556e6111b310 msgr2=0x556e61154b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1506ffd640 0 client.26312 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:58.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.086+0000 7f1530404740 1 -- 172.21.15.78:0/880589812 shutdown_connections 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.090+0000 7f1530404740 1 -- 172.21.15.78:0/880589812 wait complete. 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.091+0000 7f1530404740 20 client.26312 trim_cache size 0 max 16384 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.120+0000 7f1530404740 1 Processor -- start 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.121+0000 7f1530404740 1 -- start start 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.121+0000 7f1530404740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e6115e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.121+0000 7f1530404740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e6115ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f1530404740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115fa90 0x556e61157230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f1526ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e6115ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f1526ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e6115ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:45562/0 (socket says 172.21.15.78:45562) 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f1526ffd640 1 -- 172.21.15.78:0/84716574 learned_addr learned my addr 172.21.15.78:0/84716574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f15277fe640 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e6115e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f1530404740 1 -- 172.21.15.78:0/84716574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f150c00fc00 con 0x556e6115fa90 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f1530404740 1 -- 172.21.15.78:0/84716574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f150c011740 con 0x556e6115d520 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f1530404740 1 -- 172.21.15.78:0/84716574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0102d0 con 0x556e61153cf0 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.122+0000 7f1527fff640 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115fa90 0x556e61157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.123+0000 7f1526ffd640 1 -- 172.21.15.78:0/84716574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 msgr2=0x556e6115e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.123+0000 7f1526ffd640 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e6115e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.123+0000 7f1526ffd640 1 -- 172.21.15.78:0/84716574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115fa90 msgr2=0x556e61157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.123+0000 7f1526ffd640 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115fa90 0x556e61157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.123+0000 7f1526ffd640 1 -- 172.21.15.78:0/84716574 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6114f350 con 0x556e6115d520 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.123+0000 7f1526ffd640 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e6115ee50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1528051500 tx=0x7f1528085000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798e82de102b331a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:58.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.125+0000 7f15117fa640 1 -- 172.21.15.78:0/84716574 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528105550 con 0x556e6115d520 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.125+0000 7f15117fa640 1 -- 172.21.15.78:0/84716574 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1528114070 con 0x556e6115d520 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.125+0000 7f15117fa640 1 -- 172.21.15.78:0/84716574 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15280fd9c0 con 0x556e6115d520 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.128+0000 7f1530404740 1 -- 172.21.15.78:0/84716574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 msgr2=0x556e6115ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.128+0000 7f1530404740 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e6115ee50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1528051500 tx=0x7f1528085000 comp rx=0 tx=0).stop 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.131+0000 7f1530404740 1 -- 172.21.15.78:0/84716574 shutdown_connections 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.131+0000 7f1530404740 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e6115fa90 0x556e61157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.131+0000 7f1530404740 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e6115ee50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.131+0000 7f1530404740 1 --2- 172.21.15.78:0/84716574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e61153cf0 0x556e6115e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.131+0000 7f1530404740 1 -- 172.21.15.78:0/84716574 >> 172.21.15.78:0/84716574 conn(0x556e6111b310 msgr2=0x556e61158880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.132+0000 7f1530404740 1 -- 172.21.15.78:0/84716574 shutdown_connections 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.132+0000 7f1530404740 1 -- 172.21.15.78:0/84716574 wait complete. 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.133+0000 7f1530404740 1 -- start start 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.133+0000 7f1530404740 1 Processor -- start 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.134+0000 7f1530404740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e61153cf0 0x556e61172420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.134+0000 7f1530404740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e61172960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.134+0000 7f1530404740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6115fa90 0x556e6116f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.134+0000 7f1530404740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f150c011070 con 0x556e61153cf0 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.134+0000 7f1530404740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f150c0109a0 con 0x556e6115d520 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.134+0000 7f1530404740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f150c00dbe0 con 0x556e6115fa90 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.134+0000 7f1527fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6115fa90 0x556e6116f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1527fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6115fa90 0x556e6116f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42806/0 (socket says 172.21.15.78:42806) 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1526ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e61172960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1527fff640 1 -- 172.21.15.78:0/1414971483 learned_addr learned my addr 172.21.15.78:0/1414971483 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f15277fe640 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e61153cf0 0x556e61172420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1530404740 10 client.0 _ll_register_callbacks cb 0x7f151c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1526ffd640 1 -- 172.21.15.78:0/1414971483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6115fa90 msgr2=0x556e6116f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1526ffd640 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6115fa90 0x556e6116f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1526ffd640 1 -- 172.21.15.78:0/1414971483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e61153cf0 msgr2=0x556e61172420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1526ffd640 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e61153cf0 0x556e61172420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.135+0000 7f1526ffd640 1 -- 172.21.15.78:0/1414971483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6116f9f0 con 0x556e6115d520 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.136+0000 7f1526ffd640 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e61172960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1528101410 tx=0x7f15280fda60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed91fda26138a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.137+0000 7f1506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.137+0000 7f1506ffd640 1 -- 172.21.15.78:0/1414971483 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528105550 con 0x556e6115d520 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.137+0000 7f1506ffd640 1 -- 172.21.15.78:0/1414971483 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f15281014b0 con 0x556e6115d520 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.137+0000 7f1530404740 1 -- 172.21.15.78:0/1414971483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1528114070 con 0x556e6115d520 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.137+0000 7f1506ffd640 1 -- 172.21.15.78:0/1414971483 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1528109e10 con 0x556e6115d520 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.137+0000 7f1530404740 10 client.26318 fetch_fsmap learned FSMap version 34 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.137+0000 7f1530404740 1 -- 172.21.15.78:0/1414971483 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556e611d8180 con 0x556e6115d520 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.139+0000 7f1506ffd640 1 -- 172.21.15.78:0/1414971483 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1528114070 con 0x556e6115d520 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.139+0000 7f1506ffd640 1 -- 172.21.15.78:0/1414971483 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1528132ce0 con 0x556e6115d520 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.139+0000 7f1530404740 10 client.26318 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:21:58.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.139+0000 7f1530404740 10 client.26318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.139+0000 7f1530404740 20 client.26318 populate_metadata read hostname 'smithi078' 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.139+0000 7f1530404740 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556e611d8700 0x556e611dab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.139+0000 7f1530404740 4 client.26318 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.139+0000 7f1530404740 1 -- 172.21.15.78:0/1414971483 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e610bfd70 con 0x556e611d8700 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.140+0000 7f15277fe640 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556e611d8700 0x556e611dab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.153+0000 7f1506ffd640 10 client.26318 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.153+0000 7f15277fe640 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556e611d8700 0x556e611dab20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28de5d3a99fc8fa1 server_cookie=d146cef43a6215af in_seq=0 out_seq=0 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.153+0000 7f1506ffd640 1 -- 172.21.15.78:0/1414971483 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e610bfd70 con 0x556e611d8700 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.153+0000 7f1506ffd640 10 client.26318 handle_command_reply: tid=0 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.153+0000 7f1530404740 1 client.26318 shutdown 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 -- 172.21.15.78:0/1414971483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 msgr2=0x556e61172960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e61172960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1528101410 tx=0x7f15280fda60 comp rx=0 tx=0).stop 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 -- 172.21.15.78:0/1414971483 shutdown_connections 2024-09-16T00:21:58.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556e611d8700 0x556e611dab20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e6115fa90 0x556e6116f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.326 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:21:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e6115d520 0x556e61172960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 --2- 172.21.15.78:0/1414971483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556e61153cf0 0x556e61172420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 -- 172.21.15.78:0/1414971483 >> 172.21.15.78:0/1414971483 conn(0x556e6111b310 msgr2=0x556e6115b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1506ffd640 0 client.26318 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 -- 172.21.15.78:0/1414971483 shutdown_connections 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 1 -- 172.21.15.78:0/1414971483 wait complete. 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.154+0000 7f1530404740 20 client.26318 trim_cache size 0 max 16384 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.155+0000 7f152e02b640 1 -- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f15180a3450 msgr2=0x7f15180a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.155+0000 7f152e02b640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f15180a3450 0x7f15180a5910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1514004650 tx=0x7f15140046c0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.155+0000 7f152e02b640 1 -- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15200aa230 msgr2=0x7f1520140e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.155+0000 7f152e02b640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15200aa230 0x7f1520140e30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1528065a40 tx=0x7f152806ded0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.157+0000 7f152e02b640 1 -- 172.21.15.78:0/3696418383 shutdown_connections 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.157+0000 7f152e02b640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f15180a3450 0x7f15180a5910 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.157+0000 7f152e02b640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15200aab50 0x7f152013aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.157+0000 7f152e02b640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15200aa230 0x7f1520140e30 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.157+0000 7f152e02b640 1 --2- 172.21.15.78:0/3696418383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f15200a4d30 0x7f15201408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:21:58.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.157+0000 7f152e02b640 1 -- 172.21.15.78:0/3696418383 >> 172.21.15.78:0/3696418383 conn(0x7f152001a0a0 msgr2=0x7f15200a69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:21:58.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.157+0000 7f152e02b640 1 -- 172.21.15.78:0/3696418383 shutdown_connections 2024-09-16T00:21:58.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:21:58.157+0000 7f152e02b640 1 -- 172.21.15.78:0/3696418383 wait complete. 2024-09-16T00:22:00.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:21:59 smithi078 ceph-mon[31157]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-09-16T00:22:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:21:59 smithi193 ceph-mon[35484]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-09-16T00:22:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:21:59 smithi148 ceph-mon[35599]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-09-16T00:22:00.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 480.0 s, 966.8 tps, lat 83.686 ms stddev 148.010 2024-09-16T00:22:01.091 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:22:01.091 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:22:01.858 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:02.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:01 smithi078 ceph-mon[31157]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 783 op/s 2024-09-16T00:22:02.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:01 smithi193 ceph-mon[35484]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 783 op/s 2024-09-16T00:22:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:01 smithi148 ceph-mon[35599]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 783 op/s 2024-09-16T00:22:04.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:03 smithi078 ceph-mon[31157]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-09-16T00:22:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:03 smithi193 ceph-mon[35484]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-09-16T00:22:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:03 smithi148 ceph-mon[35599]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-09-16T00:22:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 485.0 s, 835.3 tps, lat 144.468 ms stddev 231.818 2024-09-16T00:22:06.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:05 smithi078 ceph-mon[31157]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-16T00:22:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:05 smithi193 ceph-mon[35484]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-16T00:22:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:05 smithi148 ceph-mon[35599]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-16T00:22:08.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:07 smithi193 ceph-mon[35484]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-09-16T00:22:08.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:07 smithi148 ceph-mon[35599]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-09-16T00:22:08.272 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:07 smithi078 ceph-mon[31157]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 16 MiB/s wr, 1.49k op/s 2024-09-16T00:22:08.371 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:22:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:22:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "3a3e6da9-a6f5-4540-a950-8192aa35ca06", 2024-09-16T00:22:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:22:08.371 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.394+0000 7f57ef89f640 1 -- 172.21.15.78:0/3079826045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e811b140 msgr2=0x7f57e811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.394+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3079826045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e811b140 0x7f57e811d530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f57e0009be0 tx=0x7f57e002a710 comp rx=0 tx=0).stop 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.394+0000 7f57ef89f640 1 -- 172.21.15.78:0/3079826045 shutdown_connections 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.394+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3079826045 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e811b140 0x7f57e811d530 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.394+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3079826045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57e8109830 0x7f57e8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.394+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3079826045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e8109450 0x7f57e8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.394+0000 7f57ef89f640 1 -- 172.21.15.78:0/3079826045 >> 172.21.15.78:0/3079826045 conn(0x7f57e806e140 msgr2=0x7f57e806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.400+0000 7f57ef89f640 1 -- 172.21.15.78:0/3079826045 shutdown_connections 2024-09-16T00:22:08.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.401+0000 7f57ef89f640 1 -- 172.21.15.78:0/3079826045 wait complete. 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.401+0000 7f57ef89f640 1 Processor -- start 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.408+0000 7f57ef89f640 1 -- start start 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.408+0000 7f57ef89f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e8109450 0x7f57e8118650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.408+0000 7f57ef89f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e8109830 0x7f57e8118b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.408+0000 7f57ef89f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57e811b140 0x7f57e81118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.408+0000 7f57ef89f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8120df0 con 0x7f57e811b140 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.408+0000 7f57ef89f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8120c70 con 0x7f57e8109830 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.408+0000 7f57ef89f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57e8120f70 con 0x7f57e8109450 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.409+0000 7f57ece3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e8109830 0x7f57e8118b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.409+0000 7f57ece3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e8109830 0x7f57e8118b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34588/0 (socket says 172.21.15.78:34588) 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.409+0000 7f57ece3a640 1 -- 172.21.15.78:0/3180051097 learned_addr learned my addr 172.21.15.78:0/3180051097 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:08.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.410+0000 7f57ed63b640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e8109450 0x7f57e8118650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.412+0000 7f57ed63b640 1 -- 172.21.15.78:0/3180051097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e8109830 msgr2=0x7f57e8118b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.412+0000 7f57ed63b640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e8109830 0x7f57e8118b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.412+0000 7f57ed63b640 1 -- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57e811b140 msgr2=0x7f57e81118c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.412+0000 7f57ed63b640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57e811b140 0x7f57e81118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.412+0000 7f57ed63b640 1 -- 172.21.15.78:0/3180051097 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e8112180 con 0x7f57e8109450 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.413+0000 7f57ed63b640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e8109450 0x7f57e8118650 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e790 tx=0x7f57e400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.416+0000 7f57de7fc640 1 -- 172.21.15.78:0/3180051097 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e400eea0 con 0x7f57e8109450 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.416+0000 7f57de7fc640 1 -- 172.21.15.78:0/3180051097 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e400ccc0 con 0x7f57e8109450 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.416+0000 7f57de7fc640 1 -- 172.21.15.78:0/3180051097 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4018750 con 0x7f57e8109450 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.416+0000 7f57ef89f640 1 -- 172.21.15.78:0/3180051097 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e8112410 con 0x7f57e8109450 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.416+0000 7f57ef89f640 1 -- 172.21.15.78:0/3180051097 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e81163b0 con 0x7f57e8109450 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.424+0000 7f57de7fc640 1 -- 172.21.15.78:0/3180051097 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f57e40188f0 con 0x7f57e8109450 2024-09-16T00:22:08.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.427+0000 7f57f1c78740 1 Processor -- start 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.428+0000 7f57de7fc640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57d40a3450 0x7f57d40a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.428+0000 7f57de7fc640 1 -- 172.21.15.78:0/3180051097 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f57e40cfe50 con 0x7f57e8109450 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.429+0000 7f57f1c78740 1 -- start start 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.429+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a8b60 0x560f0f6a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.429+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.429+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.429+0000 7f57f1c78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560f0f60e280 con 0x560f0f6a8b60 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.429+0000 7f57f1c78740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560f0f5b5b10 con 0x560f0f6a9480 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.429+0000 7f57f1c78740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560f0f60a240 con 0x560f0f6b3d40 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.433+0000 7f57ede3c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.433+0000 7f57ede3c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37828/0 (socket says 172.21.15.78:37828) 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.433+0000 7f57ede3c640 1 -- 172.21.15.78:0/3801128254 learned_addr learned my addr 172.21.15.78:0/3801128254 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.433+0000 7f57ece3a640 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.437+0000 7f57ed63b640 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a8b60 0x560f0f6a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.437+0000 7f57ede3c640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57d40a3450 0x7f57d40a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.437+0000 7f57ede3c640 1 -- 172.21.15.78:0/3801128254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 msgr2=0x560f0f6b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.437+0000 7f57ede3c640 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.437+0000 7f57ede3c640 1 -- 172.21.15.78:0/3801128254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a8b60 msgr2=0x560f0f6a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.437+0000 7f57ede3c640 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a8b60 0x560f0f6a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.437+0000 7f57ede3c640 1 -- 172.21.15.78:0/3801128254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0f5c21f0 con 0x560f0f6b3d40 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.437+0000 7f57ede3c640 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b6130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57e0005c40 tx=0x7f57e0030760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd16fd6dec7c8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.438+0000 7f57ede3c640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57d40a3450 0x7f57d40a5910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57e00099a0 tx=0x7f57e0048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.439+0000 7f57d37fe640 1 -- 172.21.15.78:0/3801128254 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0036030 con 0x560f0f6b3d40 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.439+0000 7f57d37fe640 1 -- 172.21.15.78:0/3801128254 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e0031bc0 con 0x560f0f6b3d40 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.440+0000 7f57f1c78740 1 -- 172.21.15.78:0/3801128254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 msgr2=0x560f0f6b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.440+0000 7f57f1c78740 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b6130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57e0005c40 tx=0x7f57e0030760 comp rx=0 tx=0).stop 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.442+0000 7f57f1c78740 1 -- 172.21.15.78:0/3801128254 shutdown_connections 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.442+0000 7f57f1c78740 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b6130 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.442+0000 7f57f1c78740 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.442+0000 7f57f1c78740 1 --2- 172.21.15.78:0/3801128254 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a8b60 0x560f0f6a8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.442+0000 7f57f1c78740 1 -- 172.21.15.78:0/3801128254 >> 172.21.15.78:0/3801128254 conn(0x560f0f6a50f0 msgr2=0x560f0f6a54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.446+0000 7f57f1c78740 1 -- 172.21.15.78:0/3801128254 shutdown_connections 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.447+0000 7f57f1c78740 1 -- 172.21.15.78:0/3801128254 wait complete. 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.447+0000 7f57f1c78740 1 -- start start 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.447+0000 7f57f1c78740 1 Processor -- start 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 0x560f0f6b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57f1c78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560f0f6b6960 con 0x560f0f6b3d40 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57f1c78740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560f0f5c8f10 con 0x560f0f6a9480 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57f1c78740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560f0f6b6ae0 con 0x560f0f6a8b60 2024-09-16T00:22:08.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57f1c78740 10 client.0 _ll_register_callbacks cb 0x560f0f591a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57ed63b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57ece3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57ece3a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34600/0 (socket says 172.21.15.78:34600) 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.448+0000 7f57ece3a640 1 -- 172.21.15.78:0/1673054787 learned_addr learned my addr 172.21.15.78:0/1673054787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.450+0000 7f57ede3c640 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 0x560f0f6b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.451+0000 7f57ed63b640 1 -- 172.21.15.78:0/1673054787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 msgr2=0x560f0f6b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.451+0000 7f57ed63b640 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.451+0000 7f57ed63b640 1 -- 172.21.15.78:0/1673054787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 msgr2=0x560f0f6b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.451+0000 7f57ed63b640 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 0x560f0f6b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.451+0000 7f57ed63b640 1 -- 172.21.15.78:0/1673054787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0f37f6a0 con 0x560f0f6a8b60 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.454+0000 7f57ed63b640 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6b37d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005fd0 tx=0x7f57e409c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8838778c45b307f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.456+0000 7f57d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.456+0000 7f57d0ff9640 1 -- 172.21.15.78:0/1673054787 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40b4060 con 0x560f0f6a8b60 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.456+0000 7f57f1c78740 1 -- 172.21.15.78:0/1673054787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e0031bc0 con 0x560f0f6a8b60 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.456+0000 7f57d0ff9640 1 -- 172.21.15.78:0/1673054787 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f57e40a0110 con 0x560f0f6a8b60 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.456+0000 7f57d0ff9640 1 -- 172.21.15.78:0/1673054787 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f57e4098720 con 0x560f0f6a8b60 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.458+0000 7f57f1c78740 10 client.26146 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.458+0000 7f57f1c78740 1 -- 172.21.15.78:0/1673054787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560f0f6c2570 con 0x560f0f6a8b60 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.461+0000 7f57d0ff9640 1 -- 172.21.15.78:0/1673054787 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e0031bc0 con 0x560f0f6a8b60 2024-09-16T00:22:08.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.461+0000 7f57d0ff9640 1 -- 172.21.15.78:0/1673054787 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f57e409b500 con 0x560f0f6a8b60 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.463+0000 7f57f1c78740 10 client.26146 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.463+0000 7f57f1c78740 10 client.26146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.463+0000 7f57f1c78740 20 client.26146 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.463+0000 7f57f1c78740 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f0f6c3980 0x560f0f6c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.463+0000 7f57f1c78740 4 client.26146 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.463+0000 7f57f1c78740 1 -- 172.21.15.78:0/1673054787 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f0f61b7f0 con 0x560f0f6c3980 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.463+0000 7f57ece3a640 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f0f6c3980 0x560f0f6c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.464+0000 7f57ece3a640 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f0f6c3980 0x560f0f6c5d70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67b53bfbb83ebd0 server_cookie=151100894a30db82 in_seq=0 out_seq=0 2024-09-16T00:22:08.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.466+0000 7f57d0ff9640 10 client.26146 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.466+0000 7f57d0ff9640 1 -- 172.21.15.78:0/1673054787 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f0f61b7f0 con 0x560f0f6c3980 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.466+0000 7f57d0ff9640 10 client.26146 handle_command_reply: tid=0 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.466+0000 7f57f1c78740 1 client.26146 shutdown 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57f1c78740 1 -- 172.21.15.78:0/1673054787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 msgr2=0x560f0f6b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57f1c78740 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6b37d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005fd0 tx=0x7f57e409c8f0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57f1c78740 1 -- 172.21.15.78:0/1673054787 shutdown_connections 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57f1c78740 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f0f6c3980 0x560f0f6c5d70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57f1c78740 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 0x560f0f6b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57f1c78740 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57f1c78740 1 --2- 172.21.15.78:0/1673054787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6b37d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57f1c78740 1 -- 172.21.15.78:0/1673054787 >> 172.21.15.78:0/1673054787 conn(0x560f0f6a50f0 msgr2=0x560f0f6aae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:08.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.470+0000 7f57d0ff9640 0 client.26146 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.473+0000 7f57f1c78740 1 -- 172.21.15.78:0/1673054787 shutdown_connections 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.473+0000 7f57f1c78740 1 -- 172.21.15.78:0/1673054787 wait complete. 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.474+0000 7f57f1c78740 20 client.26146 trim_cache size 0 max 16384 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.505+0000 7f57f1c78740 1 Processor -- start 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.505+0000 7f57f1c78740 1 -- start start 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6afb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 0x560f0f6cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ed63b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ed63b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37838/0 (socket says 172.21.15.78:37838) 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ed63b640 1 -- 172.21.15.78:0/2533653922 learned_addr learned my addr 172.21.15.78:0/2533653922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:08.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57f1c78740 1 -- 172.21.15.78:0/2533653922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81c6330 con 0x560f0f6b3d40 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57f1c78740 1 -- 172.21.15.78:0/2533653922 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81c7e70 con 0x560f0f6a9480 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57f1c78740 1 -- 172.21.15.78:0/2533653922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81c6a00 con 0x560f0f6a8b60 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ece3a640 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ede3c640 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 0x560f0f6cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ece3a640 1 -- 172.21.15.78:0/2533653922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 msgr2=0x560f0f6afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ece3a640 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ece3a640 1 -- 172.21.15.78:0/2533653922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 msgr2=0x560f0f6cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ece3a640 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 0x560f0f6cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.506+0000 7f57ece3a640 1 -- 172.21.15.78:0/2533653922 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0f6b1620 con 0x560f0f6a9480 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.507+0000 7f57ece3a640 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6cb600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x560f0f6c6570 tx=0x7f57d8013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90353d5f6dcdfb92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.507+0000 7f57d2ffd640 1 -- 172.21.15.78:0/2533653922 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8005020 con 0x560f0f6a9480 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.507+0000 7f57d2ffd640 1 -- 172.21.15.78:0/2533653922 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d80047a0 con 0x560f0f6a9480 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.507+0000 7f57d2ffd640 1 -- 172.21.15.78:0/2533653922 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d801c6c0 con 0x560f0f6a9480 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 -- 172.21.15.78:0/2533653922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 msgr2=0x560f0f6cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6cb600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x560f0f6c6570 tx=0x7f57d8013830 comp rx=0 tx=0).stop 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 -- 172.21.15.78:0/2533653922 shutdown_connections 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6b3d40 0x560f0f6cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a9480 0x560f0f6cb600 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2533653922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6a8b60 0x560f0f6afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 -- 172.21.15.78:0/2533653922 >> 172.21.15.78:0/2533653922 conn(0x560f0f6a50f0 msgr2=0x560f0f6af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 -- 172.21.15.78:0/2533653922 shutdown_connections 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 -- 172.21.15.78:0/2533653922 wait complete. 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.508+0000 7f57f1c78740 1 -- start start 2024-09-16T00:22:08.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57f1c78740 1 Processor -- start 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a8b60 0x560f0f6bc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a9480 0x560f0f6b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57f1c78740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57f1c78740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81c77a0 con 0x560f0f6a9480 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57f1c78740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81c70d0 con 0x560f0f6a8b60 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57f1c78740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57e81c4310 con 0x560f0f6b3d40 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57f1c78740 10 client.0 _ll_register_callbacks cb 0x7f57e409bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57ece3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a9480 0x560f0f6b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57ece3a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a9480 0x560f0f6b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47364/0 (socket says 172.21.15.78:47364) 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57ece3a640 1 -- 172.21.15.78:0/2976077875 learned_addr learned my addr 172.21.15.78:0/2976077875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57ed63b640 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a8b60 0x560f0f6bc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.509+0000 7f57ede3c640 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57ece3a640 1 -- 172.21.15.78:0/2976077875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 msgr2=0x560f0f6b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57ece3a640 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57ece3a640 1 -- 172.21.15.78:0/2976077875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a8b60 msgr2=0x560f0f6bc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57ece3a640 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a8b60 0x560f0f6bc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57ece3a640 1 -- 172.21.15.78:0/2976077875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0f6ba170 con 0x560f0f6a9480 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57ece3a640 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a9480 0x560f0f6b4dd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f57d8000c00 tx=0x7f57d800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c3a1bb725267df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57d0ff9640 1 -- 172.21.15.78:0/2976077875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8004d60 con 0x560f0f6a9480 2024-09-16T00:22:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57d0ff9640 1 -- 172.21.15.78:0/2976077875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f57d8005000 con 0x560f0f6a9480 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57f1c78740 1 -- 172.21.15.78:0/2976077875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d80047a0 con 0x560f0f6a9480 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57d0ff9640 1 -- 172.21.15.78:0/2976077875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f57d8037050 con 0x560f0f6a9480 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57f1c78740 10 client.16944 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57f1c78740 1 -- 172.21.15.78:0/2976077875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560f0f72e810 con 0x560f0f6a9480 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.510+0000 7f57d0ff9640 1 -- 172.21.15.78:0/2976077875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d80047a0 con 0x560f0f6a9480 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.511+0000 7f57d0ff9640 1 -- 172.21.15.78:0/2976077875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f57d801cc30 con 0x560f0f6a9480 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.511+0000 7f57f1c78740 10 client.16944 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.511+0000 7f57f1c78740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.511+0000 7f57f1c78740 20 client.16944 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:08.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.511+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f0f72ef40 0x560f0f731360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.511+0000 7f57f1c78740 4 client.16944 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.511+0000 7f57f1c78740 1 -- 172.21.15.78:0/2976077875 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560f0f615d00 con 0x560f0f72ef40 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.512+0000 7f57ed63b640 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f0f72ef40 0x560f0f731360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.512+0000 7f57d0ff9640 10 client.16944 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.512+0000 7f57ed63b640 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f0f72ef40 0x560f0f731360 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dab2473e720a816 server_cookie=3a57282176bb7d53 in_seq=0 out_seq=0 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57d0ff9640 1 -- 172.21.15.78:0/2976077875 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560f0f615d00 con 0x560f0f72ef40 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57d0ff9640 10 client.16944 handle_command_reply: tid=0 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 client.16944 shutdown 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 -- 172.21.15.78:0/2976077875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a9480 msgr2=0x560f0f6b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a9480 0x560f0f6b4dd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f57d8000c00 tx=0x7f57d800bc80 comp rx=0 tx=0).stop 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 -- 172.21.15.78:0/2976077875 shutdown_connections 2024-09-16T00:22:08.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f0f72ef40 0x560f0f731360 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f0f6b3d40 0x560f0f6b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57d0ff9640 0 client.16944 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f0f6a9480 0x560f0f6b4dd0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 --2- 172.21.15.78:0/2976077875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f0f6a8b60 0x560f0f6bc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 -- 172.21.15.78:0/2976077875 >> 172.21.15.78:0/2976077875 conn(0x560f0f6a50f0 msgr2=0x560f0f6aa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 -- 172.21.15.78:0/2976077875 shutdown_connections 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 1 -- 172.21.15.78:0/2976077875 wait complete. 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.514+0000 7f57f1c78740 20 client.16944 trim_cache size 0 max 16384 2024-09-16T00:22:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 -- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57d40a3450 msgr2=0x7f57d40a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57d40a3450 0x7f57d40a5910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57e00099a0 tx=0x7f57e0048000 comp rx=0 tx=0).stop 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 -- 172.21.15.78:0/3180051097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e8109450 msgr2=0x7f57e8118650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e8109450 0x7f57e8118650 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e790 tx=0x7f57e400ec50 comp rx=0 tx=0).stop 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 -- 172.21.15.78:0/3180051097 shutdown_connections 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f57d40a3450 0x7f57d40a5910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f57e811b140 0x7f57e81118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e8109830 0x7f57e8118b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 --2- 172.21.15.78:0/3180051097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e8109450 0x7f57e8118650 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 -- 172.21.15.78:0/3180051097 >> 172.21.15.78:0/3180051097 conn(0x7f57e806e140 msgr2=0x7f57e811c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 -- 172.21.15.78:0/3180051097 shutdown_connections 2024-09-16T00:22:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:07.515+0000 7f57ef89f640 1 -- 172.21.15.78:0/3180051097 wait complete. 2024-09-16T00:22:09.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:08 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:22:09.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:08 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:09.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:08 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:22:09.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:08 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:22:09.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:08 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:22:09.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:08 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:22:09.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:08 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:09.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:08 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:22:09.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:08 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:22:09.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:08 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:22:09.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:08 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:22:09.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:08 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:09.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:08 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:22:09.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:08 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:22:09.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:08 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:22:09.245 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:22:09.809 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:10.070 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.070 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:09 smithi078 ceph-mon[31157]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-09-16T00:22:10.071 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.071 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:09 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.131 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:22:10.131 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-16T00:22:10.131 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:22:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:09 smithi193 ceph-mon[35484]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-09-16T00:22:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:09 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:09 smithi148 ceph-mon[35599]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 1.53k op/s 2024-09-16T00:22:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:09 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:10.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.518+0000 7fcb1396c640 1 -- 172.21.15.78:0/1139744042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c103690 msgr2=0x7fcb0c10bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.518+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1139744042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c103690 0x7fcb0c10bb80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009900 tx=0x7fcafc02a480 comp rx=0 tx=0).stop 2024-09-16T00:22:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.518+0000 7fcb1396c640 1 -- 172.21.15.78:0/1139744042 shutdown_connections 2024-09-16T00:22:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.518+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1139744042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb0c10c150 0x7fcb0c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.518+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1139744042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c103690 0x7fcb0c10bb80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.518+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1139744042 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0c102d70 0x7fcb0c103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.518+0000 7fcb1396c640 1 -- 172.21.15.78:0/1139744042 >> 172.21.15.78:0/1139744042 conn(0x7fcb0c0fc790 msgr2=0x7fcb0c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.519+0000 7fcb1396c640 1 -- 172.21.15.78:0/1139744042 shutdown_connections 2024-09-16T00:22:10.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.519+0000 7fcb1396c640 1 -- 172.21.15.78:0/1139744042 wait complete. 2024-09-16T00:22:10.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.519+0000 7fcb1396c640 1 Processor -- start 2024-09-16T00:22:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.519+0000 7fcb1396c640 1 -- start start 2024-09-16T00:22:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb1396c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb0c102d70 0x7fcb0c103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb1396c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c103690 0x7fcb0c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb1396c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0c10c150 0x7fcb0c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb1396c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c111f50 con 0x7fcb0c102d70 2024-09-16T00:22:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb1396c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c111dd0 con 0x7fcb0c10c150 2024-09-16T00:22:10.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb1396c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c1120d0 con 0x7fcb0c103690 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb11708640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb0c102d70 0x7fcb0c103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb11708640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb0c102d70 0x7fcb0c103b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47382/0 (socket says 172.21.15.78:47382) 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb11708640 1 -- 172.21.15.78:0/1736162181 learned_addr learned my addr 172.21.15.78:0/1736162181 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.520+0000 7fcb11f09640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0c10c150 0x7fcb0c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.521+0000 7fcb11708640 1 -- 172.21.15.78:0/1736162181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c103690 msgr2=0x7fcb0c104080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.521+0000 7fcb11708640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c103690 0x7fcb0c104080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.521+0000 7fcb11708640 1 -- 172.21.15.78:0/1736162181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0c10c150 msgr2=0x7fcb0c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.521+0000 7fcb11708640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0c10c150 0x7fcb0c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.521+0000 7fcb11708640 1 -- 172.21.15.78:0/1736162181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c19a400 con 0x7fcb0c102d70 2024-09-16T00:22:10.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.521+0000 7fcb11708640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb0c102d70 0x7fcb0c103b40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800e9f0 tx=0x7fcb0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.522+0000 7fcb027fc640 1 -- 172.21.15.78:0/1736162181 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800cd80 con 0x7fcb0c102d70 2024-09-16T00:22:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.522+0000 7fcb027fc640 1 -- 172.21.15.78:0/1736162181 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb08010910 con 0x7fcb0c102d70 2024-09-16T00:22:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.522+0000 7fcb1396c640 1 -- 172.21.15.78:0/1736162181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c19a6f0 con 0x7fcb0c102d70 2024-09-16T00:22:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.522+0000 7fcb1396c640 1 -- 172.21.15.78:0/1736162181 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c19ac30 con 0x7fcb0c102d70 2024-09-16T00:22:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.523+0000 7fcb027fc640 1 -- 172.21.15.78:0/1736162181 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08018a00 con 0x7fcb0c102d70 2024-09-16T00:22:10.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.525+0000 7fcb027fc640 1 -- 172.21.15.78:0/1736162181 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcb08018ba0 con 0x7fcb0c102d70 2024-09-16T00:22:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.526+0000 7fcb027fc640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf80a3410 0x7fcaf80a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.526+0000 7fcb15d45740 1 Processor -- start 2024-09-16T00:22:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.526+0000 7fcb027fc640 1 -- 172.21.15.78:0/1736162181 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcb080cfe90 con 0x7fcb0c102d70 2024-09-16T00:22:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.526+0000 7fcb10f07640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf80a3410 0x7fcaf80a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.526+0000 7fcb15d45740 1 -- start start 2024-09-16T00:22:10.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.526+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0d4a0 0x55c5afd0d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd0ddc0 0x55c5afd18140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd18680 0x55c5afd1aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb15d45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5afc732e0 con 0x55c5afd0d4a0 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb15d45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c5afc1ace0 con 0x55c5afd18680 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb15d45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c5afc6f2a0 con 0x55c5afd0ddc0 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb10f07640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf80a3410 0x7fcaf80a58d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcafc02a9a0 tx=0x7fcafc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb10f07640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd0ddc0 0x55c5afd18140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb11f09640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd18680 0x55c5afd1aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb10f07640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd0ddc0 0x55c5afd18140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37862/0 (socket says 172.21.15.78:37862) 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb10f07640 1 -- 172.21.15.78:0/2084470374 learned_addr learned my addr 172.21.15.78:0/2084470374 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb11708640 1 --2- 172.21.15.78:0/2084470374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0d4a0 0x55c5afd0d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb11f09640 1 -- 172.21.15.78:0/2084470374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd0ddc0 msgr2=0x55c5afd18140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb11f09640 1 --2- 172.21.15.78:0/2084470374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd0ddc0 0x55c5afd18140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb11f09640 1 -- 172.21.15.78:0/2084470374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0d4a0 msgr2=0x55c5afd0d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb11f09640 1 --2- 172.21.15.78:0/2084470374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0d4a0 0x55c5afd0d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.527+0000 7fcb11f09640 1 -- 172.21.15.78:0/2084470374 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5afbff440 con 0x55c5afd18680 2024-09-16T00:22:10.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.528+0000 7fcb11f09640 1 --2- 172.21.15.78:0/2084470374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd18680 0x55c5afd1aa70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009f00 tx=0x7fcb04011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffcd85941cd0ae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.528+0000 7fcaf77fe640 1 -- 172.21.15.78:0/2084470374 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04004d20 con 0x55c5afd18680 2024-09-16T00:22:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.528+0000 7fcaf77fe640 1 -- 172.21.15.78:0/2084470374 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb04004050 con 0x55c5afd18680 2024-09-16T00:22:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.528+0000 7fcaf77fe640 1 -- 172.21.15.78:0/2084470374 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04005280 con 0x55c5afd18680 2024-09-16T00:22:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.529+0000 7fcb15d45740 1 -- 172.21.15.78:0/2084470374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd18680 msgr2=0x55c5afd1aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.529+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2084470374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd18680 0x55c5afd1aa70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009f00 tx=0x7fcb04011810 comp rx=0 tx=0).stop 2024-09-16T00:22:10.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.529+0000 7fcb15d45740 1 -- 172.21.15.78:0/2084470374 shutdown_connections 2024-09-16T00:22:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.529+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2084470374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd18680 0x55c5afd1aa70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.529+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2084470374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd0ddc0 0x55c5afd18140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.529+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2084470374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0d4a0 0x55c5afd0d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.529+0000 7fcb15d45740 1 -- 172.21.15.78:0/2084470374 >> 172.21.15.78:0/2084470374 conn(0x55c5afd09a30 msgr2=0x55c5afd09e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.530+0000 7fcb15d45740 1 -- 172.21.15.78:0/2084470374 shutdown_connections 2024-09-16T00:22:10.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.530+0000 7fcb15d45740 1 -- 172.21.15.78:0/2084470374 wait complete. 2024-09-16T00:22:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.531+0000 7fcb15d45740 1 -- start start 2024-09-16T00:22:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.531+0000 7fcb15d45740 1 Processor -- start 2024-09-16T00:22:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd17e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd15130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd15670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb11708640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd17e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb10f07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd15130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb11708640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd17e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34660/0 (socket says 172.21.15.78:34660) 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb10f07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd15130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47398/0 (socket says 172.21.15.78:47398) 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb11708640 1 -- 172.21.15.78:0/2616046511 learned_addr learned my addr 172.21.15.78:0/2616046511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c5afd1b250 con 0x55c5afd0ddc0 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c5afc2e050 con 0x55c5afd0d4a0 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c5afd1b3d0 con 0x55c5afd18680 2024-09-16T00:22:10.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb11f09640 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd15670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.532+0000 7fcb15d45740 10 client.0 _ll_register_callbacks cb 0x55c5afc6e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:10.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcb11708640 1 -- 172.21.15.78:0/2616046511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 msgr2=0x55c5afd15670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcb11708640 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd15670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcb11708640 1 -- 172.21.15.78:0/2616046511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 msgr2=0x55c5afd15130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcb11708640 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd15130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcb11708640 1 -- 172.21.15.78:0/2616046511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5af9e43c0 con 0x55c5afd0d4a0 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcb11708640 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd17e80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb080cd5d0 tx=0x7fcb0809ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe5bb65c74ec673 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcaf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2616046511 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08098fc0 con 0x55c5afd0d4a0 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2616046511 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcb08099160 con 0x55c5afd0d4a0 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.533+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb04004050 con 0x55c5afd0d4a0 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2616046511 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb0809c130 con 0x55c5afd0d4a0 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcb15d45740 10 client.26348 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c5afd26b50 con 0x55c5afd0d4a0 2024-09-16T00:22:10.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2616046511 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0809c2d0 con 0x55c5afd0d4a0 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2616046511 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcb080b6080 con 0x55c5afd0d4a0 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcb15d45740 10 client.26348 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcb15d45740 10 client.26348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcb15d45740 20 client.26348 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c5afd28010 0x55c5afd2a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcb15d45740 4 client.26348 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.534+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcb04004050 con 0x55c5afd28010 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.535+0000 7fcb10f07640 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c5afd28010 0x55c5afd2a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.535+0000 7fcaf4ff9640 10 client.26348 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:10.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.535+0000 7fcb10f07640 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c5afd28010 0x55c5afd2a490 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0796c2d05a2d7b server_cookie=cf96c8b5aa4e4a63 in_seq=0 out_seq=0 2024-09-16T00:22:10.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.536+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2616046511 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcb04004050 con 0x55c5afd28010 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.536+0000 7fcaf4ff9640 10 client.26348 handle_command_reply: tid=0 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.536+0000 7fcb15d45740 1 client.26348 shutdown 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.536+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 msgr2=0x55c5afd17e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.536+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd17e80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb080cd5d0 tx=0x7fcb0809ca90 comp rx=0 tx=0).stop 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 shutdown_connections 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c5afd28010 0x55c5afd2a490 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd15670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcaf4ff9640 0 client.26348 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd15130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2616046511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd17e80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 >> 172.21.15.78:0/2616046511 conn(0x55c5afd09a30 msgr2=0x55c5afd0f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 shutdown_connections 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 1 -- 172.21.15.78:0/2616046511 wait complete. 2024-09-16T00:22:10.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.537+0000 7fcb15d45740 20 client.26348 trim_cache size 0 max 16384 2024-09-16T00:22:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.561+0000 7fcb15d45740 1 Processor -- start 2024-09-16T00:22:10.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.561+0000 7fcb15d45740 1 -- start start 2024-09-16T00:22:10.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd2fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd123b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb15d45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf000fb70 con 0x55c5afd0ddc0 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb15d45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf00116b0 con 0x55c5afd0d4a0 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb15d45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0010240 con 0x55c5afd18680 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd302a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47402/0 (socket says 172.21.15.78:47402) 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 -- 172.21.15.78:0/1571233002 learned_addr learned my addr 172.21.15.78:0/1571233002 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 -- 172.21.15.78:0/1571233002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 msgr2=0x55c5afd123b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 --2- 172.21.15.78:0/1571233002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd123b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 -- 172.21.15.78:0/1571233002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 msgr2=0x55c5afd2fd60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:22:10.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 --2- 172.21.15.78:0/1571233002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd2fd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 -- 172.21.15.78:0/1571233002 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5afd09860 con 0x55c5afd0ddc0 2024-09-16T00:22:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.562+0000 7fcb10f07640 1 --2- 172.21.15.78:0/1571233002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd302a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0096c0 tx=0x7fcafc03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2299179bd7560c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.563+0000 7fcaf6ffd640 1 -- 172.21.15.78:0/1571233002 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc055070 con 0x55c5afd0ddc0 2024-09-16T00:22:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.563+0000 7fcaf6ffd640 1 -- 172.21.15.78:0/1571233002 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcafc003cd0 con 0x55c5afd0ddc0 2024-09-16T00:22:10.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.563+0000 7fcaf6ffd640 1 -- 172.21.15.78:0/1571233002 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc035040 con 0x55c5afd0ddc0 2024-09-16T00:22:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 -- 172.21.15.78:0/1571233002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 msgr2=0x55c5afd302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:10.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 --2- 172.21.15.78:0/1571233002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd302a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0096c0 tx=0x7fcafc03a000 comp rx=0 tx=0).stop 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 -- 172.21.15.78:0/1571233002 shutdown_connections 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 --2- 172.21.15.78:0/1571233002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 --2- 172.21.15.78:0/1571233002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd302a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 --2- 172.21.15.78:0/1571233002 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd2fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 -- 172.21.15.78:0/1571233002 >> 172.21.15.78:0/1571233002 conn(0x55c5afd09a30 msgr2=0x55c5afd15210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 -- 172.21.15.78:0/1571233002 shutdown_connections 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 -- 172.21.15.78:0/1571233002 wait complete. 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 -- start start 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.564+0000 7fcb15d45740 1 Processor -- start 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd20d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd1f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb15d45740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd1f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb15d45740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0010fe0 con 0x55c5afd0ddc0 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb15d45740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0010910 con 0x55c5afd0d4a0 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb15d45740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf000db50 con 0x55c5afd18680 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb10f07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd1f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb10f07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd1f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47418/0 (socket says 172.21.15.78:47418) 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb10f07640 1 -- 172.21.15.78:0/2022183738 learned_addr learned my addr 172.21.15.78:0/2022183738 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb15d45740 10 client.0 _ll_register_callbacks cb 0x7fcb0800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.565+0000 7fcb11f09640 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd1f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.566+0000 7fcb11f09640 1 -- 172.21.15.78:0/2022183738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 msgr2=0x55c5afd20d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.566+0000 7fcb11f09640 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd20d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.566+0000 7fcb11f09640 1 -- 172.21.15.78:0/2022183738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 msgr2=0x55c5afd1f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.566+0000 7fcb11f09640 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd1f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.566+0000 7fcb11f09640 1 -- 172.21.15.78:0/2022183738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5afd1fe70 con 0x55c5afd18680 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.566+0000 7fcb11f09640 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd1f670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcb04006010 tx=0x7fcb04004280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6065ca4473eeeec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.566+0000 7fcaf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.566+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2022183738 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04027070 con 0x55c5afd18680 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.567+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2022183738 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcb04004a80 con 0x55c5afd18680 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.567+0000 7fcb15d45740 1 -- 172.21.15.78:0/2022183738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcafc003cd0 con 0x55c5afd18680 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.567+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2022183738 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb04031050 con 0x55c5afd18680 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.567+0000 7fcb15d45740 10 client.26164 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.567+0000 7fcb15d45740 1 -- 172.21.15.78:0/2022183738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c5afd93d10 con 0x55c5afd18680 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.567+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2022183738 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcafc003cd0 con 0x55c5afd18680 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.568+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2022183738 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcb04023430 con 0x55c5afd18680 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.568+0000 7fcb15d45740 10 client.26164 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.568+0000 7fcb15d45740 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.568+0000 7fcb15d45740 20 client.26164 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.568+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c5afd28010 0x55c5afd963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.568+0000 7fcb15d45740 4 client.26164 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.568+0000 7fcb15d45740 1 -- 172.21.15.78:0/2022183738 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c5afc80850 con 0x55c5afd28010 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.568+0000 7fcb11708640 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c5afd28010 0x55c5afd963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcb11708640 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c5afd28010 0x55c5afd963b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2373a3a6ec2c294a server_cookie=8fd9ec8499088ae6 in_seq=0 out_seq=0 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcaf4ff9640 10 client.26164 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcaf4ff9640 1 -- 172.21.15.78:0/2022183738 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c5afc80850 con 0x55c5afd28010 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcaf4ff9640 10 client.26164 handle_command_reply: tid=0 2024-09-16T00:22:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcb15d45740 1 client.26164 shutdown 2024-09-16T00:22:10.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcb15d45740 1 -- 172.21.15.78:0/2022183738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 msgr2=0x55c5afd1f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:10.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd1f670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcb04006010 tx=0x7fcb04004280 comp rx=0 tx=0).stop 2024-09-16T00:22:10.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcb15d45740 1 -- 172.21.15.78:0/2022183738 shutdown_connections 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.569+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c5afd28010 0x55c5afd963b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.570+0000 7fcaf4ff9640 0 client.26164 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.570+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5afd18680 0x55c5afd1f670 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.570+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c5afd0ddc0 0x55c5afd1f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.570+0000 7fcb15d45740 1 --2- 172.21.15.78:0/2022183738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5afd0d4a0 0x55c5afd20d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.570+0000 7fcb15d45740 1 -- 172.21.15.78:0/2022183738 >> 172.21.15.78:0/2022183738 conn(0x55c5afd09a30 msgr2=0x55c5afd16990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.570+0000 7fcb15d45740 1 -- 172.21.15.78:0/2022183738 shutdown_connections 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.570+0000 7fcb15d45740 1 -- 172.21.15.78:0/2022183738 wait complete. 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.570+0000 7fcb15d45740 20 client.26164 trim_cache size 0 max 16384 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:22:10.574 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:22:10.575 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:22:10.575 INFO:teuthology.orchestra.run.smithi078.stdout: "3a3e6da9-a6f5-4540-a950-8192aa35ca06": { 2024-09-16T00:22:10.575 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:22:10.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "3a3e6da9-a6f5-4540-a950-8192aa35ca06", 2024-09-16T00:22:10.575 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:22:10.575 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:10.575 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:10.575 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 -- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf80a3410 msgr2=0x7fcaf80a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf80a3410 0x7fcaf80a58d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcafc02a9a0 tx=0x7fcafc000bc0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 -- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb0c102d70 msgr2=0x7fcb0c103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb0c102d70 0x7fcb0c103b40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800e9f0 tx=0x7fcb0800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 -- 172.21.15.78:0/1736162181 shutdown_connections 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcaf80a3410 0x7fcaf80a58d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcb0c10c150 0x7fcb0c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0c103690 0x7fcb0c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 --2- 172.21.15.78:0/1736162181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb0c102d70 0x7fcb0c103b40 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 -- 172.21.15.78:0/1736162181 >> 172.21.15.78:0/1736162181 conn(0x7fcb0c0fc790 msgr2=0x7fcb0c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 -- 172.21.15.78:0/1736162181 shutdown_connections 2024-09-16T00:22:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:10.571+0000 7fcb1396c640 1 -- 172.21.15.78:0/1736162181 wait complete. 2024-09-16T00:22:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 490.0 s, 620.2 tps, lat 156.937 ms stddev 179.462 2024-09-16T00:22:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:11 smithi193 ceph-mon[35484]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 MiB/s wr, 1.60k op/s 2024-09-16T00:22:12.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:11 smithi148 ceph-mon[35599]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 MiB/s wr, 1.60k op/s 2024-09-16T00:22:12.216 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:11 smithi078 ceph-mon[31157]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 17 MiB/s wr, 1.60k op/s 2024-09-16T00:22:12.242 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:12.302 INFO:tasks.cephfs.filesystem:scrub status for tag:3a3e6da9-a6f5-4540-a950-8192aa35ca06 - {'path': '/', 'tag': '3a3e6da9-a6f5-4540-a950-8192aa35ca06', 'options': 'recursive,force'} 2024-09-16T00:22:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.873+0000 7f3ffed1a640 1 -- 172.21.15.78:0/2023783202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff8108450 msgr2=0x7f3ff8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.873+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/2023783202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff8108450 0x7f3ff8108830 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009860 tx=0x7f3fec02a360 comp rx=0 tx=0).stop 2024-09-16T00:22:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.879+0000 7f3ffed1a640 1 -- 172.21.15.78:0/2023783202 shutdown_connections 2024-09-16T00:22:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.879+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/2023783202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ff810d530 0x7f3ff81150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.879+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/2023783202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff8108e00 0x7f3ff810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.879+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/2023783202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff8108450 0x7f3ff8108830 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.879+0000 7f3ffed1a640 1 -- 172.21.15.78:0/2023783202 >> 172.21.15.78:0/2023783202 conn(0x7f3ff8077ba0 msgr2=0x7f3ff8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:12.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.883+0000 7f3ffed1a640 1 -- 172.21.15.78:0/2023783202 shutdown_connections 2024-09-16T00:22:12.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.886+0000 7f3ffed1a640 1 -- 172.21.15.78:0/2023783202 wait complete. 2024-09-16T00:22:12.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.887+0000 7f3ffed1a640 1 Processor -- start 2024-09-16T00:22:12.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.887+0000 7f3ffed1a640 1 -- start start 2024-09-16T00:22:12.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.887+0000 7f3ffed1a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff8108450 0x7f3ff8073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:12.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.887+0000 7f3ffed1a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff8108e00 0x7f3ff8073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:12.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.887+0000 7f3ffed1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ff810d530 0x7f3ff806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:12.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.887+0000 7f3ffed1a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81185a0 con 0x7f3ff8108450 2024-09-16T00:22:12.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.888+0000 7f3ffed1a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8118420 con 0x7f3ff810d530 2024-09-16T00:22:12.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.888+0000 7f3ffed1a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8118720 con 0x7f3ff8108e00 2024-09-16T00:22:12.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.889+0000 7f3ffd2b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ff810d530 0x7f3ff806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:12.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.889+0000 7f3ffd2b7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ff810d530 0x7f3ff806cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34702/0 (socket says 172.21.15.78:34702) 2024-09-16T00:22:12.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.889+0000 7f3ffd2b7640 1 -- 172.21.15.78:0/1978526846 learned_addr learned my addr 172.21.15.78:0/1978526846 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:12.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.893+0000 7f3ffd2b7640 1 -- 172.21.15.78:0/1978526846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff8108e00 msgr2=0x7f3ff8073ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:12.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.893+0000 7f3ffd2b7640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff8108e00 0x7f3ff8073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.893+0000 7f3ffd2b7640 1 -- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff8108450 msgr2=0x7f3ff8073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:12.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.893+0000 7f3ffd2b7640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff8108450 0x7f3ff8073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:12.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.893+0000 7f3ffd2b7640 1 -- 172.21.15.78:0/1978526846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff806d490 con 0x7f3ff810d530 2024-09-16T00:22:12.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.896+0000 7f3ffd2b7640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ff810d530 0x7f3ff806cbd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c720 tx=0x7f3fe800cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:12.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.901+0000 7f3ff5ffb640 1 -- 172.21.15.78:0/1978526846 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8007bb0 con 0x7f3ff810d530 2024-09-16T00:22:12.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.901+0000 7f3ff5ffb640 1 -- 172.21.15.78:0/1978526846 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe800ce30 con 0x7f3ff810d530 2024-09-16T00:22:12.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.901+0000 7f3ff5ffb640 1 -- 172.21.15.78:0/1978526846 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800f450 con 0x7f3ff810d530 2024-09-16T00:22:12.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.903+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff806d780 con 0x7f3ff810d530 2024-09-16T00:22:12.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.903+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff80716e0 con 0x7f3ff810d530 2024-09-16T00:22:12.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.905+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff81098e0 con 0x7f3ff810d530 2024-09-16T00:22:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.909+0000 7f3ff5ffb640 1 -- 172.21.15.78:0/1978526846 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3fe8004ee0 con 0x7f3ff810d530 2024-09-16T00:22:12.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.909+0000 7f3ff5ffb640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3fe00a3460 0x7f3fe00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:12.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.910+0000 7f3ff5ffb640 1 -- 172.21.15.78:0/1978526846 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3fe80cfec0 con 0x7f3ff810d530 2024-09-16T00:22:12.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.910+0000 7f3ffcab6640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3fe00a3460 0x7f3fe00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:12.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.910+0000 7f3ffcab6640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3fe00a3460 0x7f3fe00a5920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3fec035040 tx=0x7f3fec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:12.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:12.918+0000 7f3ff5ffb640 1 -- 172.21.15.78:0/1978526846 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3fe801d020 con 0x7f3ff810d530 2024-09-16T00:22:13.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:12 smithi193 ceph-mon[35484]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-16T00:22:13.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:12 smithi148 ceph-mon[35599]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-16T00:22:13.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:12 smithi078 ceph-mon[31157]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 14 MiB/s wr, 1.42k op/s 2024-09-16T00:22:13.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.293+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ff8108830 con 0x7f3ff810d530 2024-09-16T00:22:13.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.296+0000 7f3ff5ffb640 1 -- 172.21.15.78:0/1978526846 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f3fe8014030 con 0x7f3ff810d530 2024-09-16T00:22:13.297 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:22:13.297 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:22:13.298 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:22:13.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.304+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3fe00a3460 msgr2=0x7f3fe00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.304+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3fe00a3460 0x7f3fe00a5920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3fec035040 tx=0x7f3fec005850 comp rx=0 tx=0).stop 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.304+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ff810d530 msgr2=0x7f3ff806cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.304+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ff810d530 0x7f3ff806cbd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c720 tx=0x7f3fe800cbe0 comp rx=0 tx=0).stop 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.305+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 shutdown_connections 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.305+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3fe00a3460 0x7f3fe00a5920 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3fec035040 tx=0x7f3fec005850 comp rx=0 tx=0).stop 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.305+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ff810d530 0x7f3ff806cbd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.305+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff8108e00 0x7f3ff8073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.305+0000 7f3ffed1a640 1 --2- 172.21.15.78:0/1978526846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff8108450 0x7f3ff8073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.305+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 >> 172.21.15.78:0/1978526846 conn(0x7f3ff8077ba0 msgr2=0x7f3ff8078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.310+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 shutdown_connections 2024-09-16T00:22:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:13.310+0000 7f3ffed1a640 1 -- 172.21.15.78:0/1978526846 wait complete. 2024-09-16T00:22:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:13 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1978526846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:13 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1978526846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:13 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1978526846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:14.790 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-16T00:22:14.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-16T00:22:15.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:14 smithi078 ceph-mon[31157]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 975 op/s 2024-09-16T00:22:15.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:14 smithi193 ceph-mon[35484]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 975 op/s 2024-09-16T00:22:15.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:14 smithi148 ceph-mon[35599]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.4 MiB/s wr, 975 op/s 2024-09-16T00:22:15.251 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:15.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 495.0 s, 731.2 tps, lat 139.082 ms stddev 173.767 2024-09-16T00:22:15.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 -- 172.21.15.78:0/860168234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc109830 msgr2=0x7f13dc109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 --2- 172.21.15.78:0/860168234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc109830 0x7f13dc109c70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009ca0 tx=0x7f13d402a8f0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 -- 172.21.15.78:0/860168234 shutdown_connections 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 --2- 172.21.15.78:0/860168234 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc11b140 0x7f13dc11d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 --2- 172.21.15.78:0/860168234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc109830 0x7f13dc109c70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 --2- 172.21.15.78:0/860168234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13dc109450 0x7f13dc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 -- 172.21.15.78:0/860168234 >> 172.21.15.78:0/860168234 conn(0x7f13dc06e2d0 msgr2=0x7f13dc06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 -- 172.21.15.78:0/860168234 shutdown_connections 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.863+0000 7f13e2b75640 1 -- 172.21.15.78:0/860168234 wait complete. 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e2b75640 1 Processor -- start 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e2b75640 1 -- start start 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e2b75640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc109450 0x7f13dc11a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e2b75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc11b140 0x7f13dc114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e2b75640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13dc114dc0 0x7f13dc115270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e2b75640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc1114b0 con 0x7f13dc114dc0 2024-09-16T00:22:15.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e2b75640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc111330 con 0x7f13dc109450 2024-09-16T00:22:15.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e2b75640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc111630 con 0x7f13dc11b140 2024-09-16T00:22:15.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.864+0000 7f13e0911640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc109450 0x7f13dc11a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.865+0000 7f13e0911640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc109450 0x7f13dc11a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34718/0 (socket says 172.21.15.78:34718) 2024-09-16T00:22:15.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.865+0000 7f13e0911640 1 -- 172.21.15.78:0/3850555943 learned_addr learned my addr 172.21.15.78:0/3850555943 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:15.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.865+0000 7f13dbfff640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc11b140 0x7f13dc114880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.866+0000 7f13e1112640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13dc114dc0 0x7f13dc115270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.866+0000 7f13dbfff640 1 -- 172.21.15.78:0/3850555943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc109450 msgr2=0x7f13dc11a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:15.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.866+0000 7f13dbfff640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc109450 0x7f13dc11a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.866+0000 7f13dbfff640 1 -- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13dc114dc0 msgr2=0x7f13dc115270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:15.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.866+0000 7f13dbfff640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13dc114dc0 0x7f13dc115270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.866+0000 7f13dbfff640 1 -- 172.21.15.78:0/3850555943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc1263b0 con 0x7f13dc11b140 2024-09-16T00:22:15.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.867+0000 7f13dbfff640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc11b140 0x7f13dc114880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009330 tx=0x7f13d4030820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:15.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.867+0000 7f13d9ffb640 1 -- 172.21.15.78:0/3850555943 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4030da0 con 0x7f13dc11b140 2024-09-16T00:22:15.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.867+0000 7f13e2b75640 1 -- 172.21.15.78:0/3850555943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc126640 con 0x7f13dc11b140 2024-09-16T00:22:15.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.867+0000 7f13e2b75640 1 -- 172.21.15.78:0/3850555943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc126bd0 con 0x7f13dc11b140 2024-09-16T00:22:15.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.871+0000 7f13e4f4e740 1 Processor -- start 2024-09-16T00:22:15.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.871+0000 7f13e4f4e740 1 -- start start 2024-09-16T00:22:15.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5b7fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5b8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b8aa30 0x5616b5b8ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5616b5ae4160 con 0x5616b5b80170 2024-09-16T00:22:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5616b5a8b720 con 0x5616b5b7f850 2024-09-16T00:22:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5616b5ae0120 con 0x5616b5b8aa30 2024-09-16T00:22:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13e0911640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5b8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13d9ffb640 1 -- 172.21.15.78:0/3850555943 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d4036040 con 0x7f13dc11b140 2024-09-16T00:22:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13dbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b8aa30 0x5616b5b8ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.872+0000 7f13dbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b8aa30 0x5616b5b8ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37914/0 (socket says 172.21.15.78:37914) 2024-09-16T00:22:15.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.873+0000 7f13dbfff640 1 -- 172.21.15.78:0/2462513140 learned_addr learned my addr 172.21.15.78:0/2462513140 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.875+0000 7f13dbfff640 1 -- 172.21.15.78:0/2462513140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 msgr2=0x5616b5b7fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.875+0000 7f13dbfff640 1 --2- 172.21.15.78:0/2462513140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5b7fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.875+0000 7f13dbfff640 1 -- 172.21.15.78:0/2462513140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 msgr2=0x5616b5b8a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.875+0000 7f13dbfff640 1 --2- 172.21.15.78:0/2462513140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5b8a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.875+0000 7f13dbfff640 1 -- 172.21.15.78:0/2462513140 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b5855ba0 con 0x5616b5b8aa30 2024-09-16T00:22:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.875+0000 7f13dbfff640 1 --2- 172.21.15.78:0/2462513140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b8aa30 0x5616b5b8ce20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f13d4117aa0 tx=0x7f13d411ab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f743e9e3ba5517ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.875+0000 7f13d9ffb640 1 -- 172.21.15.78:0/3850555943 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d403b8a0 con 0x7f13dc11b140 2024-09-16T00:22:15.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.875+0000 7f13d9ffb640 1 -- 172.21.15.78:0/3850555943 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f13d4043030 con 0x7f13dc11b140 2024-09-16T00:22:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.876+0000 7f13d9ffb640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13c40a33b0 0x7f13c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.876+0000 7f13c2ffd640 1 -- 172.21.15.78:0/2462513140 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d4125020 con 0x5616b5b8aa30 2024-09-16T00:22:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.876+0000 7f13c2ffd640 1 -- 172.21.15.78:0/2462513140 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d402d070 con 0x5616b5b8aa30 2024-09-16T00:22:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.876+0000 7f13d9ffb640 1 -- 172.21.15.78:0/3850555943 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f13d411a340 con 0x7f13dc11b140 2024-09-16T00:22:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.877+0000 7f13e1112640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13c40a33b0 0x7f13c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.878+0000 7f13c2ffd640 1 -- 172.21.15.78:0/2462513140 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d411fa30 con 0x5616b5b8aa30 2024-09-16T00:22:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.881+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2462513140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b8aa30 msgr2=0x5616b5b8ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:15.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.881+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2462513140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b8aa30 0x5616b5b8ce20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f13d4117aa0 tx=0x7f13d411ab00 comp rx=0 tx=0).stop 2024-09-16T00:22:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.882+0000 7f13e1112640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13c40a33b0 0x7f13c40a5870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f13d0006fd0 tx=0x7f13d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.882+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2462513140 shutdown_connections 2024-09-16T00:22:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.882+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2462513140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b8aa30 0x5616b5b8ce20 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f13d4117aa0 tx=0x7f13d411ab00 comp rx=0 tx=0).stop 2024-09-16T00:22:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.882+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2462513140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5b8a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.882+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2462513140 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5b7fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.882+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2462513140 >> 172.21.15.78:0/2462513140 conn(0x5616b5b7bde0 msgr2=0x5616b5b7c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:15.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.883+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2462513140 shutdown_connections 2024-09-16T00:22:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.883+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2462513140 wait complete. 2024-09-16T00:22:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.884+0000 7f13e4f4e740 1 -- start start 2024-09-16T00:22:15.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.884+0000 7f13e4f4e740 1 Processor -- start 2024-09-16T00:22:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7f850 0x5616b5b87b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5b880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b88600 0x5616b5b96830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5616b5b8d3c0 con 0x5616b5b80170 2024-09-16T00:22:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5616b5a6c900 con 0x5616b5b88600 2024-09-16T00:22:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5616b5b8d540 con 0x5616b5b7f850 2024-09-16T00:22:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e4f4e740 10 client.0 _ll_register_callbacks cb 0x5616b5a6a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:15.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e0911640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7f850 0x5616b5b87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e0911640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7f850 0x5616b5b87b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37922/0 (socket says 172.21.15.78:37922) 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e0911640 1 -- 172.21.15.78:0/494404384 learned_addr learned my addr 172.21.15.78:0/494404384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.885+0000 7f13e1112640 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b88600 0x5616b5b96830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13e0911640 1 -- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b88600 msgr2=0x5616b5b96830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13e0911640 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b88600 0x5616b5b96830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13e0911640 1 -- 172.21.15.78:0/494404384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 msgr2=0x5616b5b880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13e0911640 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5b880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13e0911640 1 -- 172.21.15.78:0/494404384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b5aacc50 con 0x5616b5b7f850 2024-09-16T00:22:15.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13e1112640 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b88600 0x5616b5b96830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13e0911640 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7f850 0x5616b5b87b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00bf30 tx=0x7f13cc0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42875aa97e863c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13bbfff640 1 -- 172.21.15.78:0/494404384 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc004230 con 0x5616b5b7f850 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13e4f4e740 1 -- 172.21.15.78:0/494404384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13d402d070 con 0x5616b5b7f850 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.886+0000 7f13bbfff640 1 -- 172.21.15.78:0/494404384 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f13cc0043d0 con 0x5616b5b7f850 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.887+0000 7f13bbfff640 1 -- 172.21.15.78:0/494404384 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f13cc024050 con 0x5616b5b7f850 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.887+0000 7f13e4f4e740 10 client.26179 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.887+0000 7f13e4f4e740 1 -- 172.21.15.78:0/494404384 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5616b5b99710 con 0x5616b5b7f850 2024-09-16T00:22:15.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.889+0000 7f13bbfff640 1 -- 172.21.15.78:0/494404384 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13d402d070 con 0x5616b5b7f850 2024-09-16T00:22:15.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.890+0000 7f13bbfff640 1 -- 172.21.15.78:0/494404384 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f13cc003b10 con 0x5616b5b7f850 2024-09-16T00:22:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.891+0000 7f13e4f4e740 10 client.26179 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.891+0000 7f13e4f4e740 10 client.26179 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:22:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.891+0000 7f13e4f4e740 20 client.26179 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.891+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5616b5b9ac20 0x5616b5b9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.891+0000 7f13e4f4e740 4 client.26179 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:15.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.891+0000 7f13e4f4e740 1 -- 172.21.15.78:0/494404384 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b5af1890 con 0x5616b5b9ac20 2024-09-16T00:22:15.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.891+0000 7f13dbfff640 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5616b5b9ac20 0x5616b5b9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.893+0000 7f13dbfff640 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5616b5b9ac20 0x5616b5b9d0a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa7a69558b4d0647 server_cookie=5c8ebc7ccbb7d8ac in_seq=0 out_seq=0 2024-09-16T00:22:15.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.893+0000 7f13bbfff640 10 client.26179 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:15.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.894+0000 7f13bbfff640 1 -- 172.21.15.78:0/494404384 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616b5af1890 con 0x5616b5b9ac20 2024-09-16T00:22:15.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.894+0000 7f13bbfff640 10 client.26179 handle_command_reply: tid=0 2024-09-16T00:22:15.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.895+0000 7f13e4f4e740 1 client.26179 shutdown 2024-09-16T00:22:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.901+0000 7f13e4f4e740 1 -- 172.21.15.78:0/494404384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7f850 msgr2=0x5616b5b87b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:15.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.901+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7f850 0x5616b5b87b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00bf30 tx=0x7f13cc0117b0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.902+0000 7f13e4f4e740 1 -- 172.21.15.78:0/494404384 shutdown_connections 2024-09-16T00:22:15.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.902+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5616b5b9ac20 0x5616b5b9d0a0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.902+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b88600 0x5616b5b96830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.902+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5b880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.902+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/494404384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7f850 0x5616b5b87b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.902+0000 7f13e4f4e740 1 -- 172.21.15.78:0/494404384 >> 172.21.15.78:0/494404384 conn(0x5616b5b7bde0 msgr2=0x5616b5b81b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:15.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.902+0000 7f13bbfff640 0 client.26179 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.904+0000 7f13e4f4e740 1 -- 172.21.15.78:0/494404384 shutdown_connections 2024-09-16T00:22:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.905+0000 7f13e4f4e740 1 -- 172.21.15.78:0/494404384 wait complete. 2024-09-16T00:22:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.905+0000 7f13e4f4e740 20 client.26179 trim_cache size 0 max 16384 2024-09-16T00:22:15.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.940+0000 7f13e4f4e740 1 Processor -- start 2024-09-16T00:22:15.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.940+0000 7f13e4f4e740 1 -- start start 2024-09-16T00:22:15.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.940+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b7bde0 0x5616b5b96620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.940+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5ba2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.940+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b80170 0x5616b5ba2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.940+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800fa10 con 0x5616b5b7bde0 2024-09-16T00:22:15.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.940+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8011550 con 0x5616b5b7f850 2024-09-16T00:22:15.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.940+0000 7f13e4f4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80100e0 con 0x5616b5b80170 2024-09-16T00:22:15.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.941+0000 7f13e0911640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b7bde0 0x5616b5b96620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.941+0000 7f13e0911640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b7bde0 0x5616b5b96620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47502/0 (socket says 172.21.15.78:47502) 2024-09-16T00:22:15.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.941+0000 7f13e0911640 1 -- 172.21.15.78:0/142827070 learned_addr learned my addr 172.21.15.78:0/142827070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.941+0000 7f13e0911640 1 -- 172.21.15.78:0/142827070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b80170 msgr2=0x5616b5ba2eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.941+0000 7f13e0911640 1 --2- 172.21.15.78:0/142827070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b80170 0x5616b5ba2eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.941+0000 7f13e0911640 1 -- 172.21.15.78:0/142827070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 msgr2=0x5616b5ba2970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.941+0000 7f13e0911640 1 --2- 172.21.15.78:0/142827070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5ba2970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.941+0000 7f13e0911640 1 -- 172.21.15.78:0/142827070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b5b88b20 con 0x5616b5b7bde0 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.942+0000 7f13e0911640 1 --2- 172.21.15.78:0/142827070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b7bde0 0x5616b5b96620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f13cc000c00 tx=0x7f13cc01c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8b0cdcd1c8877e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.942+0000 7f13c27fc640 1 -- 172.21.15.78:0/142827070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc005880 con 0x5616b5b7bde0 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.942+0000 7f13c27fc640 1 -- 172.21.15.78:0/142827070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13cc005550 con 0x5616b5b7bde0 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.942+0000 7f13c27fc640 1 -- 172.21.15.78:0/142827070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc012980 con 0x5616b5b7bde0 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.943+0000 7f13e4f4e740 1 -- 172.21.15.78:0/142827070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b7bde0 msgr2=0x5616b5b96620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.943+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/142827070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b7bde0 0x5616b5b96620 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f13cc000c00 tx=0x7f13cc01c7d0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.943+0000 7f13e4f4e740 1 -- 172.21.15.78:0/142827070 shutdown_connections 2024-09-16T00:22:15.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.944+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/142827070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b80170 0x5616b5ba2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.944+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/142827070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5ba2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.944+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/142827070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b7bde0 0x5616b5b96620 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.944+0000 7f13e4f4e740 1 -- 172.21.15.78:0/142827070 >> 172.21.15.78:0/142827070 conn(0x5616b5b88220 msgr2=0x5616b5b82db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:15.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.944+0000 7f13e4f4e740 1 -- 172.21.15.78:0/142827070 shutdown_connections 2024-09-16T00:22:15.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.944+0000 7f13e4f4e740 1 -- 172.21.15.78:0/142827070 wait complete. 2024-09-16T00:22:15.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.944+0000 7f13e4f4e740 1 -- start start 2024-09-16T00:22:15.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.944+0000 7f13e4f4e740 1 Processor -- start 2024-09-16T00:22:15.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.945+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7bde0 0x5616b5b8e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.945+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5b8ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.945+0000 7f13e4f4e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5c04940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.945+0000 7f13e0911640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7bde0 0x5616b5b8e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.945+0000 7f13dbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5b8ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.945+0000 7f13e0911640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7bde0 0x5616b5b8e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37936/0 (socket says 172.21.15.78:37936) 2024-09-16T00:22:15.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.945+0000 7f13e0911640 1 -- 172.21.15.78:0/2260942057 learned_addr learned my addr 172.21.15.78:0/2260942057 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.945+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8010e80 con 0x5616b5b80170 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80107b0 con 0x5616b5b7f850 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f13c800daa0 con 0x5616b5b7bde0 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e4f4e740 10 client.0 _ll_register_callbacks cb 0x7f13cc00b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e1112640 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5c04940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e0911640 1 -- 172.21.15.78:0/2260942057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 msgr2=0x5616b5b8ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e0911640 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5b8ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e0911640 1 -- 172.21.15.78:0/2260942057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 msgr2=0x5616b5c04940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e0911640 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5c04940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e0911640 1 -- 172.21.15.78:0/2260942057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b5b8f630 con 0x5616b5b7bde0 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e1112640 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5c04940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13e0911640 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7bde0 0x5616b5b8e840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f13cc01c840 tx=0x7f13cc0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d1aa2aceb08f08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:15.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.946+0000 7f13bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:22:15.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.947+0000 7f13bbfff640 1 -- 172.21.15.78:0/2260942057 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc005070 con 0x5616b5b7bde0 2024-09-16T00:22:15.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.947+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13cc005550 con 0x5616b5b7bde0 2024-09-16T00:22:15.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.947+0000 7f13bbfff640 1 -- 172.21.15.78:0/2260942057 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f13cc005210 con 0x5616b5b7bde0 2024-09-16T00:22:15.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.948+0000 7f13bbfff640 1 -- 172.21.15.78:0/2260942057 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f13cc01c8f0 con 0x5616b5b7bde0 2024-09-16T00:22:15.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.948+0000 7f13e4f4e740 10 client.26182 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.948+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5616b5c05f20 con 0x5616b5b7bde0 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.949+0000 7f13bbfff640 1 -- 172.21.15.78:0/2260942057 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13cc005550 con 0x5616b5b7bde0 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.949+0000 7f13bbfff640 1 -- 172.21.15.78:0/2260942057 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f13cc03b030 con 0x5616b5b7bde0 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.949+0000 7f13e4f4e740 10 client.26182 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.949+0000 7f13e4f4e740 10 client.26182 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.949+0000 7f13e4f4e740 20 client.26182 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.949+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5616b5b88600 0x5616b5c086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.949+0000 7f13e4f4e740 4 client.26182 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-16T00:22:15.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.949+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x5616b5aebbe0 con 0x5616b5b88600 2024-09-16T00:22:15.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.953+0000 7f13dbfff640 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5616b5b88600 0x5616b5c086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:15.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.955+0000 7f13dbfff640 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5616b5b88600 0x5616b5c086e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddcae60cf09e193 server_cookie=5bfa14ab8f0972 in_seq=0 out_seq=0 2024-09-16T00:22:15.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:15.955+0000 7f13bbfff640 10 client.26182 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.223+0000 7f13bbfff640 1 -- 172.21.15.78:0/2260942057 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5616b5aebbe0 con 0x5616b5b88600 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.223+0000 7f13bbfff640 10 client.26182 handle_command_reply: tid=0 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.223+0000 7f13e4f4e740 1 client.26182 shutdown 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7bde0 msgr2=0x5616b5b8e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7bde0 0x5616b5b8e840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f13cc01c840 tx=0x7f13cc0117b0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 shutdown_connections 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5616b5b88600 0x5616b5c086e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5616b5b80170 0x5616b5c04940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13bbfff640 0 client.26182 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5616b5b7f850 0x5616b5b8ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 --2- 172.21.15.78:0/2260942057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616b5b7bde0 0x5616b5b8e840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 >> 172.21.15.78:0/2260942057 conn(0x5616b5b88220 msgr2=0x5616b5b81320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 shutdown_connections 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 1 -- 172.21.15.78:0/2260942057 wait complete. 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.224+0000 7f13e4f4e740 20 client.26182 trim_cache size 0 max 16384 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 30, 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:22:16.226 INFO:teuthology.orchestra.run.smithi078.stdout: "62a7683a": { 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 30, 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 142.3, 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:22:16.227 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:22:16.228 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:16.228 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:16.228 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:16.228 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:16.228 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:16.228 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:22:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 -- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13c40a33b0 msgr2=0x7f13c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13c40a33b0 0x7f13c40a5870 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f13d0006fd0 tx=0x7f13d0008040 comp rx=0 tx=0).stop 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 -- 172.21.15.78:0/3850555943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc11b140 msgr2=0x7f13dc114880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc11b140 0x7f13dc114880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009330 tx=0x7f13d4030820 comp rx=0 tx=0).stop 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 -- 172.21.15.78:0/3850555943 shutdown_connections 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f13c40a33b0 0x7f13c40a5870 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13dc114dc0 0x7f13dc115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13dc11b140 0x7f13dc114880 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 --2- 172.21.15.78:0/3850555943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f13dc109450 0x7f13dc11a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 -- 172.21.15.78:0/3850555943 >> 172.21.15.78:0/3850555943 conn(0x7f13dc06e2d0 msgr2=0x7f13dc0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 -- 172.21.15.78:0/3850555943 shutdown_connections 2024-09-16T00:22:16.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:16.225+0000 7f13e2b75640 1 -- 172.21.15.78:0/3850555943 wait complete. 2024-09-16T00:22:17.085 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '62a7683a', quiesce took 7.0 seconds. Will release after: 15.350000000000001 2024-09-16T00:22:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:17 smithi148 ceph-mon[35599]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 647 op/s 2024-09-16T00:22:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:17 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:17.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:17 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:17.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:17 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:17.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:17 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:22:18.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:17 smithi078 ceph-mon[31157]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 647 op/s 2024-09-16T00:22:18.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:17 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:18.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:17 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:18.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:17 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:18.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:17 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:22:18.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:17 smithi193 ceph-mon[35484]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 647 op/s 2024-09-16T00:22:18.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:17 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:18.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:17 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:18.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:17 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:22:18.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:17 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:22:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:19 smithi148 ceph-mon[35599]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 608 op/s 2024-09-16T00:22:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:22:19.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:22:19.955 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:19 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:22:20.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:19 smithi078 ceph-mon[31157]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 608 op/s 2024-09-16T00:22:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:22:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:22:20.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:19 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:22:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:19 smithi193 ceph-mon[35484]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 MiB/s wr, 608 op/s 2024-09-16T00:22:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:22:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:22:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:19 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:22:20.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 500.0 s, 39.4 tps, lat 131.379 ms stddev 109.934 2024-09-16T00:22:21.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:21 smithi148 ceph-mon[35599]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-09-16T00:22:22.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:21 smithi078 ceph-mon[31157]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-09-16T00:22:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:21 smithi193 ceph-mon[35484]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.8 MiB/s wr, 547 op/s 2024-09-16T00:22:23.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:23 smithi148 ceph-mon[35599]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-09-16T00:22:24.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:23 smithi078 ceph-mon[31157]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-09-16T00:22:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:23 smithi193 ceph-mon[35484]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-09-16T00:22:25.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 505.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:22:25.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:25 smithi148 ceph-mon[35599]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-09-16T00:22:26.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:25 smithi078 ceph-mon[31157]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-09-16T00:22:26.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:25 smithi193 ceph-mon[35484]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-09-16T00:22:27.954 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:27 smithi148 ceph-mon[35599]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-16T00:22:28.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:27 smithi078 ceph-mon[31157]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-16T00:22:28.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:27 smithi193 ceph-mon[35484]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.6 MiB/s wr, 193 op/s 2024-09-16T00:22:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:29 smithi078 ceph-mon[31157]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-16T00:22:30.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:29 smithi193 ceph-mon[35484]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-16T00:22:30.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:29 smithi148 ceph-mon[35599]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-16T00:22:30.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 510.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:22:32.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:31 smithi078 ceph-mon[31157]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 545 KiB/s wr, 63 op/s 2024-09-16T00:22:32.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:31 smithi193 ceph-mon[35484]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 545 KiB/s wr, 63 op/s 2024-09-16T00:22:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:31 smithi148 ceph-mon[35599]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 545 KiB/s wr, 63 op/s 2024-09-16T00:22:32.436 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:22:32.696 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 -- 172.21.15.78:0/762006659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060100e20 msgr2=0x7f1060101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 --2- 172.21.15.78:0/762006659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060100e20 0x7f1060101200 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f105c009900 tx=0x7f105c02a480 comp rx=0 tx=0).stop 2024-09-16T00:22:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 -- 172.21.15.78:0/762006659 shutdown_connections 2024-09-16T00:22:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 --2- 172.21.15.78:0/762006659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1060102ef0 0x7f106010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 --2- 172.21.15.78:0/762006659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1060102570 0x7f10601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 --2- 172.21.15.78:0/762006659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060100e20 0x7f1060101200 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 -- 172.21.15.78:0/762006659 >> 172.21.15.78:0/762006659 conn(0x7f10600fc7d0 msgr2=0x7f10600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 -- 172.21.15.78:0/762006659 shutdown_connections 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.235+0000 7f1067ded640 1 -- 172.21.15.78:0/762006659 wait complete. 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1067ded640 1 Processor -- start 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1067ded640 1 -- start start 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1067ded640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1060100e20 0x7f10601993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1067ded640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1060102570 0x7f10601998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1067ded640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060102ef0 0x7f10601a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1067ded640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1060110fc0 con 0x7f1060102ef0 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1067ded640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1060110e40 con 0x7f1060102570 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1067ded640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1060111140 con 0x7f1060100e20 2024-09-16T00:22:33.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1065b89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1060100e20 0x7f10601993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f106638a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060102ef0 0x7f10601a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1065b89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1060100e20 0x7f10601993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47570/0 (socket says 172.21.15.78:47570) 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1065b89640 1 -- 172.21.15.78:0/2717826597 learned_addr learned my addr 172.21.15.78:0/2717826597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.236+0000 7f1065388640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1060102570 0x7f10601998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1065b89640 1 -- 172.21.15.78:0/2717826597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1060102570 msgr2=0x7f10601998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1065b89640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1060102570 0x7f10601998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1065b89640 1 -- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060102ef0 msgr2=0x7f10601a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1065b89640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060102ef0 0x7f10601a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1065b89640 1 -- 172.21.15.78:0/2717826597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10601a4960 con 0x7f1060100e20 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f106638a640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060102ef0 0x7f10601a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1065b89640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1060100e20 0x7f10601993a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f105c006010 tx=0x7f105c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:33.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1056ffd640 1 -- 172.21.15.78:0/2717826597 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c03f070 con 0x7f1060100e20 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10601a4b30 con 0x7f1060100e20 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1056ffd640 1 -- 172.21.15.78:0/2717826597 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f105c031c30 con 0x7f1060100e20 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1056ffd640 1 -- 172.21.15.78:0/2717826597 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c035030 con 0x7f1060100e20 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.237+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a4fe0 con 0x7f1060100e20 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.238+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1060101200 con 0x7f1060100e20 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.240+0000 7f1056ffd640 1 -- 172.21.15.78:0/2717826597 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f105c042070 con 0x7f1060100e20 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.240+0000 7f1056ffd640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f104c0a3240 0x7f104c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.240+0000 7f1065388640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f104c0a3240 0x7f104c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.240+0000 7f1056ffd640 1 -- 172.21.15.78:0/2717826597 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f105c0f07f0 con 0x7f1060100e20 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.241+0000 7f1065388640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f104c0a3240 0x7f104c0a5700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f106006aac0 tx=0x7f1050005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:33.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.244+0000 7f1056ffd640 1 -- 172.21.15.78:0/2717826597 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f105c0bcb00 con 0x7f1060100e20 2024-09-16T00:22:33.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.583+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1060109090 con 0x7f1060100e20 2024-09-16T00:22:33.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.584+0000 7f1056ffd640 1 -- 172.21.15.78:0/2717826597 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f105c0c0e00 con 0x7f1060100e20 2024-09-16T00:22:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:22:33.585 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:22:33.586 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:22:33.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f104c0a3240 msgr2=0x7f104c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f104c0a3240 0x7f104c0a5700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f106006aac0 tx=0x7f1050005f90 comp rx=0 tx=0).stop 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1060100e20 msgr2=0x7f10601993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1060100e20 0x7f10601993a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f105c006010 tx=0x7f105c003c10 comp rx=0 tx=0).stop 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 shutdown_connections 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f104c0a3240 0x7f104c0a5700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1060102ef0 0x7f10601a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1060102570 0x7f10601998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 --2- 172.21.15.78:0/2717826597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1060100e20 0x7f10601993a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 >> 172.21.15.78:0/2717826597 conn(0x7f10600fc7d0 msgr2=0x7f10600fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 shutdown_connections 2024-09-16T00:22:33.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:33.587+0000 7f1067ded640 1 -- 172.21.15.78:0/2717826597 wait complete. 2024-09-16T00:22:33.867 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:33 smithi078 ceph-mon[31157]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-16T00:22:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:33 smithi193 ceph-mon[35484]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-16T00:22:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:33 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2717826597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:33 smithi148 ceph-mon[35599]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-16T00:22:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:33 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2717826597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:34.223 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:33 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2717826597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:35.235 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 62a7683a --release --await' 2024-09-16T00:22:35.236 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 62a7683a --release --await 2024-09-16T00:22:35.499 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 515.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:22:35.906 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:35 smithi078 ceph-mon[31157]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 192 KiB/s wr, 31 op/s 2024-09-16T00:22:36.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 -- 172.21.15.78:0/2822937645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708100a80 msgr2=0x7fa708100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 --2- 172.21.15.78:0/2822937645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708100a80 0x7fa708100ee0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009930 tx=0x7fa6f802a420 comp rx=0 tx=0).stop 2024-09-16T00:22:36.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 -- 172.21.15.78:0/2822937645 shutdown_connections 2024-09-16T00:22:36.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 --2- 172.21.15.78:0/2822937645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa708101420 0x7fa70810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 --2- 172.21.15.78:0/2822937645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708100a80 0x7fa708100ee0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 --2- 172.21.15.78:0/2822937645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708106a80 0x7fa708106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 -- 172.21.15.78:0/2822937645 >> 172.21.15.78:0/2822937645 conn(0x7fa7080fc7a0 msgr2=0x7fa7080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 -- 172.21.15.78:0/2822937645 shutdown_connections 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.992+0000 7fa70f480640 1 -- 172.21.15.78:0/2822937645 wait complete. 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70f480640 1 Processor -- start 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70f480640 1 -- start start 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70f480640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708100a80 0x7fa708199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70f480640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa708101420 0x7fa708199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70f480640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708106a80 0x7fa7081a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70f480640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa708110e80 con 0x7fa708106a80 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70f480640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa708110d00 con 0x7fa708101420 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70f480640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa708111000 con 0x7fa708100a80 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70da1d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708106a80 0x7fa7081a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70da1d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708106a80 0x7fa7081a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49226/0 (socket says 172.21.15.78:49226) 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70da1d640 1 -- 172.21.15.78:0/3215805383 learned_addr learned my addr 172.21.15.78:0/3215805383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa708101420 0x7fa708199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70d21c640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708100a80 0x7fa708199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70da1d640 1 -- 172.21.15.78:0/3215805383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708100a80 msgr2=0x7fa708199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70da1d640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708100a80 0x7fa708199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70da1d640 1 -- 172.21.15.78:0/3215805383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa708101420 msgr2=0x7fa708199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70da1d640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa708101420 0x7fa708199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.993+0000 7fa70da1d640 1 -- 172.21.15.78:0/3215805383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7081a48d0 con 0x7fa708106a80 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.994+0000 7fa70da1d640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708106a80 0x7fa7081a4390 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa70000cad0 tx=0x7fa70000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:36.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.994+0000 7fa6fe7fc640 1 -- 172.21.15.78:0/3215805383 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70000ae20 con 0x7fa708106a80 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.994+0000 7fa6fe7fc640 1 -- 172.21.15.78:0/3215805383 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa700005020 con 0x7fa708106a80 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.994+0000 7fa70f480640 1 -- 172.21.15.78:0/3215805383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7081a4aa0 con 0x7fa708106a80 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.994+0000 7fa6fe7fc640 1 -- 172.21.15.78:0/3215805383 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70000f6f0 con 0x7fa708106a80 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.994+0000 7fa70f480640 1 -- 172.21.15.78:0/3215805383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7081a4f60 con 0x7fa708106a80 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa6fe7fc640 1 -- 172.21.15.78:0/3215805383 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa7000051d0 con 0x7fa708106a80 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa711859740 1 Processor -- start 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa711859740 1 -- start start 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa711859740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570cf20 0x55d02570d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa711859740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570d840 0x55d02571c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa711859740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa711859740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d025672620 con 0x55d02571c650 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa711859740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d025619e50 con 0x55d02570cf20 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.997+0000 7fa711859740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d02566e5e0 con 0x55d02570d840 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49240/0 (socket says 172.21.15.78:49240) 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 -- 172.21.15.78:0/203617526 learned_addr learned my addr 172.21.15.78:0/203617526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70d21c640 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570d840 0x55d02571c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570cf20 0x55d02570d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa6fe7fc640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6f40a32c0 0x7fa6f40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 -- 172.21.15.78:0/203617526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570d840 msgr2=0x55d02571c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:36.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570d840 0x55d02571c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 -- 172.21.15.78:0/203617526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570cf20 msgr2=0x55d02570d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570cf20 0x55d02570d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 -- 172.21.15.78:0/203617526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d02560ad40 con 0x55d02571c650 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70da1d640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6f40a32c0 0x7fa6f40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa6fe7fc640 1 -- 172.21.15.78:0/3215805383 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa7000cfc80 con 0x7fa708106a80 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571ea40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8005ec0 tx=0x7fa6f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1ec4ec73aa5269 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa6f37fe640 1 -- 172.21.15.78:0/203617526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8035070 con 0x55d02571c650 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa6f37fe640 1 -- 172.21.15.78:0/203617526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f8031800 con 0x55d02571c650 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.998+0000 7fa6f37fe640 1 -- 172.21.15.78:0/203617526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80047a0 con 0x55d02571c650 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 -- 172.21.15.78:0/203617526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 msgr2=0x55d02571ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa70da1d640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6f40a32c0 0x7fa6f40a5780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa70001a000 tx=0x7fa70006edc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571ea40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8005ec0 tx=0x7fa6f8005fd0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 -- 172.21.15.78:0/203617526 shutdown_connections 2024-09-16T00:22:36.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571ea40 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570d840 0x55d02571c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 --2- 172.21.15.78:0/203617526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570cf20 0x55d02570d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 -- 172.21.15.78:0/203617526 >> 172.21.15.78:0/203617526 conn(0x55d0257095f0 msgr2=0x55d0257099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 -- 172.21.15.78:0/203617526 shutdown_connections 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 -- 172.21.15.78:0/203617526 wait complete. 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:35.999+0000 7fa711859740 1 -- start start 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa711859740 1 Processor -- start 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa711859740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 0x55d025719580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa711859740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025719ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa711859740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa711859740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d02571f1c0 con 0x55d02571c650 2024-09-16T00:22:36.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa711859740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d0255faf10 con 0x55d02570d840 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa711859740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d02571f340 con 0x55d02570cf20 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa711859740 10 client.0 _ll_register_callbacks cb 0x55d02561edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa70ca1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa70ca1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49242/0 (socket says 172.21.15.78:49242) 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa70ca1b640 1 -- 172.21.15.78:0/1775056118 learned_addr learned my addr 172.21.15.78:0/1775056118 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa70d21c640 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025719ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.000+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 0x55d025719580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa70d21c640 1 -- 172.21.15.78:0/1775056118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 msgr2=0x55d025719580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa70d21c640 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 0x55d025719580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa70d21c640 1 -- 172.21.15.78:0/1775056118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 msgr2=0x55d02571a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa70d21c640 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa70d21c640 1 -- 172.21.15.78:0/1775056118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d025381580 con 0x55d02570d840 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa70d21c640 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025719ac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7040058c0 tx=0x7fa704011ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7896aede770c2bc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:36.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa6f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1775056118 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70401e060 con 0x55d02570d840 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa711859740 1 -- 172.21.15.78:0/1775056118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f8031800 con 0x55d02570d840 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1775056118 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa70400fba0 con 0x55d02570d840 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1775056118 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa70400fda0 con 0x55d02570d840 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa711859740 10 client.26369 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa711859740 1 -- 172.21.15.78:0/1775056118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d02544b960 con 0x55d02570d840 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.001+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1775056118 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f8031800 con 0x55d02570d840 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1775056118 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa704018030 con 0x55d02570d840 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa711859740 10 client.26369 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa711859740 10 client.26369 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa711859740 20 client.26369 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa711859740 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d02572c2f0 0x55d02572e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa711859740 4 client.26369 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa711859740 1 -- 172.21.15.78:0/1775056118 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d02567fb90 con 0x55d02572c2f0 2024-09-16T00:22:36.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d02572c2f0 0x55d02572e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa6f0ff9640 10 client.26369 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.002+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d02572c2f0 0x55d02572e710 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ccbd67c61b628a server_cookie=3f459756e6900859 in_seq=0 out_seq=0 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1775056118 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d02567fb90 con 0x55d02572c2f0 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa6f0ff9640 10 client.26369 handle_command_reply: tid=0 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 client.26369 shutdown 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 -- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 msgr2=0x55d025719ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025719ac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7040058c0 tx=0x7fa704011ba0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 -- 172.21.15.78:0/1775056118 shutdown_connections 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d02572c2f0 0x55d02572e710 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02571a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa6f0ff9640 0 client.26369 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025719ac0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 --2- 172.21.15.78:0/1775056118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 0x55d025719580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 -- 172.21.15.78:0/1775056118 >> 172.21.15.78:0/1775056118 conn(0x55d0257095f0 msgr2=0x55d025708750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:36.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 -- 172.21.15.78:0/1775056118 shutdown_connections 2024-09-16T00:22:36.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 1 -- 172.21.15.78:0/1775056118 wait complete. 2024-09-16T00:22:36.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.003+0000 7fa711859740 20 client.26369 trim_cache size 0 max 16384 2024-09-16T00:22:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa711859740 1 Processor -- start 2024-09-16T00:22:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa711859740 1 -- start start 2024-09-16T00:22:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa711859740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 0x55d025708a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa711859740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d02570f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa711859740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02570f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa711859740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7081ac9c0 con 0x55d02571c650 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa711859740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7081ae500 con 0x55d02570d840 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa711859740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7081ad090 con 0x55d02570cf20 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02570f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02570f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49256/0 (socket says 172.21.15.78:49256) 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 -- 172.21.15.78:0/809988772 learned_addr learned my addr 172.21.15.78:0/809988772 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70d21c640 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d02570f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 0x55d025708a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 -- 172.21.15.78:0/809988772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 msgr2=0x55d025708a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 0x55d025708a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 -- 172.21.15.78:0/809988772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 msgr2=0x55d02570f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d02570f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 -- 172.21.15.78:0/809988772 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d02571f390 con 0x55d02571c650 2024-09-16T00:22:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.021+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02570f6a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80054b0 tx=0x7fa6f8032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544be0d4b1fd59a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/809988772 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8035070 con 0x55d02571c650 2024-09-16T00:22:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/809988772 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f8031780 con 0x55d02571c650 2024-09-16T00:22:36.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa6f2ffd640 1 -- 172.21.15.78:0/809988772 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8003750 con 0x55d02571c650 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 -- 172.21.15.78:0/809988772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 msgr2=0x55d02570f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02570f6a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80054b0 tx=0x7fa6f8032f60 comp rx=0 tx=0).stop 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 -- 172.21.15.78:0/809988772 shutdown_connections 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02571c650 0x55d02570f6a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d02570f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 --2- 172.21.15.78:0/809988772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02570cf20 0x55d025708a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 -- 172.21.15.78:0/809988772 >> 172.21.15.78:0/809988772 conn(0x55d0257095f0 msgr2=0x55d02570ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 -- 172.21.15.78:0/809988772 shutdown_connections 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.022+0000 7fa711859740 1 -- 172.21.15.78:0/809988772 wait complete. 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 1 -- start start 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 1 Processor -- start 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02570cf20 0x55d025724ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025721210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02571c650 0x55d0257219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7081ade30 con 0x55d02570cf20 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7081ad760 con 0x55d02570d840 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7081aa9a0 con 0x55d02571c650 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa711859740 10 client.0 _ll_register_callbacks cb 0x7fa70400cbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa70ca1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02570cf20 0x55d025724ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa70ca1b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02570cf20 0x55d025724ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49266/0 (socket says 172.21.15.78:49266) 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa70ca1b640 1 -- 172.21.15.78:0/1003607853 learned_addr learned my addr 172.21.15.78:0/1003607853 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.023+0000 7fa70d21c640 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025721210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa70ca1b640 1 -- 172.21.15.78:0/1003607853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02571c650 msgr2=0x55d0257219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02571c650 0x55d0257219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa70ca1b640 1 -- 172.21.15.78:0/1003607853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 msgr2=0x55d025721210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025721210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa70ca1b640 1 -- 172.21.15.78:0/1003607853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d025796fa0 con 0x55d02570cf20 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa70ca1b640 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02570cf20 0x55d025724ed0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80054e0 tx=0x7fa6f8005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93dc8bbc1943fb8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa6f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1003607853 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8035070 con 0x55d02570cf20 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1003607853 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa6f80040c0 con 0x55d02570cf20 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa711859740 1 -- 172.21.15.78:0/1003607853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f8031780 con 0x55d02570cf20 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1003607853 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa6f805b040 con 0x55d02570cf20 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa711859740 10 client.17001 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa711859740 1 -- 172.21.15.78:0/1003607853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d025798130 con 0x55d02570cf20 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1003607853 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f8031780 con 0x55d02570cf20 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1003607853 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa6f805e030 con 0x55d02570cf20 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.024+0000 7fa711859740 10 client.17001 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.025+0000 7fa711859740 10 client.17001 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.025+0000 7fa711859740 20 client.17001 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.025+0000 7fa711859740 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d025798360 0x55d02579a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.025+0000 7fa711859740 4 client.17001 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "62a7683a", "release": true, "await": true}] 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.025+0000 7fa711859740 1 -- 172.21.15.78:0/1003607853 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "62a7683a", "release": true, "await": true}) -- 0x55d02567a0a0 con 0x55d025798360 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.025+0000 7fa70d21c640 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d025798360 0x55d02579a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.025+0000 7fa70d21c640 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d025798360 0x55d02579a750 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e2662bc1bd7aa7 server_cookie=8363660557f25ec2 in_seq=0 out_seq=0 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.025+0000 7fa6f0ff9640 10 client.17001 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:36.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa6f0ff9640 1 -- 172.21.15.78:0/1003607853 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d02567a0a0 con 0x55d025798360 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa6f0ff9640 10 client.17001 handle_command_reply: tid=0 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 client.17001 shutdown 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 -- 172.21.15.78:0/1003607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02570cf20 msgr2=0x55d025724ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02570cf20 0x55d025724ed0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80054e0 tx=0x7fa6f8005510 comp rx=0 tx=0).stop 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 -- 172.21.15.78:0/1003607853 shutdown_connections 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d025798360 0x55d02579a750 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d02571c650 0x55d0257219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d02570d840 0x55d025721210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 --2- 172.21.15.78:0/1003607853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d02570cf20 0x55d025724ed0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 -- 172.21.15.78:0/1003607853 >> 172.21.15.78:0/1003607853 conn(0x55d0257095f0 msgr2=0x55d02570f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:36.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa6f0ff9640 0 client.17001 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 -- 172.21.15.78:0/1003607853 shutdown_connections 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 1 -- 172.21.15.78:0/1003607853 wait complete. 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.086+0000 7fa711859740 20 client.17001 trim_cache size 0 max 16384 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 32, 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stdout: "62a7683a": { 2024-09-16T00:22:36.089 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 32, 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 142.3, 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:22:36.090 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:36.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:36.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:36.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:36.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:36.091 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:22:36.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.087+0000 7fa70f480640 1 -- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6f40a32c0 msgr2=0x7fa6f40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.087+0000 7fa70f480640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6f40a32c0 0x7fa6f40a5780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa70001a000 tx=0x7fa70006edc0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.088+0000 7fa70f480640 1 -- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708106a80 msgr2=0x7fa7081a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.088+0000 7fa70f480640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708106a80 0x7fa7081a4390 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa70000cad0 tx=0x7fa70000cf90 comp rx=0 tx=0).stop 2024-09-16T00:22:36.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.088+0000 7fa70f480640 1 -- 172.21.15.78:0/3215805383 shutdown_connections 2024-09-16T00:22:36.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.088+0000 7fa70f480640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa6f40a32c0 0x7fa6f40a5780 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.088+0000 7fa70f480640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa708106a80 0x7fa7081a4390 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.088+0000 7fa70f480640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa708101420 0x7fa708199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.088+0000 7fa70f480640 1 --2- 172.21.15.78:0/3215805383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa708100a80 0x7fa708199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.088+0000 7fa70f480640 1 -- 172.21.15.78:0/3215805383 >> 172.21.15.78:0/3215805383 conn(0x7fa7080fc7a0 msgr2=0x7fa7080fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:36.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.091+0000 7fa70f480640 1 -- 172.21.15.78:0/3215805383 shutdown_connections 2024-09-16T00:22:36.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:36.092+0000 7fa70f480640 1 -- 172.21.15.78:0/3215805383 wait complete. 2024-09-16T00:22:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:35 smithi193 ceph-mon[35484]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 192 KiB/s wr, 31 op/s 2024-09-16T00:22:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:35 smithi148 ceph-mon[35599]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 192 KiB/s wr, 31 op/s 2024-09-16T00:22:36.855 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '62a7683a', total seconds elapsed: 26.7 2024-09-16T00:22:36.855 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-16T00:22:36.856 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:22:37.288 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:37.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.881+0000 7f32c2c28640 1 -- 172.21.15.78:0/2707105298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc102570 msgr2=0x7f32bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.881+0000 7f32c2c28640 1 --2- 172.21.15.78:0/2707105298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc102570 0x7f32bc1029b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009900 tx=0x7f32ac02a480 comp rx=0 tx=0).stop 2024-09-16T00:22:37.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.881+0000 7f32c2c28640 1 -- 172.21.15.78:0/2707105298 shutdown_connections 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.881+0000 7f32c2c28640 1 --2- 172.21.15.78:0/2707105298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32bc102ef0 0x7f32bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.881+0000 7f32c2c28640 1 --2- 172.21.15.78:0/2707105298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc102570 0x7f32bc1029b0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.881+0000 7f32c2c28640 1 --2- 172.21.15.78:0/2707105298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32bc100e20 0x7f32bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.882+0000 7f32c2c28640 1 -- 172.21.15.78:0/2707105298 >> 172.21.15.78:0/2707105298 conn(0x7f32bc0fc7d0 msgr2=0x7f32bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.882+0000 7f32c2c28640 1 -- 172.21.15.78:0/2707105298 shutdown_connections 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.882+0000 7f32c2c28640 1 -- 172.21.15.78:0/2707105298 wait complete. 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.882+0000 7f32c2c28640 1 Processor -- start 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.882+0000 7f32c2c28640 1 -- start start 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c2c28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc100e20 0x7f32bc199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c2c28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32bc102570 0x7f32bc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c2c28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32bc102ef0 0x7f32bc1a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c2c28640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc110f80 con 0x7f32bc100e20 2024-09-16T00:22:37.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c2c28640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc110e00 con 0x7f32bc102570 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c2c28640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc111100 con 0x7f32bc102ef0 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc100e20 0x7f32bc199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc100e20 0x7f32bc199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60174/0 (socket says 172.21.15.78:60174) 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 -- 172.21.15.78:0/4056060064 learned_addr learned my addr 172.21.15.78:0/4056060064 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32b3fff640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32bc102570 0x7f32bc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c11c5640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32bc102ef0 0x7f32bc1a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 -- 172.21.15.78:0/4056060064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32bc102ef0 msgr2=0x7f32bc1a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32bc102ef0 0x7f32bc1a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 -- 172.21.15.78:0/4056060064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32bc102570 msgr2=0x7f32bc1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32bc102570 0x7f32bc1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 -- 172.21.15.78:0/4056060064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32bc1a4950 con 0x7f32bc100e20 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.883+0000 7f32c09c4640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc100e20 0x7f32bc199390 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f32b800e9f0 tx=0x7f32b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.884+0000 7f32b1ffb640 1 -- 172.21.15.78:0/4056060064 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b800cd80 con 0x7f32bc100e20 2024-09-16T00:22:37.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.884+0000 7f32b1ffb640 1 -- 172.21.15.78:0/4056060064 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b8010910 con 0x7f32bc100e20 2024-09-16T00:22:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.884+0000 7f32c2c28640 1 -- 172.21.15.78:0/4056060064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32bc1a4b20 con 0x7f32bc100e20 2024-09-16T00:22:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.884+0000 7f32b1ffb640 1 -- 172.21.15.78:0/4056060064 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8018990 con 0x7f32bc100e20 2024-09-16T00:22:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.884+0000 7f32c2c28640 1 -- 172.21.15.78:0/4056060064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32bc1a5030 con 0x7f32bc100e20 2024-09-16T00:22:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.885+0000 7f32c2c28640 1 -- 172.21.15.78:0/4056060064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32bc1021b0 con 0x7f32bc100e20 2024-09-16T00:22:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.887+0000 7f32b1ffb640 1 -- 172.21.15.78:0/4056060064 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f32b8010ac0 con 0x7f32bc100e20 2024-09-16T00:22:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.887+0000 7f32b1ffb640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f32a40a32c0 0x7f32a40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.888+0000 7f32b3fff640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f32a40a32c0 0x7f32a40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.888+0000 7f32b1ffb640 1 -- 172.21.15.78:0/4056060064 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f32b80cfed0 con 0x7f32bc100e20 2024-09-16T00:22:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.888+0000 7f32b3fff640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f32a40a32c0 0x7f32a40a5780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009230 tx=0x7f32ac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:37.892+0000 7f32b1ffb640 1 -- 172.21.15.78:0/4056060064 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32b809c240 con 0x7f32bc100e20 2024-09-16T00:22:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:37 smithi078 ceph-mon[31157]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 471 KiB/s wr, 68 op/s 2024-09-16T00:22:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:37 smithi193 ceph-mon[35484]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 471 KiB/s wr, 68 op/s 2024-09-16T00:22:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:37 smithi148 ceph-mon[35599]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 471 KiB/s wr, 68 op/s 2024-09-16T00:22:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.267+0000 7f32c2c28640 1 -- 172.21.15.78:0/4056060064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32bc06a560 con 0x7f32bc100e20 2024-09-16T00:22:38.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.270+0000 7f32b1ffb640 1 -- 172.21.15.78:0/4056060064 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f32b80a0540 con 0x7f32bc100e20 2024-09-16T00:22:38.271 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:22:38.271 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:22:38.271 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:22:38.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.274+0000 7f32a37fe640 1 -- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f32a40a32c0 msgr2=0x7f32a40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.274+0000 7f32a37fe640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f32a40a32c0 0x7f32a40a5780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009230 tx=0x7f32ac0058d0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.274+0000 7f32a37fe640 1 -- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc100e20 msgr2=0x7f32bc199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.274+0000 7f32a37fe640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc100e20 0x7f32bc199390 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f32b800e9f0 tx=0x7f32b800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.275+0000 7f32a37fe640 1 -- 172.21.15.78:0/4056060064 shutdown_connections 2024-09-16T00:22:38.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.275+0000 7f32a37fe640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f32a40a32c0 0x7f32a40a5780 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.275+0000 7f32a37fe640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32bc102ef0 0x7f32bc1a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.275+0000 7f32a37fe640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f32bc102570 0x7f32bc1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.275+0000 7f32a37fe640 1 --2- 172.21.15.78:0/4056060064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f32bc100e20 0x7f32bc199390 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.275+0000 7f32a37fe640 1 -- 172.21.15.78:0/4056060064 >> 172.21.15.78:0/4056060064 conn(0x7f32bc0fc7d0 msgr2=0x7f32bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:38.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.278+0000 7f32a37fe640 1 -- 172.21.15.78:0/4056060064 shutdown_connections 2024-09-16T00:22:38.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:38.278+0000 7f32a37fe640 1 -- 172.21.15.78:0/4056060064 wait complete. 2024-09-16T00:22:39.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:38 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4056060064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:39.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:38 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4056060064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:39.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:38 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4056060064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:39.909 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-16T00:22:39.909 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-16T00:22:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:39 smithi078 ceph-mon[31157]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 634 KiB/s wr, 86 op/s 2024-09-16T00:22:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:39 smithi193 ceph-mon[35484]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 634 KiB/s wr, 86 op/s 2024-09-16T00:22:40.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:39 smithi148 ceph-mon[35599]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 634 KiB/s wr, 86 op/s 2024-09-16T00:22:40.324 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 520.0 s, 773.4 tps, lat 637.220 ms stddev 3193.577 2024-09-16T00:22:42.005 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:41 smithi148 ceph-mon[35599]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-09-16T00:22:42.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:41 smithi078 ceph-mon[31157]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-09-16T00:22:42.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:41 smithi193 ceph-mon[35484]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 MiB/s wr, 195 op/s 2024-09-16T00:22:42.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:22:42.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.348+0000 7f10fa2b9640 1 -- 172.21.15.78:0/2334725797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f411b140 msgr2=0x7f10f411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.348+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/2334725797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f411b140 0x7f10f411d530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009ca0 tx=0x7f10ec02a8f0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.348+0000 7f10fa2b9640 1 -- 172.21.15.78:0/2334725797 shutdown_connections 2024-09-16T00:22:42.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.348+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/2334725797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f411b140 0x7f10f411d530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.348+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/2334725797 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10f4109830 0x7f10f4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.348+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/2334725797 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f4109450 0x7f10f4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.348+0000 7f10fa2b9640 1 -- 172.21.15.78:0/2334725797 >> 172.21.15.78:0/2334725797 conn(0x7f10f406e140 msgr2=0x7f10f406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:42.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.349+0000 7f10fa2b9640 1 -- 172.21.15.78:0/2334725797 shutdown_connections 2024-09-16T00:22:42.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.349+0000 7f10fa2b9640 1 -- 172.21.15.78:0/2334725797 wait complete. 2024-09-16T00:22:42.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.350+0000 7f10fa2b9640 1 Processor -- start 2024-09-16T00:22:42.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.351+0000 7f10fa2b9640 1 -- start start 2024-09-16T00:22:42.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.351+0000 7f10fa2b9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f4109450 0x7f10f41a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.351+0000 7f10fa2b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f4109830 0x7f10f41a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.351+0000 7f10fa2b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10f411b140 0x7f10f41abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.351+0000 7f10fa2b9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4120ed0 con 0x7f10f4109830 2024-09-16T00:22:42.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.351+0000 7f10fa2b9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4120d50 con 0x7f10f411b140 2024-09-16T00:22:42.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.351+0000 7f10fa2b9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10f4121050 con 0x7f10f4109450 2024-09-16T00:22:42.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f4109830 0x7f10f41a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f4109830 0x7f10f41a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60190/0 (socket says 172.21.15.78:60190) 2024-09-16T00:22:42.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f37fe640 1 -- 172.21.15.78:0/1744668931 learned_addr learned my addr 172.21.15.78:0/1744668931 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:42.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f8856640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10f411b140 0x7f10f41abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f3fff640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f4109450 0x7f10f41a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f8856640 1 -- 172.21.15.78:0/1744668931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f4109450 msgr2=0x7f10f41a72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f8856640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f4109450 0x7f10f41a72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f8856640 1 -- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f4109830 msgr2=0x7f10f41a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f8856640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f4109830 0x7f10f41a77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.352+0000 7f10f8856640 1 -- 172.21.15.78:0/1744668931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f41ac260 con 0x7f10f411b140 2024-09-16T00:22:42.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.353+0000 7f10f8856640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10f411b140 0x7f10f41abb80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0098d0 tx=0x7f10ec0316b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:42.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.353+0000 7f10f17fa640 1 -- 172.21.15.78:0/1744668931 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec003970 con 0x7f10f411b140 2024-09-16T00:22:42.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.353+0000 7f10fa2b9640 1 -- 172.21.15.78:0/1744668931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f41ac4f0 con 0x7f10f411b140 2024-09-16T00:22:42.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.353+0000 7f10fa2b9640 1 -- 172.21.15.78:0/1744668931 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f41b3e30 con 0x7f10f411b140 2024-09-16T00:22:42.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.353+0000 7f10f17fa640 1 -- 172.21.15.78:0/1744668931 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10ec02d070 con 0x7f10f411b140 2024-09-16T00:22:42.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.353+0000 7f10f17fa640 1 -- 172.21.15.78:0/1744668931 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec02f600 con 0x7f10f411b140 2024-09-16T00:22:42.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.360+0000 7f10fc692740 1 Processor -- start 2024-09-16T00:22:42.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.360+0000 7f10fc692740 1 -- start start 2024-09-16T00:22:42.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.360+0000 7f10fc692740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c02dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.360+0000 7f10fc692740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c0384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.360+0000 7f10fc692740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c03ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.360+0000 7f10fc692740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a55bf92160 con 0x55a55c02e170 2024-09-16T00:22:42.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.360+0000 7f10fc692740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a55bf39720 con 0x55a55c038a30 2024-09-16T00:22:42.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.360+0000 7f10fc692740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a55bf8e120 con 0x55a55c02d850 2024-09-16T00:22:42.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.361+0000 7f10f17fa640 1 -- 172.21.15.78:0/1744668931 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f10ec003b10 con 0x7f10f411b140 2024-09-16T00:22:42.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.362+0000 7f10f17fa640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10dc0a3460 0x7f10dc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.363+0000 7f10f17fa640 1 -- 172.21.15.78:0/1744668931 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f10ec0f1070 con 0x7f10f411b140 2024-09-16T00:22:42.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.363+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c0384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.363+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c0384f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60198/0 (socket says 172.21.15.78:60198) 2024-09-16T00:22:42.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.363+0000 7f10f3fff640 1 -- 172.21.15.78:0/3969893637 learned_addr learned my addr 172.21.15.78:0/3969893637 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:42.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.364+0000 7f10f37fe640 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c02dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.364+0000 7f10f37fe640 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c03ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.364+0000 7f10f8856640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10dc0a3460 0x7f10dc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.365+0000 7f10f8856640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10dc0a3460 0x7f10dc0a5920 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0b9ca0 tx=0x7f10ec0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:42.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.367+0000 7f10f3fff640 1 -- 172.21.15.78:0/3969893637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 msgr2=0x55a55c02dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:42.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.367+0000 7f10f3fff640 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c02dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.368+0000 7f10f3fff640 1 -- 172.21.15.78:0/3969893637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 msgr2=0x55a55c03ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:42.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.368+0000 7f10f3fff640 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c03ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.368+0000 7f10f3fff640 1 -- 172.21.15.78:0/3969893637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55bd03ba0 con 0x55a55c02e170 2024-09-16T00:22:42.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.368+0000 7f10f3fff640 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c0384f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f10e400b820 tx=0x7f10e400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7416159adb724f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:42.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.368+0000 7f10da7fc640 1 -- 172.21.15.78:0/3969893637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e400d950 con 0x55a55c02e170 2024-09-16T00:22:42.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.369+0000 7f10da7fc640 1 -- 172.21.15.78:0/3969893637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e4019070 con 0x55a55c02e170 2024-09-16T00:22:42.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.369+0000 7f10da7fc640 1 -- 172.21.15.78:0/3969893637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e40123e0 con 0x55a55c02e170 2024-09-16T00:22:42.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.370+0000 7f10fc692740 1 -- 172.21.15.78:0/3969893637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 msgr2=0x55a55c0384f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.370+0000 7f10fc692740 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c0384f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f10e400b820 tx=0x7f10e400bce0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.370+0000 7f10fc692740 1 -- 172.21.15.78:0/3969893637 shutdown_connections 2024-09-16T00:22:42.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.370+0000 7f10fc692740 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c03ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.370+0000 7f10fc692740 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c0384f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.370+0000 7f10fc692740 1 --2- 172.21.15.78:0/3969893637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c02dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.370+0000 7f10fc692740 1 -- 172.21.15.78:0/3969893637 >> 172.21.15.78:0/3969893637 conn(0x55a55c029de0 msgr2=0x55a55c02a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:42.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.373+0000 7f10fc692740 1 -- 172.21.15.78:0/3969893637 shutdown_connections 2024-09-16T00:22:42.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.373+0000 7f10fc692740 1 -- 172.21.15.78:0/3969893637 wait complete. 2024-09-16T00:22:42.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.373+0000 7f10fc692740 1 -- start start 2024-09-16T00:22:42.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.374+0000 7f10fc692740 1 Processor -- start 2024-09-16T00:22:42.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.375+0000 7f10fc692740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02d850 0x55a55c0358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.375+0000 7f10fc692740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02e170 0x55a55c035e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.375+0000 7f10fc692740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c036340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.375+0000 7f10fc692740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a55c03b600 con 0x55a55c02d850 2024-09-16T00:22:42.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.375+0000 7f10fc692740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a55bf1a900 con 0x55a55c038a30 2024-09-16T00:22:42.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.375+0000 7f10fc692740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a55c03b780 con 0x55a55c02e170 2024-09-16T00:22:42.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.375+0000 7f10fc692740 10 client.0 _ll_register_callbacks cb 0x55a55bf18780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:42.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.375+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02e170 0x55a55c035e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.376+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02e170 0x55a55c035e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:43268/0 (socket says 172.21.15.78:43268) 2024-09-16T00:22:42.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.376+0000 7f10f3fff640 1 -- 172.21.15.78:0/1284481738 learned_addr learned my addr 172.21.15.78:0/1284481738 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:42.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.376+0000 7f10f3fff640 1 -- 172.21.15.78:0/1284481738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 msgr2=0x55a55c036340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:22:42.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.376+0000 7f10f3fff640 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c036340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.376+0000 7f10f3fff640 1 -- 172.21.15.78:0/1284481738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02d850 msgr2=0x55a55c0358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:42.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.376+0000 7f10f3fff640 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02d850 0x55a55c0358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.377+0000 7f10f3fff640 1 -- 172.21.15.78:0/1284481738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55bf5ac50 con 0x55a55c02e170 2024-09-16T00:22:42.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.377+0000 7f10f3fff640 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02e170 0x55a55c035e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10e4014bd0 tx=0x7f10e4014660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=425e0adbcebec09f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:42.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.377+0000 7f10d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:22:42.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.377+0000 7f10d37fe640 1 -- 172.21.15.78:0/1284481738 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4023070 con 0x55a55c02e170 2024-09-16T00:22:42.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.377+0000 7f10fc692740 1 -- 172.21.15.78:0/1284481738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10e4019070 con 0x55a55c02e170 2024-09-16T00:22:42.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.378+0000 7f10d37fe640 1 -- 172.21.15.78:0/1284481738 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f10e401f440 con 0x55a55c02e170 2024-09-16T00:22:42.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.378+0000 7f10d37fe640 1 -- 172.21.15.78:0/1284481738 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f10e402f050 con 0x55a55c02e170 2024-09-16T00:22:42.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.378+0000 7f10fc692740 10 client.26218 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:42.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.378+0000 7f10fc692740 1 -- 172.21.15.78:0/1284481738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a55c0472d0 con 0x55a55c02e170 2024-09-16T00:22:42.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.379+0000 7f10d37fe640 1 -- 172.21.15.78:0/1284481738 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e4019070 con 0x55a55c02e170 2024-09-16T00:22:42.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.379+0000 7f10d37fe640 1 -- 172.21.15.78:0/1284481738 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f10e4013070 con 0x55a55c02e170 2024-09-16T00:22:42.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.379+0000 7f10fc692740 10 client.26218 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:42.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.379+0000 7f10fc692740 10 client.26218 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:22:42.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.379+0000 7f10fc692740 20 client.26218 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:42.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.379+0000 7f10fc692740 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a55c048730 0x55a55c04ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.379+0000 7f10fc692740 4 client.26218 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:42.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.379+0000 7f10fc692740 1 -- 172.21.15.78:0/1284481738 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a55bf9f890 con 0x55a55c048730 2024-09-16T00:22:42.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.385+0000 7f10f37fe640 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a55c048730 0x55a55c04ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.386+0000 7f10f37fe640 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a55c048730 0x55a55c04ab20 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d4375aadab8e0e0 server_cookie=ab29c93e5c11e454 in_seq=0 out_seq=0 2024-09-16T00:22:42.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.386+0000 7f10d37fe640 10 client.26218 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:42.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.387+0000 7f10d37fe640 1 -- 172.21.15.78:0/1284481738 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a55bf9f890 con 0x55a55c048730 2024-09-16T00:22:42.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.387+0000 7f10d37fe640 10 client.26218 handle_command_reply: tid=0 2024-09-16T00:22:42.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.387+0000 7f10fc692740 1 client.26218 shutdown 2024-09-16T00:22:42.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.388+0000 7f10fc692740 1 -- 172.21.15.78:0/1284481738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02e170 msgr2=0x55a55c035e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.388+0000 7f10fc692740 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02e170 0x55a55c035e00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10e4014bd0 tx=0x7f10e4014660 comp rx=0 tx=0).stop 2024-09-16T00:22:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.390+0000 7f10fc692740 1 -- 172.21.15.78:0/1284481738 shutdown_connections 2024-09-16T00:22:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.390+0000 7f10fc692740 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a55c048730 0x55a55c04ab20 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.390+0000 7f10fc692740 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c036340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.390+0000 7f10fc692740 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02e170 0x55a55c035e00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.390+0000 7f10fc692740 1 --2- 172.21.15.78:0/1284481738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02d850 0x55a55c0358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.390+0000 7f10fc692740 1 -- 172.21.15.78:0/1284481738 >> 172.21.15.78:0/1284481738 conn(0x55a55c029de0 msgr2=0x55a55c02fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.390+0000 7f10d37fe640 0 client.26218 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.391+0000 7f10fc692740 1 -- 172.21.15.78:0/1284481738 shutdown_connections 2024-09-16T00:22:42.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.391+0000 7f10fc692740 1 -- 172.21.15.78:0/1284481738 wait complete. 2024-09-16T00:22:42.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.391+0000 7f10fc692740 20 client.26218 trim_cache size 0 max 16384 2024-09-16T00:22:42.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.421+0000 7f10fc692740 1 Processor -- start 2024-09-16T00:22:42.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10fc692740 1 -- start start 2024-09-16T00:22:42.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10fc692740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c02d850 0x55a55c030e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10fc692740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c037830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10fc692740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c038a30 0x55a55c050150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10fc692740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10e000fb70 con 0x55a55c02e170 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10fc692740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10e00116b0 con 0x55a55c02d850 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10fc692740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0010240 con 0x55a55c038a30 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10f37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c02d850 0x55a55c030e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c038a30 0x55a55c050150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10f37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c02d850 0x55a55c030e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48934/0 (socket says 172.21.15.78:48934) 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10f3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c037830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.422+0000 7f10f37fe640 1 -- 172.21.15.78:0/2016643984 learned_addr learned my addr 172.21.15.78:0/2016643984 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.423+0000 7f10f37fe640 1 -- 172.21.15.78:0/2016643984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c038a30 msgr2=0x55a55c050150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.423+0000 7f10f37fe640 1 --2- 172.21.15.78:0/2016643984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c038a30 0x55a55c050150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.423+0000 7f10f37fe640 1 -- 172.21.15.78:0/2016643984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 msgr2=0x55a55c037830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.423+0000 7f10f37fe640 1 --2- 172.21.15.78:0/2016643984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c037830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.423+0000 7f10f37fe640 1 -- 172.21.15.78:0/2016643984 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55c034870 con 0x55a55c02d850 2024-09-16T00:22:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.423+0000 7f10f37fe640 1 --2- 172.21.15.78:0/2016643984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c02d850 0x55a55c030e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f10f411d860 tx=0x7f10e8004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc0b76e579864b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.423+0000 7f10d9ffb640 1 -- 172.21.15.78:0/2016643984 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8028070 con 0x55a55c02d850 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.423+0000 7f10d9ffb640 1 -- 172.21.15.78:0/2016643984 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e8010040 con 0x55a55c02d850 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.424+0000 7f10d9ffb640 1 -- 172.21.15.78:0/2016643984 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8023690 con 0x55a55c02d850 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.424+0000 7f10fc692740 1 -- 172.21.15.78:0/2016643984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c02d850 msgr2=0x55a55c030e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.424+0000 7f10fc692740 1 --2- 172.21.15.78:0/2016643984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c02d850 0x55a55c030e00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f10f411d860 tx=0x7f10e8004700 comp rx=0 tx=0).stop 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.424+0000 7f10fc692740 1 -- 172.21.15.78:0/2016643984 shutdown_connections 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.425+0000 7f10fc692740 1 --2- 172.21.15.78:0/2016643984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c038a30 0x55a55c050150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.425+0000 7f10fc692740 1 --2- 172.21.15.78:0/2016643984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c037830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.425+0000 7f10fc692740 1 --2- 172.21.15.78:0/2016643984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c02d850 0x55a55c030e00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.425+0000 7f10fc692740 1 -- 172.21.15.78:0/2016643984 >> 172.21.15.78:0/2016643984 conn(0x55a55c029de0 msgr2=0x55a55c037120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:42.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.425+0000 7f10fc692740 1 -- 172.21.15.78:0/2016643984 shutdown_connections 2024-09-16T00:22:42.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.425+0000 7f10fc692740 1 -- 172.21.15.78:0/2016643984 wait complete. 2024-09-16T00:22:42.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.425+0000 7f10fc692740 1 -- start start 2024-09-16T00:22:42.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.425+0000 7f10fc692740 1 Processor -- start 2024-09-16T00:22:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10fc692740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c041320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10fc692740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c03c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10fc692740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c03ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10fc692740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0010fe0 con 0x55a55c02e170 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10fc692740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0010910 con 0x55a55c038a30 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10fc692740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10e000db50 con 0x55a55c02d850 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10fc692740 10 client.0 _ll_register_callbacks cb 0x7f10e400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c041320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c041320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:43284/0 (socket says 172.21.15.78:43284) 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10f37fe640 1 -- 172.21.15.78:0/3438656205 learned_addr learned my addr 172.21.15.78:0/3438656205 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10f3fff640 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c03c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.426+0000 7f10f37fe640 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c03ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10f37fe640 1 -- 172.21.15.78:0/3438656205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 msgr2=0x55a55c03ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10f37fe640 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c03ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10f37fe640 1 -- 172.21.15.78:0/3438656205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 msgr2=0x55a55c03c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:42.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10f37fe640 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c03c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10f37fe640 1 -- 172.21.15.78:0/3438656205 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55c02dfa0 con 0x55a55c02d850 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10f37fe640 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c041320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f10e801a080 tx=0x7f10e801a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db3883ae49bbcd4c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10d37fe640 1 -- 172.21.15.78:0/3438656205 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8028070 con 0x55a55c02d850 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10d37fe640 1 -- 172.21.15.78:0/3438656205 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f10e800f070 con 0x55a55c02d850 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.427+0000 7f10fc692740 1 -- 172.21.15.78:0/3438656205 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10e8010040 con 0x55a55c02d850 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.428+0000 7f10d37fe640 1 -- 172.21.15.78:0/3438656205 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f10e8028070 con 0x55a55c02d850 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.428+0000 7f10fc692740 10 client.26224 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.428+0000 7f10fc692740 1 -- 172.21.15.78:0/3438656205 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a55c0b30b0 con 0x55a55c02d850 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.428+0000 7f10d37fe640 1 -- 172.21.15.78:0/3438656205 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e8010040 con 0x55a55c02d850 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.428+0000 7f10d37fe640 1 -- 172.21.15.78:0/3438656205 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f10e8015050 con 0x55a55c02d850 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.428+0000 7f10fc692740 10 client.26224 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.428+0000 7f10fc692740 10 client.26224 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.429+0000 7f10fc692740 20 client.26224 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.429+0000 7f10fc692740 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a55c0b3690 0x55a55c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.429+0000 7f10fc692740 4 client.26224 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-16T00:22:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.429+0000 7f10fc692740 1 -- 172.21.15.78:0/3438656205 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x55a55bf99be0 con 0x55a55c0b3690 2024-09-16T00:22:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.429+0000 7f10f3fff640 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a55c0b3690 0x55a55c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.430+0000 7f10d37fe640 10 client.26224 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.430+0000 7f10f3fff640 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a55c0b3690 0x55a55c0b5a80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c46aef1f7f61f43 server_cookie=7982bb2e05946a2e in_seq=0 out_seq=0 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10d37fe640 1 -- 172.21.15.78:0/3438656205 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a55bf99be0 con 0x55a55c0b3690 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10d37fe640 10 client.26224 handle_command_reply: tid=0 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 client.26224 shutdown 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 -- 172.21.15.78:0/3438656205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 msgr2=0x55a55c041320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c041320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f10e801a080 tx=0x7f10e801a180 comp rx=0 tx=0).stop 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 -- 172.21.15.78:0/3438656205 shutdown_connections 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a55c0b3690 0x55a55c0b5a80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10d37fe640 0 client.26224 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a55c038a30 0x55a55c03ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a55c02e170 0x55a55c03c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 --2- 172.21.15.78:0/3438656205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55c02d850 0x55a55c041320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 -- 172.21.15.78:0/3438656205 >> 172.21.15.78:0/3438656205 conn(0x55a55c029de0 msgr2=0x55a55c0360a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 -- 172.21.15.78:0/3438656205 shutdown_connections 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 1 -- 172.21.15.78:0/3438656205 wait complete. 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.669+0000 7f10fc692740 20 client.26224 trim_cache size 0 max 16384 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:22:42.671 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 34, 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "9e4e6ded": { 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 34, 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 142.3, 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:22:42.672 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:42.673 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 -- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10dc0a3460 msgr2=0x7f10dc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10dc0a3460 0x7f10dc0a5920 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0b9ca0 tx=0x7f10ec0095c0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 -- 172.21.15.78:0/1744668931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10f411b140 msgr2=0x7f10f41abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10f411b140 0x7f10f41abb80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0098d0 tx=0x7f10ec0316b0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 -- 172.21.15.78:0/1744668931 shutdown_connections 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f10dc0a3460 0x7f10dc0a5920 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10f411b140 0x7f10f41abb80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f4109830 0x7f10f41a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 --2- 172.21.15.78:0/1744668931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f4109450 0x7f10f41a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 -- 172.21.15.78:0/1744668931 >> 172.21.15.78:0/1744668931 conn(0x7f10f406e140 msgr2=0x7f10f411bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:42.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 -- 172.21.15.78:0/1744668931 shutdown_connections 2024-09-16T00:22:42.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:42.670+0000 7f10fa2b9640 1 -- 172.21.15.78:0/1744668931 wait complete. 2024-09-16T00:22:43.321 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:43.810 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9e4e6ded', quiesce took 7.0 seconds. Will release after: 15.350000000000001 2024-09-16T00:22:43.997 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:43 smithi078 ceph-mon[31157]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-16T00:22:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:43 smithi193 ceph-mon[35484]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-16T00:22:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:43 smithi148 ceph-mon[35599]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.0 MiB/s wr, 336 op/s 2024-09-16T00:22:44.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 -- 172.21.15.78:0/3841415008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988100e20 msgr2=0x7fe988101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 --2- 172.21.15.78:0/3841415008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988100e20 0x7fe988101200 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe978009900 tx=0x7fe97802a480 comp rx=0 tx=0).stop 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 -- 172.21.15.78:0/3841415008 shutdown_connections 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 --2- 172.21.15.78:0/3841415008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe988102ef0 0x7fe98810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 --2- 172.21.15.78:0/3841415008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe988102570 0x7fe9881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 --2- 172.21.15.78:0/3841415008 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988100e20 0x7fe988101200 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 -- 172.21.15.78:0/3841415008 >> 172.21.15.78:0/3841415008 conn(0x7fe9880fc7d0 msgr2=0x7fe9880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 -- 172.21.15.78:0/3841415008 shutdown_connections 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.366+0000 7fe98c938640 1 -- 172.21.15.78:0/3841415008 wait complete. 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe98c938640 1 Processor -- start 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe98c938640 1 -- start start 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe98c938640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe988100e20 0x7fe988198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe98c938640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988102570 0x7fe988198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe98c938640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe988102ef0 0x7fe98819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe98c938640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe988110fa0 con 0x7fe988102570 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe98c938640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe988110e20 con 0x7fe988102ef0 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe98c938640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe988111120 con 0x7fe988100e20 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988102570 0x7fe988198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988102570 0x7fe988198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60250/0 (socket says 172.21.15.78:60250) 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 -- 172.21.15.78:0/3158081529 learned_addr learned my addr 172.21.15.78:0/3158081529 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 -- 172.21.15.78:0/3158081529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe988100e20 msgr2=0x7fe988198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe988100e20 0x7fe988198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 -- 172.21.15.78:0/3158081529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe988102ef0 msgr2=0x7fe98819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe988102ef0 0x7fe98819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 -- 172.21.15.78:0/3158081529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98819d4c0 con 0x7fe988102570 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.367+0000 7fe985d9b640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988102570 0x7fe988198a50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00b4a0 tx=0x7fe97c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:44.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.368+0000 7fe9777fe640 1 -- 172.21.15.78:0/3158081529 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c014070 con 0x7fe988102570 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.368+0000 7fe98c938640 1 -- 172.21.15.78:0/3158081529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98819d7b0 con 0x7fe988102570 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.368+0000 7fe9777fe640 1 -- 172.21.15.78:0/3158081529 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c010af0 con 0x7fe988102570 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.368+0000 7fe9777fe640 1 -- 172.21.15.78:0/3158081529 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c017d10 con 0x7fe988102570 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.368+0000 7fe98c938640 1 -- 172.21.15.78:0/3158081529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9881a5090 con 0x7fe988102570 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.370+0000 7fe9777fe640 1 -- 172.21.15.78:0/3158081529 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe97c017070 con 0x7fe988102570 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98ed11740 1 Processor -- start 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98ed11740 1 -- start start 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff13760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff22550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff24e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98ed11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c73fe792e0 con 0x55c73ff13380 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98ed11740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c73fe20ce0 con 0x55c73ff13ca0 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98ed11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c73fe752a0 con 0x55c73ff22a90 2024-09-16T00:22:44.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe986d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff13760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe986d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff13760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60252/0 (socket says 172.21.15.78:60252) 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe986d9d640 1 -- 172.21.15.78:0/3429903180 learned_addr learned my addr 172.21.15.78:0/3429903180 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe9777fe640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe9700a3240 0x7fe9700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe98659c640 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff22550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.371+0000 7fe986d9d640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe9700a3240 0x7fe9700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe9777fe640 1 -- 172.21.15.78:0/3158081529 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe97c0cf0f0 con 0x7fe988102570 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe985d9b640 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff24e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe986d9d640 1 -- 172.21.15.78:0/3429903180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 msgr2=0x55c73ff24e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe986d9d640 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff24e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe986d9d640 1 -- 172.21.15.78:0/3429903180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 msgr2=0x55c73ff22550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe986d9d640 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff22550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe986d9d640 1 -- 172.21.15.78:0/3429903180 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73fe05440 con 0x55c73ff13380 2024-09-16T00:22:44.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe985d9b640 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff24e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe986d9d640 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff13760 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe980013890 tx=0x7fe980013d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133e959816a45b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe986d9d640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe9700a3240 0x7fe9700a5700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe98000a000 tx=0x7fe980019000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe974ff9640 1 -- 172.21.15.78:0/3429903180 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980007e60 con 0x55c73ff13380 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe974ff9640 1 -- 172.21.15.78:0/3429903180 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98000ee20 con 0x55c73ff13380 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.372+0000 7fe974ff9640 1 -- 172.21.15.78:0/3429903180 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980004660 con 0x55c73ff13380 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 -- 172.21.15.78:0/3429903180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 msgr2=0x55c73ff13760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff13760 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe980013890 tx=0x7fe980013d50 comp rx=0 tx=0).stop 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 -- 172.21.15.78:0/3429903180 shutdown_connections 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff24e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff22550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3429903180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff13760 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 -- 172.21.15.78:0/3429903180 >> 172.21.15.78:0/3429903180 conn(0x55c73ff0fa30 msgr2=0x55c73ff0fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 -- 172.21.15.78:0/3429903180 shutdown_connections 2024-09-16T00:22:44.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 -- 172.21.15.78:0/3429903180 wait complete. 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 -- start start 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.373+0000 7fe98ed11740 1 Processor -- start 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff22420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff1f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff1fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98ed11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c73ff25660 con 0x55c73ff13380 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98ed11740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c73fe34050 con 0x55c73ff13ca0 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98ed11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c73ff257e0 con 0x55c73ff22a90 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98ed11740 10 client.0 _ll_register_callbacks cb 0x55c73fe74160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:44.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff22420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff22420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60264/0 (socket says 172.21.15.78:60264) 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 -- 172.21.15.78:0/284395118 learned_addr learned my addr 172.21.15.78:0/284395118 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe985d9b640 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff1f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 -- 172.21.15.78:0/284395118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 msgr2=0x55c73ff1fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff1fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 -- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 msgr2=0x55c73ff1f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff1f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 -- 172.21.15.78:0/284395118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73fbea3c0 con 0x55c73ff13380 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe985d9b640 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff1f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98659c640 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff22420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe978005250 tx=0x7fe978005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe135de55add38a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe96dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe96dffb640 1 -- 172.21.15.78:0/284395118 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978040070 con 0x55c73ff13380 2024-09-16T00:22:44.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe98ed11740 1 -- 172.21.15.78:0/284395118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe98000ee20 con 0x55c73ff13380 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.374+0000 7fe96dffb640 1 -- 172.21.15.78:0/284395118 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe9780048c0 con 0x55c73ff13380 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe96dffb640 1 -- 172.21.15.78:0/284395118 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe97804c050 con 0x55c73ff13380 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe98ed11740 10 client.17031 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe98ed11740 1 -- 172.21.15.78:0/284395118 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c73ff31050 con 0x55c73ff13380 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe96dffb640 1 -- 172.21.15.78:0/284395118 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98000ee20 con 0x55c73ff13380 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe96dffb640 1 -- 172.21.15.78:0/284395118 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe97803a040 con 0x55c73ff13380 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe98ed11740 10 client.17031 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe98ed11740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe98ed11740 20 client.17031 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe98ed11740 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c73ff32460 0x55c73ff348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe98ed11740 4 client.17031 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe98ed11740 1 -- 172.21.15.78:0/284395118 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73fe86850 con 0x55c73ff32460 2024-09-16T00:22:44.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.375+0000 7fe985d9b640 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c73ff32460 0x55c73ff348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.376+0000 7fe985d9b640 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c73ff32460 0x55c73ff348d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1028978ff14ad1da server_cookie=71112b7b7838806c in_seq=0 out_seq=0 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.376+0000 7fe96dffb640 10 client.17031 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe96dffb640 1 -- 172.21.15.78:0/284395118 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c73fe86850 con 0x55c73ff32460 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe96dffb640 10 client.17031 handle_command_reply: tid=0 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 client.17031 shutdown 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 -- 172.21.15.78:0/284395118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 msgr2=0x55c73ff22420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff22420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe978005250 tx=0x7fe978005280 comp rx=0 tx=0).stop 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 -- 172.21.15.78:0/284395118 shutdown_connections 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c73ff32460 0x55c73ff348d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff1fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff1f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 --2- 172.21.15.78:0/284395118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff22420 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe96dffb640 0 client.17031 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:44.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 -- 172.21.15.78:0/284395118 >> 172.21.15.78:0/284395118 conn(0x55c73ff0fa30 msgr2=0x55c73ff15790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:44.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 -- 172.21.15.78:0/284395118 shutdown_connections 2024-09-16T00:22:44.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 1 -- 172.21.15.78:0/284395118 wait complete. 2024-09-16T00:22:44.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.377+0000 7fe98ed11740 20 client.17031 trim_cache size 0 max 16384 2024-09-16T00:22:44.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.394+0000 7fe98ed11740 1 Processor -- start 2024-09-16T00:22:44.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.394+0000 7fe98ed11740 1 -- start start 2024-09-16T00:22:44.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13380 0x55c73ff2e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff13ca0 0x55c73ff3a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff22a90 0x55c73ff3a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe98ed11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881ac9e0 con 0x55c73ff22a90 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe98ed11740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881ae520 con 0x55c73ff13380 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe98ed11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881ad0b0 con 0x55c73ff13ca0 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff22a90 0x55c73ff3a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff22a90 0x55c73ff3a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60278/0 (socket says 172.21.15.78:60278) 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 -- 172.21.15.78:0/4175170586 learned_addr learned my addr 172.21.15.78:0/4175170586 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe98659c640 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13380 0x55c73ff2e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe985d9b640 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff13ca0 0x55c73ff3a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 -- 172.21.15.78:0/4175170586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff13ca0 msgr2=0x55c73ff3a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff13ca0 0x55c73ff3a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 -- 172.21.15.78:0/4175170586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13380 msgr2=0x55c73ff2e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13380 0x55c73ff2e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 -- 172.21.15.78:0/4175170586 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73ff1e680 con 0x55c73ff22a90 2024-09-16T00:22:44.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe986d9d640 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff22a90 0x55c73ff3a6e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe980013730 tx=0x7fe980005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81274aac70c567f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:44.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.395+0000 7fe96ffff640 1 -- 172.21.15.78:0/4175170586 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980013250 con 0x55c73ff22a90 2024-09-16T00:22:44.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe96ffff640 1 -- 172.21.15.78:0/4175170586 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe980003e50 con 0x55c73ff22a90 2024-09-16T00:22:44.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe96ffff640 1 -- 172.21.15.78:0/4175170586 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980004030 con 0x55c73ff22a90 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 -- 172.21.15.78:0/4175170586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff22a90 msgr2=0x55c73ff3a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff22a90 0x55c73ff3a6e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe980013730 tx=0x7fe980005e90 comp rx=0 tx=0).stop 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 -- 172.21.15.78:0/4175170586 shutdown_connections 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff22a90 0x55c73ff3a6e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff13ca0 0x55c73ff3a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 --2- 172.21.15.78:0/4175170586 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13380 0x55c73ff2e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 -- 172.21.15.78:0/4175170586 >> 172.21.15.78:0/4175170586 conn(0x55c73ff0fa30 msgr2=0x55c73ff1ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 -- 172.21.15.78:0/4175170586 shutdown_connections 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.396+0000 7fe98ed11740 1 -- 172.21.15.78:0/4175170586 wait complete. 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 1 -- start start 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 1 Processor -- start 2024-09-16T00:22:44.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff2aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff2afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff27a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881ade50 con 0x55c73ff13380 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881ad780 con 0x55c73ff13ca0 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9881aa9c0 con 0x55c73ff22a90 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98ed11740 10 client.0 _ll_register_callbacks cb 0x7fe978009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff2aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff2aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60284/0 (socket says 172.21.15.78:60284) 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98659c640 1 -- 172.21.15.78:0/3080695950 learned_addr learned my addr 172.21.15.78:0/3080695950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe986d9d640 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff27a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe985d9b640 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff2afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98659c640 1 -- 172.21.15.78:0/3080695950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 msgr2=0x55c73ff27a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98659c640 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff27a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98659c640 1 -- 172.21.15.78:0/3080695950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 msgr2=0x55c73ff2afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98659c640 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff2afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe98659c640 1 -- 172.21.15.78:0/3080695950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73ff2b5f0 con 0x55c73ff13380 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.397+0000 7fe986d9d640 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff27a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98659c640 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff2aa70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe978001eb0 tx=0x7fe978004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9420830cf477ec3e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe96dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe96dffb640 1 -- 172.21.15.78:0/3080695950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978054020 con 0x55c73ff13380 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe96dffb640 1 -- 172.21.15.78:0/3080695950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe978003a00 con 0x55c73ff13380 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 1 -- 172.21.15.78:0/3080695950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe980003e50 con 0x55c73ff13380 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe96dffb640 1 -- 172.21.15.78:0/3080695950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe97805f050 con 0x55c73ff13380 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 10 client.17037 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 1 -- 172.21.15.78:0/3080695950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c73ff9d700 con 0x55c73ff13380 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe96dffb640 1 -- 172.21.15.78:0/3080695950 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe980003e50 con 0x55c73ff13380 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe96dffb640 1 -- 172.21.15.78:0/3080695950 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe9780046c0 con 0x55c73ff13380 2024-09-16T00:22:44.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 10 client.17037 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 20 client.17037 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c73ff9ded0 0x55c73ffa02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 4 client.17037 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.398+0000 7fe98ed11740 1 -- 172.21.15.78:0/3080695950 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c73fe80d60 con 0x55c73ff9ded0 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe985d9b640 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c73ff9ded0 0x55c73ffa02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe985d9b640 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c73ff9ded0 0x55c73ffa02f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cbeb0ff43b4029f server_cookie=8d47a5a3807ae839 in_seq=0 out_seq=0 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe96dffb640 10 client.17037 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe96dffb640 1 -- 172.21.15.78:0/3080695950 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c73fe80d60 con 0x55c73ff9ded0 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe96dffb640 10 client.17037 handle_command_reply: tid=0 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 client.17037 shutdown 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 -- 172.21.15.78:0/3080695950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 msgr2=0x55c73ff2aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff2aa70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe978001eb0 tx=0x7fe978004490 comp rx=0 tx=0).stop 2024-09-16T00:22:44.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 -- 172.21.15.78:0/3080695950 shutdown_connections 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c73ff9ded0 0x55c73ffa02f0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe96dffb640 0 client.17037 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c73ff22a90 0x55c73ff27a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c73ff13ca0 0x55c73ff2afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 --2- 172.21.15.78:0/3080695950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c73ff13380 0x55c73ff2aa70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.399+0000 7fe98ed11740 1 -- 172.21.15.78:0/3080695950 >> 172.21.15.78:0/3080695950 conn(0x55c73ff0fa30 msgr2=0x55c73ff0e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98ed11740 1 -- 172.21.15.78:0/3080695950 shutdown_connections 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98ed11740 1 -- 172.21.15.78:0/3080695950 wait complete. 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98ed11740 20 client.17037 trim_cache size 0 max 16384 2024-09-16T00:22:44.406 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:22:44.407 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:22:44.407 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:22:44.407 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:22:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 -- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe9700a3240 msgr2=0x7fe9700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe9700a3240 0x7fe9700a5700 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe98000a000 tx=0x7fe980019000 comp rx=0 tx=0).stop 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 -- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988102570 msgr2=0x7fe988198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988102570 0x7fe988198a50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe97c00b4a0 tx=0x7fe97c00b960 comp rx=0 tx=0).stop 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 -- 172.21.15.78:0/3158081529 shutdown_connections 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe9700a3240 0x7fe9700a5700 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe988102ef0 0x7fe98819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe988102570 0x7fe988198a50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 --2- 172.21.15.78:0/3158081529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe988100e20 0x7fe988198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 -- 172.21.15.78:0/3158081529 >> 172.21.15.78:0/3158081529 conn(0x7fe9880fc7d0 msgr2=0x7fe98806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 -- 172.21.15.78:0/3158081529 shutdown_connections 2024-09-16T00:22:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:44.400+0000 7fe98c938640 1 -- 172.21.15.78:0/3158081529 wait complete. 2024-09-16T00:22:45.289 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:22:45.290 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:22:45.540 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:45.861 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 525.0 s, 181.5 tps, lat 210.305 ms stddev 298.508 2024-09-16T00:22:45.962 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:45 smithi078 ceph-mon[31157]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 -- 172.21.15.78:0/2118116043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0102790 msgr2=0x7f3ae01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/2118116043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0102790 0x7f3ae01067e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009900 tx=0x7f3ad002a410 comp rx=0 tx=0).stop 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 -- 172.21.15.78:0/2118116043 shutdown_connections 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/2118116043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ae0106f10 0x7f3ae010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/2118116043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0102790 0x7f3ae01067e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/2118116043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ae0101de0 0x7f3ae01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 -- 172.21.15.78:0/2118116043 >> 172.21.15.78:0/2118116043 conn(0x7f3ae00fd8c0 msgr2=0x7f3ae00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 -- 172.21.15.78:0/2118116043 shutdown_connections 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 -- 172.21.15.78:0/2118116043 wait complete. 2024-09-16T00:22:46.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.041+0000 7f3ae4f47640 1 Processor -- start 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ae4f47640 1 -- start start 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ae4f47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0101de0 0x7f3ae0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ae4f47640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ae0102790 0x7f3ae01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ae4f47640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ae0106f10 0x7f3ae019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ae4f47640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0111f90 con 0x7f3ae0101de0 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ae4f47640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0111e10 con 0x7f3ae0102790 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ae4f47640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0112110 con 0x7f3ae0106f10 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0101de0 0x7f3ae0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0101de0 0x7f3ae0198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60298/0 (socket says 172.21.15.78:60298) 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 -- 172.21.15.78:0/3974279026 learned_addr learned my addr 172.21.15.78:0/3974279026 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3aded9d640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ae0106f10 0x7f3ae019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3addd9b640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ae0102790 0x7f3ae01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 -- 172.21.15.78:0/3974279026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ae0106f10 msgr2=0x7f3ae019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ae0106f10 0x7f3ae019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 -- 172.21.15.78:0/3974279026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ae0102790 msgr2=0x7f3ae01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ae0102790 0x7f3ae01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 -- 172.21.15.78:0/3974279026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae019d440 con 0x7f3ae0101de0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.042+0000 7f3ade59c640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0101de0 0x7f3ae0198490 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400b4a0 tx=0x7f3ad400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.043+0000 7f3acf7fe640 1 -- 172.21.15.78:0/3974279026 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4014070 con 0x7f3ae0101de0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.043+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae019d730 con 0x7f3ae0101de0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.043+0000 7f3acf7fe640 1 -- 172.21.15.78:0/3974279026 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ad4010af0 con 0x7f3ae0101de0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.043+0000 7f3acf7fe640 1 -- 172.21.15.78:0/3974279026 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4017d10 con 0x7f3ae0101de0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.043+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae01a5010 con 0x7f3ae0101de0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.044+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae01021c0 con 0x7f3ae0101de0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.045+0000 7f3acf7fe640 1 -- 172.21.15.78:0/3974279026 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3ad4017070 con 0x7f3ae0101de0 2024-09-16T00:22:46.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.045+0000 7f3acf7fe640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac80a3240 0x7f3ac80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:46.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.045+0000 7f3addd9b640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac80a3240 0x7f3ac80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:46.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.045+0000 7f3acf7fe640 1 -- 172.21.15.78:0/3974279026 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3ad40cf0f0 con 0x7f3ae0101de0 2024-09-16T00:22:46.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.046+0000 7f3addd9b640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac80a3240 0x7f3ac80a5700 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00fdce0 tx=0x7f3ad0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:46.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.050+0000 7f3acf7fe640 1 -- 172.21.15.78:0/3974279026 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ad409bab0 con 0x7f3ae0101de0 2024-09-16T00:22:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:45 smithi193 ceph-mon[35484]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-16T00:22:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:45 smithi148 ceph-mon[35599]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-16T00:22:46.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.251+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ae01997f0 con 0x7f3ae0101de0 2024-09-16T00:22:46.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.252+0000 7f3acf7fe640 1 -- 172.21.15.78:0/3974279026 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f3ad409b3b0 con 0x7f3ae0101de0 2024-09-16T00:22:46.253 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:22:46.253 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:22:46.253 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:22:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.254+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac80a3240 msgr2=0x7f3ac80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:46.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.254+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac80a3240 0x7f3ac80a5700 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00fdce0 tx=0x7f3ad0005a20 comp rx=0 tx=0).stop 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.254+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0101de0 msgr2=0x7f3ae0198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.254+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0101de0 0x7f3ae0198490 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400b4a0 tx=0x7f3ad400b960 comp rx=0 tx=0).stop 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.255+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 shutdown_connections 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.255+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ac80a3240 0x7f3ac80a5700 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.255+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ae0106f10 0x7f3ae019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.255+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3ae0102790 0x7f3ae01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.255+0000 7f3ae4f47640 1 --2- 172.21.15.78:0/3974279026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ae0101de0 0x7f3ae0198490 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.255+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 >> 172.21.15.78:0/3974279026 conn(0x7f3ae00fd8c0 msgr2=0x7f3ae00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.255+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 shutdown_connections 2024-09-16T00:22:46.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:46.255+0000 7f3ae4f47640 1 -- 172.21.15.78:0/3974279026 wait complete. 2024-09-16T00:22:46.876 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:46 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3974279026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:46.904 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:22:47.149 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:47.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:46 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3974279026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:46 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3974279026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:22:47.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/1947499105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102ef0 msgr2=0x7fe6a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/1947499105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102ef0 0x7fe6a810da40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009ca0 tx=0x7fe6a002a950 comp rx=0 tx=0).stop 2024-09-16T00:22:47.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/1947499105 shutdown_connections 2024-09-16T00:22:47.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/1947499105 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102ef0 0x7fe6a810da40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/1947499105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a8102570 0x7fe6a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/1947499105 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a8100e20 0x7fe6a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/1947499105 >> 172.21.15.78:0/1947499105 conn(0x7fe6a80fc7d0 msgr2=0x7fe6a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/1947499105 shutdown_connections 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.644+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/1947499105 wait complete. 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6ae4ca640 1 Processor -- start 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6ae4ca640 1 -- start start 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6ae4ca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a8100e20 0x7fe6a8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6ae4ca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102570 0x7fe6a8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6ae4ca640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a8102ef0 0x7fe6a819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6ae4ca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8110f80 con 0x7fe6a8102570 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6ae4ca640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8110e00 con 0x7fe6a8100e20 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6ae4ca640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8111100 con 0x7fe6a8102ef0 2024-09-16T00:22:47.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102570 0x7fe6a8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102570 0x7fe6a8198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43236/0 (socket says 172.21.15.78:43236) 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6a77fe640 1 -- 172.21.15.78:0/947572613 learned_addr learned my addr 172.21.15.78:0/947572613 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6aca67640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a8102ef0 0x7fe6a819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.645+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a8100e20 0x7fe6a8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a77fe640 1 -- 172.21.15.78:0/947572613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a8102ef0 msgr2=0x7fe6a819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a8102ef0 0x7fe6a819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a77fe640 1 -- 172.21.15.78:0/947572613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a8100e20 msgr2=0x7fe6a8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a8100e20 0x7fe6a8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a77fe640 1 -- 172.21.15.78:0/947572613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a819d4e0 con 0x7fe6a8102570 2024-09-16T00:22:47.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a8100e20 0x7fe6a8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102570 0x7fe6a8198a70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d770 tx=0x7fe69000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a57fa640 1 -- 172.21.15.78:0/947572613 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690017070 con 0x7fe6a8102570 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/947572613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a819d7d0 con 0x7fe6a8102570 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a57fa640 1 -- 172.21.15.78:0/947572613 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe690005500 con 0x7fe6a8102570 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6a57fa640 1 -- 172.21.15.78:0/947572613 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690004e00 con 0x7fe6a8102570 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.646+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/947572613 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a81a50b0 con 0x7fe6a8102570 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.648+0000 7fe6a57fa640 1 -- 172.21.15.78:0/947572613 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe690004fa0 con 0x7fe6a8102570 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6b08a3740 1 Processor -- start 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6b08a3740 1 -- start start 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb7b40 0x558e71cb6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558e71c1c2f0 con 0x558e71cb65d0 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558e71bc3cf0 con 0x558e71cc1370 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558e71c182b0 con 0x558e71cb7b40 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a57fa640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6940a3240 0x7fe6940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43238/0 (socket says 172.21.15.78:43238) 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a7fff640 1 -- 172.21.15.78:0/2362651865 learned_addr learned my addr 172.21.15.78:0/2362651865 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6aca67640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6940a3240 0x7fe6940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a57fa640 1 -- 172.21.15.78:0/947572613 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe6900cf6f0 con 0x7fe6a8102570 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6aca67640 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb7b40 0x558e71cb6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a7fff640 1 -- 172.21.15.78:0/2362651865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb7b40 msgr2=0x558e71cb6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb7b40 0x558e71cb6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a7fff640 1 -- 172.21.15.78:0/2362651865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 msgr2=0x558e71cc3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:47.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cc3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a7fff640 1 -- 172.21.15.78:0/2362651865 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e71ba83a0 con 0x558e71cb65d0 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.649+0000 7fe6aca67640 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb7b40 0x558e71cb6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.650+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb6a10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe69800cca0 tx=0x7fe69800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8542ad7ba5c429 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.650+0000 7fe69e7fc640 1 -- 172.21.15.78:0/2362651865 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698005220 con 0x558e71cb65d0 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.650+0000 7fe6aca67640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6940a3240 0x7fe6940a5700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0030040 tx=0x7fe6a003e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.650+0000 7fe69e7fc640 1 -- 172.21.15.78:0/2362651865 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe698004c80 con 0x558e71cb65d0 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.650+0000 7fe69e7fc640 1 -- 172.21.15.78:0/2362651865 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698010800 con 0x558e71cb65d0 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.650+0000 7fe6b08a3740 1 -- 172.21.15.78:0/2362651865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 msgr2=0x558e71cb6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.650+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb6a10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe69800cca0 tx=0x7fe69800e720 comp rx=0 tx=0).stop 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 -- 172.21.15.78:0/2362651865 shutdown_connections 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cc3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb6a10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/2362651865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb7b40 0x558e71cb6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 -- 172.21.15.78:0/2362651865 >> 172.21.15.78:0/2362651865 conn(0x558e71c7f310 msgr2=0x558e71cb2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 -- 172.21.15.78:0/2362651865 shutdown_connections 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 -- 172.21.15.78:0/2362651865 wait complete. 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 -- start start 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 Processor -- start 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71cbdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cc1370 0x558e71cbe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558e71cc4070 con 0x558e71cc1370 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558e71bd7060 con 0x558e71cb7b40 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558e71cc41f0 con 0x558e71cb65d0 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.651+0000 7fe6b08a3740 10 client.0 _ll_register_callbacks cb 0x558e7192a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cc1370 0x558e71cbe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cc1370 0x558e71cbe4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43252/0 (socket says 172.21.15.78:43252) 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 -- 172.21.15.78:0/808232548 learned_addr learned my addr 172.21.15.78:0/808232548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71cbdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 -- 172.21.15.78:0/808232548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 msgr2=0x558e71cc0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 -- 172.21.15.78:0/808232548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 msgr2=0x558e71cbdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71cbdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 -- 172.21.15.78:0/808232548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e7198d3c0 con 0x558e71cc1370 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71cbdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6aca67640 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cc1370 0x558e71cbe4a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009ca0 tx=0x7fe6a002fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539db372de6b149c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6877fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6877fe640 1 -- 172.21.15.78:0/808232548 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0034040 con 0x558e71cc1370 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6877fe640 1 -- 172.21.15.78:0/808232548 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe6a0004250 con 0x558e71cc1370 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6b08a3740 1 -- 172.21.15.78:0/808232548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe698004c80 con 0x558e71cc1370 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6877fe640 1 -- 172.21.15.78:0/808232548 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe6a0059050 con 0x558e71cc1370 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6b08a3740 10 client.17055 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.652+0000 7fe6b08a3740 1 -- 172.21.15.78:0/808232548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558e71ccfa20 con 0x558e71cc1370 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6877fe640 1 -- 172.21.15.78:0/808232548 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe698004c80 con 0x558e71cc1370 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6877fe640 1 -- 172.21.15.78:0/808232548 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe6a003bca0 con 0x558e71cc1370 2024-09-16T00:22:47.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6b08a3740 10 client.17055 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6b08a3740 10 client.17055 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6b08a3740 20 client.17055 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558e71cd0ee0 0x558e71cd3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6b08a3740 4 client.17055 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6b08a3740 1 -- 172.21.15.78:0/808232548 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e71c29860 con 0x558e71cd0ee0 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.653+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558e71cd0ee0 0x558e71cd3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.654+0000 7fe6877fe640 10 client.17055 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.654+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558e71cd0ee0 0x558e71cd3330 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a58d464925e35d2 server_cookie=84dd4fb1a7c331a3 in_seq=0 out_seq=0 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.654+0000 7fe6877fe640 1 -- 172.21.15.78:0/808232548 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e71c29860 con 0x558e71cd0ee0 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.654+0000 7fe6877fe640 10 client.17055 handle_command_reply: tid=0 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.654+0000 7fe6b08a3740 1 client.17055 shutdown 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.654+0000 7fe6b08a3740 1 -- 172.21.15.78:0/808232548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cc1370 msgr2=0x558e71cbe4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.654+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cc1370 0x558e71cbe4a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009ca0 tx=0x7fe6a002fe00 comp rx=0 tx=0).stop 2024-09-16T00:22:47.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 1 -- 172.21.15.78:0/808232548 shutdown_connections 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cc1370 0x558e71cbe4a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71cbdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558e71cd0ee0 0x558e71cd3330 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/808232548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6877fe640 0 client.17055 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 1 -- 172.21.15.78:0/808232548 >> 172.21.15.78:0/808232548 conn(0x558e71c7f310 msgr2=0x558e71cb8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 1 -- 172.21.15.78:0/808232548 shutdown_connections 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 1 -- 172.21.15.78:0/808232548 wait complete. 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.655+0000 7fe6b08a3740 20 client.17055 trim_cache size 0 max 16384 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.667+0000 7fe6b08a3740 1 Processor -- start 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.667+0000 7fe6b08a3740 1 -- start start 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.667+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.667+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71ccd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.667+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cc1370 0x558e71cc9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.667+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81aeb50 con 0x558e71cb65d0 2024-09-16T00:22:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81b0690 con 0x558e71cb7b40 2024-09-16T00:22:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81af220 con 0x558e71cc1370 2024-09-16T00:22:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43262/0 (socket says 172.21.15.78:43262) 2024-09-16T00:22:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 -- 172.21.15.78:0/3144574627 learned_addr learned my addr 172.21.15.78:0/3144574627 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/3144574627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71ccd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 -- 172.21.15.78:0/3144574627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cc1370 msgr2=0x558e71cc9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/3144574627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cc1370 0x558e71cc9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 -- 172.21.15.78:0/3144574627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 msgr2=0x558e71ccd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/3144574627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71ccd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 -- 172.21.15.78:0/3144574627 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e71cbc4b0 con 0x558e71cb65d0 2024-09-16T00:22:47.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/3144574627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb2e10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe69800cbc0 tx=0x7fe69800c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe747e192a375704 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe69dffb640 1 -- 172.21.15.78:0/3144574627 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6980050c0 con 0x558e71cb65d0 2024-09-16T00:22:47.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe69dffb640 1 -- 172.21.15.78:0/3144574627 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe698005260 con 0x558e71cb65d0 2024-09-16T00:22:47.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.668+0000 7fe69dffb640 1 -- 172.21.15.78:0/3144574627 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698004de0 con 0x558e71cb65d0 2024-09-16T00:22:47.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 -- 172.21.15.78:0/3144574627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 msgr2=0x558e71cb2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/3144574627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb2e10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe69800cbc0 tx=0x7fe69800c590 comp rx=0 tx=0).stop 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 -- 172.21.15.78:0/3144574627 shutdown_connections 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/3144574627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cc1370 0x558e71cc9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/3144574627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cb7b40 0x558e71ccd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/3144574627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb65d0 0x558e71cb2e10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 -- 172.21.15.78:0/3144574627 >> 172.21.15.78:0/3144574627 conn(0x558e71c7f310 msgr2=0x558e71cb2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 -- 172.21.15.78:0/3144574627 shutdown_connections 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 -- 172.21.15.78:0/3144574627 wait complete. 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 -- start start 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.669+0000 7fe6b08a3740 1 Processor -- start 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb7b40 0x558e71cc29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6b08a3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cd7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81affc0 con 0x558e71cb7b40 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81af8f0 con 0x558e71cc1370 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6b08a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a81acb30 con 0x558e71cb65d0 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6b08a3740 10 client.0 _ll_register_callbacks cb 0x7fe6a0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb7b40 0x558e71cc29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb7b40 0x558e71cc29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43268/0 (socket says 172.21.15.78:43268) 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 -- 172.21.15.78:0/1290754879 learned_addr learned my addr 172.21.15.78:0/1290754879 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6aca67640 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cd7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 -- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 msgr2=0x558e71cc2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 -- 172.21.15.78:0/1290754879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 msgr2=0x558e71cd7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cd7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 -- 172.21.15.78:0/1290754879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e71cd8130 con 0x558e71cb7b40 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6a77fe640 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb7b40 0x558e71cc29d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d740 tx=0x7fe69009c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2572cd6d124ed238 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6877fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6877fe640 1 -- 172.21.15.78:0/1290754879 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690020040 con 0x558e71cb7b40 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6877fe640 1 -- 172.21.15.78:0/1290754879 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe6900ccf40 con 0x558e71cb7b40 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.670+0000 7fe6b08a3740 1 -- 172.21.15.78:0/1290754879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe698005260 con 0x558e71cb7b40 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6877fe640 1 -- 172.21.15.78:0/1290754879 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe6900a45c0 con 0x558e71cb7b40 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6b08a3740 10 client.17061 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:47.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6b08a3740 1 -- 172.21.15.78:0/1290754879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558e71d40300 con 0x558e71cb7b40 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6877fe640 1 -- 172.21.15.78:0/1290754879 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe698005260 con 0x558e71cb7b40 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6877fe640 1 -- 172.21.15.78:0/1290754879 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe6900ce040 con 0x558e71cb7b40 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6b08a3740 10 client.17061 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6b08a3740 10 client.17061 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6b08a3740 20 client.17061 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558e71d40a20 0x558e71d42e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6b08a3740 4 client.17061 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6b08a3740 1 -- 172.21.15.78:0/1290754879 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e71cbe830 con 0x558e71d40a20 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.671+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558e71d40a20 0x558e71d42e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6877fe640 10 client.17061 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:22:47.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6a7fff640 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558e71d40a20 0x558e71d42e10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fde93d003fa6ba3b server_cookie=8cb0a75f10bb2c8 in_seq=0 out_seq=0 2024-09-16T00:22:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6877fe640 1 -- 172.21.15.78:0/1290754879 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e71cbe830 con 0x558e71d40a20 2024-09-16T00:22:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6877fe640 10 client.17061 handle_command_reply: tid=0 2024-09-16T00:22:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 client.17061 shutdown 2024-09-16T00:22:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 -- 172.21.15.78:0/1290754879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb7b40 msgr2=0x558e71cc29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb7b40 0x558e71cc29d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d740 tx=0x7fe69009c050 comp rx=0 tx=0).stop 2024-09-16T00:22:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 -- 172.21.15.78:0/1290754879 shutdown_connections 2024-09-16T00:22:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e71cc1370 0x558e71cd7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558e71cb7b40 0x558e71cc29d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558e71d40a20 0x558e71d42e10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 --2- 172.21.15.78:0/1290754879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e71cb65d0 0x558e71cc2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 -- 172.21.15.78:0/1290754879 >> 172.21.15.78:0/1290754879 conn(0x558e71c7f310 msgr2=0x558e71cbf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6877fe640 0 client.17061 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:22:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 -- 172.21.15.78:0/1290754879 shutdown_connections 2024-09-16T00:22:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 1 -- 172.21.15.78:0/1290754879 wait complete. 2024-09-16T00:22:47.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.672+0000 7fe6b08a3740 20 client.17061 trim_cache size 0 max 16384 2024-09-16T00:22:47.694 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:22:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6940a3240 msgr2=0x7fe6940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6940a3240 0x7fe6940a5700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0030040 tx=0x7fe6a003e040 comp rx=0 tx=0).stop 2024-09-16T00:22:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102570 msgr2=0x7fe6a8198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102570 0x7fe6a8198a70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d770 tx=0x7fe69000dc30 comp rx=0 tx=0).stop 2024-09-16T00:22:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/947572613 shutdown_connections 2024-09-16T00:22:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6940a3240 0x7fe6940a5700 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a8102ef0 0x7fe6a819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a8102570 0x7fe6a8198a70 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 --2- 172.21.15.78:0/947572613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a8100e20 0x7fe6a8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:47.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/947572613 >> 172.21.15.78:0/947572613 conn(0x7fe6a80fc7d0 msgr2=0x7fe6a806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:47.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/947572613 shutdown_connections 2024-09-16T00:22:47.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:47.673+0000 7fe6ae4ca640 1 -- 172.21.15.78:0/947572613 wait complete. 2024-09-16T00:22:48.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:47 smithi078 ceph-mon[31157]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-09-16T00:22:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:47 smithi193 ceph-mon[35484]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-09-16T00:22:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:47 smithi148 ceph-mon[35599]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-09-16T00:22:48.583 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:22:48.832 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 -- 172.21.15.78:0/1805898029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102ef0 msgr2=0x7f5fd810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1805898029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102ef0 0x7f5fd810da40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009ca0 tx=0x7f5fd002a950 comp rx=0 tx=0).stop 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 -- 172.21.15.78:0/1805898029 shutdown_connections 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1805898029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102ef0 0x7f5fd810da40 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1805898029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8102570 0x7f5fd81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1805898029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fd8100e20 0x7f5fd8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 -- 172.21.15.78:0/1805898029 >> 172.21.15.78:0/1805898029 conn(0x7f5fd80fc7d0 msgr2=0x7f5fd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 -- 172.21.15.78:0/1805898029 shutdown_connections 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.365+0000 7f5fdcced640 1 -- 172.21.15.78:0/1805898029 wait complete. 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fdcced640 1 Processor -- start 2024-09-16T00:22:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fdcced640 1 -- start start 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fdcced640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fd8100e20 0x7f5fd80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fdcced640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102570 0x7f5fd80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fdcced640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8102ef0 0x7f5fd80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fdcced640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8110fa0 con 0x7f5fd8102570 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fdcced640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8110e20 con 0x7f5fd8102ef0 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fdcced640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8111120 con 0x7f5fd8100e20 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fcffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102570 0x7f5fd80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fcffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102570 0x7f5fd80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43294/0 (socket says 172.21.15.78:43294) 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fcffff640 1 -- 172.21.15.78:0/1117311102 learned_addr learned my addr 172.21.15.78:0/1117311102 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fcffff640 1 -- 172.21.15.78:0/1117311102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fd8100e20 msgr2=0x7f5fd80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fd8100e20 0x7f5fd80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fcffff640 1 -- 172.21.15.78:0/1117311102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8102ef0 msgr2=0x7f5fd80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8102ef0 0x7f5fd80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.366+0000 7f5fcffff640 1 -- 172.21.15.78:0/1117311102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd81a6990 con 0x7f5fd8102570 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.367+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102570 0x7f5fd80fd1c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c920 tx=0x7f5fc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.367+0000 7f5fcf7fe640 1 -- 172.21.15.78:0/1117311102 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0007c70 con 0x7f5fd8102570 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.367+0000 7f5fcf7fe640 1 -- 172.21.15.78:0/1117311102 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc0007e10 con 0x7f5fd8102570 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.367+0000 7f5fdcced640 1 -- 172.21.15.78:0/1117311102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd81a6c80 con 0x7f5fd8102570 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.367+0000 7f5fcf7fe640 1 -- 172.21.15.78:0/1117311102 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0018890 con 0x7f5fd8102570 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.367+0000 7f5fdcced640 1 -- 172.21.15.78:0/1117311102 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd81a71c0 con 0x7f5fd8102570 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.369+0000 7f5fcf7fe640 1 -- 172.21.15.78:0/1117311102 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5fc0018a70 con 0x7f5fd8102570 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.369+0000 7f5fdf0c6740 1 Processor -- start 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.369+0000 7f5fdf0c6740 1 -- start start 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a46b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 0x55651a46ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a478760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55651a3d12f0 con 0x55651a46cb40 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55651a378cf0 con 0x55651a476370 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55651a3cd2b0 con 0x55651a46b5d0 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fcf7fe640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5fc40a32c0 0x7f5fc40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a46b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a46b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43296/0 (socket says 172.21.15.78:43296) 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 -- 172.21.15.78:0/3008266733 learned_addr learned my addr 172.21.15.78:0/3008266733 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5fc40a32c0 0x7f5fc40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd659c640 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 0x55651a46ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fcf7fe640 1 -- 172.21.15.78:0/1117311102 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5fc00cfe40 con 0x7f5fd8102570 2024-09-16T00:22:49.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fcffff640 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a478760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 -- 172.21.15.78:0/3008266733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 msgr2=0x55651a46ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 0x55651a46ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 -- 172.21.15.78:0/3008266733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 msgr2=0x55651a478760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a478760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 -- 172.21.15.78:0/3008266733 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651a35d3a0 con 0x55651a46cb40 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a46b090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0006010 tx=0x7f5fd002f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8676def9b2b1641 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.370+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5fc40a32c0 0x7f5fc40a5780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5fd002ae70 tx=0x7f5fd0034000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fccff9640 1 -- 172.21.15.78:0/3008266733 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0034070 con 0x55651a46cb40 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fccff9640 1 -- 172.21.15.78:0/3008266733 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd002fb60 con 0x55651a46cb40 2024-09-16T00:22:49.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fccff9640 1 -- 172.21.15.78:0/3008266733 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00486e0 con 0x55651a46cb40 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/3008266733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 msgr2=0x55651a46b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a46b090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0006010 tx=0x7f5fd002f5e0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/3008266733 shutdown_connections 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a478760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 0x55651a46ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/3008266733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a46b090 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/3008266733 >> 172.21.15.78:0/3008266733 conn(0x55651a434310 msgr2=0x55651a467c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/3008266733 shutdown_connections 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.371+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/3008266733 wait complete. 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 1 -- start start 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 1 Processor -- start 2024-09-16T00:22:49.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a475d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a46cb40 0x55651a473020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a476370 0x55651a473610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55651a479010 con 0x55651a46b5d0 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55651a38c060 con 0x55651a46cb40 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55651a479190 con 0x55651a476370 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fdf0c6740 10 client.0 _ll_register_callbacks cb 0x55651a0df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a475d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a475d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43312/0 (socket says 172.21.15.78:43312) 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fd659c640 1 -- 172.21.15.78:0/1509649142 learned_addr learned my addr 172.21.15.78:0/1509649142 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a476370 0x55651a473610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a46cb40 0x55651a473020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fd659c640 1 -- 172.21.15.78:0/1509649142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a476370 msgr2=0x55651a473610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fd659c640 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a476370 0x55651a473610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fd659c640 1 -- 172.21.15.78:0/1509649142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a46cb40 msgr2=0x55651a473020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.372+0000 7f5fd659c640 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a46cb40 0x55651a473020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fd659c640 1 -- 172.21.15.78:0/1509649142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651a1423c0 con 0x55651a46b5d0 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a46cb40 0x55651a473020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fd659c640 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a475d70 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800e240 tx=0x7f5fc800e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8069a998b0565ed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fbdffb640 1 -- 172.21.15.78:0/1509649142 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8017070 con 0x55651a46b5d0 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fbdffb640 1 -- 172.21.15.78:0/1509649142 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5fc8013680 con 0x55651a46b5d0 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1509649142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55651a3de860 con 0x55651a46b5d0 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fbdffb640 1 -- 172.21.15.78:0/1509649142 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fc8023050 con 0x55651a46b5d0 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 10 client.17073 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:49.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1509649142 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55651a484b10 con 0x55651a46b5d0 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fbdffb640 1 -- 172.21.15.78:0/1509649142 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55651a3de860 con 0x55651a46b5d0 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fbdffb640 1 -- 172.21.15.78:0/1509649142 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5fc80129a0 con 0x55651a46b5d0 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 10 client.17073 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 10 client.17073 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 20 client.17073 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55651a485f20 0x55651a488380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 4 client.17073 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.373+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1509649142 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55651a3d8d70 con 0x55651a485f20 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.374+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55651a485f20 0x55651a488380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.374+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55651a485f20 0x55651a488380 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90fe33067d9973a6 server_cookie=98a870fd5284c9e9 in_seq=0 out_seq=0 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.374+0000 7f5fbdffb640 10 client.17073 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fbdffb640 1 -- 172.21.15.78:0/1509649142 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55651a3d8d70 con 0x55651a485f20 2024-09-16T00:22:49.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fbdffb640 10 client.17073 handle_command_reply: tid=0 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 client.17073 shutdown 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1509649142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 msgr2=0x55651a475d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a475d70 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800e240 tx=0x7f5fc800e700 comp rx=0 tx=0).stop 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1509649142 shutdown_connections 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a476370 0x55651a473610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55651a485f20 0x55651a488380 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a46cb40 0x55651a473020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fbdffb640 0 client.17073 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1509649142 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a475d70 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1509649142 >> 172.21.15.78:0/1509649142 conn(0x55651a434310 msgr2=0x55651a46d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.375+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1509649142 shutdown_connections 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.376+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1509649142 wait complete. 2024-09-16T00:22:49.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.376+0000 7f5fdf0c6740 20 client.17073 trim_cache size 0 max 16384 2024-09-16T00:22:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.388+0000 7f5fdf0c6740 1 Processor -- start 2024-09-16T00:22:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.388+0000 7f5fdf0c6740 1 -- start start 2024-09-16T00:22:49.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 0x55651a4728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a466c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a467150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81b0d90 con 0x55651a46cb40 2024-09-16T00:22:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81b28d0 con 0x55651a476370 2024-09-16T00:22:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81b1460 con 0x55651a46b5d0 2024-09-16T00:22:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a466c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a466c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43326/0 (socket says 172.21.15.78:43326) 2024-09-16T00:22:49.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 -- 172.21.15.78:0/1673871360 learned_addr learned my addr 172.21.15.78:0/1673871360 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a467150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fd659c640 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 0x55651a4728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 -- 172.21.15.78:0/1673871360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 msgr2=0x55651a4728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 0x55651a4728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 -- 172.21.15.78:0/1673871360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 msgr2=0x55651a467150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a467150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 -- 172.21.15.78:0/1673871360 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651a468960 con 0x55651a46cb40 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a467150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fcffff640 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a466c10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5fc009c090 tx=0x7f5fc00cd4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72623a4cd43bfed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fbffff640 1 -- 172.21.15.78:0/1673871360 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00a04b0 con 0x55651a46cb40 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fbffff640 1 -- 172.21.15.78:0/1673871360 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc00a4d70 con 0x55651a46cb40 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.389+0000 7f5fbffff640 1 -- 172.21.15.78:0/1673871360 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00bc0b0 con 0x55651a46cb40 2024-09-16T00:22:49.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1673871360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 msgr2=0x55651a466c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a466c10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5fc009c090 tx=0x7f5fc00cd4b0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1673871360 shutdown_connections 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a467150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46cb40 0x55651a466c10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/1673871360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46b5d0 0x55651a4728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1673871360 >> 172.21.15.78:0/1673871360 conn(0x55651a434310 msgr2=0x55651a466500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1673871360 shutdown_connections 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/1673871360 wait complete. 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.390+0000 7f5fdf0c6740 1 -- start start 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fdf0c6740 1 Processor -- start 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a47e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46cb40 0x55651a479c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fdf0c6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a47a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81b2200 con 0x55651a46b5d0 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81b1b30 con 0x55651a476370 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fdf0c6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81aed70 con 0x55651a46cb40 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fdf0c6740 10 client.0 _ll_register_callbacks cb 0x7f5fc8009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a47e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a47e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43328/0 (socket says 172.21.15.78:43328) 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 -- 172.21.15.78:0/2140184786 learned_addr learned my addr 172.21.15.78:0/2140184786 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd6d9d640 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a47a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 -- 172.21.15.78:0/2140184786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46cb40 msgr2=0x55651a479c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46cb40 0x55651a479c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 -- 172.21.15.78:0/2140184786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 msgr2=0x55651a47a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a47a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 -- 172.21.15.78:0/2140184786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651a47a700 con 0x55651a46b5d0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fd659c640 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a47e8b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800e240 tx=0x7f5fc8005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c157639b9ef867 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.391+0000 7f5fbdffb640 1 -- 172.21.15.78:0/2140184786 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc802c060 con 0x55651a46b5d0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fbdffb640 1 -- 172.21.15.78:0/2140184786 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5fc80127e0 con 0x55651a46b5d0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/2140184786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc00a4d70 con 0x55651a46b5d0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fbdffb640 1 -- 172.21.15.78:0/2140184786 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fc8033050 con 0x55651a46b5d0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 10 client.17079 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/2140184786 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55651a4f09c0 con 0x55651a46b5d0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fbdffb640 1 -- 172.21.15.78:0/2140184786 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc00a4d70 con 0x55651a46b5d0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fbdffb640 1 -- 172.21.15.78:0/2140184786 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5fc801c440 con 0x55651a46b5d0 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 10 client.17079 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 10 client.17079 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:22:49.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 20 client.17079 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55651a4f1190 0x55651a4f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 4 client.17079 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/2140184786 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55651a355170 con 0x55651a4f1190 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.392+0000 7f5fcffff640 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55651a4f1190 0x55651a4f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fbdffb640 10 client.17079 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fcffff640 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55651a4f1190 0x55651a4f35f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51b88d6e0fb98638 server_cookie=64d6e0df1b30cd24 in_seq=0 out_seq=0 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fbdffb640 1 -- 172.21.15.78:0/2140184786 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55651a355170 con 0x55651a4f1190 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fbdffb640 10 client.17079 handle_command_reply: tid=0 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 client.17079 shutdown 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/2140184786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 msgr2=0x55651a47e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a47e8b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800e240 tx=0x7f5fc8005ce0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/2140184786 shutdown_connections 2024-09-16T00:22:49.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55651a4f1190 0x55651a4f35f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55651a476370 0x55651a47a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fbdffb640 0 client.17079 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55651a46cb40 0x55651a479c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 --2- 172.21.15.78:0/2140184786 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55651a46b5d0 0x55651a47e8b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/2140184786 >> 172.21.15.78:0/2140184786 conn(0x55651a434310 msgr2=0x55651a466d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/2140184786 shutdown_connections 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 1 -- 172.21.15.78:0/2140184786 wait complete. 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.393+0000 7f5fdf0c6740 20 client.17079 trim_cache size 0 max 16384 2024-09-16T00:22:49.406 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:22:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 -- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5fc40a32c0 msgr2=0x7f5fc40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5fc40a32c0 0x7f5fc40a5780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5fd002ae70 tx=0x7f5fd0034000 comp rx=0 tx=0).stop 2024-09-16T00:22:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 -- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102570 msgr2=0x7f5fd80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102570 0x7f5fd80fd1c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c920 tx=0x7f5fc000cde0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 -- 172.21.15.78:0/1117311102 shutdown_connections 2024-09-16T00:22:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5fc40a32c0 0x7f5fc40a5780 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5fd8102ef0 0x7f5fd80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5fd8102570 0x7f5fd80fd1c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 --2- 172.21.15.78:0/1117311102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fd8100e20 0x7f5fd80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 -- 172.21.15.78:0/1117311102 >> 172.21.15.78:0/1117311102 conn(0x7f5fd80fc7d0 msgr2=0x7f5fd806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 -- 172.21.15.78:0/1117311102 shutdown_connections 2024-09-16T00:22:49.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:49.394+0000 7f5fdcced640 1 -- 172.21.15.78:0/1117311102 wait complete. 2024-09-16T00:22:50.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:49 smithi078 ceph-mon[31157]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-16T00:22:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:49 smithi193 ceph-mon[35484]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-16T00:22:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:49 smithi148 ceph-mon[35599]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.7 MiB/s wr, 413 op/s 2024-09-16T00:22:50.207 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:22:50.454 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 530.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:22:50.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 -- 172.21.15.78:0/4131889669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64102570 msgr2=0x7faf641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 --2- 172.21.15.78:0/4131889669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64102570 0x7faf641029b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faf54009900 tx=0x7faf5402a480 comp rx=0 tx=0).stop 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 -- 172.21.15.78:0/4131889669 shutdown_connections 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 --2- 172.21.15.78:0/4131889669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf64102ef0 0x7faf6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 --2- 172.21.15.78:0/4131889669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64102570 0x7faf641029b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 --2- 172.21.15.78:0/4131889669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf64100e20 0x7faf64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 -- 172.21.15.78:0/4131889669 >> 172.21.15.78:0/4131889669 conn(0x7faf640fc7d0 msgr2=0x7faf640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 -- 172.21.15.78:0/4131889669 shutdown_connections 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.944+0000 7faf6b389640 1 -- 172.21.15.78:0/4131889669 wait complete. 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf6b389640 1 Processor -- start 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf6b389640 1 -- start start 2024-09-16T00:22:50.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf6b389640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64100e20 0x7faf64198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf6b389640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf64102570 0x7faf64198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf6b389640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf64102ef0 0x7faf6419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf6b389640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf64110f80 con 0x7faf64100e20 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf6b389640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faf64110e00 con 0x7faf64102570 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf6b389640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111100 con 0x7faf64102ef0 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf69125640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64100e20 0x7faf64198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf69125640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64100e20 0x7faf64198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43352/0 (socket says 172.21.15.78:43352) 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.945+0000 7faf69125640 1 -- 172.21.15.78:0/2855803087 learned_addr learned my addr 172.21.15.78:0/2855803087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf69926640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf64102ef0 0x7faf6419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf68924640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf64102570 0x7faf64198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf69125640 1 -- 172.21.15.78:0/2855803087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf64102ef0 msgr2=0x7faf6419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf69125640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf64102ef0 0x7faf6419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf69125640 1 -- 172.21.15.78:0/2855803087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf64102570 msgr2=0x7faf64198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf69125640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf64102570 0x7faf64198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf69125640 1 -- 172.21.15.78:0/2855803087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6419d500 con 0x7faf64100e20 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf69125640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64100e20 0x7faf64198550 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faf6000d950 tx=0x7faf6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf5a7fc640 1 -- 172.21.15.78:0/2855803087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60017070 con 0x7faf64100e20 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf5a7fc640 1 -- 172.21.15.78:0/2855803087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf60009c30 con 0x7faf64100e20 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf6b389640 1 -- 172.21.15.78:0/2855803087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6419d7f0 con 0x7faf64100e20 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf5a7fc640 1 -- 172.21.15.78:0/2855803087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6000f6b0 con 0x7faf64100e20 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.946+0000 7faf6b389640 1 -- 172.21.15.78:0/2855803087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a50d0 con 0x7faf64100e20 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.948+0000 7faf5a7fc640 1 -- 172.21.15.78:0/2855803087 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7faf600109d0 con 0x7faf64100e20 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf6d762740 1 Processor -- start 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf6d762740 1 -- start start 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf6d762740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c41b40 0x55d893c40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf6d762740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c405d0 0x55d893c40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf6d762740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 0x55d893c4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf6d762740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d893ba62f0 con 0x55d893c41b40 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf6d762740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d893b4dcf0 con 0x55d893c405d0 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf6d762740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d893ba22b0 con 0x55d893c4b370 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf69926640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c41b40 0x55d893c40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf69926640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c41b40 0x55d893c40090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43368/0 (socket says 172.21.15.78:43368) 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf69926640 1 -- 172.21.15.78:0/4273047338 learned_addr learned my addr 172.21.15.78:0/4273047338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf69125640 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c405d0 0x55d893c40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf5a7fc640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf500a3240 0x7faf500a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf68924640 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 0x55d893c4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.949+0000 7faf69926640 1 -- 172.21.15.78:0/4273047338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 msgr2=0x55d893c4d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf69926640 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 0x55d893c4d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf69926640 1 -- 172.21.15.78:0/4273047338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c405d0 msgr2=0x55d893c40a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf69926640 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c405d0 0x55d893c40a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf69926640 1 -- 172.21.15.78:0/4273047338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d893b323a0 con 0x55d893c41b40 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf69926640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf500a3240 0x7faf500a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf5a7fc640 1 -- 172.21.15.78:0/2855803087 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faf600cf900 con 0x7faf64100e20 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf69926640 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c41b40 0x55d893c40090 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0104a0 tx=0x7faf5c010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4c396ddb6eb015 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf4f7fe640 1 -- 172.21.15.78:0/4273047338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c022070 con 0x55d893c41b40 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf4f7fe640 1 -- 172.21.15.78:0/4273047338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5c013c60 con 0x55d893c41b40 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.950+0000 7faf4f7fe640 1 -- 172.21.15.78:0/4273047338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c01b3c0 con 0x55d893c41b40 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 -- 172.21.15.78:0/4273047338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c41b40 msgr2=0x55d893c40090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf69926640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf500a3240 0x7faf500a5700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7faf5c01f000 tx=0x7faf5c01c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c41b40 0x55d893c40090 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0104a0 tx=0x7faf5c010960 comp rx=0 tx=0).stop 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 -- 172.21.15.78:0/4273047338 shutdown_connections 2024-09-16T00:22:50.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 0x55d893c4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c405d0 0x55d893c40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 --2- 172.21.15.78:0/4273047338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c41b40 0x55d893c40090 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 -- 172.21.15.78:0/4273047338 >> 172.21.15.78:0/4273047338 conn(0x55d893c09310 msgr2=0x55d893c3cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 -- 172.21.15.78:0/4273047338 shutdown_connections 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 -- 172.21.15.78:0/4273047338 wait complete. 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 -- start start 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.951+0000 7faf6d762740 1 Processor -- start 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c4ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c48020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 0x55d893c48610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d893c4e070 con 0x55d893c405d0 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d893b61060 con 0x55d893c41b40 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d893c4e1f0 con 0x55d893c4b370 2024-09-16T00:22:50.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 10 client.0 _ll_register_callbacks cb 0x55d8938b48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c4ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c4ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43384/0 (socket says 172.21.15.78:43384) 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 -- 172.21.15.78:0/536955128 learned_addr learned my addr 172.21.15.78:0/536955128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf68924640 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c48020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69926640 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 0x55d893c48610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 -- 172.21.15.78:0/536955128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 msgr2=0x55d893c48610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 0x55d893c48610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 -- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 msgr2=0x55d893c48020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c48020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 -- 172.21.15.78:0/536955128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8939173c0 con 0x55d893c405d0 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf68924640 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c48020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf69125640 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c4ad70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faf600cd410 tx=0x7faf6009b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cc51b970e2d94a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf4cff9640 1 -- 172.21.15.78:0/536955128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60020040 con 0x55d893c405d0 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf4cff9640 1 -- 172.21.15.78:0/536955128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faf6009b010 con 0x55d893c405d0 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 1 -- 172.21.15.78:0/536955128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf5c013c60 con 0x55d893c405d0 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf4cff9640 1 -- 172.21.15.78:0/536955128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faf60009de0 con 0x55d893c405d0 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 10 client.17091 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.952+0000 7faf6d762740 1 -- 172.21.15.78:0/536955128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d893c59b60 con 0x55d893c405d0 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf4cff9640 1 -- 172.21.15.78:0/536955128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5c013c60 con 0x55d893c405d0 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf4cff9640 1 -- 172.21.15.78:0/536955128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7faf60097910 con 0x55d893c405d0 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf6d762740 10 client.17091 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf6d762740 10 client.17091 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf6d762740 20 client.17091 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf6d762740 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55d893c5b020 0x55d893c5d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf6d762740 4 client.17091 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:50.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf6d762740 1 -- 172.21.15.78:0/536955128 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d893bb3860 con 0x55d893c5b020 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf68924640 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55d893c5b020 0x55d893c5d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf4cff9640 10 client.17091 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.953+0000 7faf68924640 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55d893c5b020 0x55d893c5d470 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a2fe55e543867cd server_cookie=3ebf1233e98ca2b1 in_seq=0 out_seq=0 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf4cff9640 1 -- 172.21.15.78:0/536955128 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d893bb3860 con 0x55d893c5b020 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf4cff9640 10 client.17091 handle_command_reply: tid=0 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 client.17091 shutdown 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 -- 172.21.15.78:0/536955128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 msgr2=0x55d893c4ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c4ad70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faf600cd410 tx=0x7faf6009b780 comp rx=0 tx=0).stop 2024-09-16T00:22:50.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 -- 172.21.15.78:0/536955128 shutdown_connections 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c4b370 0x55d893c48610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55d893c5b020 0x55d893c5d470 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c48020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 --2- 172.21.15.78:0/536955128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c4ad70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf4cff9640 0 client.17091 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 -- 172.21.15.78:0/536955128 >> 172.21.15.78:0/536955128 conn(0x55d893c09310 msgr2=0x55d893c42960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 -- 172.21.15.78:0/536955128 shutdown_connections 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.954+0000 7faf6d762740 1 -- 172.21.15.78:0/536955128 wait complete. 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.955+0000 7faf6d762740 20 client.17091 trim_cache size 0 max 16384 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.967+0000 7faf6d762740 1 Processor -- start 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.967+0000 7faf6d762740 1 -- start start 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.967+0000 7faf6d762740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.967+0000 7faf6d762740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c41b40 0x55d893c43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.967+0000 7faf6d762740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c4b370 0x55d893c43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf6d762740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf641aebd0 con 0x55d893c405d0 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf6d762740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faf641b0710 con 0x55d893c4b370 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf6d762740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf641af2a0 con 0x55d893c41b40 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43400/0 (socket says 172.21.15.78:43400) 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 -- 172.21.15.78:0/4263838807 learned_addr learned my addr 172.21.15.78:0/4263838807 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69926640 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c4b370 0x55d893c43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 -- 172.21.15.78:0/4263838807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c41b40 msgr2=0x55d893c43120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf68924640 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c41b40 0x55d893c43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c41b40 0x55d893c43120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 -- 172.21.15.78:0/4263838807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c4b370 msgr2=0x55d893c43660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:50.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c4b370 0x55d893c43660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 -- 172.21.15.78:0/4263838807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d893c46fd0 con 0x55d893c405d0 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69926640 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c4b370 0x55d893c43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf68924640 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c41b40 0x55d893c43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf69125640 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c468f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faf60098020 tx=0x7faf6009a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aee684e286757b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf4effd640 1 -- 172.21.15.78:0/4263838807 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60020070 con 0x55d893c405d0 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf4effd640 1 -- 172.21.15.78:0/4263838807 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf60015070 con 0x55d893c405d0 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.968+0000 7faf4effd640 1 -- 172.21.15.78:0/4263838807 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf600860c0 con 0x55d893c405d0 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 -- 172.21.15.78:0/4263838807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 msgr2=0x55d893c468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c468f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faf60098020 tx=0x7faf6009a0d0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 -- 172.21.15.78:0/4263838807 shutdown_connections 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c4b370 0x55d893c43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c41b40 0x55d893c43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 --2- 172.21.15.78:0/4263838807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c405d0 0x55d893c468f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 -- 172.21.15.78:0/4263838807 >> 172.21.15.78:0/4263838807 conn(0x55d893c09310 msgr2=0x55d893c4a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 -- 172.21.15.78:0/4263838807 shutdown_connections 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 -- 172.21.15.78:0/4263838807 wait complete. 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 -- start start 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.969+0000 7faf6d762740 1 Processor -- start 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf6d762740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c405d0 0x55d893c4f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf6d762740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c4fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf6d762740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c4b370 0x55d893cc4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf6d762740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf641b0040 con 0x55d893c4b370 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf6d762740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faf641af970 con 0x55d893c41b40 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf6d762740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf641acbb0 con 0x55d893c405d0 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf6d762740 10 client.0 _ll_register_callbacks cb 0x7faf6009ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c4b370 0x55d893cc4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c4b370 0x55d893cc4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43412/0 (socket says 172.21.15.78:43412) 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 -- 172.21.15.78:0/3186129647 learned_addr learned my addr 172.21.15.78:0/3186129647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf68924640 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c4fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69125640 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c405d0 0x55d893c4f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 -- 172.21.15.78:0/3186129647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c405d0 msgr2=0x55d893c4f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c405d0 0x55d893c4f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 -- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 msgr2=0x55d893c4fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c4fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 -- 172.21.15.78:0/3186129647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d893c48980 con 0x55d893c4b370 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf68924640 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c4fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf69926640 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c4b370 0x55d893cc4d00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0104a0 tx=0x7faf5c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dee9b9f609c4d05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.970+0000 7faf4cff9640 1 -- 172.21.15.78:0/3186129647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c035060 con 0x55d893c4b370 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf4cff9640 1 -- 172.21.15.78:0/3186129647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faf5c00e430 con 0x55d893c4b370 2024-09-16T00:22:50.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 1 -- 172.21.15.78:0/3186129647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf60015070 con 0x55d893c4b370 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf4cff9640 1 -- 172.21.15.78:0/3186129647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faf5c03c050 con 0x55d893c4b370 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 10 client.17097 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 1 -- 172.21.15.78:0/3186129647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d893cc61f0 con 0x55d893c4b370 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf4cff9640 1 -- 172.21.15.78:0/3186129647 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf60015070 con 0x55d893c4b370 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf4cff9640 1 -- 172.21.15.78:0/3186129647 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7faf5c02f030 con 0x55d893c4b370 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 10 client.17097 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 10 client.17097 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 20 client.17097 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55d893cc66c0 0x55d893cc8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 4 client.17097 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf6d762740 1 -- 172.21.15.78:0/3186129647 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d893badd70 con 0x55d893cc66c0 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.971+0000 7faf69125640 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55d893cc66c0 0x55d893cc8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf4cff9640 10 client.17097 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf69125640 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55d893cc66c0 0x55d893cc8ab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31c67327704b3646 server_cookie=e37e62756eee104e in_seq=0 out_seq=0 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf4cff9640 1 -- 172.21.15.78:0/3186129647 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d893badd70 con 0x55d893cc66c0 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf4cff9640 10 client.17097 handle_command_reply: tid=0 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 client.17097 shutdown 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 -- 172.21.15.78:0/3186129647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c4b370 msgr2=0x55d893cc4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c4b370 0x55d893cc4d00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0104a0 tx=0x7faf5c009810 comp rx=0 tx=0).stop 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 -- 172.21.15.78:0/3186129647 shutdown_connections 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d893c4b370 0x55d893cc4d00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55d893cc66c0 0x55d893cc8ab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d893c41b40 0x55d893c4fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 --2- 172.21.15.78:0/3186129647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d893c405d0 0x55d893c4f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf4cff9640 0 client.17097 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:22:50.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 -- 172.21.15.78:0/3186129647 >> 172.21.15.78:0/3186129647 conn(0x55d893c09310 msgr2=0x55d893c49880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 -- 172.21.15.78:0/3186129647 shutdown_connections 2024-09-16T00:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 1 -- 172.21.15.78:0/3186129647 wait complete. 2024-09-16T00:22:50.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.972+0000 7faf6d762740 20 client.17097 trim_cache size 0 max 16384 2024-09-16T00:22:50.990 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 -- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf500a3240 msgr2=0x7faf500a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf500a3240 0x7faf500a5700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7faf5c01f000 tx=0x7faf5c01c000 comp rx=0 tx=0).stop 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 -- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64100e20 msgr2=0x7faf64198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64100e20 0x7faf64198550 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faf6000d950 tx=0x7faf6000de10 comp rx=0 tx=0).stop 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 -- 172.21.15.78:0/2855803087 shutdown_connections 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf500a3240 0x7faf500a5700 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf64102ef0 0x7faf6419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf64102570 0x7faf64198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 --2- 172.21.15.78:0/2855803087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf64100e20 0x7faf64198550 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 -- 172.21.15.78:0/2855803087 >> 172.21.15.78:0/2855803087 conn(0x7faf640fc7d0 msgr2=0x7faf6406a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 -- 172.21.15.78:0/2855803087 shutdown_connections 2024-09-16T00:22:50.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:50.973+0000 7faf6b389640 1 -- 172.21.15.78:0/2855803087 wait complete. 2024-09-16T00:22:51.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:22:52.000 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:52.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:51 smithi078 ceph-mon[31157]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-09-16T00:22:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:51 smithi193 ceph-mon[35484]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-09-16T00:22:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:51 smithi148 ceph-mon[35599]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.6 MiB/s wr, 394 op/s 2024-09-16T00:22:52.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 -- 172.21.15.78:0/3226626093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528106f10 msgr2=0x7f552810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 --2- 172.21.15.78:0/3226626093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528106f10 0x7f552810ea90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5520009ca0 tx=0x7f552002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 -- 172.21.15.78:0/3226626093 shutdown_connections 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 --2- 172.21.15.78:0/3226626093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528106f10 0x7f552810ea90 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 --2- 172.21.15.78:0/3226626093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528102790 0x7f55281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 --2- 172.21.15.78:0/3226626093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5528101de0 0x7f55281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 -- 172.21.15.78:0/3226626093 >> 172.21.15.78:0/3226626093 conn(0x7f55280fd8c0 msgr2=0x7f55280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 -- 172.21.15.78:0/3226626093 shutdown_connections 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 -- 172.21.15.78:0/3226626093 wait complete. 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 Processor -- start 2024-09-16T00:22:52.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.486+0000 7f553066d640 1 -- start start 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f553066d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5528101de0 0x7f55280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f553066d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528102790 0x7f5528074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f553066d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528106f10 0x7f5528074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f553066d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5528111f90 con 0x7f5528102790 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f553066d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5528111e10 con 0x7f5528101de0 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f553066d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5528112110 con 0x7f5528106f10 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528102790 0x7f5528074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528102790 0x7f5528074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43430/0 (socket says 172.21.15.78:43430) 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 -- 172.21.15.78:0/628155512 learned_addr learned my addr 172.21.15.78:0/628155512 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552e409640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5528101de0 0x7f55280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552ec0a640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528106f10 0x7f5528074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 -- 172.21.15.78:0/628155512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528106f10 msgr2=0x7f5528074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528106f10 0x7f5528074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 -- 172.21.15.78:0/628155512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5528101de0 msgr2=0x7f55280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5528101de0 0x7f55280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 -- 172.21.15.78:0/628155512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55281a68d0 con 0x7f5528102790 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552ec0a640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528106f10 0x7f5528074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f552dc08640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528102790 0x7f5528074a30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f551800b4a0 tx=0x7f551800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f551f7fe640 1 -- 172.21.15.78:0/628155512 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518015020 con 0x7f5528102790 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f553066d640 1 -- 172.21.15.78:0/628155512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55281a6bc0 con 0x7f5528102790 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f551f7fe640 1 -- 172.21.15.78:0/628155512 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5518011d10 con 0x7f5528102790 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.487+0000 7f551f7fe640 1 -- 172.21.15.78:0/628155512 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551800f2e0 con 0x7f5528102790 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.488+0000 7f553066d640 1 -- 172.21.15.78:0/628155512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55281a70b0 con 0x7f5528102790 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f551f7fe640 1 -- 172.21.15.78:0/628155512 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5518018020 con 0x7f5528102790 2024-09-16T00:22:52.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f5532a46740 1 Processor -- start 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f5532a46740 1 -- start start 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f5532a46740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f5532a46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05eda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f5532a46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05f8370 0x5604e05fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f5532a46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5604e05532f0 con 0x5604e05f8370 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f5532a46740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5604e04facf0 con 0x5604e05eeb40 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.490+0000 7f5532a46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5604e054f2b0 con 0x5604e05ed5d0 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552dc08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05f8370 0x5604e05fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552dc08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05f8370 0x5604e05fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43436/0 (socket says 172.21.15.78:43436) 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552dc08640 1 -- 172.21.15.78:0/4220117282 learned_addr learned my addr 172.21.15.78:0/4220117282 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552e409640 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f551f7fe640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55140a3240 0x7f55140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 -- 172.21.15.78:0/4220117282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 msgr2=0x5604e05eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 -- 172.21.15.78:0/4220117282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05f8370 msgr2=0x5604e05fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05f8370 0x5604e05fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 -- 172.21.15.78:0/4220117282 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e04df3a0 con 0x5604e05eeb40 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552dc08640 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05f8370 0x5604e05fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f551f7fe640 1 -- 172.21.15.78:0/628155512 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f55180cf5b0 con 0x7f5528102790 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55140a3240 0x7f55140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05ed090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5520035000 tx=0x7f5520031bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d4600d4a8d6efa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f551cff9640 1 -- 172.21.15.78:0/4220117282 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520004040 con 0x5604e05eeb40 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f552ec0a640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55140a3240 0x7f55140a5700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f552004b000 tx=0x7f5520048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f551cff9640 1 -- 172.21.15.78:0/4220117282 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5520036070 con 0x5604e05eeb40 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.491+0000 7f551cff9640 1 -- 172.21.15.78:0/4220117282 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552002fc50 con 0x5604e05eeb40 2024-09-16T00:22:52.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 -- 172.21.15.78:0/4220117282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 msgr2=0x5604e05ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05ed090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5520035000 tx=0x7f5520031bb0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 -- 172.21.15.78:0/4220117282 shutdown_connections 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05f8370 0x5604e05fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 --2- 172.21.15.78:0/4220117282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05ed090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 -- 172.21.15.78:0/4220117282 >> 172.21.15.78:0/4220117282 conn(0x5604e05b6310 msgr2=0x5604e05e9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 -- 172.21.15.78:0/4220117282 shutdown_connections 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.492+0000 7f5532a46740 1 -- 172.21.15.78:0/4220117282 wait complete. 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 1 -- start start 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 1 Processor -- start 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05f5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05f8370 0x5604e05f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5604e05faef0 con 0x5604e05eeb40 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5604e050e060 con 0x5604e05f8370 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5604e05fb070 con 0x5604e05ed5d0 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f5532a46740 10 client.0 _ll_register_callbacks cb 0x5604e02618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552dc08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552dc08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43442/0 (socket says 172.21.15.78:43442) 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552dc08640 1 -- 172.21.15.78:0/3171302367 learned_addr learned my addr 172.21.15.78:0/3171302367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552ec0a640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05f8370 0x5604e05f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552e409640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552dc08640 1 -- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 msgr2=0x5604e05f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552dc08640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552dc08640 1 -- 172.21.15.78:0/3171302367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05f8370 msgr2=0x5604e05f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552dc08640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05f8370 0x5604e05f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552dc08640 1 -- 172.21.15.78:0/3171302367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e02c43c0 con 0x5604e05eeb40 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552e409640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.493+0000 7f552ec0a640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05f8370 0x5604e05f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f552dc08640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05f5070 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f551800b470 tx=0x7f5518010400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c1a55c4f99299d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5511ffb640 1 -- 172.21.15.78:0/3171302367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551809b1d0 con 0x5604e05eeb40 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5511ffb640 1 -- 172.21.15.78:0/3171302367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5518099b70 con 0x5604e05eeb40 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 1 -- 172.21.15.78:0/3171302367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5520036070 con 0x5604e05eeb40 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5511ffb640 1 -- 172.21.15.78:0/3171302367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f55180a2fa0 con 0x5604e05eeb40 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 10 client.17109 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 1 -- 172.21.15.78:0/3171302367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5604e0606a90 con 0x5604e05eeb40 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5511ffb640 1 -- 172.21.15.78:0/3171302367 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5520036070 con 0x5604e05eeb40 2024-09-16T00:22:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5511ffb640 1 -- 172.21.15.78:0/3171302367 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f55180a4050 con 0x5604e05eeb40 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 10 client.17109 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 10 client.17109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 20 client.17109 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5604e0607fa0 0x5604e060a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 4 client.17109 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.494+0000 7f5532a46740 1 -- 172.21.15.78:0/3171302367 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604e0560860 con 0x5604e0607fa0 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.495+0000 7f552e409640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5604e0607fa0 0x5604e060a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.495+0000 7f552e409640 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5604e0607fa0 0x5604e060a3d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31d8bafb505575a1 server_cookie=7b3ea0482c21351f in_seq=0 out_seq=0 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.495+0000 7f5511ffb640 10 client.17109 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5511ffb640 1 -- 172.21.15.78:0/3171302367 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604e0560860 con 0x5604e0607fa0 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5511ffb640 10 client.17109 handle_command_reply: tid=0 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 client.17109 shutdown 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 -- 172.21.15.78:0/3171302367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 msgr2=0x5604e05f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05f5070 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f551800b470 tx=0x7f5518010400 comp rx=0 tx=0).stop 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 -- 172.21.15.78:0/3171302367 shutdown_connections 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5604e0607fa0 0x5604e060a3d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05f8370 0x5604e05f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05f5070 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5511ffb640 0 client.17109 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 --2- 172.21.15.78:0/3171302367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05ed5d0 0x5604e05f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 -- 172.21.15.78:0/3171302367 >> 172.21.15.78:0/3171302367 conn(0x5604e05b6310 msgr2=0x5604e05ef960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 -- 172.21.15.78:0/3171302367 shutdown_connections 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 1 -- 172.21.15.78:0/3171302367 wait complete. 2024-09-16T00:22:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.496+0000 7f5532a46740 20 client.17109 trim_cache size 0 max 16384 2024-09-16T00:22:52.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f5532a46740 1 Processor -- start 2024-09-16T00:22:52.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f5532a46740 1 -- start start 2024-09-16T00:22:52.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f5532a46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05ed5d0 0x5604e05f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f5532a46740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05f0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f5532a46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 0x5604e060fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f5532a46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55281b0d30 con 0x5604e05ed5d0 2024-09-16T00:22:52.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f5532a46740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55281b2870 con 0x5604e05eeb40 2024-09-16T00:22:52.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f5532a46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55281b1400 con 0x5604e05f8370 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552e409640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05ed5d0 0x5604e05f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552e409640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05ed5d0 0x5604e05f0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43444/0 (socket says 172.21.15.78:43444) 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552e409640 1 -- 172.21.15.78:0/3806383041 learned_addr learned my addr 172.21.15.78:0/3806383041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552e409640 1 -- 172.21.15.78:0/3806383041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 msgr2=0x5604e060fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552e409640 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 0x5604e060fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552dc08640 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552e409640 1 -- 172.21.15.78:0/3806383041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 msgr2=0x5604e05f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552e409640 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05f0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552e409640 1 -- 172.21.15.78:0/3806383041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e05f5b70 con 0x5604e05ed5d0 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.509+0000 7f552dc08640 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f552e409640 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05ed5d0 0x5604e05f0120 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5604e060ac10 tx=0x7f5524010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c51b646dfe76100 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5513fff640 1 -- 172.21.15.78:0/3806383041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524005220 con 0x5604e05ed5d0 2024-09-16T00:22:52.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5513fff640 1 -- 172.21.15.78:0/3806383041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5524004c80 con 0x5604e05ed5d0 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5513fff640 1 -- 172.21.15.78:0/3806383041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552401c8c0 con 0x5604e05ed5d0 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 -- 172.21.15.78:0/3806383041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05ed5d0 msgr2=0x5604e05f0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05ed5d0 0x5604e05f0120 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5604e060ac10 tx=0x7f5524010800 comp rx=0 tx=0).stop 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 -- 172.21.15.78:0/3806383041 shutdown_connections 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 0x5604e060fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05eeb40 0x5604e05f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 --2- 172.21.15.78:0/3806383041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05ed5d0 0x5604e05f0120 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 -- 172.21.15.78:0/3806383041 >> 172.21.15.78:0/3806383041 conn(0x5604e05b6310 msgr2=0x5604e05f4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 -- 172.21.15.78:0/3806383041 shutdown_connections 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.510+0000 7f5532a46740 1 -- 172.21.15.78:0/3806383041 wait complete. 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 1 -- start start 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 1 Processor -- start 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05ed5d0 0x5604e0600790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 0x5604e05fc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55281b21a0 con 0x5604e05eeb40 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55281b1ad0 con 0x5604e05ed5d0 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55281aed10 con 0x5604e05f8370 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f5532a46740 10 client.0 _ll_register_callbacks cb 0x7f551800bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552dc08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552dc08640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05fbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43454/0 (socket says 172.21.15.78:43454) 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552dc08640 1 -- 172.21.15.78:0/2153561499 learned_addr learned my addr 172.21.15.78:0/2153561499 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552e409640 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05ed5d0 0x5604e0600790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552ec0a640 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 0x5604e05fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552dc08640 1 -- 172.21.15.78:0/2153561499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 msgr2=0x5604e05fc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552dc08640 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 0x5604e05fc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552dc08640 1 -- 172.21.15.78:0/2153561499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05ed5d0 msgr2=0x5604e0600790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552dc08640 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05ed5d0 0x5604e0600790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.511+0000 7f552dc08640 1 -- 172.21.15.78:0/2153561499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e05fc540 con 0x5604e05eeb40 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f552dc08640 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05fbac0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f551809b570 tx=0x7f55180974c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e077601dfbf815 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5511ffb640 1 -- 172.21.15.78:0/2153561499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55180d2e60 con 0x5604e05eeb40 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5511ffb640 1 -- 172.21.15.78:0/2153561499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5518084bc0 con 0x5604e05eeb40 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 1 -- 172.21.15.78:0/2153561499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5524004c80 con 0x5604e05eeb40 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5511ffb640 1 -- 172.21.15.78:0/2153561499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f551809b1d0 con 0x5604e05eeb40 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 10 client.17115 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 1 -- 172.21.15.78:0/2153561499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5604e06728f0 con 0x5604e05eeb40 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5511ffb640 1 -- 172.21.15.78:0/2153561499 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5524004c80 con 0x5604e05eeb40 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5511ffb640 1 -- 172.21.15.78:0/2153561499 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f55180ba0d0 con 0x5604e05eeb40 2024-09-16T00:22:52.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 10 client.17115 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 10 client.17115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 20 client.17115 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5604e0672e50 0x5604e0675270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 4 client.17115 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.512+0000 7f5532a46740 1 -- 172.21.15.78:0/2153561499 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604e055ad70 con 0x5604e0672e50 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f552e409640 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5604e0672e50 0x5604e0675270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5511ffb640 10 client.17115 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f552e409640 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5604e0672e50 0x5604e0675270 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f61f807943ee0b1 server_cookie=e30422c03cfe067f in_seq=0 out_seq=0 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5511ffb640 1 -- 172.21.15.78:0/2153561499 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604e055ad70 con 0x5604e0672e50 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5511ffb640 10 client.17115 handle_command_reply: tid=0 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 client.17115 shutdown 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 -- 172.21.15.78:0/2153561499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 msgr2=0x5604e05fbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05fbac0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f551809b570 tx=0x7f55180974c0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 -- 172.21.15.78:0/2153561499 shutdown_connections 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5604e0672e50 0x5604e0675270 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604e05f8370 0x5604e05fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5604e05eeb40 0x5604e05fbac0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 --2- 172.21.15.78:0/2153561499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5604e05ed5d0 0x5604e0600790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5532a46740 1 -- 172.21.15.78:0/2153561499 >> 172.21.15.78:0/2153561499 conn(0x5604e05b6310 msgr2=0x5604e05f68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.513+0000 7f5511ffb640 0 client.17115 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f5532a46740 1 -- 172.21.15.78:0/2153561499 shutdown_connections 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f5532a46740 1 -- 172.21.15.78:0/2153561499 wait complete. 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f5532a46740 20 client.17115 trim_cache size 0 max 16384 2024-09-16T00:22:52.531 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:22:52.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 -- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55140a3240 msgr2=0x7f55140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55140a3240 0x7f55140a5700 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f552004b000 tx=0x7f5520048000 comp rx=0 tx=0).stop 2024-09-16T00:22:52.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 -- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528102790 msgr2=0x7f5528074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528102790 0x7f5528074a30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f551800b4a0 tx=0x7f551800b8a0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 -- 172.21.15.78:0/628155512 shutdown_connections 2024-09-16T00:22:52.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f55140a3240 0x7f55140a5700 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5528106f10 0x7f5528074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5528102790 0x7f5528074a30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 --2- 172.21.15.78:0/628155512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5528101de0 0x7f55280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 -- 172.21.15.78:0/628155512 >> 172.21.15.78:0/628155512 conn(0x7f55280fd8c0 msgr2=0x7f55280ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 -- 172.21.15.78:0/628155512 shutdown_connections 2024-09-16T00:22:52.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:52.514+0000 7f553066d640 1 -- 172.21.15.78:0/628155512 wait complete. 2024-09-16T00:22:53.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:22:53.607 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:54.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:53 smithi078 ceph-mon[31157]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-09-16T00:22:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:53 smithi193 ceph-mon[35484]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-09-16T00:22:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:53 smithi148 ceph-mon[35599]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.3 MiB/s wr, 287 op/s 2024-09-16T00:22:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 -- 172.21.15.78:0/3591011643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648100a80 msgr2=0x7f5648100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 --2- 172.21.15.78:0/3591011643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648100a80 0x7f5648100ee0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f563c009930 tx=0x7f563c02a420 comp rx=0 tx=0).stop 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 -- 172.21.15.78:0/3591011643 shutdown_connections 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 --2- 172.21.15.78:0/3591011643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5648101420 0x7f564810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 --2- 172.21.15.78:0/3591011643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648100a80 0x7f5648100ee0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 --2- 172.21.15.78:0/3591011643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5648106a80 0x7f5648106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 -- 172.21.15.78:0/3591011643 >> 172.21.15.78:0/3591011643 conn(0x7f56480fc7a0 msgr2=0x7f56480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 -- 172.21.15.78:0/3591011643 shutdown_connections 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.272+0000 7f564d7c4640 1 -- 172.21.15.78:0/3591011643 wait complete. 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f564d7c4640 1 Processor -- start 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f564d7c4640 1 -- start start 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f564d7c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5648100a80 0x7f5648197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f564d7c4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648101420 0x7f5648197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f564d7c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5648106a80 0x7f5648197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f564d7c4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5648110ea0 con 0x7f5648101420 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f564d7c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5648110d20 con 0x7f5648106a80 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f564d7c4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5648111020 con 0x7f5648100a80 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f56467fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648101420 0x7f5648197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f56467fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648101420 0x7f5648197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43476/0 (socket says 172.21.15.78:43476) 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.273+0000 7f56467fc640 1 -- 172.21.15.78:0/2680735346 learned_addr learned my addr 172.21.15.78:0/2680735346 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f5646ffd640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5648100a80 0x7f5648197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f56477fe640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5648106a80 0x7f5648197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f56467fc640 1 -- 172.21.15.78:0/2680735346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5648100a80 msgr2=0x7f5648197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f56467fc640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5648100a80 0x7f5648197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f56467fc640 1 -- 172.21.15.78:0/2680735346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5648106a80 msgr2=0x7f5648197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f56467fc640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5648106a80 0x7f5648197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f56467fc640 1 -- 172.21.15.78:0/2680735346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56481a69b0 con 0x7f5648101420 2024-09-16T00:22:54.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f56467fc640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648101420 0x7f5648197680 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f563c009a50 tx=0x7f563c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f562ffff640 1 -- 172.21.15.78:0/2680735346 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c03f030 con 0x7f5648101420 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f564d7c4640 1 -- 172.21.15.78:0/2680735346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56481a6c40 con 0x7f5648101420 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f562ffff640 1 -- 172.21.15.78:0/2680735346 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f563c003ce0 con 0x7f5648101420 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f562ffff640 1 -- 172.21.15.78:0/2680735346 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c035070 con 0x7f5648101420 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.274+0000 7f564d7c4640 1 -- 172.21.15.78:0/2680735346 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56481a7170 con 0x7f5648101420 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.276+0000 7f562ffff640 1 -- 172.21.15.78:0/2680735346 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f563c042030 con 0x7f5648101420 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f564fb9d740 1 Processor -- start 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f564fb9d740 1 -- start start 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c5094090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c50945d0 0x5619c5094a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c50a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f564fb9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619c4ffa2f0 con 0x5619c509f370 2024-09-16T00:22:54.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f564fb9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5619c4fa1cf0 con 0x5619c50945d0 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f564fb9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5619c4ff62b0 con 0x5619c5095b40 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f562ffff640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f56300a3240 0x7f56300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f56467fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c50a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f56467fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c50a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43478/0 (socket says 172.21.15.78:43478) 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f56467fc640 1 -- 172.21.15.78:0/4073990584 learned_addr learned my addr 172.21.15.78:0/4073990584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f56477fe640 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c5094090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f5646ffd640 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c50945d0 0x5619c5094a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f56477fe640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f56300a3240 0x7f56300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.277+0000 7f562ffff640 1 -- 172.21.15.78:0/2680735346 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f563c0f02a0 con 0x7f5648101420 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f56467fc640 1 -- 172.21.15.78:0/4073990584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 msgr2=0x5619c5094090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f56467fc640 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c5094090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f56467fc640 1 -- 172.21.15.78:0/4073990584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c50945d0 msgr2=0x5619c5094a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:54.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f56467fc640 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c50945d0 0x5619c5094a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f56467fc640 1 -- 172.21.15.78:0/4073990584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c4f863a0 con 0x5619c509f370 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f56467fc640 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c50a1760 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f563c009900 tx=0x7f563c08f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a8cecda4a25b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f56477fe640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f56300a3240 0x7f56300a5700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f564000afd0 tx=0x7f5640005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f562d7fa640 1 -- 172.21.15.78:0/4073990584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c049020 con 0x5619c509f370 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f562d7fa640 1 -- 172.21.15.78:0/4073990584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f563c0bcc60 con 0x5619c509f370 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.278+0000 7f562d7fa640 1 -- 172.21.15.78:0/4073990584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c03bd00 con 0x5619c509f370 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- 172.21.15.78:0/4073990584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 msgr2=0x5619c50a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c50a1760 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f563c009900 tx=0x7f563c08f3e0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- 172.21.15.78:0/4073990584 shutdown_connections 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c50a1760 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c50945d0 0x5619c5094a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 --2- 172.21.15.78:0/4073990584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c5094090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- 172.21.15.78:0/4073990584 >> 172.21.15.78:0/4073990584 conn(0x5619c505d310 msgr2=0x5619c5090c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:54.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- 172.21.15.78:0/4073990584 shutdown_connections 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- 172.21.15.78:0/4073990584 wait complete. 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- start start 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 Processor -- start 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c50945d0 0x5619c509ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 0x5619c509bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c509c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619c4ffa2f0 con 0x5619c509f370 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5619c4fb5060 con 0x5619c5095b40 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5619c50a1e30 con 0x5619c50945d0 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f564fb9d740 10 client.0 _ll_register_callbacks cb 0x5619c4d088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f56477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c509c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f56477fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c509c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43488/0 (socket says 172.21.15.78:43488) 2024-09-16T00:22:54.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.279+0000 7f56477fe640 1 -- 172.21.15.78:0/899220206 learned_addr learned my addr 172.21.15.78:0/899220206 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f5646ffd640 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c50945d0 0x5619c509ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f56467fc640 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 0x5619c509bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f56477fe640 1 -- 172.21.15.78:0/899220206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c50945d0 msgr2=0x5619c509ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f56477fe640 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c50945d0 0x5619c509ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f56477fe640 1 -- 172.21.15.78:0/899220206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 msgr2=0x5619c509bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f56477fe640 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 0x5619c509bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f56477fe640 1 -- 172.21.15.78:0/899220206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c4d6b3c0 con 0x5619c509f370 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f5646ffd640 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c50945d0 0x5619c509ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f56477fe640 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c509c5d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f564000e720 tx=0x7f5640004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792110deef36177 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f562aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f562aff5640 1 -- 172.21.15.78:0/899220206 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640004d20 con 0x5619c509f370 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f562aff5640 1 -- 172.21.15.78:0/899220206 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5640003970 con 0x5619c509f370 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f564fb9d740 1 -- 172.21.15.78:0/899220206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f563c0bcc60 con 0x5619c509f370 2024-09-16T00:22:54.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f562aff5640 1 -- 172.21.15.78:0/899220206 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5640030050 con 0x5619c509f370 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f564fb9d740 10 client.17127 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f564fb9d740 1 -- 172.21.15.78:0/899220206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5619c50ad7a0 con 0x5619c509f370 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.280+0000 7f562aff5640 1 -- 172.21.15.78:0/899220206 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f563c0bcc60 con 0x5619c509f370 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f562aff5640 1 -- 172.21.15.78:0/899220206 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5640005470 con 0x5619c509f370 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f564fb9d740 10 client.17127 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f564fb9d740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f564fb9d740 20 client.17127 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f564fb9d740 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5619c50aec00 0x5619c50b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f564fb9d740 4 client.17127 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f564fb9d740 1 -- 172.21.15.78:0/899220206 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619c5007860 con 0x5619c50aec00 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f5646ffd640 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5619c50aec00 0x5619c50b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f562aff5640 10 client.17127 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.281+0000 7f5646ffd640 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5619c50aec00 0x5619c50b1060 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee9b472e6e76061 server_cookie=df3ee51b376ec1ef in_seq=0 out_seq=0 2024-09-16T00:22:54.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f562aff5640 1 -- 172.21.15.78:0/899220206 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619c5007860 con 0x5619c50aec00 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f562aff5640 10 client.17127 handle_command_reply: tid=0 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 client.17127 shutdown 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 -- 172.21.15.78:0/899220206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 msgr2=0x5619c509c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c509c5d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f564000e720 tx=0x7f5640004ab0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 -- 172.21.15.78:0/899220206 shutdown_connections 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5619c50aec00 0x5619c50b1060 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c509f370 0x5619c509c5d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 0x5619c509bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 --2- 172.21.15.78:0/899220206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c50945d0 0x5619c509ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 -- 172.21.15.78:0/899220206 >> 172.21.15.78:0/899220206 conn(0x5619c505d310 msgr2=0x5619c5096990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f562aff5640 0 client.17127 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 -- 172.21.15.78:0/899220206 shutdown_connections 2024-09-16T00:22:54.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 1 -- 172.21.15.78:0/899220206 wait complete. 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.282+0000 7f564fb9d740 20 client.17127 trim_cache size 0 max 16384 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f564fb9d740 1 Processor -- start 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f564fb9d740 1 -- start start 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c509df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 0x5619c5097180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c509f370 0x5619c50976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f564fb9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b0e50 con 0x5619c50945d0 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f564fb9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b2990 con 0x5619c5095b40 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f564fb9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b1520 con 0x5619c509f370 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f5646ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c509df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f5646ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c509df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43492/0 (socket says 172.21.15.78:43492) 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f5646ffd640 1 -- 172.21.15.78:0/1933679409 learned_addr learned my addr 172.21.15.78:0/1933679409 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.295+0000 7f5646ffd640 1 -- 172.21.15.78:0/1933679409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c509f370 msgr2=0x5619c50976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:54.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f5646ffd640 1 --2- 172.21.15.78:0/1933679409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c509f370 0x5619c50976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f5646ffd640 1 -- 172.21.15.78:0/1933679409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 msgr2=0x5619c5097180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f5646ffd640 1 --2- 172.21.15.78:0/1933679409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 0x5619c5097180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f5646ffd640 1 -- 172.21.15.78:0/1933679409 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c509a530 con 0x5619c50945d0 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f5646ffd640 1 --2- 172.21.15.78:0/1933679409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c509df50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f563800b840 tx=0x7f5638008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213e769712be320c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f562cff9640 1 -- 172.21.15.78:0/1933679409 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638020030 con 0x5619c50945d0 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f562cff9640 1 -- 172.21.15.78:0/1933679409 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f563801c250 con 0x5619c50945d0 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f562cff9640 1 -- 172.21.15.78:0/1933679409 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56380243c0 con 0x5619c50945d0 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 -- 172.21.15.78:0/1933679409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 msgr2=0x5619c509df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1933679409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c509df50 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f563800b840 tx=0x7f5638008040 comp rx=0 tx=0).stop 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 -- 172.21.15.78:0/1933679409 shutdown_connections 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1933679409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c509f370 0x5619c50976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1933679409 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c5095b40 0x5619c5097180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1933679409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c509df50 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 -- 172.21.15.78:0/1933679409 >> 172.21.15.78:0/1933679409 conn(0x5619c505d310 msgr2=0x5619c509d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 -- 172.21.15.78:0/1933679409 shutdown_connections 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.296+0000 7f564fb9d740 1 -- 172.21.15.78:0/1933679409 wait complete. 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 1 -- start start 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 1 Processor -- start 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c50a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c50a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c509f370 0x5619c50a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b22c0 con 0x5619c50945d0 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b1bf0 con 0x5619c509f370 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56481aee30 con 0x5619c5095b40 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f564fb9d740 10 client.0 _ll_register_callbacks cb 0x7f5640010ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f5646ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c50a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f5646ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c50a3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43508/0 (socket says 172.21.15.78:43508) 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f5646ffd640 1 -- 172.21.15.78:0/1224300147 learned_addr learned my addr 172.21.15.78:0/1224300147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f56467fc640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c50a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f5646ffd640 1 -- 172.21.15.78:0/1224300147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 msgr2=0x5619c50a35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f5646ffd640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c50a35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.297+0000 7f5646ffd640 1 -- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c509f370 msgr2=0x5619c50a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f56477fe640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c509f370 0x5619c50a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f5646ffd640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c509f370 0x5619c50a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f5646ffd640 1 -- 172.21.15.78:0/1224300147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c5118b00 con 0x5619c50945d0 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f56467fc640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c50a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f56477fe640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c509f370 0x5619c50a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f5646ffd640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c50a3090 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f563800b370 tx=0x7f563801c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a1886a47835cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:54.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f562aff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f562aff5640 1 -- 172.21.15.78:0/1224300147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638020070 con 0x5619c50945d0 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f562aff5640 1 -- 172.21.15.78:0/1224300147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5638024940 con 0x5619c50945d0 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f564fb9d740 1 -- 172.21.15.78:0/1224300147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f563801c250 con 0x5619c50945d0 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f562aff5640 1 -- 172.21.15.78:0/1224300147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5638033050 con 0x5619c50945d0 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f564fb9d740 10 client.17133 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f564fb9d740 1 -- 172.21.15.78:0/1224300147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5619c5119dc0 con 0x5619c50945d0 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f562aff5640 1 -- 172.21.15.78:0/1224300147 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f563801c250 con 0x5619c50945d0 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f562aff5640 1 -- 172.21.15.78:0/1224300147 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5638025b00 con 0x5619c50945d0 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f564fb9d740 10 client.17133 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f564fb9d740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f564fb9d740 20 client.17133 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.298+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5619c511a470 0x5619c511c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f564fb9d740 4 client.17133 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:22:54.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f564fb9d740 1 -- 172.21.15.78:0/1224300147 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619c5001d70 con 0x5619c511a470 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f56467fc640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5619c511a470 0x5619c511c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f562aff5640 10 client.17133 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f56467fc640 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5619c511a470 0x5619c511c890 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f77168c69038e1 server_cookie=612d13c5a8277383 in_seq=0 out_seq=0 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f562aff5640 1 -- 172.21.15.78:0/1224300147 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619c5001d70 con 0x5619c511a470 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f562aff5640 10 client.17133 handle_command_reply: tid=0 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f564fb9d740 1 client.17133 shutdown 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f564fb9d740 1 -- 172.21.15.78:0/1224300147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 msgr2=0x5619c50a3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.299+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c50a3090 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f563800b370 tx=0x7f563801c7f0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 1 -- 172.21.15.78:0/1224300147 shutdown_connections 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5619c511a470 0x5619c511c890 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619c509f370 0x5619c50a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c5095b40 0x5619c50a35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 1 --2- 172.21.15.78:0/1224300147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619c50945d0 0x5619c50a3090 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 1 -- 172.21.15.78:0/1224300147 >> 172.21.15.78:0/1224300147 conn(0x5619c505d310 msgr2=0x5619c508fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f562aff5640 0 client.17133 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 1 -- 172.21.15.78:0/1224300147 shutdown_connections 2024-09-16T00:22:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 1 -- 172.21.15.78:0/1224300147 wait complete. 2024-09-16T00:22:54.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564fb9d740 20 client.17133 trim_cache size 0 max 16384 2024-09-16T00:22:54.316 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:22:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 -- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f56300a3240 msgr2=0x7f56300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f56300a3240 0x7f56300a5700 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f564000afd0 tx=0x7f5640005bd0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 -- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648101420 msgr2=0x7f5648197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:54.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648101420 0x7f5648197680 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f563c009a50 tx=0x7f563c004270 comp rx=0 tx=0).stop 2024-09-16T00:22:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 -- 172.21.15.78:0/2680735346 shutdown_connections 2024-09-16T00:22:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f56300a3240 0x7f56300a5700 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5648106a80 0x7f5648197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5648101420 0x7f5648197680 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 --2- 172.21.15.78:0/2680735346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5648100a80 0x7f5648197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:54.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.300+0000 7f564d7c4640 1 -- 172.21.15.78:0/2680735346 >> 172.21.15.78:0/2680735346 conn(0x7f56480fc7a0 msgr2=0x7f56480fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.301+0000 7f564d7c4640 1 -- 172.21.15.78:0/2680735346 shutdown_connections 2024-09-16T00:22:54.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:54.301+0000 7f564d7c4640 1 -- 172.21.15.78:0/2680735346 wait complete. 2024-09-16T00:22:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 535.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:22:56.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:55 smithi078 ceph-mon[31157]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-09-16T00:22:56.117 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:22:56.117 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:22:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:55 smithi193 ceph-mon[35484]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-09-16T00:22:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:55 smithi148 ceph-mon[35599]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-09-16T00:22:56.387 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 -- 172.21.15.78:0/563433441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 msgr2=0x7fce7010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 --2- 172.21.15.78:0/563433441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 0x7fce7010eaa0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fce68009ca0 tx=0x7fce6802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 -- 172.21.15.78:0/563433441 shutdown_connections 2024-09-16T00:22:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 --2- 172.21.15.78:0/563433441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 0x7fce7010eaa0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 --2- 172.21.15.78:0/563433441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce701027a0 0x7fce701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 --2- 172.21.15.78:0/563433441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce70101df0 0x7fce701021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 -- 172.21.15.78:0/563433441 >> 172.21.15.78:0/563433441 conn(0x7fce700fd8b0 msgr2=0x7fce700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 -- 172.21.15.78:0/563433441 shutdown_connections 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 -- 172.21.15.78:0/563433441 wait complete. 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 Processor -- start 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.893+0000 7fce7605a640 1 -- start start 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce7605a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101df0 0x7fce70198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce7605a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce701027a0 0x7fce70198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce7605a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 0x7fce7019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce7605a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce70111fa0 con 0x7fce70106f20 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce7605a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce70111e20 con 0x7fce70101df0 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce7605a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce70112120 con 0x7fce701027a0 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 0x7fce7019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 0x7fce7019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40758/0 (socket says 172.21.15.78:40758) 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 -- 172.21.15.78:0/356929016 learned_addr learned my addr 172.21.15.78:0/356929016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101df0 0x7fce70198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6effd640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce701027a0 0x7fce70198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 -- 172.21.15.78:0/356929016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce701027a0 msgr2=0x7fce70198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce701027a0 0x7fce70198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 -- 172.21.15.78:0/356929016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101df0 msgr2=0x7fce70198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101df0 0x7fce70198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 -- 172.21.15.78:0/356929016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7019d3b0 con 0x7fce70106f20 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6ffff640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 0x7fce7019ccd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fce68031a60 tx=0x7fce68031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce6cff9640 1 -- 172.21.15.78:0/356929016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68003b70 con 0x7fce70106f20 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.894+0000 7fce7605a640 1 -- 172.21.15.78:0/356929016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7019d640 con 0x7fce70106f20 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.895+0000 7fce6cff9640 1 -- 172.21.15.78:0/356929016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce68003d10 con 0x7fce70106f20 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.895+0000 7fce6cff9640 1 -- 172.21.15.78:0/356929016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68004770 con 0x7fce70106f20 2024-09-16T00:22:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.895+0000 7fce7605a640 1 -- 172.21.15.78:0/356929016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701a4f80 con 0x7fce70106f20 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.897+0000 7fce78433740 1 Processor -- start 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.897+0000 7fce6cff9640 1 -- 172.21.15.78:0/356929016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fce6802f790 con 0x7fce70106f20 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.897+0000 7fce78433740 1 -- start start 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.897+0000 7fce78433740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a526b60 0x55f21a526f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.897+0000 7fce78433740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a531800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.897+0000 7fce78433740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 0x55f21a534130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce78433740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f21a48c280 con 0x55f21a526b60 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce78433740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f21a433b10 con 0x55f21a531d40 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce78433740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f21a488240 con 0x55f21a527480 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a526b60 0x55f21a526f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a526b60 0x55f21a526f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40760/0 (socket says 172.21.15.78:40760) 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 -- 172.21.15.78:0/484717563 learned_addr learned my addr 172.21.15.78:0/484717563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a531800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 0x55f21a534130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6cff9640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce580a3240 0x7fce580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 -- 172.21.15.78:0/484717563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 msgr2=0x55f21a531800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a531800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 -- 172.21.15.78:0/484717563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 msgr2=0x55f21a534130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 0x55f21a534130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 -- 172.21.15.78:0/484717563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21a4401f0 con 0x55f21a526b60 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a531800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6cff9640 1 -- 172.21.15.78:0/356929016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fce680f0fa0 con 0x7fce70106f20 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6ffff640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce580a3240 0x7fce580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce6effd640 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a526b60 0x55f21a526f40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fce64005d90 tx=0x7fce6400c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201fa41a7a1f61c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce55ffb640 1 -- 172.21.15.78:0/484717563 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64018030 con 0x55f21a526b60 2024-09-16T00:22:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce55ffb640 1 -- 172.21.15.78:0/484717563 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce64005500 con 0x55f21a526b60 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.898+0000 7fce55ffb640 1 -- 172.21.15.78:0/484717563 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6400d7b0 con 0x55f21a526b60 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 -- 172.21.15.78:0/484717563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a526b60 msgr2=0x55f21a526f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a526b60 0x55f21a526f40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fce64005d90 tx=0x7fce6400c5e0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce6ffff640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce580a3240 0x7fce580a5700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fce68031f80 tx=0x7fce680095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 -- 172.21.15.78:0/484717563 shutdown_connections 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 0x55f21a534130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a531800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 --2- 172.21.15.78:0/484717563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a526b60 0x55f21a526f40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 -- 172.21.15.78:0/484717563 >> 172.21.15.78:0/484717563 conn(0x55f21a5230f0 msgr2=0x55f21a5234d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 -- 172.21.15.78:0/484717563 shutdown_connections 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 -- 172.21.15.78:0/484717563 wait complete. 2024-09-16T00:22:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.899+0000 7fce78433740 1 -- start start 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce78433740 1 Processor -- start 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce78433740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 0x55f21a52ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce78433740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a52f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce78433740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a531d40 0x55f21a52f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce78433740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f21a534910 con 0x55f21a531d40 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce78433740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f21a446f10 con 0x55f21a526b60 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce78433740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f21a534a90 con 0x55f21a527480 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce78433740 10 client.0 _ll_register_callbacks cb 0x55f21a40fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a531d40 0x55f21a52f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a531d40 0x55f21a52f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40772/0 (socket says 172.21.15.78:40772) 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 -- 172.21.15.78:0/3309996288 learned_addr learned my addr 172.21.15.78:0/3309996288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 0x55f21a52ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a52f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 -- 172.21.15.78:0/3309996288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 msgr2=0x55f21a52f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a52f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 -- 172.21.15.78:0/3309996288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 msgr2=0x55f21a52ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 0x55f21a52ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6effd640 1 -- 172.21.15.78:0/3309996288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21a1fd6a0 con 0x55f21a531d40 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.900+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a52f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce6effd640 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a531d40 0x55f21a52f6f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c3f0 tx=0x7fce64004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e21d75f06ff459d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce4effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce4effd640 1 -- 172.21.15.78:0/3309996288 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6400d750 con 0x55f21a531d40 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce4effd640 1 -- 172.21.15.78:0/3309996288 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fce6400d8f0 con 0x55f21a531d40 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 1 -- 172.21.15.78:0/3309996288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce64005500 con 0x55f21a531d40 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce4effd640 1 -- 172.21.15.78:0/3309996288 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fce64030050 con 0x55f21a531d40 2024-09-16T00:22:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 10 client.17145 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 1 -- 172.21.15.78:0/3309996288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f21a540550 con 0x55f21a531d40 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce4effd640 1 -- 172.21.15.78:0/3309996288 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce64005500 con 0x55f21a531d40 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce4effd640 1 -- 172.21.15.78:0/3309996288 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fce64029060 con 0x55f21a531d40 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 10 client.17145 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 20 client.17145 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f21a5419b0 0x55f21a543da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 4 client.17145 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.901+0000 7fce78433740 1 -- 172.21.15.78:0/3309996288 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f21a4997f0 con 0x55f21a5419b0 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.902+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f21a5419b0 0x55f21a543da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.902+0000 7fce4effd640 10 client.17145 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.902+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f21a5419b0 0x55f21a543da0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71505406dc6f341 server_cookie=15e565093703695c in_seq=0 out_seq=0 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce4effd640 1 -- 172.21.15.78:0/3309996288 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f21a4997f0 con 0x55f21a5419b0 2024-09-16T00:22:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce4effd640 10 client.17145 handle_command_reply: tid=0 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 client.17145 shutdown 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 -- 172.21.15.78:0/3309996288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a531d40 msgr2=0x55f21a52f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a531d40 0x55f21a52f6f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c3f0 tx=0x7fce64004dc0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 -- 172.21.15.78:0/3309996288 shutdown_connections 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f21a5419b0 0x55f21a543da0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a531d40 0x55f21a52f6f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a527480 0x55f21a52f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 --2- 172.21.15.78:0/3309996288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 0x55f21a52ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 -- 172.21.15.78:0/3309996288 >> 172.21.15.78:0/3309996288 conn(0x55f21a5230f0 msgr2=0x55f21a528e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce4effd640 0 client.17145 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 -- 172.21.15.78:0/3309996288 shutdown_connections 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 1 -- 172.21.15.78:0/3309996288 wait complete. 2024-09-16T00:22:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.903+0000 7fce78433740 20 client.17145 trim_cache size 0 max 16384 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.920+0000 7fce78433740 1 Processor -- start 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.920+0000 7fce78433740 1 -- start start 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.920+0000 7fce78433740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 0x55f21a5295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.920+0000 7fce78433740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a549560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.920+0000 7fce78433740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a531d40 0x55f21a549aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.920+0000 7fce78433740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b0d50 con 0x55f21a527480 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.920+0000 7fce78433740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b2890 con 0x55f21a526b60 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.920+0000 7fce78433740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b1420 con 0x55f21a531d40 2024-09-16T00:22:56.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a549560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a549560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40780/0 (socket says 172.21.15.78:40780) 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 -- 172.21.15.78:0/2892543342 learned_addr learned my addr 172.21.15.78:0/2892543342 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6effd640 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 0x55f21a5295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6effd640 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a531d40 0x55f21a549aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 -- 172.21.15.78:0/2892543342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a531d40 msgr2=0x55f21a549aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a531d40 0x55f21a549aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 -- 172.21.15.78:0/2892543342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 msgr2=0x55f21a5295c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 0x55f21a5295c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 -- 172.21.15.78:0/2892543342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21a5272b0 con 0x55f21a527480 2024-09-16T00:22:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6effd640 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a531d40 0x55f21a549aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a549560 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55f21a523f50 tx=0x7fce60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a26b08f1bbf947 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce557fa640 1 -- 172.21.15.78:0/2892543342 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60020070 con 0x55f21a527480 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce557fa640 1 -- 172.21.15.78:0/2892543342 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce6001c730 con 0x55f21a527480 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.921+0000 7fce557fa640 1 -- 172.21.15.78:0/2892543342 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60014870 con 0x55f21a527480 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 -- 172.21.15.78:0/2892543342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 msgr2=0x55f21a549560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a549560 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55f21a523f50 tx=0x7fce60008040 comp rx=0 tx=0).stop 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 -- 172.21.15.78:0/2892543342 shutdown_connections 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a531d40 0x55f21a549aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a549560 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 --2- 172.21.15.78:0/2892543342 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a526b60 0x55f21a5295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 -- 172.21.15.78:0/2892543342 >> 172.21.15.78:0/2892543342 conn(0x55f21a5230f0 msgr2=0x55f21a52dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 -- 172.21.15.78:0/2892543342 shutdown_connections 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 -- 172.21.15.78:0/2892543342 wait complete. 2024-09-16T00:22:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.922+0000 7fce78433740 1 -- start start 2024-09-16T00:22:56.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce78433740 1 Processor -- start 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce78433740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a526b60 0x55f21a535240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce78433740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a535780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce78433740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 0x55f21a5aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce78433740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b21c0 con 0x55f21a527480 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce78433740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fce701b1af0 con 0x55f21a531d40 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce78433740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce701aed30 con 0x55f21a526b60 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce78433740 10 client.0 _ll_register_callbacks cb 0x7fce6400cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a535780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a535780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40796/0 (socket says 172.21.15.78:40796) 2024-09-16T00:22:56.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 -- 172.21.15.78:0/3352068125 learned_addr learned my addr 172.21.15.78:0/3352068125 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6effd640 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 0x55f21a5aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6effd640 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a526b60 0x55f21a535240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 -- 172.21.15.78:0/3352068125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a526b60 msgr2=0x55f21a535240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a526b60 0x55f21a535240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 -- 172.21.15.78:0/3352068125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 msgr2=0x55f21a5aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 0x55f21a5aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 -- 172.21.15.78:0/3352068125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21a5aaa60 con 0x55f21a527480 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6effd640 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a526b60 0x55f21a535240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce6f7fe640 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a535780 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fce60022000 tx=0x7fce6001c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f98a2887d4b997d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.923+0000 7fce4effd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce4effd640 1 -- 172.21.15.78:0/3352068125 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60020040 con 0x55f21a527480 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce4effd640 1 -- 172.21.15.78:0/3352068125 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fce6001cae0 con 0x55f21a527480 2024-09-16T00:22:56.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 1 -- 172.21.15.78:0/3352068125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce6001c730 con 0x55f21a527480 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce4effd640 1 -- 172.21.15.78:0/3352068125 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fce60034050 con 0x55f21a527480 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 10 client.17151 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 1 -- 172.21.15.78:0/3352068125 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f21a5abda0 con 0x55f21a527480 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce4effd640 1 -- 172.21.15.78:0/3352068125 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce6001c730 con 0x55f21a527480 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce4effd640 1 -- 172.21.15.78:0/3352068125 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fce600256c0 con 0x55f21a527480 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 10 client.17151 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 20 client.17151 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f21a5ac3c0 0x55f21a5ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 4 client.17151 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.924+0000 7fce78433740 1 -- 172.21.15.78:0/3352068125 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f21a493d00 con 0x55f21a5ac3c0 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.925+0000 7fce6effd640 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f21a5ac3c0 0x55f21a5ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.925+0000 7fce6effd640 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f21a5ac3c0 0x55f21a5ae7b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8bb4f597fc32b2c server_cookie=fa06d4c1c4dc98ae in_seq=0 out_seq=0 2024-09-16T00:22:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.925+0000 7fce4effd640 10 client.17151 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce4effd640 1 -- 172.21.15.78:0/3352068125 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f21a493d00 con 0x55f21a5ac3c0 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce4effd640 10 client.17151 handle_command_reply: tid=0 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 client.17151 shutdown 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 -- 172.21.15.78:0/3352068125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 msgr2=0x55f21a535780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a535780 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fce60022000 tx=0x7fce6001c250 comp rx=0 tx=0).stop 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 -- 172.21.15.78:0/3352068125 shutdown_connections 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f21a5ac3c0 0x55f21a5ae7b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f21a531d40 0x55f21a5aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f21a527480 0x55f21a535780 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 --2- 172.21.15.78:0/3352068125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f21a526b60 0x55f21a535240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 -- 172.21.15.78:0/3352068125 >> 172.21.15.78:0/3352068125 conn(0x55f21a5230f0 msgr2=0x55f21a52ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce4effd640 0 client.17151 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 -- 172.21.15.78:0/3352068125 shutdown_connections 2024-09-16T00:22:56.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.926+0000 7fce78433740 1 -- 172.21.15.78:0/3352068125 wait complete. 2024-09-16T00:22:56.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce78433740 20 client.17151 trim_cache size 0 max 16384 2024-09-16T00:22:56.933 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:22:56.933 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:22:56.933 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "8f745501-b7ad-4e66-abf0-41e4b7c61e1c", 2024-09-16T00:22:56.933 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:22:56.933 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 -- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce580a3240 msgr2=0x7fce580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce580a3240 0x7fce580a5700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fce68031f80 tx=0x7fce680095c0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 -- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 msgr2=0x7fce7019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 0x7fce7019ccd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fce68031a60 tx=0x7fce68031a90 comp rx=0 tx=0).stop 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 -- 172.21.15.78:0/356929016 shutdown_connections 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fce580a3240 0x7fce580a5700 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fce70106f20 0x7fce7019ccd0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce701027a0 0x7fce70198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 --2- 172.21.15.78:0/356929016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fce70101df0 0x7fce70198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 -- 172.21.15.78:0/356929016 >> 172.21.15.78:0/356929016 conn(0x7fce700fd8b0 msgr2=0x7fce700feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 -- 172.21.15.78:0/356929016 shutdown_connections 2024-09-16T00:22:56.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:56.927+0000 7fce7605a640 1 -- 172.21.15.78:0/356929016 wait complete. 2024-09-16T00:22:57.726 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:22:57.974 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:58.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:57 smithi078 ceph-mon[31157]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 319 KiB/s wr, 52 op/s 2024-09-16T00:22:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:57 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:22:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:57 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:57 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:22:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:57 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:22:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:57 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:22:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:57 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:57 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:57 smithi193 ceph-mon[35484]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 319 KiB/s wr, 52 op/s 2024-09-16T00:22:58.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:57 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:22:58.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:57 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:57 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:22:58.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:57 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:22:58.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:57 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:22:58.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:57 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:57 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:57 smithi148 ceph-mon[35599]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 319 KiB/s wr, 52 op/s 2024-09-16T00:22:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:57 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:22:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:57 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:57 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:22:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:57 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:22:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:57 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:22:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:57 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:57 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 -- 172.21.15.78:0/94362352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 msgr2=0x7f17301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 --2- 172.21.15.78:0/94362352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 0x7f17301029b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1724009900 tx=0x7f172402a480 comp rx=0 tx=0).stop 2024-09-16T00:22:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 -- 172.21.15.78:0/94362352 shutdown_connections 2024-09-16T00:22:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 --2- 172.21.15.78:0/94362352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730102ef0 0x7f173010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 --2- 172.21.15.78:0/94362352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 0x7f17301029b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 --2- 172.21.15.78:0/94362352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1730100e20 0x7f1730101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 -- 172.21.15.78:0/94362352 >> 172.21.15.78:0/94362352 conn(0x7f17300fc7d0 msgr2=0x7f17300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 -- 172.21.15.78:0/94362352 shutdown_connections 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.483+0000 7f1737404640 1 -- 172.21.15.78:0/94362352 wait complete. 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f1737404640 1 Processor -- start 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f1737404640 1 -- start start 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f1737404640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1730100e20 0x7f1730199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f1737404640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 0x7f1730199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f1737404640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730102ef0 0x7f17301a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f1737404640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1730110fa0 con 0x7f1730102570 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f1737404640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1730110e20 con 0x7f1730102ef0 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f1737404640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1730111120 con 0x7f1730100e20 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f173499f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 0x7f1730199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f173499f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 0x7f1730199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40810/0 (socket says 172.21.15.78:40810) 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f173499f640 1 -- 172.21.15.78:0/3978018482 learned_addr learned my addr 172.21.15.78:0/3978018482 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.484+0000 7f17359a1640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730102ef0 0x7f17301a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f17351a0640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1730100e20 0x7f1730199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f173499f640 1 -- 172.21.15.78:0/3978018482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1730100e20 msgr2=0x7f1730199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f173499f640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1730100e20 0x7f1730199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f173499f640 1 -- 172.21.15.78:0/3978018482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730102ef0 msgr2=0x7f17301a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f173499f640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730102ef0 0x7f17301a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f173499f640 1 -- 172.21.15.78:0/3978018482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17301a49f0 con 0x7f1730102570 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f173499f640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 0x7f1730199970 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1724006010 tx=0x7f1724003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f172e7fc640 1 -- 172.21.15.78:0/3978018482 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172403f070 con 0x7f1730102570 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f172e7fc640 1 -- 172.21.15.78:0/3978018482 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1724031c30 con 0x7f1730102570 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f1737404640 1 -- 172.21.15.78:0/3978018482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17301a4bc0 con 0x7f1730102570 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f172e7fc640 1 -- 172.21.15.78:0/3978018482 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724035030 con 0x7f1730102570 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.485+0000 7f1737404640 1 -- 172.21.15.78:0/3978018482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301a5010 con 0x7f1730102570 2024-09-16T00:22:58.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.487+0000 7f172e7fc640 1 -- 172.21.15.78:0/3978018482 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1724042070 con 0x7f1730102570 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17397dd740 1 Processor -- start 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17397dd740 1 -- start start 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17397dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 0x55c6348c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17397dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17397dd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17397dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6348282e0 con 0x55c6348c2dc0 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17397dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c6347cfce0 con 0x55c6348c24a0 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17397dd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c6348242a0 con 0x55c6348cd680 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f172e7fc640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f17180a3240 0x7f17180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17351a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17351a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40824/0 (socket says 172.21.15.78:40824) 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17351a0640 1 -- 172.21.15.78:0/1190633811 learned_addr learned my addr 172.21.15.78:0/1190633811 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17359a1640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f17180a3240 0x7f17180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.488+0000 7f17359a1640 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 0x55c6348c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f172e7fc640 1 -- 172.21.15.78:0/3978018482 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f17240f04d0 con 0x7f1730102570 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17351a0640 1 -- 172.21.15.78:0/1190633811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 msgr2=0x55c6348cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f173499f640 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17351a0640 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17351a0640 1 -- 172.21.15.78:0/1190633811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 msgr2=0x55c6348c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17351a0640 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 0x55c6348c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17351a0640 1 -- 172.21.15.78:0/1190633811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6347b4440 con 0x55c6348c2dc0 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f173499f640 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17359a1640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f17180a3240 0x7f17180a5700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f172800bfd0 tx=0x7f172800ba60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17351a0640 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348cd140 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1720009fa0 tx=0x7f172000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d55d2abc59f87f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17177fe640 1 -- 172.21.15.78:0/1190633811 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172001a070 con 0x55c6348c2dc0 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17177fe640 1 -- 172.21.15.78:0/1190633811 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17200158f0 con 0x55c6348c2dc0 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.489+0000 7f17177fe640 1 -- 172.21.15.78:0/1190633811 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172000a8f0 con 0x55c6348c2dc0 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 -- 172.21.15.78:0/1190633811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 msgr2=0x55c6348cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348cd140 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1720009fa0 tx=0x7f172000c8b0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 -- 172.21.15.78:0/1190633811 shutdown_connections 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348cd140 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 --2- 172.21.15.78:0/1190633811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 0x55c6348c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 -- 172.21.15.78:0/1190633811 >> 172.21.15.78:0/1190633811 conn(0x55c6348bea30 msgr2=0x55c6348bee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 -- 172.21.15.78:0/1190633811 shutdown_connections 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 -- 172.21.15.78:0/1190633811 wait complete. 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 -- start start 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.490+0000 7f17397dd740 1 Processor -- start 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17397dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c24a0 0x55c6348ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17397dd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c2dc0 0x55c6348ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17397dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348cd680 0x55c6348ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17397dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c6348d0250 con 0x55c6348c24a0 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17397dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c6347e3050 con 0x55c6348cd680 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17397dd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c6348d03d0 con 0x55c6348c2dc0 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17397dd740 10 client.0 _ll_register_callbacks cb 0x55c634823160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c24a0 0x55c6348ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c24a0 0x55c6348ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40834/0 (socket says 172.21.15.78:40834) 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 -- 172.21.15.78:0/1280752328 learned_addr learned my addr 172.21.15.78:0/1280752328 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17359a1640 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348cd680 0x55c6348ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 -- 172.21.15.78:0/1280752328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c2dc0 msgr2=0x55c6348ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c2dc0 0x55c6348ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 -- 172.21.15.78:0/1280752328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348cd680 msgr2=0x55c6348ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348cd680 0x55c6348ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 -- 172.21.15.78:0/1280752328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6345993c0 con 0x55c6348c24a0 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f17351a0640 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c24a0 0x55c6348ccf30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1720018040 tx=0x7f1720015640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936f1ca779ca61b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.491+0000 7f1714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f1714ff9640 1 -- 172.21.15.78:0/1280752328 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172001a040 con 0x55c6348c24a0 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f1714ff9640 1 -- 172.21.15.78:0/1280752328 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f172000a5f0 con 0x55c6348c24a0 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 1 -- 172.21.15.78:0/1280752328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17200158f0 con 0x55c6348c24a0 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f1714ff9640 1 -- 172.21.15.78:0/1280752328 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f172002e050 con 0x55c6348c24a0 2024-09-16T00:22:58.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 10 client.17163 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 1 -- 172.21.15.78:0/1280752328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c6348dbb10 con 0x55c6348c24a0 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f1714ff9640 1 -- 172.21.15.78:0/1280752328 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17200158f0 con 0x55c6348c24a0 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f1714ff9640 1 -- 172.21.15.78:0/1280752328 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1720020570 con 0x55c6348c24a0 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 10 client.17163 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 20 client.17163 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c6348dcfd0 0x55c6348df3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 4 client.17163 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.492+0000 7f17397dd740 1 -- 172.21.15.78:0/1280752328 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c634835850 con 0x55c6348dcfd0 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.493+0000 7f173499f640 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c6348dcfd0 0x55c6348df3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.493+0000 7f173499f640 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c6348dcfd0 0x55c6348df3c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395399aa59cd68e3 server_cookie=ae48ce12a281c8cf in_seq=0 out_seq=0 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.493+0000 7f1714ff9640 10 client.17163 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:58.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f1714ff9640 1 -- 172.21.15.78:0/1280752328 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c634835850 con 0x55c6348dcfd0 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f1714ff9640 10 client.17163 handle_command_reply: tid=0 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 client.17163 shutdown 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 -- 172.21.15.78:0/1280752328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c24a0 msgr2=0x55c6348ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c24a0 0x55c6348ccf30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1720018040 tx=0x7f1720015640 comp rx=0 tx=0).stop 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 -- 172.21.15.78:0/1280752328 shutdown_connections 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c6348dcfd0 0x55c6348df3c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348cd680 0x55c6348ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c2dc0 0x55c6348ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 --2- 172.21.15.78:0/1280752328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c24a0 0x55c6348ccf30 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 -- 172.21.15.78:0/1280752328 >> 172.21.15.78:0/1280752328 conn(0x55c6348bea30 msgr2=0x55c6348c4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f1714ff9640 0 client.17163 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 -- 172.21.15.78:0/1280752328 shutdown_connections 2024-09-16T00:22:58.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 1 -- 172.21.15.78:0/1280752328 wait complete. 2024-09-16T00:22:58.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.494+0000 7f17397dd740 20 client.17163 trim_cache size 0 max 16384 2024-09-16T00:22:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17397dd740 1 Processor -- start 2024-09-16T00:22:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17397dd740 1 -- start start 2024-09-16T00:22:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17397dd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c24a0 0x55c6348d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17397dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c2dc0 0x55c6348e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17397dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348cd680 0x55c6348e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17397dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17301aeba0 con 0x55c6348cd680 2024-09-16T00:22:58.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17397dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f17301b06e0 con 0x55c6348c2dc0 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17397dd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17301af270 con 0x55c6348c24a0 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17359a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348cd680 0x55c6348e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17359a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348cd680 0x55c6348e4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40846/0 (socket says 172.21.15.78:40846) 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.511+0000 7f17359a1640 1 -- 172.21.15.78:0/3803128579 learned_addr learned my addr 172.21.15.78:0/3803128579 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f17351a0640 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c24a0 0x55c6348d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f173499f640 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c2dc0 0x55c6348e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f17359a1640 1 -- 172.21.15.78:0/3803128579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c24a0 msgr2=0x55c6348d94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f17359a1640 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c24a0 0x55c6348d94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f17359a1640 1 -- 172.21.15.78:0/3803128579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c2dc0 msgr2=0x55c6348e49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f17359a1640 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c2dc0 0x55c6348e49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f17359a1640 1 -- 172.21.15.78:0/3803128579 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6348be860 con 0x55c6348cd680 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f17351a0640 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c24a0 0x55c6348d94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f17359a1640 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348cd680 0x55c6348e4f30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f172800bdb0 tx=0x7f172801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba97f6fdb35d676 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f1716ffd640 1 -- 172.21.15.78:0/3803128579 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17280044c0 con 0x55c6348cd680 2024-09-16T00:22:58.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f1716ffd640 1 -- 172.21.15.78:0/3803128579 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172800bde0 con 0x55c6348cd680 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.512+0000 7f1716ffd640 1 -- 172.21.15.78:0/3803128579 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172802a9b0 con 0x55c6348cd680 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 -- 172.21.15.78:0/3803128579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348cd680 msgr2=0x55c6348e4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348cd680 0x55c6348e4f30 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f172800bdb0 tx=0x7f172801f000 comp rx=0 tx=0).stop 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 -- 172.21.15.78:0/3803128579 shutdown_connections 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348cd680 0x55c6348e4f30 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c2dc0 0x55c6348e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 --2- 172.21.15.78:0/3803128579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348c24a0 0x55c6348d94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 -- 172.21.15.78:0/3803128579 >> 172.21.15.78:0/3803128579 conn(0x55c6348bea30 msgr2=0x55c6348c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 -- 172.21.15.78:0/3803128579 shutdown_connections 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 -- 172.21.15.78:0/3803128579 wait complete. 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 -- start start 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.513+0000 7f17397dd740 1 Processor -- start 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17397dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 0x55c6348d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17397dd740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17397dd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17397dd740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17301b0010 con 0x55c6348c2dc0 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17397dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f17301af940 con 0x55c6348c24a0 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17397dd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17301acb80 con 0x55c6348cd680 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17397dd740 10 client.0 _ll_register_callbacks cb 0x7f1720018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348d1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40858/0 (socket says 172.21.15.78:40858) 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 -- 172.21.15.78:0/2089019128 learned_addr learned my addr 172.21.15.78:0/2089019128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17351a0640 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 0x55c6348d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17359a1640 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 -- 172.21.15.78:0/2089019128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 msgr2=0x55c6348d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 -- 172.21.15.78:0/2089019128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 msgr2=0x55c6348d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 0x55c6348d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 -- 172.21.15.78:0/2089019128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6348d5ae0 con 0x55c6348c2dc0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f173499f640 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348d1330 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1724000c00 tx=0x7f1724005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badbef2c8ef973a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f1714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f1714ff9640 1 -- 172.21.15.78:0/2089019128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724049040 con 0x55c6348c2dc0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f1714ff9640 1 -- 172.21.15.78:0/2089019128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f17240f0210 con 0x55c6348c2dc0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.514+0000 7f17397dd740 1 -- 172.21.15.78:0/2089019128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172800bde0 con 0x55c6348c2dc0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f1714ff9640 1 -- 172.21.15.78:0/2089019128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17240bce90 con 0x55c6348c2dc0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17397dd740 10 client.17169 fetch_fsmap learned FSMap version 34 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17397dd740 1 -- 172.21.15.78:0/2089019128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c634947550 con 0x55c6348c2dc0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f1714ff9640 1 -- 172.21.15.78:0/2089019128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172800bde0 con 0x55c6348c2dc0 2024-09-16T00:22:58.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f1714ff9640 1 -- 172.21.15.78:0/2089019128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f17240b8e00 con 0x55c6348c2dc0 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17397dd740 10 client.17169 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17397dd740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17397dd740 20 client.17169 populate_metadata read hostname 'smithi078' 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17397dd740 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c634947c70 0x55c63494a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17397dd740 4 client.17169 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17397dd740 1 -- 172.21.15.78:0/2089019128 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c63482fd60 con 0x55c634947c70 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.515+0000 7f17351a0640 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c634947c70 0x55c63494a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f1714ff9640 10 client.17169 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17351a0640 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c634947c70 0x55c63494a060 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82371209d270b6 server_cookie=6bf401111115b8e4 in_seq=0 out_seq=0 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f1714ff9640 1 -- 172.21.15.78:0/2089019128 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c63482fd60 con 0x55c634947c70 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f1714ff9640 10 client.17169 handle_command_reply: tid=0 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 client.17169 shutdown 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 -- 172.21.15.78:0/2089019128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 msgr2=0x55c6348d1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348d1330 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1724000c00 tx=0x7f1724005b50 comp rx=0 tx=0).stop 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 -- 172.21.15.78:0/2089019128 shutdown_connections 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c634947c70 0x55c63494a060 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6348cd680 0x55c6348d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c6348c2dc0 0x55c6348d1330 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 --2- 172.21.15.78:0/2089019128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c6348c24a0 0x55c6348d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 -- 172.21.15.78:0/2089019128 >> 172.21.15.78:0/2089019128 conn(0x55c6348bea30 msgr2=0x55c6348bd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f1714ff9640 0 client.17169 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 -- 172.21.15.78:0/2089019128 shutdown_connections 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 1 -- 172.21.15.78:0/2089019128 wait complete. 2024-09-16T00:22:58.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.516+0000 7f17397dd740 20 client.17169 trim_cache size 0 max 16384 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout: "8f745501-b7ad-4e66-abf0-41e4b7c61e1c": { 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "8f745501-b7ad-4e66-abf0-41e4b7c61e1c", 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:22:58.533 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:22:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 -- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f17180a3240 msgr2=0x7f17180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f17180a3240 0x7f17180a5700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f172800bfd0 tx=0x7f172800ba60 comp rx=0 tx=0).stop 2024-09-16T00:22:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 -- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 msgr2=0x7f1730199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 0x7f1730199970 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1724006010 tx=0x7f1724003c10 comp rx=0 tx=0).stop 2024-09-16T00:22:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 -- 172.21.15.78:0/3978018482 shutdown_connections 2024-09-16T00:22:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f17180a3240 0x7f17180a5700 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1730102ef0 0x7f17301a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1730102570 0x7f1730199970 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 --2- 172.21.15.78:0/3978018482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1730100e20 0x7f1730199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 -- 172.21.15.78:0/3978018482 >> 172.21.15.78:0/3978018482 conn(0x7f17300fc7d0 msgr2=0x7f17300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 -- 172.21.15.78:0/3978018482 shutdown_connections 2024-09-16T00:22:58.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:58.517+0000 7f1737404640 1 -- 172.21.15.78:0/3978018482 wait complete. 2024-09-16T00:22:59.161 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:22:59.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:58 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:59.185 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:58 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:59.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:58 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:22:59.358 INFO:tasks.cephfs.filesystem:scrub status for tag:8f745501-b7ad-4e66-abf0-41e4b7c61e1c - {'path': '/', 'tag': '8f745501-b7ad-4e66-abf0-41e4b7c61e1c', 'options': 'recursive,force'} 2024-09-16T00:22:59.413 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:22:59.870 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:22:59 smithi078 ceph-mon[31157]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 -- 172.21.15.78:0/1234276847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48100a80 msgr2=0x7fed48100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 --2- 172.21.15.78:0/1234276847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48100a80 0x7fed48100ee0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fed38009cd0 tx=0x7fed3802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 -- 172.21.15.78:0/1234276847 shutdown_connections 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 --2- 172.21.15.78:0/1234276847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed48101420 0x7fed4810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 --2- 172.21.15.78:0/1234276847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48100a80 0x7fed48100ee0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 --2- 172.21.15.78:0/1234276847 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed48106a80 0x7fed48106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 -- 172.21.15.78:0/1234276847 >> 172.21.15.78:0/1234276847 conn(0x7fed480fc7a0 msgr2=0x7fed480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 -- 172.21.15.78:0/1234276847 shutdown_connections 2024-09-16T00:22:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 -- 172.21.15.78:0/1234276847 wait complete. 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 Processor -- start 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.949+0000 7fed4ef95640 1 -- start start 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4ef95640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed48100a80 0x7fed480fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4ef95640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48101420 0x7fed480fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4ef95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed48106a80 0x7fed480fd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4ef95640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed48110e80 con 0x7fed48101420 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4ef95640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fed48110d00 con 0x7fed48106a80 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4ef95640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fed48111000 con 0x7fed48100a80 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48101420 0x7fed480fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48101420 0x7fed480fd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40866/0 (socket says 172.21.15.78:40866) 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 -- 172.21.15.78:0/2408774001 learned_addr learned my addr 172.21.15.78:0/2408774001 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4d532640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed48106a80 0x7fed480fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4cd31640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed48100a80 0x7fed480fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 -- 172.21.15.78:0/2408774001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed48100a80 msgr2=0x7fed480fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed48100a80 0x7fed480fccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 -- 172.21.15.78:0/2408774001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed48106a80 msgr2=0x7fed480fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed48106a80 0x7fed480fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 -- 172.21.15.78:0/2408774001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed481a68d0 con 0x7fed48101420 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed4cd31640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed48100a80 0x7fed480fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed47fff640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48101420 0x7fed480fd230 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fed3802ae00 tx=0x7fed38003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.950+0000 7fed45ffb640 1 -- 172.21.15.78:0/2408774001 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38004240 con 0x7fed48101420 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.951+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed481a6b60 con 0x7fed48101420 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.951+0000 7fed45ffb640 1 -- 172.21.15.78:0/2408774001 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed380305a0 con 0x7fed48101420 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.951+0000 7fed45ffb640 1 -- 172.21.15.78:0/2408774001 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3803a660 con 0x7fed48101420 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.951+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed481a70f0 con 0x7fed48101420 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.952+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed48100ee0 con 0x7fed48101420 2024-09-16T00:22:59.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.953+0000 7fed45ffb640 1 -- 172.21.15.78:0/2408774001 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fed38030ac0 con 0x7fed48101420 2024-09-16T00:22:59.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.953+0000 7fed45ffb640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed340a3240 0x7fed340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:59.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.954+0000 7fed45ffb640 1 -- 172.21.15.78:0/2408774001 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fed3802e030 con 0x7fed48101420 2024-09-16T00:22:59.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.954+0000 7fed4cd31640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed340a3240 0x7fed340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:59.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.954+0000 7fed4cd31640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed340a3240 0x7fed340a5700 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fed40005e10 tx=0x7fed40005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:59.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:22:59.957+0000 7fed45ffb640 1 -- 172.21.15.78:0/2408774001 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed380098b0 con 0x7fed48101420 2024-09-16T00:23:00.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.158+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed4810a050 con 0x7fed48101420 2024-09-16T00:23:00.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.159+0000 7fed45ffb640 1 -- 172.21.15.78:0/2408774001 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fed380bdec0 con 0x7fed48101420 2024-09-16T00:23:00.160 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:23:00.160 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:23:00.160 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:23:00.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed340a3240 msgr2=0x7fed340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:00.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed340a3240 0x7fed340a5700 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fed40005e10 tx=0x7fed40005da0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48101420 msgr2=0x7fed480fd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:00.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48101420 0x7fed480fd230 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fed3802ae00 tx=0x7fed38003ba0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 shutdown_connections 2024-09-16T00:23:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fed340a3240 0x7fed340a5700 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fed48106a80 0x7fed480fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed48101420 0x7fed480fd230 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 --2- 172.21.15.78:0/2408774001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed48100a80 0x7fed480fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.161+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 >> 172.21.15.78:0/2408774001 conn(0x7fed480fc7a0 msgr2=0x7fed4806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.162+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 shutdown_connections 2024-09-16T00:23:00.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:00.162+0000 7fed4ef95640 1 -- 172.21.15.78:0/2408774001 wait complete. 2024-09-16T00:23:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:22:59 smithi193 ceph-mon[35484]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-09-16T00:23:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:22:59 smithi148 ceph-mon[35599]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 60 KiB/s wr, 17 op/s 2024-09-16T00:23:00.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 540.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:23:00.953 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 9e4e6ded --release --await' 2024-09-16T00:23:00.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 9e4e6ded --release --await 2024-09-16T00:23:01.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:00 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2408774001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:01.202 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:01.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:00 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2408774001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:01.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:00 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2408774001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 -- 172.21.15.78:0/1263795017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4102790 msgr2=0x7f9eb41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 --2- 172.21.15.78:0/1263795017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4102790 0x7f9eb41067e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009900 tx=0x7f9ea402a410 comp rx=0 tx=0).stop 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 -- 172.21.15.78:0/1263795017 shutdown_connections 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 --2- 172.21.15.78:0/1263795017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb4106f10 0x7f9eb410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 --2- 172.21.15.78:0/1263795017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4102790 0x7f9eb41067e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 --2- 172.21.15.78:0/1263795017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb4101de0 0x7f9eb41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 -- 172.21.15.78:0/1263795017 >> 172.21.15.78:0/1263795017 conn(0x7f9eb40fd8c0 msgr2=0x7f9eb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 -- 172.21.15.78:0/1263795017 shutdown_connections 2024-09-16T00:23:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 -- 172.21.15.78:0/1263795017 wait complete. 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 Processor -- start 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.717+0000 7f9ebb300640 1 -- start start 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9ebb300640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb4101de0 0x7f9eb4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9ebb300640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb4102790 0x7f9eb41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9ebb300640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9ebb300640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4111f30 con 0x7f9eb4106f10 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9ebb300640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4111db0 con 0x7f9eb4101de0 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9ebb300640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41120b0 con 0x7f9eb4102790 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40878/0 (socket says 172.21.15.78:40878) 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 -- 172.21.15.78:0/3596353633 learned_addr learned my addr 172.21.15.78:0/3596353633 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb909c640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb4101de0 0x7f9eb4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb4102790 0x7f9eb41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 -- 172.21.15.78:0/3596353633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb4102790 msgr2=0x7f9eb41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb4102790 0x7f9eb41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 -- 172.21.15.78:0/3596353633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb4101de0 msgr2=0x7f9eb4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb4101de0 0x7f9eb4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 -- 172.21.15.78:0/3596353633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb419d440 con 0x7f9eb4106f10 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eb989d640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cd60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00eaa0 tx=0x7f9eac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eaa7fc640 1 -- 172.21.15.78:0/3596353633 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00cdb0 con 0x7f9eb4106f10 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eaa7fc640 1 -- 172.21.15.78:0/3596353633 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eac005020 con 0x7f9eb4106f10 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9eaa7fc640 1 -- 172.21.15.78:0/3596353633 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0116b0 con 0x7f9eb4106f10 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9ebb300640 1 -- 172.21.15.78:0/3596353633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb419d730 con 0x7f9eb4106f10 2024-09-16T00:23:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.718+0000 7f9ebb300640 1 -- 172.21.15.78:0/3596353633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb41a5010 con 0x7f9eb4106f10 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9eaa7fc640 1 -- 172.21.15.78:0/3596353633 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9eac005560 con 0x7f9eb4106f10 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9ebd6d9740 1 Processor -- start 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9ebd6d9740 1 -- start start 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 0x5581850b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55818501b620 con 0x5581850c5650 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558184fc2e50 con 0x5581850b6840 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.721+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581850175e0 con 0x5581850b5f20 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40882/0 (socket says 172.21.15.78:40882) 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 -- 172.21.15.78:0/3972828945 learned_addr learned my addr 172.21.15.78:0/3972828945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 0x5581850b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb909c640 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eaa7fc640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ea00a3240 0x7f9ea00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb989d640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ea00a3240 0x7f9ea00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 -- 172.21.15.78:0/3972828945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 msgr2=0x5581850b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 0x5581850b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 -- 172.21.15.78:0/3972828945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 msgr2=0x5581850c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 -- 172.21.15.78:0/3972828945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558184fb3d40 con 0x5581850c5650 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eaa7fc640 1 -- 172.21.15.78:0/3596353633 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9eac0cfdb0 con 0x7f9eb4106f10 2024-09-16T00:23:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c7a40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4031550 tx=0x7f9ea40314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df413cf2eb15945 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9e9f7fe640 1 -- 172.21.15.78:0/3972828945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4035070 con 0x5581850c5650 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9e9f7fe640 1 -- 172.21.15.78:0/3972828945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea4005020 con 0x5581850c5650 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.722+0000 7f9e9f7fe640 1 -- 172.21.15.78:0/3972828945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4004a00 con 0x5581850c5650 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3972828945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 msgr2=0x5581850c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c7a40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4031550 tx=0x7f9ea40314e0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9eb989d640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ea00a3240 0x7f9ea00a5700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cd8c0 tx=0x7f9eac0c6e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3972828945 shutdown_connections 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c7a40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/3972828945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 0x5581850b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3972828945 >> 172.21.15.78:0/3972828945 conn(0x5581850b25f0 msgr2=0x5581850b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3972828945 shutdown_connections 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3972828945 wait complete. 2024-09-16T00:23:01.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.723+0000 7f9ebd6d9740 1 -- start start 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9ebd6d9740 1 Processor -- start 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b5f20 0x5581850c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b6840 0x5581850c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 0x5581850c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581850c81c0 con 0x5581850b6840 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558184fa3f10 con 0x5581850b5f20 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581850c8340 con 0x5581850c5650 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9ebd6d9740 10 client.0 _ll_register_callbacks cb 0x558184fc7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb909c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b6840 0x5581850c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb909c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b6840 0x5581850c2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40896/0 (socket says 172.21.15.78:40896) 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb909c640 1 -- 172.21.15.78:0/2889537883 learned_addr learned my addr 172.21.15.78:0/2889537883 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb889b640 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 0x5581850c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb909c640 1 -- 172.21.15.78:0/2889537883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 msgr2=0x5581850c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb909c640 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 0x5581850c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb889b640 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b5f20 0x5581850c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb909c640 1 -- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b5f20 msgr2=0x5581850c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb909c640 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b5f20 0x5581850c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb909c640 1 -- 172.21.15.78:0/2889537883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558184d2a580 con 0x5581850b6840 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.724+0000 7f9eb889b640 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b5f20 0x5581850c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9eb909c640 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b6840 0x5581850c2b80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0011440 tx=0x7f9eb0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55faeb3140e3af9a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9e9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9e9cff9640 1 -- 172.21.15.78:0/2889537883 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0005020 con 0x5581850b6840 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9e9cff9640 1 -- 172.21.15.78:0/2889537883 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9eb00051c0 con 0x5581850b6840 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/2889537883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea4005020 con 0x5581850b6840 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9e9cff9640 1 -- 172.21.15.78:0/2889537883 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9eb0025050 con 0x5581850b6840 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 10 client.17187 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/2889537883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558184df4960 con 0x5581850b6840 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9e9cff9640 1 -- 172.21.15.78:0/2889537883 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea4005020 con 0x5581850b6840 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9e9cff9640 1 -- 172.21.15.78:0/2889537883 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9eb0010850 con 0x5581850b6840 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 10 client.17187 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 10 client.17187 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 20 client.17187 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581850d5300 0x5581850d7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 4 client.17187 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.725+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/2889537883 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558185028b90 con 0x5581850d5300 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.726+0000 7f9eb889b640 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581850d5300 0x5581850d7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.726+0000 7f9e9cff9640 10 client.17187 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.726+0000 7f9eb889b640 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581850d5300 0x5581850d7780 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd9d8b28edc3048a server_cookie=59106cf42681fe22 in_seq=0 out_seq=0 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9e9cff9640 1 -- 172.21.15.78:0/2889537883 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558185028b90 con 0x5581850d5300 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9e9cff9640 10 client.17187 handle_command_reply: tid=0 2024-09-16T00:23:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 client.17187 shutdown 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/2889537883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b6840 msgr2=0x5581850c2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b6840 0x5581850c2b80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0011440 tx=0x7f9eb0011900 comp rx=0 tx=0).stop 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/2889537883 shutdown_connections 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581850d5300 0x5581850d7780 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 0x5581850c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b6840 0x5581850c2b80 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/2889537883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b5f20 0x5581850c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9e9cff9640 0 client.17187 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/2889537883 >> 172.21.15.78:0/2889537883 conn(0x5581850b25f0 msgr2=0x5581850b8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/2889537883 shutdown_connections 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/2889537883 wait complete. 2024-09-16T00:23:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.727+0000 7f9ebd6d9740 20 client.17187 trim_cache size 0 max 16384 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9ebd6d9740 1 Processor -- start 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9ebd6d9740 1 -- start start 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b5f20 0x5581850dd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850dd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 0x5581850bf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41aca00 con 0x5581850b5f20 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41ae540 con 0x5581850b6840 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41ad0d0 con 0x5581850c5650 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9eb889b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b5f20 0x5581850dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9eb889b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b5f20 0x5581850dd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40908/0 (socket says 172.21.15.78:40908) 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9eb889b640 1 -- 172.21.15.78:0/3142366086 learned_addr learned my addr 172.21.15.78:0/3142366086 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9eb909c640 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 0x5581850bf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9eb889b640 1 -- 172.21.15.78:0/3142366086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 msgr2=0x5581850bf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.745+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 0x5581850bf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.746+0000 7f9eb889b640 1 -- 172.21.15.78:0/3142366086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 msgr2=0x5581850dd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.746+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850dd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.746+0000 7f9eb889b640 1 -- 172.21.15.78:0/3142366086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581850b8a60 con 0x5581850b5f20 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.746+0000 7f9eb909c640 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.746+0000 7f9eb889b640 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b5f20 0x5581850dd050 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4031770 tx=0x7f9ea40314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2ecfcae5e0518f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.746+0000 7f9e9effd640 1 -- 172.21.15.78:0/3142366086 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4035040 con 0x5581850b5f20 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.746+0000 7f9e9effd640 1 -- 172.21.15.78:0/3142366086 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea4005520 con 0x5581850b5f20 2024-09-16T00:23:01.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.746+0000 7f9e9effd640 1 -- 172.21.15.78:0/3142366086 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4004a40 con 0x5581850b5f20 2024-09-16T00:23:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3142366086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b5f20 msgr2=0x5581850dd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b5f20 0x5581850dd050 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4031770 tx=0x7f9ea40314e0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3142366086 shutdown_connections 2024-09-16T00:23:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850c5650 0x5581850bf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/3142366086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850b5f20 0x5581850dd050 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3142366086 >> 172.21.15.78:0/3142366086 conn(0x5581850b25f0 msgr2=0x5581850c46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3142366086 shutdown_connections 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/3142366086 wait complete. 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 -- start start 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 Processor -- start 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.747+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 0x5581850c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9ebd6d9740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41ade70 con 0x5581850c5650 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41ad7a0 con 0x5581850b6840 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9ebd6d9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb41aa9e0 con 0x5581850b5f20 2024-09-16T00:23:01.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9ebd6d9740 10 client.0 _ll_register_callbacks cb 0x7f9eb000bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40918/0 (socket says 172.21.15.78:40918) 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 -- 172.21.15.78:0/763569695 learned_addr learned my addr 172.21.15.78:0/763569695 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb909c640 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 0x5581850c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 -- 172.21.15.78:0/763569695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 msgr2=0x5581850c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 0x5581850c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 -- 172.21.15.78:0/763569695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 msgr2=0x5581850c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 -- 172.21.15.78:0/763569695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55818513f180 con 0x5581850c5650 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9eb889b640 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c9f00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9ea402a930 tx=0x7f9ea4057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cae8bb81bf53280 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9e9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9e9cff9640 1 -- 172.21.15.78:0/763569695 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4035040 con 0x5581850c5650 2024-09-16T00:23:01.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9e9cff9640 1 -- 172.21.15.78:0/763569695 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9ea4004750 con 0x5581850c5650 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.748+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/763569695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea4005520 con 0x5581850c5650 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9e9cff9640 1 -- 172.21.15.78:0/763569695 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9ea4004950 con 0x5581850c5650 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9ebd6d9740 10 client.17193 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/763569695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558185140480 con 0x5581850c5650 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9e9cff9640 1 -- 172.21.15.78:0/763569695 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea4005520 con 0x5581850c5650 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9e9cff9640 1 -- 172.21.15.78:0/763569695 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9ea4061c40 con 0x5581850c5650 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9ebd6d9740 10 client.17193 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9ebd6d9740 10 client.17193 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9ebd6d9740 20 client.17193 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558185140700 0x558185142af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9ebd6d9740 4 client.17193 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9e4e6ded", "release": true, "await": true}] 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/763569695 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9e4e6ded", "release": true, "await": true}) -- 0x5581850230a0 con 0x558185140700 2024-09-16T00:23:01.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.749+0000 7f9eb909c640 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558185140700 0x558185142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:01.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.750+0000 7f9e9cff9640 10 client.17193 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:01.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.750+0000 7f9eb909c640 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558185140700 0x558185142af0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05a1961550ba0e4 server_cookie=9fa39a67dd50495b in_seq=0 out_seq=0 2024-09-16T00:23:01.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.794+0000 7f9e9cff9640 1 -- 172.21.15.78:0/763569695 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5581850230a0 con 0x558185140700 2024-09-16T00:23:01.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.794+0000 7f9e9cff9640 10 client.17193 handle_command_reply: tid=0 2024-09-16T00:23:01.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.794+0000 7f9ebd6d9740 1 client.17193 shutdown 2024-09-16T00:23:01.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.795+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/763569695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 msgr2=0x5581850c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.795+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c9f00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9ea402a930 tx=0x7f9ea4057040 comp rx=0 tx=0).stop 2024-09-16T00:23:01.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.795+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/763569695 shutdown_connections 2024-09-16T00:23:01.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.795+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558185140700 0x558185142af0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.795+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581850c5650 0x5581850c9f00 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.795+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581850b6840 0x5581850c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.796+0000 7f9ebd6d9740 1 --2- 172.21.15.78:0/763569695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581850b5f20 0x5581850c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.796+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/763569695 >> 172.21.15.78:0/763569695 conn(0x5581850b25f0 msgr2=0x5581850c46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:01.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.797+0000 7f9e9cff9640 0 client.17193 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:01.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.798+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/763569695 shutdown_connections 2024-09-16T00:23:01.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.799+0000 7f9ebd6d9740 1 -- 172.21.15.78:0/763569695 wait complete. 2024-09-16T00:23:01.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.799+0000 7f9ebd6d9740 20 client.17193 trim_cache size 0 max 16384 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 36, 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "9e4e6ded": { 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 36, 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:23:01.800 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 142.3, 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:23:01.801 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:23:01.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 -- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ea00a3240 msgr2=0x7f9ea00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ea00a3240 0x7f9ea00a5700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cd8c0 tx=0x7f9eac0c6e40 comp rx=0 tx=0).stop 2024-09-16T00:23:01.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 -- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4106f10 msgr2=0x7f9eb419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:01.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cd60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9eac00eaa0 tx=0x7f9eac00ef60 comp rx=0 tx=0).stop 2024-09-16T00:23:01.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 -- 172.21.15.78:0/3596353633 shutdown_connections 2024-09-16T00:23:01.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ea00a3240 0x7f9ea00a5700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9eb4106f10 0x7f9eb419cd60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb4102790 0x7f9eb41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 --2- 172.21.15.78:0/3596353633 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9eb4101de0 0x7f9eb4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:01.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.800+0000 7f9ebb300640 1 -- 172.21.15.78:0/3596353633 >> 172.21.15.78:0/3596353633 conn(0x7f9eb40fd8c0 msgr2=0x7f9eb40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:01.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.803+0000 7f9ebb300640 1 -- 172.21.15.78:0/3596353633 shutdown_connections 2024-09-16T00:23:01.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:01.803+0000 7f9ebb300640 1 -- 172.21.15.78:0/3596353633 wait complete. 2024-09-16T00:23:02.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:01 smithi078 ceph-mon[31157]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-16T00:23:02.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:01 smithi193 ceph-mon[35484]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-16T00:23:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:01 smithi148 ceph-mon[35599]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-16T00:23:02.623 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9e4e6ded', total seconds elapsed: 25.8 2024-09-16T00:23:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:03 smithi193 ceph-mon[35484]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-16T00:23:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:03 smithi148 ceph-mon[35599]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-16T00:23:04.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:03 smithi078 ceph-mon[31157]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-09-16T00:23:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 545.0 s, 810.6 tps, lat 574.596 ms stddev 3001.907 2024-09-16T00:23:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:05 smithi193 ceph-mon[35484]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 402 KiB/s wr, 44 op/s 2024-09-16T00:23:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:05 smithi148 ceph-mon[35599]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 402 KiB/s wr, 44 op/s 2024-09-16T00:23:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:05 smithi078 ceph-mon[31157]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 402 KiB/s wr, 44 op/s 2024-09-16T00:23:08.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:07 smithi193 ceph-mon[35484]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-16T00:23:08.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:07 smithi148 ceph-mon[35599]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-16T00:23:08.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:07 smithi078 ceph-mon[31157]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-16T00:23:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:09 smithi193 ceph-mon[35484]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.7 MiB/s wr, 308 op/s 2024-09-16T00:23:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:09 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:23:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:09 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:23:10.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:09 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:23:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:09 smithi148 ceph-mon[35599]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.7 MiB/s wr, 308 op/s 2024-09-16T00:23:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:09 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:23:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:09 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:23:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:09 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:23:10.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:09 smithi078 ceph-mon[31157]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 2.7 MiB/s wr, 308 op/s 2024-09-16T00:23:10.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:09 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:23:10.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:09 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:23:10.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:09 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:23:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 550.0 s, 945.2 tps, lat 104.888 ms stddev 130.720 2024-09-16T00:23:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:11 smithi193 ceph-mon[35484]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.1 MiB/s wr, 492 op/s 2024-09-16T00:23:12.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:11 smithi148 ceph-mon[35599]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.1 MiB/s wr, 492 op/s 2024-09-16T00:23:12.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:11 smithi078 ceph-mon[31157]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.1 MiB/s wr, 492 op/s 2024-09-16T00:23:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:13 smithi193 ceph-mon[35484]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.8 MiB/s wr, 672 op/s 2024-09-16T00:23:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:13 smithi148 ceph-mon[35599]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.8 MiB/s wr, 672 op/s 2024-09-16T00:23:14.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:13 smithi078 ceph-mon[31157]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.8 MiB/s wr, 672 op/s 2024-09-16T00:23:15.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 555.0 s, 760.0 tps, lat 130.198 ms stddev 137.867 2024-09-16T00:23:16.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:15 smithi193 ceph-mon[35484]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.4 MiB/s wr, 746 op/s 2024-09-16T00:23:16.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:15 smithi148 ceph-mon[35599]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.4 MiB/s wr, 746 op/s 2024-09-16T00:23:16.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:15 smithi078 ceph-mon[31157]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.4 MiB/s wr, 746 op/s 2024-09-16T00:23:18.145 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:17 smithi193 ceph-mon[35484]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.6 MiB/s wr, 792 op/s 2024-09-16T00:23:18.145 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:17 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:18.170 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:17 smithi148 ceph-mon[35599]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.6 MiB/s wr, 792 op/s 2024-09-16T00:23:18.170 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:17 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:17 smithi078 ceph-mon[31157]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.6 MiB/s wr, 792 op/s 2024-09-16T00:23:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:17 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:19 smithi193 ceph-mon[35484]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-09-16T00:23:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:19 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:19 smithi148 ceph-mon[35599]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-09-16T00:23:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:19 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:19 smithi078 ceph-mon[31157]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.8 MiB/s wr, 794 op/s 2024-09-16T00:23:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:19 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 560.0 s, 696.0 tps, lat 144.906 ms stddev 162.165 2024-09-16T00:23:21.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:20 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:21.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:20 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:21.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:20 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:21.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:20 smithi193 ceph-mon[35484]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.3 MiB/s wr, 726 op/s 2024-09-16T00:23:21.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:20 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:21.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:20 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:21.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:20 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:21.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:20 smithi148 ceph-mon[35599]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.3 MiB/s wr, 726 op/s 2024-09-16T00:23:21.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:20 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:21.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:20 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:21.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:20 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:23:21.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:20 smithi078 ceph-mon[31157]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.3 MiB/s wr, 726 op/s 2024-09-16T00:23:24.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:23 smithi078 ceph-mon[31157]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 693 op/s 2024-09-16T00:23:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:23 smithi193 ceph-mon[35484]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 693 op/s 2024-09-16T00:23:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:23 smithi148 ceph-mon[35599]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.0 MiB/s wr, 693 op/s 2024-09-16T00:23:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 565.0 s, 667.8 tps, lat 150.623 ms stddev 186.670 2024-09-16T00:23:26.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:25 smithi078 ceph-mon[31157]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 MiB/s wr, 628 op/s 2024-09-16T00:23:26.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:25 smithi193 ceph-mon[35484]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 MiB/s wr, 628 op/s 2024-09-16T00:23:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:25 smithi148 ceph-mon[35599]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 MiB/s wr, 628 op/s 2024-09-16T00:23:28.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:27 smithi078 ceph-mon[31157]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.7 MiB/s wr, 678 op/s 2024-09-16T00:23:28.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:27 smithi193 ceph-mon[35484]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.7 MiB/s wr, 678 op/s 2024-09-16T00:23:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:27 smithi148 ceph-mon[35599]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.7 MiB/s wr, 678 op/s 2024-09-16T00:23:29.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:23:29.771 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:29 smithi078 ceph-mon[31157]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 721 op/s 2024-09-16T00:23:30.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:29 smithi193 ceph-mon[35484]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 721 op/s 2024-09-16T00:23:30.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:29 smithi148 ceph-mon[35599]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 MiB/s wr, 721 op/s 2024-09-16T00:23:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 570.0 s, 624.2 tps, lat 158.547 ms stddev 202.497 2024-09-16T00:23:31.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.264+0000 7ff927958640 1 -- 172.21.15.78:0/3994517880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff92010ea80 msgr2=0x7ff9201154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.264+0000 7ff927958640 1 --2- 172.21.15.78:0/3994517880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff92010ea80 0x7ff9201154f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff910009860 tx=0x7ff91002a340 comp rx=0 tx=0).stop 2024-09-16T00:23:31.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.265+0000 7ff927958640 1 -- 172.21.15.78:0/3994517880 shutdown_connections 2024-09-16T00:23:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.265+0000 7ff927958640 1 --2- 172.21.15.78:0/3994517880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff92010ea80 0x7ff9201154f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.265+0000 7ff927958640 1 --2- 172.21.15.78:0/3994517880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff920109e00 0x7ff92010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.265+0000 7ff927958640 1 --2- 172.21.15.78:0/3994517880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff920109450 0x7ff920109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.265+0000 7ff927958640 1 -- 172.21.15.78:0/3994517880 >> 172.21.15.78:0/3994517880 conn(0x7ff92006e2d0 msgr2=0x7ff92006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:31.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.266+0000 7ff927958640 1 -- 172.21.15.78:0/3994517880 shutdown_connections 2024-09-16T00:23:31.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.269+0000 7ff927958640 1 -- 172.21.15.78:0/3994517880 wait complete. 2024-09-16T00:23:31.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.270+0000 7ff927958640 1 Processor -- start 2024-09-16T00:23:31.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.277+0000 7ff927958640 1 -- start start 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.277+0000 7ff927958640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff920109450 0x7ff920113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.277+0000 7ff927958640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff920109e00 0x7ff9201144a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff927958640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff92010ea80 0x7ff9201149e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff9256f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff920109450 0x7ff920113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff9256f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff920109450 0x7ff920113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:39374/0 (socket says 172.21.15.78:39374) 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff9256f4640 1 -- 172.21.15.78:0/3096636682 learned_addr learned my addr 172.21.15.78:0/3096636682 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff925ef5640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff92010ea80 0x7ff9201149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff924ef3640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff920109e00 0x7ff9201144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff920117bc0 con 0x7ff92010ea80 2024-09-16T00:23:31.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff920117a40 con 0x7ff920109e00 2024-09-16T00:23:31.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.278+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff920117d40 con 0x7ff920109450 2024-09-16T00:23:31.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.283+0000 7ff925ef5640 1 -- 172.21.15.78:0/3096636682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff920109450 msgr2=0x7ff920113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.283+0000 7ff925ef5640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff920109450 0x7ff920113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.283+0000 7ff925ef5640 1 -- 172.21.15.78:0/3096636682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff920109e00 msgr2=0x7ff9201144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.283+0000 7ff925ef5640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff920109e00 0x7ff9201144a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.283+0000 7ff925ef5640 1 -- 172.21.15.78:0/3096636682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9201bbcb0 con 0x7ff92010ea80 2024-09-16T00:23:31.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.284+0000 7ff9256f4640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff920109450 0x7ff920113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:31.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.284+0000 7ff925ef5640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff92010ea80 0x7ff9201149e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff91002a860 tx=0x7ff910033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:31.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.288+0000 7ff9167fc640 1 -- 172.21.15.78:0/3096636682 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910037040 con 0x7ff92010ea80 2024-09-16T00:23:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.288+0000 7ff9167fc640 1 -- 172.21.15.78:0/3096636682 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff910030550 con 0x7ff92010ea80 2024-09-16T00:23:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.288+0000 7ff9167fc640 1 -- 172.21.15.78:0/3096636682 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910005410 con 0x7ff92010ea80 2024-09-16T00:23:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.288+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9201bbf40 con 0x7ff92010ea80 2024-09-16T00:23:31.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.288+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9201bc2e0 con 0x7ff92010ea80 2024-09-16T00:23:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.292+0000 7ff9167fc640 1 -- 172.21.15.78:0/3096636682 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff910042020 con 0x7ff92010ea80 2024-09-16T00:23:31.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.293+0000 7ff9167fc640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff90c0a3390 0x7ff90c0a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.293+0000 7ff9167fc640 1 -- 172.21.15.78:0/3096636682 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff9100b9650 con 0x7ff92010ea80 2024-09-16T00:23:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.294+0000 7ff9256f4640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff90c0a3390 0x7ff90c0a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.295+0000 7ff929d31740 1 Processor -- start 2024-09-16T00:23:31.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.296+0000 7ff929d31740 1 -- start start 2024-09-16T00:23:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.296+0000 7ff929d31740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb35f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.296+0000 7ff929d31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.296+0000 7ff929d31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb36ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.296+0000 7ff929d31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4fb2c52e0 con 0x55f4fb36a680 2024-09-16T00:23:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.296+0000 7ff929d31740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f4fb26cce0 con 0x55f4fb35f4a0 2024-09-16T00:23:31.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.296+0000 7ff929d31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f4fb2c12a0 con 0x55f4fb35fdc0 2024-09-16T00:23:31.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.299+0000 7ff925ef5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb36ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.299+0000 7ff925ef5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb36ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58920/0 (socket says 172.21.15.78:58920) 2024-09-16T00:23:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.299+0000 7ff924ef3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.299+0000 7ff924ef3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:39380/0 (socket says 172.21.15.78:39380) 2024-09-16T00:23:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.299+0000 7ff924ef3640 1 -- 172.21.15.78:0/2604491523 learned_addr learned my addr 172.21.15.78:0/2604491523 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:31.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.299+0000 7ff9256f4640 1 --2- 172.21.15.78:0/2604491523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb35f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.300+0000 7ff925ef5640 1 -- 172.21.15.78:0/2604491523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 msgr2=0x55f4fb36a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.300+0000 7ff925ef5640 1 --2- 172.21.15.78:0/2604491523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.300+0000 7ff925ef5640 1 -- 172.21.15.78:0/2604491523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 msgr2=0x55f4fb35f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.300+0000 7ff925ef5640 1 --2- 172.21.15.78:0/2604491523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb35f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.300+0000 7ff925ef5640 1 -- 172.21.15.78:0/2604491523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4fb251440 con 0x55f4fb36a680 2024-09-16T00:23:31.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.300+0000 7ff925ef5640 1 --2- 172.21.15.78:0/2604491523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb36ca70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff910043000 tx=0x7ff91002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f28bdb6daf2b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:31.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.303+0000 7ff90b7fe640 1 -- 172.21.15.78:0/2604491523 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100bb5b0 con 0x55f4fb36a680 2024-09-16T00:23:31.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.303+0000 7ff90b7fe640 1 -- 172.21.15.78:0/2604491523 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9100bf960 con 0x55f4fb36a680 2024-09-16T00:23:31.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.304+0000 7ff929d31740 1 -- 172.21.15.78:0/2604491523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 msgr2=0x55f4fb36ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.304+0000 7ff929d31740 1 --2- 172.21.15.78:0/2604491523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb36ca70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff910043000 tx=0x7ff91002a340 comp rx=0 tx=0).stop 2024-09-16T00:23:31.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.305+0000 7ff929d31740 1 -- 172.21.15.78:0/2604491523 shutdown_connections 2024-09-16T00:23:31.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.305+0000 7ff929d31740 1 --2- 172.21.15.78:0/2604491523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb36ca70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.305+0000 7ff929d31740 1 --2- 172.21.15.78:0/2604491523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.305+0000 7ff929d31740 1 --2- 172.21.15.78:0/2604491523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb35f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.305+0000 7ff929d31740 1 -- 172.21.15.78:0/2604491523 >> 172.21.15.78:0/2604491523 conn(0x55f4fb35ba30 msgr2=0x55f4fb35be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.308+0000 7ff929d31740 1 -- 172.21.15.78:0/2604491523 shutdown_connections 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.308+0000 7ff929d31740 1 -- 172.21.15.78:0/2604491523 wait complete. 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.309+0000 7ff929d31740 1 -- start start 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.309+0000 7ff929d31740 1 Processor -- start 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.309+0000 7ff929d31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.309+0000 7ff929d31740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 0x55f4fb3672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.309+0000 7ff929d31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb36a680 0x55f4fb3678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.309+0000 7ff929d31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4fb36d010 con 0x55f4fb35f4a0 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.309+0000 7ff929d31740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f4fb280050 con 0x55f4fb35fdc0 2024-09-16T00:23:31.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.309+0000 7ff929d31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f4fb36d190 con 0x55f4fb36a680 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff929d31740 10 client.0 _ll_register_callbacks cb 0x55f4fb2c0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff925ef5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb36a680 0x55f4fb3678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff925ef5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb36a680 0x55f4fb3678e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:39382/0 (socket says 172.21.15.78:39382) 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff925ef5640 1 -- 172.21.15.78:0/3987911195 learned_addr learned my addr 172.21.15.78:0/3987911195 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff9256f4640 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff924ef3640 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 0x55f4fb3672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff9256f4640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff90c0a3390 0x7ff90c0a5850 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff918005c40 tx=0x7ff9180128f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff9256f4640 1 -- 172.21.15.78:0/3987911195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb36a680 msgr2=0x55f4fb3678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff9256f4640 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb36a680 0x55f4fb3678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff9256f4640 1 -- 172.21.15.78:0/3987911195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 msgr2=0x55f4fb3672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff9256f4640 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 0x55f4fb3672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.310+0000 7ff9256f4640 1 -- 172.21.15.78:0/3987911195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4fb0363c0 con 0x55f4fb35f4a0 2024-09-16T00:23:31.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.313+0000 7ff9256f4640 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36a040 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff918012df0 tx=0x7ff918004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc7a25017e94ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.314+0000 7ff908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.314+0000 7ff908ff9640 1 -- 172.21.15.78:0/3987911195 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918022030 con 0x55f4fb35f4a0 2024-09-16T00:23:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.314+0000 7ff929d31740 1 -- 172.21.15.78:0/3987911195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4fb2d2850 con 0x55f4fb35f4a0 2024-09-16T00:23:31.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.314+0000 7ff908ff9640 1 -- 172.21.15.78:0/3987911195 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff918003970 con 0x55f4fb35f4a0 2024-09-16T00:23:31.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.321+0000 7ff908ff9640 1 -- 172.21.15.78:0/3987911195 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff91802c050 con 0x55f4fb35f4a0 2024-09-16T00:23:31.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.323+0000 7ff929d31740 10 client.17205 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:31.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.323+0000 7ff929d31740 1 -- 172.21.15.78:0/3987911195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f4fb39aba0 con 0x55f4fb35f4a0 2024-09-16T00:23:31.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.330+0000 7ff908ff9640 1 -- 172.21.15.78:0/3987911195 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f4fb2d2850 con 0x55f4fb35f4a0 2024-09-16T00:23:31.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.330+0000 7ff908ff9640 1 -- 172.21.15.78:0/3987911195 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff918027450 con 0x55f4fb35f4a0 2024-09-16T00:23:31.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.330+0000 7ff929d31740 10 client.17205 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:31.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.330+0000 7ff929d31740 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:23:31.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.330+0000 7ff929d31740 20 client.17205 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:31.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.330+0000 7ff929d31740 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f4fb39c530 0x55f4fb39e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.330+0000 7ff929d31740 4 client.17205 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:31.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.330+0000 7ff929d31740 1 -- 172.21.15.78:0/3987911195 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4fb2ccd60 con 0x55f4fb39c530 2024-09-16T00:23:31.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.333+0000 7ff924ef3640 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f4fb39c530 0x55f4fb39e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.339+0000 7ff908ff9640 10 client.17205 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:31.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.339+0000 7ff924ef3640 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f4fb39c530 0x55f4fb39e9b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa3b385b832435f server_cookie=f617ae9841345026 in_seq=0 out_seq=0 2024-09-16T00:23:31.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.340+0000 7ff908ff9640 1 -- 172.21.15.78:0/3987911195 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4fb2ccd60 con 0x55f4fb39c530 2024-09-16T00:23:31.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.340+0000 7ff908ff9640 10 client.17205 handle_command_reply: tid=0 2024-09-16T00:23:31.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.340+0000 7ff929d31740 1 client.17205 shutdown 2024-09-16T00:23:31.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.342+0000 7ff929d31740 1 -- 172.21.15.78:0/3987911195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb35f4a0 msgr2=0x55f4fb36a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.342+0000 7ff929d31740 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36a040 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff918012df0 tx=0x7ff918004ab0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 1 -- 172.21.15.78:0/3987911195 shutdown_connections 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f4fb39c530 0x55f4fb39e9b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb36a680 0x55f4fb3678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 0x55f4fb3672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 1 --2- 172.21.15.78:0/3987911195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36a040 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 1 -- 172.21.15.78:0/3987911195 >> 172.21.15.78:0/3987911195 conn(0x55f4fb35ba30 msgr2=0x55f4fb361790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff908ff9640 0 client.17205 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 1 -- 172.21.15.78:0/3987911195 shutdown_connections 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 1 -- 172.21.15.78:0/3987911195 wait complete. 2024-09-16T00:23:31.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.346+0000 7ff929d31740 20 client.17205 trim_cache size 0 max 16384 2024-09-16T00:23:31.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.379+0000 7ff929d31740 1 Processor -- start 2024-09-16T00:23:31.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.380+0000 7ff929d31740 1 -- start start 2024-09-16T00:23:31.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.380+0000 7ff929d31740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb376310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.380+0000 7ff929d31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb3627a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.381+0000 7ff929d31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb3a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.381+0000 7ff929d31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff90400fbb0 con 0x55f4fb36a680 2024-09-16T00:23:31.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.381+0000 7ff925ef5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb3a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.381+0000 7ff925ef5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb3a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58930/0 (socket says 172.21.15.78:58930) 2024-09-16T00:23:31.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.381+0000 7ff925ef5640 1 -- 172.21.15.78:0/588718717 learned_addr learned my addr 172.21.15.78:0/588718717 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:31.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff929d31740 1 -- 172.21.15.78:0/588718717 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040116f0 con 0x55f4fb35f4a0 2024-09-16T00:23:31.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff9256f4640 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb376310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff924ef3640 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb3627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff929d31740 1 -- 172.21.15.78:0/588718717 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff904010280 con 0x55f4fb35fdc0 2024-09-16T00:23:31.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff925ef5640 1 -- 172.21.15.78:0/588718717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 msgr2=0x55f4fb3627a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:31.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff925ef5640 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb3627a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff925ef5640 1 -- 172.21.15.78:0/588718717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 msgr2=0x55f4fb376310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:31.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff925ef5640 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb376310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff925ef5640 1 -- 172.21.15.78:0/588718717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4fb36a3b0 con 0x55f4fb36a680 2024-09-16T00:23:31.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.382+0000 7ff9256f4640 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb376310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:23:31.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.383+0000 7ff925ef5640 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb3a42c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff910009230 tx=0x7ff9100b9ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3348cc391680e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:31.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.384+0000 7ff90affd640 1 -- 172.21.15.78:0/588718717 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100bf960 con 0x55f4fb36a680 2024-09-16T00:23:31.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.384+0000 7ff90affd640 1 -- 172.21.15.78:0/588718717 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9100c4220 con 0x55f4fb36a680 2024-09-16T00:23:31.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.385+0000 7ff929d31740 1 -- 172.21.15.78:0/588718717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 msgr2=0x55f4fb3a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.385+0000 7ff929d31740 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb3a42c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff910009230 tx=0x7ff9100b9ac0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.387+0000 7ff929d31740 1 -- 172.21.15.78:0/588718717 shutdown_connections 2024-09-16T00:23:31.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.387+0000 7ff929d31740 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb3a42c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.387+0000 7ff929d31740 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35fdc0 0x55f4fb3627a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.387+0000 7ff929d31740 1 --2- 172.21.15.78:0/588718717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35f4a0 0x55f4fb376310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.387+0000 7ff929d31740 1 -- 172.21.15.78:0/588718717 >> 172.21.15.78:0/588718717 conn(0x55f4fb35ba30 msgr2=0x55f4fb362090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:31.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.387+0000 7ff929d31740 1 -- 172.21.15.78:0/588718717 shutdown_connections 2024-09-16T00:23:31.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.387+0000 7ff929d31740 1 -- 172.21.15.78:0/588718717 wait complete. 2024-09-16T00:23:31.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.388+0000 7ff929d31740 1 -- start start 2024-09-16T00:23:31.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.388+0000 7ff929d31740 1 Processor -- start 2024-09-16T00:23:31.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.388+0000 7ff929d31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff929d31740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff929d31740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb405280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff929d31740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff904011020 con 0x55f4fb36a680 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff929d31740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff904010950 con 0x55f4fb35fdc0 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff929d31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff90400db90 con 0x55f4fb35f4a0 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff929d31740 10 client.0 _ll_register_callbacks cb 0x7ff918007920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff9256f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff9256f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:39398/0 (socket says 172.21.15.78:39398) 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff9256f4640 1 -- 172.21.15.78:0/4017136505 learned_addr learned my addr 172.21.15.78:0/4017136505 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff924ef3640 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff9256f4640 1 -- 172.21.15.78:0/4017136505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 msgr2=0x55f4fb36da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff9256f4640 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff9256f4640 1 -- 172.21.15.78:0/4017136505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 msgr2=0x55f4fb405280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff9256f4640 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb405280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.389+0000 7ff9256f4640 1 -- 172.21.15.78:0/4017136505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4fb36e2b0 con 0x55f4fb35f4a0 2024-09-16T00:23:31.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.390+0000 7ff9256f4640 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36d4e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff918010cc0 tx=0x7ff918027680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da923c4b83238103 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.390+0000 7ff908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:23:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.390+0000 7ff908ff9640 1 -- 172.21.15.78:0/4017136505 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918022040 con 0x55f4fb35f4a0 2024-09-16T00:23:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.390+0000 7ff908ff9640 1 -- 172.21.15.78:0/4017136505 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff918004b20 con 0x55f4fb35f4a0 2024-09-16T00:23:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.390+0000 7ff929d31740 1 -- 172.21.15.78:0/4017136505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9100c4220 con 0x55f4fb35f4a0 2024-09-16T00:23:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.391+0000 7ff908ff9640 1 -- 172.21.15.78:0/4017136505 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff918027e60 con 0x55f4fb35f4a0 2024-09-16T00:23:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.391+0000 7ff929d31740 10 client.26344 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:31.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.391+0000 7ff929d31740 1 -- 172.21.15.78:0/4017136505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f4fb4068a0 con 0x55f4fb35f4a0 2024-09-16T00:23:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.391+0000 7ff908ff9640 1 -- 172.21.15.78:0/4017136505 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9100c4220 con 0x55f4fb35f4a0 2024-09-16T00:23:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.392+0000 7ff908ff9640 1 -- 172.21.15.78:0/4017136505 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff918004020 con 0x55f4fb35f4a0 2024-09-16T00:23:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.392+0000 7ff929d31740 10 client.26344 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.392+0000 7ff929d31740 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:23:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.392+0000 7ff929d31740 20 client.26344 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.392+0000 7ff929d31740 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f4fb407020 0x55f4fb409480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.392+0000 7ff929d31740 4 client.26344 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:23:31.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.392+0000 7ff929d31740 1 -- 172.21.15.78:0/4017136505 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4fb249210 con 0x55f4fb407020 2024-09-16T00:23:31.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.393+0000 7ff924ef3640 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f4fb407020 0x55f4fb409480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:31.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.401+0000 7ff924ef3640 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f4fb407020 0x55f4fb409480 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233c274cf7806ab3 server_cookie=104a11121c257790 in_seq=0 out_seq=0 2024-09-16T00:23:31.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.401+0000 7ff908ff9640 10 client.26344 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:31.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.402+0000 7ff908ff9640 1 -- 172.21.15.78:0/4017136505 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f4fb249210 con 0x55f4fb407020 2024-09-16T00:23:31.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.402+0000 7ff908ff9640 10 client.26344 handle_command_reply: tid=0 2024-09-16T00:23:31.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.402+0000 7ff929d31740 1 client.26344 shutdown 2024-09-16T00:23:31.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.403+0000 7ff929d31740 1 -- 172.21.15.78:0/4017136505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35f4a0 msgr2=0x55f4fb36d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.403+0000 7ff929d31740 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36d4e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff918010cc0 tx=0x7ff918027680 comp rx=0 tx=0).stop 2024-09-16T00:23:31.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.403+0000 7ff929d31740 1 -- 172.21.15.78:0/4017136505 shutdown_connections 2024-09-16T00:23:31.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.403+0000 7ff929d31740 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55f4fb407020 0x55f4fb409480 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.403+0000 7ff929d31740 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4fb36a680 0x55f4fb405280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.403+0000 7ff929d31740 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f4fb35fdc0 0x55f4fb36da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.403+0000 7ff929d31740 1 --2- 172.21.15.78:0/4017136505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4fb35f4a0 0x55f4fb36d4e0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff918010cc0 tx=0x7ff918027680 comp rx=0 tx=0).stop 2024-09-16T00:23:31.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.403+0000 7ff929d31740 1 -- 172.21.15.78:0/4017136505 >> 172.21.15.78:0/4017136505 conn(0x55f4fb35ba30 msgr2=0x55f4fb361760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:31.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.404+0000 7ff908ff9640 0 client.26344 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:31.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.404+0000 7ff929d31740 1 -- 172.21.15.78:0/4017136505 shutdown_connections 2024-09-16T00:23:31.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.404+0000 7ff929d31740 1 -- 172.21.15.78:0/4017136505 wait complete. 2024-09-16T00:23:31.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.404+0000 7ff929d31740 20 client.26344 trim_cache size 0 max 16384 2024-09-16T00:23:31.406 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:23:31.406 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:23:31.406 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:23:31.407 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff90c0a3390 msgr2=0x7ff90c0a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff90c0a3390 0x7ff90c0a5850 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff918005c40 tx=0x7ff9180128f0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff92010ea80 msgr2=0x7ff9201149e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff92010ea80 0x7ff9201149e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff91002a860 tx=0x7ff910033d10 comp rx=0 tx=0).stop 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 shutdown_connections 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff90c0a3390 0x7ff90c0a5850 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff92010ea80 0x7ff9201149e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff920109e00 0x7ff9201144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 --2- 172.21.15.78:0/3096636682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff920109450 0x7ff920113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 >> 172.21.15.78:0/3096636682 conn(0x7ff92006e2d0 msgr2=0x7ff9200707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:31.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 shutdown_connections 2024-09-16T00:23:31.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:31.407+0000 7ff927958640 1 -- 172.21.15.78:0/3096636682 wait complete. 2024-09-16T00:23:32.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:31 smithi078 ceph-mon[31157]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-09-16T00:23:32.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:31 smithi193 ceph-mon[35484]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-09-16T00:23:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:31 smithi148 ceph-mon[35599]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.2 MiB/s wr, 728 op/s 2024-09-16T00:23:34.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:33 smithi078 ceph-mon[31157]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 746 op/s 2024-09-16T00:23:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:33 smithi193 ceph-mon[35484]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 746 op/s 2024-09-16T00:23:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:33 smithi148 ceph-mon[35599]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 746 op/s 2024-09-16T00:23:34.308 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:23:34.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:23:34.811 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:35.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.481+0000 7fd90ce1f640 1 -- 172.21.15.78:0/1363792026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 msgr2=0x7fd908109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:35.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.481+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/1363792026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 0x7fd908109c90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009900 tx=0x7fd8f002a410 comp rx=0 tx=0).stop 2024-09-16T00:23:35.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.483+0000 7fd90ce1f640 1 -- 172.21.15.78:0/1363792026 shutdown_connections 2024-09-16T00:23:35.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.483+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/1363792026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd90811b140 0x7fd90811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.483+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/1363792026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 0x7fd908109c90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.483+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/1363792026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd908109450 0x7fd908110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.483+0000 7fd90ce1f640 1 -- 172.21.15.78:0/1363792026 >> 172.21.15.78:0/1363792026 conn(0x7fd90806e140 msgr2=0x7fd90806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.483+0000 7fd90ce1f640 1 -- 172.21.15.78:0/1363792026 shutdown_connections 2024-09-16T00:23:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.483+0000 7fd90ce1f640 1 -- 172.21.15.78:0/1363792026 wait complete. 2024-09-16T00:23:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.484+0000 7fd90ce1f640 1 Processor -- start 2024-09-16T00:23:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.484+0000 7fd90ce1f640 1 -- start start 2024-09-16T00:23:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.484+0000 7fd90ce1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908109450 0x7fd9081a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.485+0000 7fd90ce1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 0x7fd9081a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.485+0000 7fd90ce1f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd90811b140 0x7fd9081abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.485+0000 7fd90ce1f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd908120e30 con 0x7fd90811b140 2024-09-16T00:23:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.485+0000 7fd90ce1f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd908120cb0 con 0x7fd908109450 2024-09-16T00:23:35.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.485+0000 7fd90ce1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd908120fb0 con 0x7fd908109830 2024-09-16T00:23:35.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.486+0000 7fd905d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 0x7fd9081a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:35.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.486+0000 7fd90659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908109450 0x7fd9081a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:35.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.486+0000 7fd90659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908109450 0x7fd9081a7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59412/0 (socket says 172.21.15.78:59412) 2024-09-16T00:23:35.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.486+0000 7fd90659c640 1 -- 172.21.15.78:0/837256518 learned_addr learned my addr 172.21.15.78:0/837256518 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:35.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.489+0000 7fd905d9b640 1 -- 172.21.15.78:0/837256518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908109450 msgr2=0x7fd9081a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.489+0000 7fd905d9b640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908109450 0x7fd9081a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.489+0000 7fd905d9b640 1 -- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd90811b140 msgr2=0x7fd9081abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.489+0000 7fd905d9b640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd90811b140 0x7fd9081abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.489+0000 7fd905d9b640 1 -- 172.21.15.78:0/837256518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9081ac320 con 0x7fd908109830 2024-09-16T00:23:35.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.491+0000 7fd905d9b640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 0x7fd9081a78b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8f002aa00 tx=0x7fd8f0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:35.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.493+0000 7fd8ff7fe640 1 -- 172.21.15.78:0/837256518 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f003f070 con 0x7fd908109830 2024-09-16T00:23:35.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.493+0000 7fd8ff7fe640 1 -- 172.21.15.78:0/837256518 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8f0031c90 con 0x7fd908109830 2024-09-16T00:23:35.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.494+0000 7fd8ff7fe640 1 -- 172.21.15.78:0/837256518 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0035030 con 0x7fd908109830 2024-09-16T00:23:35.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.494+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9081ac5b0 con 0x7fd908109830 2024-09-16T00:23:35.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.494+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9081aca10 con 0x7fd908109830 2024-09-16T00:23:35.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.495+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd908073b00 con 0x7fd908109830 2024-09-16T00:23:35.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.497+0000 7fd8ff7fe640 1 -- 172.21.15.78:0/837256518 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd8f0042070 con 0x7fd908109830 2024-09-16T00:23:35.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.499+0000 7fd8ff7fe640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8f40a32c0 0x7fd8f40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.499+0000 7fd8ff7fe640 1 -- 172.21.15.78:0/837256518 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd8f00f03c0 con 0x7fd908109830 2024-09-16T00:23:35.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.499+0000 7fd90659c640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8f40a32c0 0x7fd8f40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:35.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.500+0000 7fd90659c640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8f40a32c0 0x7fd8f40a5780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd900009c90 tx=0x7fd900005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.501+0000 7fd8ff7fe640 1 -- 172.21.15.78:0/837256518 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8f0049020 con 0x7fd908109830 2024-09-16T00:23:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 575.0 s, 538.0 tps, lat 187.009 ms stddev 227.517 2024-09-16T00:23:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.922+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd90810b3a0 con 0x7fd908109830 2024-09-16T00:23:35.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.923+0000 7fd8ff7fe640 1 -- 172.21.15.78:0/837256518 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fd8f00bcda0 con 0x7fd908109830 2024-09-16T00:23:35.923 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:23:35.924 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:23:35.924 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:23:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8f40a32c0 msgr2=0x7fd8f40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8f40a32c0 0x7fd8f40a5780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd900009c90 tx=0x7fd900005f90 comp rx=0 tx=0).stop 2024-09-16T00:23:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 msgr2=0x7fd9081a78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 0x7fd9081a78b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8f002aa00 tx=0x7fd8f0003b90 comp rx=0 tx=0).stop 2024-09-16T00:23:35.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 shutdown_connections 2024-09-16T00:23:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8f40a32c0 0x7fd8f40a5780 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd90811b140 0x7fd9081abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd908109830 0x7fd9081a78b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 --2- 172.21.15.78:0/837256518 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd908109450 0x7fd9081a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 >> 172.21.15.78:0/837256518 conn(0x7fd90806e140 msgr2=0x7fd90811bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 shutdown_connections 2024-09-16T00:23:35.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:35.926+0000 7fd90ce1f640 1 -- 172.21.15.78:0/837256518 wait complete. 2024-09-16T00:23:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:35 smithi078 ceph-mon[31157]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 MiB/s wr, 666 op/s 2024-09-16T00:23:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:35 smithi193 ceph-mon[35484]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 MiB/s wr, 666 op/s 2024-09-16T00:23:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:35 smithi148 ceph-mon[35599]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 MiB/s wr, 666 op/s 2024-09-16T00:23:36.695 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:23:37.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:36 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/837256518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:37.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:36 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/837256518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:37.183 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:37.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:36 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/837256518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:23:37.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.827+0000 7f16d6b7f640 1 -- 172.21.15.78:0/3702088691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0101430 msgr2=0x7f16d010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:37.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.827+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/3702088691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0101430 0x7f16d010d960 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009c10 tx=0x7f16c802a700 comp rx=0 tx=0).stop 2024-09-16T00:23:37.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.836+0000 7f16d6b7f640 1 -- 172.21.15.78:0/3702088691 shutdown_connections 2024-09-16T00:23:37.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.836+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/3702088691 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0101430 0x7f16d010d960 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.836+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/3702088691 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d0100a90 0x7f16d0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.836+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/3702088691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16d0106a90 0x7f16d0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.836+0000 7f16d6b7f640 1 -- 172.21.15.78:0/3702088691 >> 172.21.15.78:0/3702088691 conn(0x7f16d00fc7d0 msgr2=0x7f16d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:37.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.838+0000 7f16d6b7f640 1 -- 172.21.15.78:0/3702088691 shutdown_connections 2024-09-16T00:23:37.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.839+0000 7f16d6b7f640 1 -- 172.21.15.78:0/3702088691 wait complete. 2024-09-16T00:23:37.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.840+0000 7f16d6b7f640 1 Processor -- start 2024-09-16T00:23:37.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.844+0000 7f16d6b7f640 1 -- start start 2024-09-16T00:23:37.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.844+0000 7f16d6b7f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d0100a90 0x7f16d0073940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.844+0000 7f16d6b7f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0101430 0x7f16d0073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.844+0000 7f16d6b7f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16d0106a90 0x7f16d006cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.844+0000 7f16d6b7f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010fd80 con 0x7f16d0101430 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.844+0000 7f16d6b7f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010fc00 con 0x7f16d0100a90 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.844+0000 7f16d6b7f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16d010ff00 con 0x7f16d0106a90 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d511c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16d0106a90 0x7f16d006cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d491b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d0100a90 0x7f16d0073940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d511c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16d0106a90 0x7f16d006cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42704/0 (socket says 172.21.15.78:42704) 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d511c640 1 -- 172.21.15.78:0/1218471954 learned_addr learned my addr 172.21.15.78:0/1218471954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d491b640 1 -- 172.21.15.78:0/1218471954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16d0106a90 msgr2=0x7f16d006cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d491b640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16d0106a90 0x7f16d006cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d491b640 1 -- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0101430 msgr2=0x7f16d0073e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d491b640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0101430 0x7f16d0073e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.849+0000 7f16d491b640 1 -- 172.21.15.78:0/1218471954 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d006d470 con 0x7f16d0100a90 2024-09-16T00:23:37.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.850+0000 7f16d491b640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d0100a90 0x7f16d0073940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009760 tx=0x7f16c4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:37.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.858+0000 7f16cdffb640 1 -- 172.21.15.78:0/1218471954 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c401b070 con 0x7f16d0100a90 2024-09-16T00:23:37.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.858+0000 7f16cdffb640 1 -- 172.21.15.78:0/1218471954 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c4016410 con 0x7f16d0100a90 2024-09-16T00:23:37.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.858+0000 7f16cdffb640 1 -- 172.21.15.78:0/1218471954 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4015610 con 0x7f16d0100a90 2024-09-16T00:23:37.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.860+0000 7f16d6b7f640 1 -- 172.21.15.78:0/1218471954 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d006d700 con 0x7f16d0100a90 2024-09-16T00:23:37.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.861+0000 7f16d6b7f640 1 -- 172.21.15.78:0/1218471954 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d00716a0 con 0x7f16d0100a90 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.875+0000 7f16d8f58740 1 Processor -- start 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.875+0000 7f16d8f58740 1 -- start start 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.866+0000 7f16cdffb640 1 -- 172.21.15.78:0/1218471954 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f16c401e070 con 0x7f16d0100a90 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.875+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 0x559c71beb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.875+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71beb5d0 0x559c71beba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.875+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bf8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.875+0000 7f16d8f58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c71b512f0 con 0x559c71bf6370 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.876+0000 7f16d8f58740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559c71af8cf0 con 0x559c71beb5d0 2024-09-16T00:23:37.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.876+0000 7f16d8f58740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c71b4d2b0 con 0x559c71becb40 2024-09-16T00:23:37.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.876+0000 7f16d491b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71beb5d0 0x559c71beba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.876+0000 7f16d491b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71beb5d0 0x559c71beba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52150/0 (socket says 172.21.15.78:52150) 2024-09-16T00:23:37.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.876+0000 7f16d491b640 1 -- 172.21.15.78:0/2179280486 learned_addr learned my addr 172.21.15.78:0/2179280486 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:37.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.876+0000 7f16d491b640 1 -- 172.21.15.78:0/2179280486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 msgr2=0x559c71beb090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:23:37.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.876+0000 7f16d491b640 1 --2- 172.21.15.78:0/2179280486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 0x559c71beb090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.876+0000 7f16d491b640 1 -- 172.21.15.78:0/2179280486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 msgr2=0x559c71bf8760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:23:37.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.877+0000 7f16d491b640 1 --2- 172.21.15.78:0/2179280486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bf8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.877+0000 7f16d491b640 1 -- 172.21.15.78:0/2179280486 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c71add3a0 con 0x559c71beb5d0 2024-09-16T00:23:37.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.877+0000 7f16cdffb640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16c00a3450 0x7f16c00a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.878+0000 7f16d491b640 1 --2- 172.21.15.78:0/2179280486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71beb5d0 0x559c71beba10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16c40d2e00 tx=0x7f16c40d77a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f00be8ad7b896a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:37.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.878+0000 7f16cdffb640 1 -- 172.21.15.78:0/1218471954 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f16c40cedd0 con 0x7f16d0100a90 2024-09-16T00:23:37.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.879+0000 7f16baffd640 1 -- 172.21.15.78:0/2179280486 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4025030 con 0x559c71beb5d0 2024-09-16T00:23:37.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.879+0000 7f16baffd640 1 -- 172.21.15.78:0/2179280486 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c40d0070 con 0x559c71beb5d0 2024-09-16T00:23:37.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.880+0000 7f16d8f58740 1 -- 172.21.15.78:0/2179280486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71beb5d0 msgr2=0x559c71beba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:37.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.880+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2179280486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71beb5d0 0x559c71beba10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16c40d2e00 tx=0x7f16c40d77a0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.880+0000 7f16d511c640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16c00a3450 0x7f16c00a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.881+0000 7f16d511c640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16c00a3450 0x7f16c00a5910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f16c8006010 tx=0x7f16c802f320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:37.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.881+0000 7f16d8f58740 1 -- 172.21.15.78:0/2179280486 shutdown_connections 2024-09-16T00:23:37.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.881+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2179280486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bf8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.881+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2179280486 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71beb5d0 0x559c71beba10 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16c40d2e00 tx=0x7f16c40d77a0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.881+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2179280486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 0x559c71beb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.881+0000 7f16d8f58740 1 -- 172.21.15.78:0/2179280486 >> 172.21.15.78:0/2179280486 conn(0x559c71bb4310 msgr2=0x559c71be7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.885+0000 7f16d8f58740 1 -- 172.21.15.78:0/2179280486 shutdown_connections 2024-09-16T00:23:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.886+0000 7f16d8f58740 1 -- 172.21.15.78:0/2179280486 wait complete. 2024-09-16T00:23:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.886+0000 7f16d8f58740 1 -- start start 2024-09-16T00:23:37.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.886+0000 7f16d8f58740 1 Processor -- start 2024-09-16T00:23:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.887+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bf3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.887+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.887+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71c02650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.887+0000 7f16d8f58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559c71bf90c0 con 0x559c71becb40 2024-09-16T00:23:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.887+0000 7f16d8f58740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559c71b0c060 con 0x559c71bf3c00 2024-09-16T00:23:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.887+0000 7f16d8f58740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c71bf9240 con 0x559c71bf6370 2024-09-16T00:23:37.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.887+0000 7f16d8f58740 10 client.0 _ll_register_callbacks cb 0x559c7185f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:37.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.889+0000 7f16d511c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71c02650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.889+0000 7f16d511c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71c02650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52166/0 (socket says 172.21.15.78:52166) 2024-09-16T00:23:37.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.889+0000 7f16d511c640 1 -- 172.21.15.78:0/2622776279 learned_addr learned my addr 172.21.15.78:0/2622776279 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:37.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.890+0000 7f16cffff640 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.892+0000 7f16cffff640 1 -- 172.21.15.78:0/2622776279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 msgr2=0x559c71c02650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:37.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.892+0000 7f16cffff640 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71c02650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.892+0000 7f16cffff640 1 -- 172.21.15.78:0/2622776279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 msgr2=0x559c71bf3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:37.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.893+0000 7f16cffff640 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bf3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.893+0000 7f16cffff640 1 -- 172.21.15.78:0/2622776279 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c718c23c0 con 0x559c71bf6370 2024-09-16T00:23:37.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.896+0000 7f16cffff640 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf36c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x559c71bec070 tx=0x7f16bc00e850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c3fea2cbf86eda server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:37.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.896+0000 7f16affff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:23:37.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.897+0000 7f16affff640 1 -- 172.21.15.78:0/2622776279 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc005460 con 0x559c71bf6370 2024-09-16T00:23:37.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.897+0000 7f16d8f58740 1 -- 172.21.15.78:0/2622776279 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16c40d0070 con 0x559c71bf6370 2024-09-16T00:23:37.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.897+0000 7f16affff640 1 -- 172.21.15.78:0/2622776279 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f16bc004020 con 0x559c71bf6370 2024-09-16T00:23:37.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.897+0000 7f16affff640 1 -- 172.21.15.78:0/2622776279 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f16bc021050 con 0x559c71bf6370 2024-09-16T00:23:37.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.900+0000 7f16d8f58740 10 client.26359 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:37.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.900+0000 7f16d8f58740 1 -- 172.21.15.78:0/2622776279 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559c71c05210 con 0x559c71bf6370 2024-09-16T00:23:37.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.902+0000 7f16affff640 1 -- 172.21.15.78:0/2622776279 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c40d0070 con 0x559c71bf6370 2024-09-16T00:23:37.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.902+0000 7f16affff640 1 -- 172.21.15.78:0/2622776279 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f16bc003750 con 0x559c71bf6370 2024-09-16T00:23:37.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.902+0000 7f16d8f58740 10 client.26359 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:37.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.902+0000 7f16d8f58740 10 client.26359 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:23:37.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.902+0000 7f16d8f58740 20 client.26359 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:37.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.902+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559c71c06af0 0x559c71c08f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.902+0000 7f16d8f58740 4 client.26359 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:37.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.902+0000 7f16d8f58740 1 -- 172.21.15.78:0/2622776279 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c71b5e860 con 0x559c71c06af0 2024-09-16T00:23:37.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.904+0000 7f16d491b640 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559c71c06af0 0x559c71c08f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.905+0000 7f16d491b640 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559c71c06af0 0x559c71c08f90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a4ccca10ee9bd226 server_cookie=d07de26daf44e280 in_seq=0 out_seq=0 2024-09-16T00:23:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.906+0000 7f16affff640 10 client.26359 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:23:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.906+0000 7f16affff640 1 -- 172.21.15.78:0/2622776279 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c71b5e860 con 0x559c71c06af0 2024-09-16T00:23:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.906+0000 7f16affff640 10 client.26359 handle_command_reply: tid=0 2024-09-16T00:23:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.907+0000 7f16d8f58740 1 client.26359 shutdown 2024-09-16T00:23:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.913+0000 7f16d8f58740 1 -- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 msgr2=0x559c71bf36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:37.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.913+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf36c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x559c71bec070 tx=0x7f16bc00e850 comp rx=0 tx=0).stop 2024-09-16T00:23:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.914+0000 7f16d8f58740 1 -- 172.21.15.78:0/2622776279 shutdown_connections 2024-09-16T00:23:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.914+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71c02650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.914+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559c71c06af0 0x559c71c08f90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.914+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf36c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.914+0000 7f16d8f58740 1 --2- 172.21.15.78:0/2622776279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bf3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.914+0000 7f16d8f58740 1 -- 172.21.15.78:0/2622776279 >> 172.21.15.78:0/2622776279 conn(0x559c71bb4310 msgr2=0x559c71bed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.914+0000 7f16affff640 0 client.26359 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:23:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.915+0000 7f16d8f58740 1 -- 172.21.15.78:0/2622776279 shutdown_connections 2024-09-16T00:23:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.915+0000 7f16d8f58740 1 -- 172.21.15.78:0/2622776279 wait complete. 2024-09-16T00:23:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.915+0000 7f16d8f58740 20 client.26359 trim_cache size 0 max 16384 2024-09-16T00:23:37.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d8f58740 1 Processor -- start 2024-09-16T00:23:37.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d8f58740 1 -- start start 2024-09-16T00:23:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 0x559c71bf50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71bee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d8f58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16b400fc00 con 0x559c71bf6370 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d8f58740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4011740 con 0x559c71bf3c00 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d8f58740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16b40102d0 con 0x559c71becb40 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d511c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71bee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16cffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71bee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52170/0 (socket says 172.21.15.78:52170) 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d511c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35400/0 (socket says 172.21.15.78:35400) 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16cffff640 1 -- 172.21.15.78:0/1730127574 learned_addr learned my addr 172.21.15.78:0/1730127574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.938+0000 7f16d491b640 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 0x559c71bf50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.939+0000 7f16d511c640 1 -- 172.21.15.78:0/1730127574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 msgr2=0x559c71bf50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.939+0000 7f16d511c640 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 0x559c71bf50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.939+0000 7f16d511c640 1 -- 172.21.15.78:0/1730127574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 msgr2=0x559c71bee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.939+0000 7f16d511c640 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71bee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.939+0000 7f16d511c640 1 -- 172.21.15.78:0/1730127574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c71bf2130 con 0x559c71bf6370 2024-09-16T00:23:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.939+0000 7f16cffff640 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71bee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:23:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.939+0000 7f16d511c640 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bee660 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f16c8034040 tx=0x7f16c803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebb40f0f45cdc4f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.940+0000 7f16ba7fc640 1 -- 172.21.15.78:0/1730127574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c804f070 con 0x559c71bf6370 2024-09-16T00:23:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.940+0000 7f16ba7fc640 1 -- 172.21.15.78:0/1730127574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c8030880 con 0x559c71bf6370 2024-09-16T00:23:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.940+0000 7f16ba7fc640 1 -- 172.21.15.78:0/1730127574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8043070 con 0x559c71bf6370 2024-09-16T00:23:37.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 -- 172.21.15.78:0/1730127574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 msgr2=0x559c71bee660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bee660 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f16c8034040 tx=0x7f16c803d000 comp rx=0 tx=0).stop 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 -- 172.21.15.78:0/1730127574 shutdown_connections 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71bf6370 0x559c71bee660 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71bee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 --2- 172.21.15.78:0/1730127574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71becb40 0x559c71bf50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 -- 172.21.15.78:0/1730127574 >> 172.21.15.78:0/1730127574 conn(0x559c71bb4310 msgr2=0x559c71bf49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 -- 172.21.15.78:0/1730127574 shutdown_connections 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 -- 172.21.15.78:0/1730127574 wait complete. 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 -- start start 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.941+0000 7f16d8f58740 1 Processor -- start 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bfec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71befe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d8f58740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d8f58740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4011070 con 0x559c71becb40 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d8f58740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16b40109a0 con 0x559c71bf3c00 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d8f58740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16b400dbe0 con 0x559c71bf6370 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d8f58740 10 client.0 _ll_register_callbacks cb 0x7f16bc009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d491b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bfec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d491b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bfec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35402/0 (socket says 172.21.15.78:35402) 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d491b640 1 -- 172.21.15.78:0/3557739080 learned_addr learned my addr 172.21.15.78:0/3557739080 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d511c640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d491b640 1 -- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 msgr2=0x559c71bf03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d491b640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16d491b640 1 -- 172.21.15.78:0/3557739080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 msgr2=0x559c71befe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.942+0000 7f16cffff640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71befe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.943+0000 7f16d491b640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71befe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.943+0000 7f16d491b640 1 -- 172.21.15.78:0/3557739080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c71bf0b40 con 0x559c71becb40 2024-09-16T00:23:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.943+0000 7f16d511c640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:23:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.943+0000 7f16cffff640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71befe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:23:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.943+0000 7f16d491b640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bfec30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f16c4099390 tx=0x7f16c400f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5395f17b63eb5c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.943+0000 7f16affff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:37.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.944+0000 7f16affff640 1 -- 172.21.15.78:0/3557739080 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c409aa40 con 0x559c71becb40 2024-09-16T00:23:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.944+0000 7f16affff640 1 -- 172.21.15.78:0/3557739080 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f16c40d2990 con 0x559c71becb40 2024-09-16T00:23:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.944+0000 7f16d8f58740 1 -- 172.21.15.78:0/3557739080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c71b58d70 con 0x559c71becb40 2024-09-16T00:23:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.945+0000 7f16affff640 1 -- 172.21.15.78:0/3557739080 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f16c40a9050 con 0x559c71becb40 2024-09-16T00:23:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.945+0000 7f16d8f58740 10 client.17220 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:37.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.945+0000 7f16d8f58740 1 -- 172.21.15.78:0/3557739080 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559c71c71840 con 0x559c71becb40 2024-09-16T00:23:37.948 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:37 smithi078 ceph-mon[31157]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 646 op/s 2024-09-16T00:23:37.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.946+0000 7f16affff640 1 -- 172.21.15.78:0/3557739080 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16c40d0040 con 0x559c71becb40 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.946+0000 7f16affff640 1 -- 172.21.15.78:0/3557739080 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f16c4097750 con 0x559c71becb40 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.946+0000 7f16d8f58740 10 client.17220 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.947+0000 7f16d8f58740 10 client.17220 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.947+0000 7f16d8f58740 20 client.17220 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.947+0000 7f16d8f58740 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559c71c71ad0 0x559c71c73ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.947+0000 7f16d8f58740 4 client.17220 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.947+0000 7f16d8f58740 1 -- 172.21.15.78:0/3557739080 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c71b58d70 con 0x559c71c71ad0 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.947+0000 7f16cffff640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559c71c71ad0 0x559c71c73ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16affff640 10 client.17220 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16cffff640 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559c71c71ad0 0x559c71c73ec0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb22f9051d8ceb4d server_cookie=bc3cf8641cbe08d4 in_seq=0 out_seq=0 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16affff640 1 -- 172.21.15.78:0/3557739080 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c71b58d70 con 0x559c71c71ad0 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16affff640 10 client.17220 handle_command_reply: tid=0 2024-09-16T00:23:37.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 client.17220 shutdown 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 -- 172.21.15.78:0/3557739080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 msgr2=0x559c71bfec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bfec30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f16c4099390 tx=0x7f16c400f040 comp rx=0 tx=0).stop 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 -- 172.21.15.78:0/3557739080 shutdown_connections 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x559c71c71ad0 0x559c71c73ec0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16affff640 0 client.17220 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c71bf6370 0x559c71bf03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559c71bf3c00 0x559c71befe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 --2- 172.21.15.78:0/3557739080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559c71becb40 0x559c71bfec30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.948+0000 7f16d8f58740 1 -- 172.21.15.78:0/3557739080 >> 172.21.15.78:0/3557739080 conn(0x559c71bb4310 msgr2=0x559c71bf5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.949+0000 7f16d8f58740 1 -- 172.21.15.78:0/3557739080 shutdown_connections 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.949+0000 7f16d8f58740 1 -- 172.21.15.78:0/3557739080 wait complete. 2024-09-16T00:23:37.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.949+0000 7f16d8f58740 20 client.17220 trim_cache size 0 max 16384 2024-09-16T00:23:37.953 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:23:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.950+0000 7f16d6b7f640 1 -- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16c00a3450 msgr2=0x7f16c00a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.950+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16c00a3450 0x7f16c00a5910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f16c8006010 tx=0x7f16c802f320 comp rx=0 tx=0).stop 2024-09-16T00:23:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.950+0000 7f16d6b7f640 1 -- 172.21.15.78:0/1218471954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d0100a90 msgr2=0x7f16d0073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.950+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d0100a90 0x7f16d0073940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009760 tx=0x7f16c4009c20 comp rx=0 tx=0).stop 2024-09-16T00:23:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.950+0000 7f16d6b7f640 1 -- 172.21.15.78:0/1218471954 shutdown_connections 2024-09-16T00:23:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.950+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f16c00a3450 0x7f16c00a5910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.950+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16d0106a90 0x7f16d006cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.951+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16d0101430 0x7f16d0073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.951+0000 7f16d6b7f640 1 --2- 172.21.15.78:0/1218471954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16d0100a90 0x7f16d0073940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.951+0000 7f16d6b7f640 1 -- 172.21.15.78:0/1218471954 >> 172.21.15.78:0/1218471954 conn(0x7f16d00fc7d0 msgr2=0x7f16d00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.951+0000 7f16d6b7f640 1 -- 172.21.15.78:0/1218471954 shutdown_connections 2024-09-16T00:23:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:37.951+0000 7f16d6b7f640 1 -- 172.21.15.78:0/1218471954 wait complete. 2024-09-16T00:23:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:37 smithi193 ceph-mon[35484]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 646 op/s 2024-09-16T00:23:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:37 smithi148 ceph-mon[35599]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 MiB/s wr, 646 op/s 2024-09-16T00:23:38.703 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:23:39.173 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.787+0000 7f8b64245640 1 -- 172.21.15.78:0/4031872374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109450 msgr2=0x7f8b5c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.787+0000 7f8b64245640 1 --2- 172.21.15.78:0/4031872374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109450 0x7f8b5c110250 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009ca0 tx=0x7f8b5402a8d0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.789+0000 7f8b64245640 1 -- 172.21.15.78:0/4031872374 shutdown_connections 2024-09-16T00:23:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.789+0000 7f8b64245640 1 --2- 172.21.15.78:0/4031872374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c11b140 0x7f8b5c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.789+0000 7f8b64245640 1 --2- 172.21.15.78:0/4031872374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b5c109830 0x7f8b5c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.789+0000 7f8b64245640 1 --2- 172.21.15.78:0/4031872374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109450 0x7f8b5c110250 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.789+0000 7f8b64245640 1 -- 172.21.15.78:0/4031872374 >> 172.21.15.78:0/4031872374 conn(0x7f8b5c06e140 msgr2=0x7f8b5c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.789+0000 7f8b64245640 1 -- 172.21.15.78:0/4031872374 shutdown_connections 2024-09-16T00:23:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.789+0000 7f8b64245640 1 -- 172.21.15.78:0/4031872374 wait complete. 2024-09-16T00:23:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b64245640 1 Processor -- start 2024-09-16T00:23:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b64245640 1 -- start start 2024-09-16T00:23:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b64245640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c109450 0x7f8b5c1a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b64245640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109830 0x7f8b5c1a7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b64245640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b5c11b140 0x7f8b5c1abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b64245640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c120eb0 con 0x7f8b5c11b140 2024-09-16T00:23:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b64245640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c120d30 con 0x7f8b5c109450 2024-09-16T00:23:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b64245640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c121030 con 0x7f8b5c109830 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b627e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b5c11b140 0x7f8b5c1abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b617e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109830 0x7f8b5c1a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b627e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b5c11b140 0x7f8b5c1abd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35416/0 (socket says 172.21.15.78:35416) 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b627e2640 1 -- 172.21.15.78:0/248760976 learned_addr learned my addr 172.21.15.78:0/248760976 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.790+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c109450 0x7f8b5c1a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b617e0640 1 -- 172.21.15.78:0/248760976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c109450 msgr2=0x7f8b5c1a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b617e0640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c109450 0x7f8b5c1a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b617e0640 1 -- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b5c11b140 msgr2=0x7f8b5c1abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b617e0640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b5c11b140 0x7f8b5c1abd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b617e0640 1 -- 172.21.15.78:0/248760976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c1ac400 con 0x7f8b5c109830 2024-09-16T00:23:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b617e0640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109830 0x7f8b5c1a7990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00b3f0 tx=0x7f8b4c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b52ffd640 1 -- 172.21.15.78:0/248760976 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c015020 con 0x7f8b5c109830 2024-09-16T00:23:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b64245640 1 -- 172.21.15.78:0/248760976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5c1ac6f0 con 0x7f8b5c109830 2024-09-16T00:23:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b52ffd640 1 -- 172.21.15.78:0/248760976 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b4c011d10 con 0x7f8b5c109830 2024-09-16T00:23:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b52ffd640 1 -- 172.21.15.78:0/248760976 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00f2e0 con 0x7f8b5c109830 2024-09-16T00:23:39.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.791+0000 7f8b64245640 1 -- 172.21.15.78:0/248760976 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5c1b3fd0 con 0x7f8b5c109830 2024-09-16T00:23:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.794+0000 7f8b52ffd640 1 -- 172.21.15.78:0/248760976 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8b4c018020 con 0x7f8b5c109830 2024-09-16T00:23:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.794+0000 7f8b6661e740 1 Processor -- start 2024-09-16T00:23:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.794+0000 7f8b52ffd640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b480a3340 0x7f8b480a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.794+0000 7f8b52ffd640 1 -- 172.21.15.78:0/248760976 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8b4c0cf3d0 con 0x7f8b5c109830 2024-09-16T00:23:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.795+0000 7f8b6661e740 1 -- start start 2024-09-16T00:23:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.795+0000 7f8b627e2640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b480a3340 0x7f8b480a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb4a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b6661e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637bb4062f0 con 0x5637bb4ab370 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b6661e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5637bb3adcf0 con 0x5637bb4a1b40 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b6661e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5637bb4022b0 con 0x5637bb4a05d0 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b61fe1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b61fe1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52190/0 (socket says 172.21.15.78:52190) 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b61fe1640 1 -- 172.21.15.78:0/2376551210 learned_addr learned my addr 172.21.15.78:0/2376551210 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b627e2640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b480a3340 0x7f8b480a5800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8b580099a0 tx=0x7f8b580090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.796+0000 7f8b627e2640 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.797+0000 7f8b61fe1640 1 -- 172.21.15.78:0/2376551210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 msgr2=0x5637bb4a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.797+0000 7f8b617e0640 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb4a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.797+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb4a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.797+0000 7f8b61fe1640 1 -- 172.21.15.78:0/2376551210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 msgr2=0x5637bb4ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.797+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.797+0000 7f8b61fe1640 1 -- 172.21.15.78:0/2376551210 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637bb3923a0 con 0x5637bb4a1b40 2024-09-16T00:23:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.797+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a0090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009ca0 tx=0x7f8b5402f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a2108767513321 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.798+0000 7f8b47fff640 1 -- 172.21.15.78:0/2376551210 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54003970 con 0x5637bb4a1b40 2024-09-16T00:23:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.798+0000 7f8b47fff640 1 -- 172.21.15.78:0/2376551210 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b54036030 con 0x5637bb4a1b40 2024-09-16T00:23:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.798+0000 7f8b47fff640 1 -- 172.21.15.78:0/2376551210 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54004460 con 0x5637bb4a1b40 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 -- 172.21.15.78:0/2376551210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 msgr2=0x5637bb4a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a0090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009ca0 tx=0x7f8b5402f3a0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 -- 172.21.15.78:0/2376551210 shutdown_connections 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb4a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 --2- 172.21.15.78:0/2376551210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a0090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 -- 172.21.15.78:0/2376551210 >> 172.21.15.78:0/2376551210 conn(0x5637bb469310 msgr2=0x5637bb49cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 -- 172.21.15.78:0/2376551210 shutdown_connections 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.799+0000 7f8b6661e740 1 -- 172.21.15.78:0/2376551210 wait complete. 2024-09-16T00:23:39.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.800+0000 7f8b6661e740 1 -- start start 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.800+0000 7f8b6661e740 1 Processor -- start 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.800+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4a05d0 0x5637bb4aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.800+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a1b40 0x5637bb4a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.800+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4ab370 0x5637bb4a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.800+0000 7f8b6661e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5637bb4ae060 con 0x5637bb4a05d0 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.800+0000 7f8b6661e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5637bb3c1060 con 0x5637bb4ab370 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.800+0000 7f8b6661e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5637bb4ae1e0 con 0x5637bb4a1b40 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b6661e740 10 client.0 _ll_register_callbacks cb 0x5637bb1148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4ab370 0x5637bb4a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4ab370 0x5637bb4a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52192/0 (socket says 172.21.15.78:52192) 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 -- 172.21.15.78:0/3712439742 learned_addr learned my addr 172.21.15.78:0/3712439742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b617e0640 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a1b40 0x5637bb4a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 -- 172.21.15.78:0/3712439742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a1b40 msgr2=0x5637bb4a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a1b40 0x5637bb4a7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 -- 172.21.15.78:0/3712439742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4a05d0 msgr2=0x5637bb4aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4a05d0 0x5637bb4aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 -- 172.21.15.78:0/3712439742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637bb1773c0 con 0x5637bb4ab370 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b627e2640 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4ab370 0x5637bb4a84d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800f010 tx=0x7f8b58004810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4617037a0971cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b457fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b457fa640 1 -- 172.21.15.78:0/3712439742 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58025020 con 0x5637bb4ab370 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b6661e740 1 -- 172.21.15.78:0/3712439742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b54036030 con 0x5637bb4ab370 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.801+0000 7f8b457fa640 1 -- 172.21.15.78:0/3712439742 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8b58015d20 con 0x5637bb4ab370 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b457fa640 1 -- 172.21.15.78:0/3712439742 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8b58030050 con 0x5637bb4ab370 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b6661e740 10 client.26558 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b6661e740 1 -- 172.21.15.78:0/3712439742 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5637bb4b9b20 con 0x5637bb4ab370 2024-09-16T00:23:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b457fa640 1 -- 172.21.15.78:0/3712439742 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b54036030 con 0x5637bb4ab370 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b457fa640 1 -- 172.21.15.78:0/3712439742 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8b58021b90 con 0x5637bb4ab370 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b6661e740 10 client.26558 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b6661e740 10 client.26558 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b6661e740 20 client.26558 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b6661e740 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5637bb4bb030 0x5637bb4bd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b6661e740 4 client.26558 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b6661e740 1 -- 172.21.15.78:0/3712439742 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637bb413860 con 0x5637bb4bb030 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.802+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5637bb4bb030 0x5637bb4bd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.803+0000 7f8b457fa640 10 client.26558 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.803+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5637bb4bb030 0x5637bb4bd4c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ebe17e3c92cb3714 server_cookie=b6c2374936716067 in_seq=0 out_seq=0 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.803+0000 7f8b457fa640 1 -- 172.21.15.78:0/3712439742 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637bb413860 con 0x5637bb4bb030 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.803+0000 7f8b457fa640 10 client.26558 handle_command_reply: tid=0 2024-09-16T00:23:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.803+0000 7f8b6661e740 1 client.26558 shutdown 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 -- 172.21.15.78:0/3712439742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4ab370 msgr2=0x5637bb4a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4ab370 0x5637bb4a84d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800f010 tx=0x7f8b58004810 comp rx=0 tx=0).stop 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 -- 172.21.15.78:0/3712439742 shutdown_connections 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5637bb4bb030 0x5637bb4bd4c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b457fa640 0 client.26558 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4ab370 0x5637bb4a84d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a1b40 0x5637bb4a7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 --2- 172.21.15.78:0/3712439742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4a05d0 0x5637bb4aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 -- 172.21.15.78:0/3712439742 >> 172.21.15.78:0/3712439742 conn(0x5637bb469310 msgr2=0x5637bb4a2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 -- 172.21.15.78:0/3712439742 shutdown_connections 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 1 -- 172.21.15.78:0/3712439742 wait complete. 2024-09-16T00:23:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.804+0000 7f8b6661e740 20 client.26558 trim_cache size 0 max 16384 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.821+0000 7f8b6661e740 1 Processor -- start 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.821+0000 7f8b6661e740 1 -- start start 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb497c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b6661e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1bda70 con 0x5637bb4ab370 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b6661e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1bf5b0 con 0x5637bb4a1b40 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b6661e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1be140 con 0x5637bb4a05d0 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb497c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb497c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42758/0 (socket says 172.21.15.78:42758) 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 -- 172.21.15.78:0/2398181033 learned_addr learned my addr 172.21.15.78:0/2398181033 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:39.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b627e2640 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b617e0640 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 -- 172.21.15.78:0/2398181033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 msgr2=0x5637bb4a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 -- 172.21.15.78:0/2398181033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 msgr2=0x5637bb4a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4a3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 -- 172.21.15.78:0/2398181033 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637bb4a64c0 con 0x5637bb4a05d0 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.822+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb497c60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b54031af0 tx=0x7f8b5402f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eee0cce615d8b3a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.823+0000 7f8b477fe640 1 -- 172.21.15.78:0/2398181033 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54055030 con 0x5637bb4a05d0 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.823+0000 7f8b477fe640 1 -- 172.21.15.78:0/2398181033 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b54036040 con 0x5637bb4a05d0 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.823+0000 7f8b477fe640 1 -- 172.21.15.78:0/2398181033 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540042f0 con 0x5637bb4a05d0 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.823+0000 7f8b6661e740 1 -- 172.21.15.78:0/2398181033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 msgr2=0x5637bb497c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.823+0000 7f8b6661e740 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb497c60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b54031af0 tx=0x7f8b5402f510 comp rx=0 tx=0).stop 2024-09-16T00:23:39.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 -- 172.21.15.78:0/2398181033 shutdown_connections 2024-09-16T00:23:39.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 --2- 172.21.15.78:0/2398181033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb497c60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 -- 172.21.15.78:0/2398181033 >> 172.21.15.78:0/2398181033 conn(0x5637bb469310 msgr2=0x5637bb4a97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 -- 172.21.15.78:0/2398181033 shutdown_connections 2024-09-16T00:23:39.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 -- 172.21.15.78:0/2398181033 wait complete. 2024-09-16T00:23:39.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 -- start start 2024-09-16T00:23:39.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.824+0000 7f8b6661e740 1 Processor -- start 2024-09-16T00:23:39.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb4af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b6661e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b6661e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1beee0 con 0x5637bb4ab370 2024-09-16T00:23:39.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b6661e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1be810 con 0x5637bb4a1b40 2024-09-16T00:23:39.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b6661e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1bba50 con 0x5637bb4a05d0 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b6661e740 10 client.0 _ll_register_callbacks cb 0x7f8b580096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b617e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b617e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52200/0 (socket says 172.21.15.78:52200) 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b617e0640 1 -- 172.21.15.78:0/1167174770 learned_addr learned my addr 172.21.15.78:0/1167174770 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b627e2640 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb4af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b627e2640 1 -- 172.21.15.78:0/1167174770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 msgr2=0x5637bb4af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b627e2640 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb4af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b627e2640 1 -- 172.21.15.78:0/1167174770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 msgr2=0x5637bb4af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b627e2640 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.825+0000 7f8b627e2640 1 -- 172.21.15.78:0/1167174770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637bb524e60 con 0x5637bb4ab370 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.826+0000 7f8b627e2640 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4afda0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8b58021f80 tx=0x7f8b580044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77292ef904f2c173 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.826+0000 7f8b457fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.827+0000 7f8b457fa640 1 -- 172.21.15.78:0/1167174770 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58025040 con 0x5637bb4ab370 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.827+0000 7f8b457fa640 1 -- 172.21.15.78:0/1167174770 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8b58013c30 con 0x5637bb4ab370 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.827+0000 7f8b6661e740 1 -- 172.21.15.78:0/1167174770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b54036040 con 0x5637bb4ab370 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b457fa640 1 -- 172.21.15.78:0/1167174770 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8b58042050 con 0x5637bb4ab370 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b6661e740 10 client.17232 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b6661e740 1 -- 172.21.15.78:0/1167174770 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5637bb526140 con 0x5637bb4ab370 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b457fa640 1 -- 172.21.15.78:0/1167174770 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58039070 con 0x5637bb4ab370 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b457fa640 1 -- 172.21.15.78:0/1167174770 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8b58021690 con 0x5637bb4ab370 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b6661e740 10 client.17232 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b6661e740 10 client.17232 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b6661e740 20 client.17232 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b6661e740 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5637bb4bb030 0x5637bb528890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b6661e740 4 client.17232 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.828+0000 7f8b6661e740 1 -- 172.21.15.78:0/1167174770 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8b54036040 con 0x5637bb4bb030 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.829+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5637bb4bb030 0x5637bb528890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.829+0000 7f8b457fa640 10 client.17232 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.829+0000 7f8b61fe1640 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5637bb4bb030 0x5637bb528890 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=86fc1d7707f6bf67 server_cookie=715c2595ebe05be2 in_seq=0 out_seq=0 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.829+0000 7f8b457fa640 1 -- 172.21.15.78:0/1167174770 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8b54036040 con 0x5637bb4bb030 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.829+0000 7f8b457fa640 10 client.17232 handle_command_reply: tid=0 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.829+0000 7f8b6661e740 1 client.17232 shutdown 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.830+0000 7f8b6661e740 1 -- 172.21.15.78:0/1167174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 msgr2=0x5637bb4afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.830+0000 7f8b6661e740 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4afda0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8b58021f80 tx=0x7f8b580044c0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.830+0000 7f8b6661e740 1 -- 172.21.15.78:0/1167174770 shutdown_connections 2024-09-16T00:23:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.830+0000 7f8b6661e740 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5637bb4ab370 0x5637bb4afda0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.830+0000 7f8b6661e740 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5637bb4bb030 0x5637bb528890 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b6661e740 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5637bb4a1b40 0x5637bb4af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b6661e740 1 --2- 172.21.15.78:0/1167174770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5637bb4a05d0 0x5637bb4af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b6661e740 1 -- 172.21.15.78:0/1167174770 >> 172.21.15.78:0/1167174770 conn(0x5637bb469310 msgr2=0x5637bb49bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b457fa640 0 client.17232 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:23:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b6661e740 1 -- 172.21.15.78:0/1167174770 shutdown_connections 2024-09-16T00:23:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b6661e740 1 -- 172.21.15.78:0/1167174770 wait complete. 2024-09-16T00:23:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b6661e740 20 client.17232 trim_cache size 0 max 16384 2024-09-16T00:23:39.835 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b64245640 1 -- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b480a3340 msgr2=0x7f8b480a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b64245640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b480a3340 0x7f8b480a5800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8b580099a0 tx=0x7f8b580090e0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b64245640 1 -- 172.21.15.78:0/248760976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109830 msgr2=0x7f8b5c1a7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.831+0000 7f8b64245640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109830 0x7f8b5c1a7990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00b3f0 tx=0x7f8b4c00b8b0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.832+0000 7f8b64245640 1 -- 172.21.15.78:0/248760976 shutdown_connections 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.832+0000 7f8b64245640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8b480a3340 0x7f8b480a5800 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.832+0000 7f8b64245640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b5c11b140 0x7f8b5c1abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.832+0000 7f8b64245640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b5c109830 0x7f8b5c1a7990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.832+0000 7f8b64245640 1 --2- 172.21.15.78:0/248760976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b5c109450 0x7f8b5c1a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.832+0000 7f8b64245640 1 -- 172.21.15.78:0/248760976 >> 172.21.15.78:0/248760976 conn(0x7f8b5c06e140 msgr2=0x7f8b5c11bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.832+0000 7f8b64245640 1 -- 172.21.15.78:0/248760976 shutdown_connections 2024-09-16T00:23:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:39.832+0000 7f8b64245640 1 -- 172.21.15.78:0/248760976 wait complete. 2024-09-16T00:23:40.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:39 smithi078 ceph-mon[31157]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 603 op/s 2024-09-16T00:23:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:39 smithi193 ceph-mon[35484]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 603 op/s 2024-09-16T00:23:40.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:39 smithi148 ceph-mon[35599]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 603 op/s 2024-09-16T00:23:40.590 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:23:40.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 580.0 s, 745.2 tps, lat 132.808 ms stddev 149.868 2024-09-16T00:23:41.069 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:41.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.677+0000 7fa84096f640 1 -- 172.21.15.78:0/324633932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 msgr2=0x7fa83c106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.677+0000 7fa84096f640 1 --2- 172.21.15.78:0/324633932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 0x7fa83c106de0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa834009ca0 tx=0x7fa83402a8f0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.678+0000 7fa84096f640 1 -- 172.21.15.78:0/324633932 shutdown_connections 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.678+0000 7fa84096f640 1 --2- 172.21.15.78:0/324633932 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c1008c0 0x7fa83c104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.678+0000 7fa84096f640 1 --2- 172.21.15.78:0/324633932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa83c0fff40 0x7fa83c100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.678+0000 7fa84096f640 1 --2- 172.21.15.78:0/324633932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 0x7fa83c106de0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.678+0000 7fa84096f640 1 -- 172.21.15.78:0/324633932 >> 172.21.15.78:0/324633932 conn(0x7fa83c077070 msgr2=0x7fa83c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.678+0000 7fa84096f640 1 -- 172.21.15.78:0/324633932 shutdown_connections 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.678+0000 7fa84096f640 1 -- 172.21.15.78:0/324633932 wait complete. 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.679+0000 7fa84096f640 1 Processor -- start 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa84096f640 1 -- start start 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa84096f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa83c0fff40 0x7fa83c115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa84096f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c1008c0 0x7fa83c115840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa84096f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 0x7fa83c10e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa84096f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c118450 con 0x7fa83c106a00 2024-09-16T00:23:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa84096f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1182d0 con 0x7fa83c1008c0 2024-09-16T00:23:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa84096f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1185d0 con 0x7fa83c0fff40 2024-09-16T00:23:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa83b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 0x7fa83c10e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa83b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 0x7fa83c10e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35492/0 (socket says 172.21.15.78:35492) 2024-09-16T00:23:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa83b59e640 1 -- 172.21.15.78:0/4212141693 learned_addr learned my addr 172.21.15.78:0/4212141693 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa83a59c640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c1008c0 0x7fa83c115840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.680+0000 7fa83ad9d640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa83c0fff40 0x7fa83c115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa83b59e640 1 -- 172.21.15.78:0/4212141693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa83c0fff40 msgr2=0x7fa83c115300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa83b59e640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa83c0fff40 0x7fa83c115300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa83b59e640 1 -- 172.21.15.78:0/4212141693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c1008c0 msgr2=0x7fa83c115840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa83b59e640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c1008c0 0x7fa83c115840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa83b59e640 1 -- 172.21.15.78:0/4212141693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c10ed00 con 0x7fa83c106a00 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa83b59e640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 0x7fa83c10e440 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa83000d720 tx=0x7fa83000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa823fff640 1 -- 172.21.15.78:0/4212141693 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830017070 con 0x7fa83c106a00 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa823fff640 1 -- 172.21.15.78:0/4212141693 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa83000de30 con 0x7fa83c106a00 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.681+0000 7fa823fff640 1 -- 172.21.15.78:0/4212141693 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004d90 con 0x7fa83c106a00 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.682+0000 7fa84096f640 1 -- 172.21.15.78:0/4212141693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83c10eff0 con 0x7fa83c106a00 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.682+0000 7fa84096f640 1 -- 172.21.15.78:0/4212141693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83c113000 con 0x7fa83c106a00 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.687+0000 7fa843549740 1 Processor -- start 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.688+0000 7fa823fff640 1 -- 172.21.15.78:0/4212141693 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa830009bd0 con 0x7fa83c106a00 2024-09-16T00:23:41.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.688+0000 7fa843549740 1 -- start start 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.688+0000 7fa843549740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dc0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.688+0000 7fa843549740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dc0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.688+0000 7fa843549740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dcb520 0x55c381dcd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.689+0000 7fa843549740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c381d262f0 con 0x55c381dc1cf0 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.689+0000 7fa843549740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c381ccdcf0 con 0x55c381dc0780 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.689+0000 7fa843549740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c381d222b0 con 0x55c381dcb520 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.689+0000 7fa83ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dc0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.689+0000 7fa83ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dc0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:52224/0 (socket says 172.21.15.78:52224) 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.689+0000 7fa83ad9d640 1 -- 172.21.15.78:0/1057499481 learned_addr learned my addr 172.21.15.78:0/1057499481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.689+0000 7fa83a59c640 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dcb520 0x55c381dcd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa823fff640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8240a33e0 0x7fa8240a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa83a59c640 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dc0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa83b59e640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8240a33e0 0x7fa8240a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa83a59c640 1 -- 172.21.15.78:0/1057499481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dcb520 msgr2=0x55c381dcd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa83a59c640 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dcb520 0x55c381dcd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa83a59c640 1 -- 172.21.15.78:0/1057499481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 msgr2=0x55c381dc0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa83a59c640 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dc0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa83a59c640 1 -- 172.21.15.78:0/1057499481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c381cb23a0 con 0x55c381dc1cf0 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa83b59e640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8240a33e0 0x7fa8240a58a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa830098010 tx=0x7fa8300971b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.690+0000 7fa823fff640 1 -- 172.21.15.78:0/4212141693 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa8300cf380 con 0x7fa83c106a00 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.691+0000 7fa83a59c640 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dc0240 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55c381dc1220 tx=0x7fa82c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242f466ba6b405fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.692+0000 7fa8217fa640 1 -- 172.21.15.78:0/1057499481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c005020 con 0x55c381dc1cf0 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.692+0000 7fa8217fa640 1 -- 172.21.15.78:0/1057499481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82c0047a0 con 0x55c381dc1cf0 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 -- 172.21.15.78:0/1057499481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 msgr2=0x55c381dc0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dc0240 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55c381dc1220 tx=0x7fa82c00b390 comp rx=0 tx=0).stop 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 -- 172.21.15.78:0/1057499481 shutdown_connections 2024-09-16T00:23:41.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dcb520 0x55c381dcd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dc0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 --2- 172.21.15.78:0/1057499481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dc0240 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 -- 172.21.15.78:0/1057499481 >> 172.21.15.78:0/1057499481 conn(0x55c381d89310 msgr2=0x55c381dbce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:41.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 -- 172.21.15.78:0/1057499481 shutdown_connections 2024-09-16T00:23:41.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 -- 172.21.15.78:0/1057499481 wait complete. 2024-09-16T00:23:41.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 -- start start 2024-09-16T00:23:41.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.693+0000 7fa843549740 1 Processor -- start 2024-09-16T00:23:41.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa843549740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc0780 0x55c381dcad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa843549740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dc1cf0 0x55c381dc8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa843549740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dcb520 0x55c381dc85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa843549740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c381dce270 con 0x55c381dc0780 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa843549740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c381ce1060 con 0x55c381dcb520 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa843549740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c381dce3f0 con 0x55c381dc1cf0 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa843549740 10 client.0 _ll_register_callbacks cb 0x55c381a348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa83a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc0780 0x55c381dcad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa83a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc0780 0x55c381dcad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35516/0 (socket says 172.21.15.78:35516) 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa83a59c640 1 -- 172.21.15.78:0/2805644653 learned_addr learned my addr 172.21.15.78:0/2805644653 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa83a59c640 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dcb520 0x55c381dc85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa83ad9d640 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dc1cf0 0x55c381dc8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa83a59c640 1 -- 172.21.15.78:0/2805644653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dc1cf0 msgr2=0x55c381dc8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.694+0000 7fa83a59c640 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dc1cf0 0x55c381dc8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.695+0000 7fa83a59c640 1 -- 172.21.15.78:0/2805644653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dcb520 msgr2=0x55c381dc85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.695+0000 7fa83a59c640 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dcb520 0x55c381dc85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.695+0000 7fa83a59c640 1 -- 172.21.15.78:0/2805644653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c381a973c0 con 0x55c381dc0780 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.695+0000 7fa83a59c640 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc0780 0x55c381dcad90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00bc40 tx=0x7fa82c00bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135eb13b8e110338 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.695+0000 7fa81eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.696+0000 7fa81eff5640 1 -- 172.21.15.78:0/2805644653 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c023070 con 0x55c381dc0780 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.696+0000 7fa81eff5640 1 -- 172.21.15.78:0/2805644653 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa82c004450 con 0x55c381dc0780 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.696+0000 7fa843549740 1 -- 172.21.15.78:0/2805644653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82c0047a0 con 0x55c381dc0780 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.696+0000 7fa81eff5640 1 -- 172.21.15.78:0/2805644653 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa82c033050 con 0x55c381dc0780 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.696+0000 7fa843549740 10 client.17244 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.696+0000 7fa843549740 1 -- 172.21.15.78:0/2805644653 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c381dd9cc0 con 0x55c381dc0780 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.696+0000 7fa81eff5640 1 -- 172.21.15.78:0/2805644653 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82c0047a0 con 0x55c381dc0780 2024-09-16T00:23:41.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.697+0000 7fa81eff5640 1 -- 172.21.15.78:0/2805644653 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa82c01f560 con 0x55c381dc0780 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.697+0000 7fa843549740 10 client.17244 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.697+0000 7fa843549740 10 client.17244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.697+0000 7fa843549740 20 client.17244 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.697+0000 7fa843549740 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55c381ddb010 0x55c381ddd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.697+0000 7fa843549740 4 client.17244 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.697+0000 7fa843549740 1 -- 172.21.15.78:0/2805644653 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c381d33860 con 0x55c381ddb010 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.697+0000 7fa83ad9d640 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55c381ddb010 0x55c381ddd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.698+0000 7fa83ad9d640 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55c381ddb010 0x55c381ddd400 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ae3b088d9273a14f server_cookie=81c283938b90d3c8 in_seq=0 out_seq=0 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.698+0000 7fa81eff5640 10 client.17244 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa81eff5640 1 -- 172.21.15.78:0/2805644653 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c381d33860 con 0x55c381ddb010 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa81eff5640 10 client.17244 handle_command_reply: tid=0 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 client.17244 shutdown 2024-09-16T00:23:41.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 -- 172.21.15.78:0/2805644653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc0780 msgr2=0x55c381dcad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc0780 0x55c381dcad90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00bc40 tx=0x7fa82c00bbd0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 -- 172.21.15.78:0/2805644653 shutdown_connections 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55c381ddb010 0x55c381ddd400 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dcb520 0x55c381dc85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381dc1cf0 0x55c381dc8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 --2- 172.21.15.78:0/2805644653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc0780 0x55c381dcad90 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa843549740 1 -- 172.21.15.78:0/2805644653 >> 172.21.15.78:0/2805644653 conn(0x55c381d89310 msgr2=0x55c381dc2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.699+0000 7fa81eff5640 0 client.17244 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.700+0000 7fa843549740 1 -- 172.21.15.78:0/2805644653 shutdown_connections 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.700+0000 7fa843549740 1 -- 172.21.15.78:0/2805644653 wait complete. 2024-09-16T00:23:41.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.700+0000 7fa843549740 20 client.17244 trim_cache size 0 max 16384 2024-09-16T00:23:41.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.725+0000 7fa843549740 1 Processor -- start 2024-09-16T00:23:41.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.725+0000 7fa843549740 1 -- start start 2024-09-16T00:23:41.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.725+0000 7fa843549740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dd75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.725+0000 7fa843549740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dcd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa843549740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dcc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa83a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dd75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa83a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dd75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42802/0 (socket says 172.21.15.78:42802) 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa83a59c640 1 -- 172.21.15.78:0/3614598636 learned_addr learned my addr 172.21.15.78:0/3614598636 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa843549740 1 -- 172.21.15.78:0/3614598636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa82800fc00 con 0x55c381dc1cf0 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa843549740 1 -- 172.21.15.78:0/3614598636 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa828011740 con 0x55c381dc0780 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa843549740 1 -- 172.21.15.78:0/3614598636 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280102d0 con 0x55c381d89310 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa83a59c640 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dcc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.726+0000 7fa83ad9d640 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dcd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.727+0000 7fa83a59c640 1 -- 172.21.15.78:0/3614598636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 msgr2=0x55c381dd75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.727+0000 7fa83a59c640 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dd75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.727+0000 7fa83a59c640 1 -- 172.21.15.78:0/3614598636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 msgr2=0x55c381dcd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.727+0000 7fa83a59c640 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dcd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.727+0000 7fa83a59c640 1 -- 172.21.15.78:0/3614598636 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c381dc90b0 con 0x55c381dc1cf0 2024-09-16T00:23:41.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.727+0000 7fa83a59c640 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dcc910 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa82c01fad0 tx=0x7fa82c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256ed2b287dae8cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:41.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.732+0000 7fa820ff9640 1 -- 172.21.15.78:0/3614598636 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c016070 con 0x55c381dc1cf0 2024-09-16T00:23:41.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.733+0000 7fa820ff9640 1 -- 172.21.15.78:0/3614598636 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa82c03d070 con 0x55c381dc1cf0 2024-09-16T00:23:41.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.733+0000 7fa820ff9640 1 -- 172.21.15.78:0/3614598636 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c023040 con 0x55c381dc1cf0 2024-09-16T00:23:41.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.734+0000 7fa843549740 1 -- 172.21.15.78:0/3614598636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 msgr2=0x55c381dcc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.734+0000 7fa843549740 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dcc910 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa82c01fad0 tx=0x7fa82c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.735+0000 7fa843549740 1 -- 172.21.15.78:0/3614598636 shutdown_connections 2024-09-16T00:23:41.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.735+0000 7fa843549740 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dc1cf0 0x55c381dcc910 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.735+0000 7fa843549740 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dcd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.735+0000 7fa843549740 1 --2- 172.21.15.78:0/3614598636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dd75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.735+0000 7fa843549740 1 -- 172.21.15.78:0/3614598636 >> 172.21.15.78:0/3614598636 conn(0x55c381dc8730 msgr2=0x55c381dbbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:41.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.739+0000 7fa843549740 1 -- 172.21.15.78:0/3614598636 shutdown_connections 2024-09-16T00:23:41.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.742+0000 7fa843549740 1 -- 172.21.15.78:0/3614598636 wait complete. 2024-09-16T00:23:41.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.742+0000 7fa843549740 1 -- start start 2024-09-16T00:23:41.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.742+0000 7fa843549740 1 Processor -- start 2024-09-16T00:23:41.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.743+0000 7fa843549740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.743+0000 7fa843549740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dcd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.743+0000 7fa83a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa83a59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:42806/0 (socket says 172.21.15.78:42806) 2024-09-16T00:23:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa83a59c640 1 -- 172.21.15.78:0/2014525082 learned_addr learned my addr 172.21.15.78:0/2014525082 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa83ad9d640 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dcd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa843549740 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dcb520 0x55c381dcd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa828011070 con 0x55c381dcb520 2024-09-16T00:23:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa8280109a0 con 0x55c381dc0780 2024-09-16T00:23:41.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa82800dbe0 con 0x55c381d89310 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa83a59c640 1 -- 172.21.15.78:0/2014525082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 msgr2=0x55c381dcd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa83a59c640 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dcd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.744+0000 7fa83a59c640 1 -- 172.21.15.78:0/2014525082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dcb520 msgr2=0x55c381dcd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.745+0000 7fa83a59c640 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dcb520 0x55c381dcd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.745+0000 7fa83a59c640 1 -- 172.21.15.78:0/2014525082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c381dd2210 con 0x55c381d89310 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.745+0000 7fa843549740 10 client.0 _ll_register_callbacks cb 0x7fa834009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.745+0000 7fa83a59c640 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dccb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009c90 tx=0x7fa82c00b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa584e556d9f2123 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.745+0000 7fa81eff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.745+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82c03d070 con 0x55c381d89310 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.745+0000 7fa81eff5640 1 -- 172.21.15.78:0/2014525082 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c046060 con 0x55c381d89310 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.746+0000 7fa81eff5640 1 -- 172.21.15.78:0/2014525082 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa82c0052d0 con 0x55c381d89310 2024-09-16T00:23:41.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.746+0000 7fa81eff5640 1 -- 172.21.15.78:0/2014525082 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa82c04d050 con 0x55c381d89310 2024-09-16T00:23:41.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.751+0000 7fa843549740 10 client.26395 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:41.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.751+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55c381e45f40 con 0x55c381d89310 2024-09-16T00:23:41.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.752+0000 7fa81eff5640 1 -- 172.21.15.78:0/2014525082 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa82c0054d0 con 0x55c381d89310 2024-09-16T00:23:41.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.752+0000 7fa843549740 10 client.26395 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:41.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.752+0000 7fa843549740 10 client.26395 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:23:41.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.752+0000 7fa843549740 20 client.26395 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:41.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.752+0000 7fa843549740 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55c381e46400 0x55c381e48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.752+0000 7fa843549740 4 client.26395 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:23:41.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.752+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa82c03d070 con 0x55c381e46400 2024-09-16T00:23:41.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.752+0000 7fa83ad9d640 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55c381e46400 0x55c381e48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.753+0000 7fa83ad9d640 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55c381e46400 0x55c381e48810 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58f5d5840353a601 server_cookie=81e08010ec6fc708 in_seq=0 out_seq=0 2024-09-16T00:23:41.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.753+0000 7fa81eff5640 10 client.26395 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:23:41.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.753+0000 7fa81eff5640 1 -- 172.21.15.78:0/2014525082 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa82c03d070 con 0x55c381e46400 2024-09-16T00:23:41.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.753+0000 7fa81eff5640 10 client.26395 handle_command_reply: tid=0 2024-09-16T00:23:41.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.753+0000 7fa843549740 1 client.26395 shutdown 2024-09-16T00:23:41.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.759+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 msgr2=0x55c381dccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.759+0000 7fa843549740 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dccb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009c90 tx=0x7fa82c00b570 comp rx=0 tx=0).stop 2024-09-16T00:23:41.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.759+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 shutdown_connections 2024-09-16T00:23:41.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.759+0000 7fa843549740 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c381dcb520 0x55c381dcd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.759+0000 7fa843549740 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55c381e46400 0x55c381e48810 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.760+0000 7fa843549740 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c381dc0780 0x55c381dcd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.760+0000 7fa81eff5640 0 client.26395 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:23:41.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.760+0000 7fa843549740 1 --2- 172.21.15.78:0/2014525082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c381d89310 0x55c381dccb30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.760+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 >> 172.21.15.78:0/2014525082 conn(0x55c381dc8730 msgr2=0x55c381dc98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:41.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.760+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 shutdown_connections 2024-09-16T00:23:41.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.760+0000 7fa843549740 1 -- 172.21.15.78:0/2014525082 wait complete. 2024-09-16T00:23:41.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.760+0000 7fa843549740 20 client.26395 trim_cache size 0 max 16384 2024-09-16T00:23:41.765 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:23:41.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.766+0000 7fa84096f640 1 -- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8240a33e0 msgr2=0x7fa8240a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.766+0000 7fa84096f640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8240a33e0 0x7fa8240a58a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa830098010 tx=0x7fa8300971b0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.766+0000 7fa84096f640 1 -- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 msgr2=0x7fa83c10e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.766+0000 7fa84096f640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 0x7fa83c10e440 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa83000d720 tx=0x7fa83000dbe0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.768+0000 7fa84096f640 1 -- 172.21.15.78:0/4212141693 shutdown_connections 2024-09-16T00:23:41.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.768+0000 7fa84096f640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa8240a33e0 0x7fa8240a58a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.768+0000 7fa84096f640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa83c106a00 0x7fa83c10e440 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.768+0000 7fa84096f640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa83c1008c0 0x7fa83c115840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.768+0000 7fa84096f640 1 --2- 172.21.15.78:0/4212141693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa83c0fff40 0x7fa83c115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.768+0000 7fa84096f640 1 -- 172.21.15.78:0/4212141693 >> 172.21.15.78:0/4212141693 conn(0x7fa83c077070 msgr2=0x7fa83c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:41.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.769+0000 7fa84096f640 1 -- 172.21.15.78:0/4212141693 shutdown_connections 2024-09-16T00:23:41.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:41.769+0000 7fa84096f640 1 -- 172.21.15.78:0/4212141693 wait complete. 2024-09-16T00:23:41.793 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:41 smithi078 ceph-mon[31157]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 577 op/s 2024-09-16T00:23:42.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:41 smithi193 ceph-mon[35484]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 577 op/s 2024-09-16T00:23:42.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:41 smithi148 ceph-mon[35599]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 MiB/s wr, 577 op/s 2024-09-16T00:23:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:43 smithi193 ceph-mon[35484]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 610 op/s 2024-09-16T00:23:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:43 smithi148 ceph-mon[35599]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 610 op/s 2024-09-16T00:23:44.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:43 smithi078 ceph-mon[31157]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 610 op/s 2024-09-16T00:23:45.637 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:23:45.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 585.0 s, 557.8 tps, lat 176.966 ms stddev 223.569 2024-09-16T00:23:46.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:45 smithi078 ceph-mon[31157]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-09-16T00:23:46.061 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:45 smithi193 ceph-mon[35484]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-09-16T00:23:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:45 smithi148 ceph-mon[35599]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-09-16T00:23:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.682+0000 7fb3b1ffb640 1 -- 172.21.15.78:0/1657050840 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac030d30 con 0x7fb3b4109830 2024-09-16T00:23:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 -- 172.21.15.78:0/1657050840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b4109830 msgr2=0x7fb3b4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/1657050840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b4109830 0x7fb3b4109c90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009ca0 tx=0x7fb3ac02a8b0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 -- 172.21.15.78:0/1657050840 shutdown_connections 2024-09-16T00:23:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/1657050840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b411b140 0x7fb3b411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/1657050840 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b4109830 0x7fb3b4109c90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/1657050840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b4109450 0x7fb3b4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 -- 172.21.15.78:0/1657050840 >> 172.21.15.78:0/1657050840 conn(0x7fb3b406e140 msgr2=0x7fb3b406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:46.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 -- 172.21.15.78:0/1657050840 shutdown_connections 2024-09-16T00:23:46.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.683+0000 7fb3b99b1640 1 -- 172.21.15.78:0/1657050840 wait complete. 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.684+0000 7fb3b99b1640 1 Processor -- start 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.684+0000 7fb3b99b1640 1 -- start start 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.684+0000 7fb3b99b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b4109450 0x7fb3b41a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.684+0000 7fb3b99b1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b4109830 0x7fb3b41a7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.684+0000 7fb3b99b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b411b140 0x7fb3b41abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.684+0000 7fb3b99b1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4120e10 con 0x7fb3b4109450 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.684+0000 7fb3b99b1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4120c90 con 0x7fb3b411b140 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.684+0000 7fb3b99b1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4120f90 con 0x7fb3b4109830 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b4109830 0x7fb3b41a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b4109830 0x7fb3b41a7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:58068/0 (socket says 172.21.15.78:58068) 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 -- 172.21.15.78:0/731303679 learned_addr learned my addr 172.21.15.78:0/731303679 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b4109450 0x7fb3b41a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b37fe640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b411b140 0x7fb3b41abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 -- 172.21.15.78:0/731303679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b411b140 msgr2=0x7fb3b41abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b411b140 0x7fb3b41abba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 -- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b4109450 msgr2=0x7fb3b41a72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b4109450 0x7fb3b41a72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 -- 172.21.15.78:0/731303679 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b41ac280 con 0x7fb3b4109830 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b4109450 0x7fb3b41a72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.685+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b4109830 0x7fb3b41a7810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac02aea0 tx=0x7fb3ac003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.686+0000 7fb3b89af640 1 -- 172.21.15.78:0/731303679 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac030e80 con 0x7fb3b4109830 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.686+0000 7fb3b89af640 1 -- 172.21.15.78:0/731303679 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac040070 con 0x7fb3b4109830 2024-09-16T00:23:46.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.686+0000 7fb3b89af640 1 -- 172.21.15.78:0/731303679 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac045b20 con 0x7fb3b4109830 2024-09-16T00:23:46.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.686+0000 7fb3b99b1640 1 -- 172.21.15.78:0/731303679 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b41ac510 con 0x7fb3b4109830 2024-09-16T00:23:46.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.686+0000 7fb3b99b1640 1 -- 172.21.15.78:0/731303679 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b41b3e50 con 0x7fb3b4109830 2024-09-16T00:23:46.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.692+0000 7fb3b89af640 1 -- 172.21.15.78:0/731303679 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb3ac045cc0 con 0x7fb3b4109830 2024-09-16T00:23:46.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.693+0000 7fb3b89af640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb39c0a3380 0x7fb39c0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.693+0000 7fb3b89af640 1 -- 172.21.15.78:0/731303679 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb3ac0f13d0 con 0x7fb3b4109830 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.694+0000 7fb3bbd8a740 1 Processor -- start 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.694+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb39c0a3380 0x7fb39c0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.694+0000 7fb3bbd8a740 1 -- start start 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.694+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136dbb40 0x5594136da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136da5d0 0x5594136daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3bbd8a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594136402f0 con 0x5594136dbb40 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3bbd8a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5594135e7cf0 con 0x5594136da5d0 2024-09-16T00:23:46.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3bbd8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55941363c2b0 con 0x5594136e5370 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136da5d0 0x5594136daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136da5d0 0x5594136daa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37306/0 (socket says 172.21.15.78:37306) 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b27fc640 1 -- 172.21.15.78:0/3040555154 learned_addr learned my addr 172.21.15.78:0/3040555154 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b37fe640 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136dbb40 0x5594136da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b27fc640 1 -- 172.21.15.78:0/3040555154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 msgr2=0x5594136e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b27fc640 1 -- 172.21.15.78:0/3040555154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136dbb40 msgr2=0x5594136da090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136dbb40 0x5594136da090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.695+0000 7fb3b27fc640 1 -- 172.21.15.78:0/3040555154 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594135cc3a0 con 0x5594136da5d0 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.696+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136da5d0 0x5594136daa10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ba000 tx=0x7fb3ac0455e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15b770d8a5c5c002 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.696+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb39c0a3380 0x7fb39c0a5840 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009fd0 tx=0x7fb3a80098b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:46.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.696+0000 7fb399ffb640 1 -- 172.21.15.78:0/3040555154 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0bd630 con 0x5594136da5d0 2024-09-16T00:23:46.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.696+0000 7fb399ffb640 1 -- 172.21.15.78:0/3040555154 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac036030 con 0x5594136da5d0 2024-09-16T00:23:46.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.696+0000 7fb399ffb640 1 -- 172.21.15.78:0/3040555154 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0b9ca0 con 0x5594136da5d0 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3040555154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136da5d0 msgr2=0x5594136daa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136da5d0 0x5594136daa10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ba000 tx=0x7fb3ac0455e0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3040555154 shutdown_connections 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136da5d0 0x5594136daa10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3040555154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136dbb40 0x5594136da090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3040555154 >> 172.21.15.78:0/3040555154 conn(0x5594136a3310 msgr2=0x5594136d6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3040555154 shutdown_connections 2024-09-16T00:23:46.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3040555154 wait complete. 2024-09-16T00:23:46.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.697+0000 7fb3bbd8a740 1 -- start start 2024-09-16T00:23:46.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3bbd8a740 1 Processor -- start 2024-09-16T00:23:46.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136e2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 0x5594136e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3bbd8a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594136e7e30 con 0x5594136e5370 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3bbd8a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5594135fb060 con 0x5594136dbb40 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3bbd8a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594136e7fb0 con 0x5594136da5d0 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3bbd8a740 10 client.0 _ll_register_callbacks cb 0x55941334e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.698+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 0x5594136e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 0x5594136e2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47322/0 (socket says 172.21.15.78:47322) 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b37fe640 1 -- 172.21.15.78:0/3445431756 learned_addr learned my addr 172.21.15.78:0/3445431756 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136e2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b2ffd640 1 -- 172.21.15.78:0/3445431756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 msgr2=0x5594136e2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136e2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b2ffd640 1 -- 172.21.15.78:0/3445431756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 msgr2=0x5594136e2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:46.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 0x5594136e2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b2ffd640 1 -- 172.21.15.78:0/3445431756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594133b13c0 con 0x5594136da5d0 2024-09-16T00:23:46.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136e4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8018da0 tx=0x7fb3a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20f67693a62595e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:46.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:23:46.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb392ffd640 1 -- 172.21.15.78:0/3445431756 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8029060 con 0x5594136da5d0 2024-09-16T00:23:46.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb392ffd640 1 -- 172.21.15.78:0/3445431756 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb3a8017d70 con 0x5594136da5d0 2024-09-16T00:23:46.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.699+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3445431756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3ac036030 con 0x5594136da5d0 2024-09-16T00:23:46.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb392ffd640 1 -- 172.21.15.78:0/3445431756 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb3a8030050 con 0x5594136da5d0 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3bbd8a740 10 client.26404 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3445431756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5594136f3840 con 0x5594136da5d0 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb392ffd640 1 -- 172.21.15.78:0/3445431756 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac036030 con 0x5594136da5d0 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb392ffd640 1 -- 172.21.15.78:0/3445431756 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb3a8022410 con 0x5594136da5d0 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3bbd8a740 10 client.26404 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3bbd8a740 10 client.26404 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3bbd8a740 20 client.26404 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5594136f4d00 0x5594136f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3bbd8a740 4 client.26404 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3445431756 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55941364d860 con 0x5594136f4d00 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.700+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5594136f4d00 0x5594136f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.701+0000 7fb392ffd640 10 client.26404 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:23:46.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.701+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5594136f4d00 0x5594136f7110 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3403de8704347e3 server_cookie=cc9587f0a7ccbf5c in_seq=0 out_seq=0 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb392ffd640 1 -- 172.21.15.78:0/3445431756 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55941364d860 con 0x5594136f4d00 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb392ffd640 10 client.26404 handle_command_reply: tid=0 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 client.26404 shutdown 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 msgr2=0x5594136e4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136e4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8018da0 tx=0x7fb3a80046c0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3445431756 shutdown_connections 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5594136f4d00 0x5594136f7110 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 0x5594136e2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136e2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/3445431756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136e4dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3445431756 >> 172.21.15.78:0/3445431756 conn(0x5594136a3310 msgr2=0x5594136dc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:46.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb392ffd640 0 client.26404 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:23:46.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3445431756 shutdown_connections 2024-09-16T00:23:46.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/3445431756 wait complete. 2024-09-16T00:23:46.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.702+0000 7fb3bbd8a740 20 client.26404 trim_cache size 0 max 16384 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.719+0000 7fb3bbd8a740 1 Processor -- start 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.719+0000 7fb3bbd8a740 1 -- start start 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136da5d0 0x5594136d6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136d6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3bbd8a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41bda10 con 0x5594136da5d0 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3bbd8a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41bf550 con 0x5594136dbb40 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3b27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136d6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3b27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136d6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37312/0 (socket says 172.21.15.78:37312) 2024-09-16T00:23:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3b27fc640 1 -- 172.21.15.78:0/299951216 learned_addr learned my addr 172.21.15.78:0/299951216 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/299951216 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41be0e0 con 0x5594136e5370 2024-09-16T00:23:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.720+0000 7fb3b37fe640 1 --2- 172.21.15.78:0/299951216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.721+0000 7fb3b37fe640 1 -- 172.21.15.78:0/299951216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 msgr2=0x5594136d6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.721+0000 7fb3b37fe640 1 --2- 172.21.15.78:0/299951216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136d6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.721+0000 7fb3b37fe640 1 -- 172.21.15.78:0/299951216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136da5d0 msgr2=0x5594136d6700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:23:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.721+0000 7fb3b37fe640 1 --2- 172.21.15.78:0/299951216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136da5d0 0x5594136d6700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.721+0000 7fb3b37fe640 1 -- 172.21.15.78:0/299951216 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594136d7180 con 0x5594136e5370 2024-09-16T00:23:46.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.725+0000 7fb3b37fe640 1 --2- 172.21.15.78:0/299951216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136fc9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009c50 tx=0x7fb3a40127c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4457f83ac59ec9d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:46.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.728+0000 7fb3997fa640 1 -- 172.21.15.78:0/299951216 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4009d40 con 0x5594136e5370 2024-09-16T00:23:46.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.728+0000 7fb3997fa640 1 -- 172.21.15.78:0/299951216 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a4005020 con 0x5594136e5370 2024-09-16T00:23:46.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.728+0000 7fb3997fa640 1 -- 172.21.15.78:0/299951216 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4004b20 con 0x5594136e5370 2024-09-16T00:23:46.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.729+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/299951216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 msgr2=0x5594136fc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.729+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/299951216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136fc9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009c50 tx=0x7fb3a40127c0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.729+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/299951216 shutdown_connections 2024-09-16T00:23:46.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.729+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/299951216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136e5370 0x5594136fc9e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.729+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/299951216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136d6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.729+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/299951216 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136da5d0 0x5594136d6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.729+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/299951216 >> 172.21.15.78:0/299951216 conn(0x5594136a3310 msgr2=0x5594136dd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:46.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.733+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/299951216 shutdown_connections 2024-09-16T00:23:46.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.733+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/299951216 wait complete. 2024-09-16T00:23:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.733+0000 7fb3bbd8a740 1 -- start start 2024-09-16T00:23:46.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.734+0000 7fb3bbd8a740 1 Processor -- start 2024-09-16T00:23:46.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.734+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.734+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136deaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3bbd8a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 0x5594136defe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3b2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3b2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136ed580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:58100/0 (socket says 172.21.15.78:58100) 2024-09-16T00:23:46.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3b27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136deaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3b27fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136deaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37322/0 (socket says 172.21.15.78:37322) 2024-09-16T00:23:46.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3b2ffd640 1 -- 172.21.15.78:0/2521089152 learned_addr learned my addr 172.21.15.78:0/2521089152 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:46.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41bee80 con 0x5594136e5370 2024-09-16T00:23:46.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41be7b0 con 0x5594136dbb40 2024-09-16T00:23:46.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41bb9f0 con 0x5594136da5d0 2024-09-16T00:23:46.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.735+0000 7fb3bbd8a740 10 client.0 _ll_register_callbacks cb 0x7fb3a801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:46.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.737+0000 7fb3b27fc640 1 -- 172.21.15.78:0/2521089152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 msgr2=0x5594136ed580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:46.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.737+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136ed580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.737+0000 7fb3b27fc640 1 -- 172.21.15.78:0/2521089152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 msgr2=0x5594136defe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:46.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.737+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 0x5594136defe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.737+0000 7fb3b27fc640 1 -- 172.21.15.78:0/2521089152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594136df520 con 0x5594136dbb40 2024-09-16T00:23:46.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.740+0000 7fb3b27fc640 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136deaa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0bd5c0 tx=0x7fb3ac0bd170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b61fd3356e4a35f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:46.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.741+0000 7fb392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:23:46.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.741+0000 7fb392ffd640 1 -- 172.21.15.78:0/2521089152 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0d5020 con 0x5594136dbb40 2024-09-16T00:23:46.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.741+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a4005020 con 0x5594136dbb40 2024-09-16T00:23:46.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.744+0000 7fb392ffd640 1 -- 172.21.15.78:0/2521089152 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb3ac0b9ca0 con 0x5594136dbb40 2024-09-16T00:23:46.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.744+0000 7fb392ffd640 1 -- 172.21.15.78:0/2521089152 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb3ac0bd890 con 0x5594136dbb40 2024-09-16T00:23:46.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.746+0000 7fb3bbd8a740 10 client.26597 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:46.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.746+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55941375f6a0 con 0x5594136dbb40 2024-09-16T00:23:46.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.750+0000 7fb392ffd640 1 -- 172.21.15.78:0/2521089152 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac0a5070 con 0x5594136dbb40 2024-09-16T00:23:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.752+0000 7fb392ffd640 1 -- 172.21.15.78:0/2521089152 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb3ac04d070 con 0x5594136dbb40 2024-09-16T00:23:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.752+0000 7fb3bbd8a740 10 client.26597 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.752+0000 7fb3bbd8a740 10 client.26597 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:23:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.752+0000 7fb3bbd8a740 20 client.26597 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.752+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55941375fb70 0x559413761f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:46.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.753+0000 7fb3bbd8a740 4 client.26597 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:23:46.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.753+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb3a4005020 con 0x55941375fb70 2024-09-16T00:23:46.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.760+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55941375fb70 0x559413761f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:46.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.770+0000 7fb3b2ffd640 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55941375fb70 0x559413761f90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee336109a585afb7 server_cookie=7d2d691b6beb3d18 in_seq=0 out_seq=0 2024-09-16T00:23:46.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.770+0000 7fb392ffd640 10 client.26597 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:23:46.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.771+0000 7fb392ffd640 1 -- 172.21.15.78:0/2521089152 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb3a4005020 con 0x55941375fb70 2024-09-16T00:23:46.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.771+0000 7fb392ffd640 10 client.26597 handle_command_reply: tid=0 2024-09-16T00:23:46.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.771+0000 7fb3bbd8a740 1 client.26597 shutdown 2024-09-16T00:23:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.771+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 msgr2=0x5594136deaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.771+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136deaa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0bd5c0 tx=0x7fb3ac0bd170 comp rx=0 tx=0).stop 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 shutdown_connections 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55941375fb70 0x559413761f90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594136e5370 0x5594136defe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594136dbb40 0x5594136deaa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb3bbd8a740 1 --2- 172.21.15.78:0/2521089152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594136da5d0 0x5594136ed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 >> 172.21.15.78:0/2521089152 conn(0x5594136a3310 msgr2=0x5594136dc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb392ffd640 0 client.26597 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 shutdown_connections 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.772+0000 7fb3bbd8a740 1 -- 172.21.15.78:0/2521089152 wait complete. 2024-09-16T00:23:46.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.773+0000 7fb3bbd8a740 20 client.26597 trim_cache size 0 max 16384 2024-09-16T00:23:46.776 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:23:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 -- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb39c0a3380 msgr2=0x7fb39c0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb39c0a3380 0x7fb39c0a5840 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009fd0 tx=0x7fb3a80098b0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 -- 172.21.15.78:0/731303679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b4109830 msgr2=0x7fb3b41a7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b4109830 0x7fb3b41a7810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac02aea0 tx=0x7fb3ac003f10 comp rx=0 tx=0).stop 2024-09-16T00:23:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 -- 172.21.15.78:0/731303679 shutdown_connections 2024-09-16T00:23:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb39c0a3380 0x7fb39c0a5840 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3b411b140 0x7fb3b41abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b4109830 0x7fb3b41a7810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 --2- 172.21.15.78:0/731303679 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3b4109450 0x7fb3b41a72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 -- 172.21.15.78:0/731303679 >> 172.21.15.78:0/731303679 conn(0x7fb3b406e140 msgr2=0x7fb3b411be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 -- 172.21.15.78:0/731303679 shutdown_connections 2024-09-16T00:23:46.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:46.776+0000 7fb3b99b1640 1 -- 172.21.15.78:0/731303679 wait complete. 2024-09-16T00:23:47.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:23:48.023 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:47 smithi078 ceph-mon[31157]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-09-16T00:23:48.150 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:47 smithi193 ceph-mon[35484]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-09-16T00:23:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:47 smithi148 ceph-mon[35599]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-09-16T00:23:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 -- 172.21.15.78:0/2370911554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff318113940 msgr2=0x7ff318115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2370911554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff318113940 0x7ff318115d30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff310009ca0 tx=0x7ff31002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 -- 172.21.15.78:0/2370911554 shutdown_connections 2024-09-16T00:23:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2370911554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff318113940 0x7ff318115d30 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2370911554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff31810ae80 0x7ff318113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2370911554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff31810a560 0x7ff31810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 -- 172.21.15.78:0/2370911554 >> 172.21.15.78:0/2370911554 conn(0x7ff31806e140 msgr2=0x7ff31806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 -- 172.21.15.78:0/2370911554 shutdown_connections 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.781+0000 7ff31d04d640 1 -- 172.21.15.78:0/2370911554 wait complete. 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.782+0000 7ff31d04d640 1 Processor -- start 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.782+0000 7ff31d04d640 1 -- start start 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31d04d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff31810a560 0x7ff3181a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31d04d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31810ae80 0x7ff3181a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31d04d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff318113940 0x7ff3181a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31d04d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff318119630 con 0x7ff31810ae80 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31d04d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181194b0 con 0x7ff31810a560 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31d04d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181197b0 con 0x7ff318113940 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff31810a560 0x7ff3181a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31810ae80 0x7ff3181a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff31810a560 0x7ff3181a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37346/0 (socket says 172.21.15.78:37346) 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 -- 172.21.15.78:0/2314176504 learned_addr learned my addr 172.21.15.78:0/2314176504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31759e640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff318113940 0x7ff3181a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 -- 172.21.15.78:0/2314176504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff318113940 msgr2=0x7ff3181a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff318113940 0x7ff3181a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 -- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31810ae80 msgr2=0x7ff3181a6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31810ae80 0x7ff3181a6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 -- 172.21.15.78:0/2314176504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3181a0f90 con 0x7ff31810a560 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff31659c640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31810ae80 0x7ff3181a6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.783+0000 7ff316d9d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff31810a560 0x7ff3181a6670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9f0 tx=0x7ff30800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.784+0000 7ff2fffff640 1 -- 172.21.15.78:0/2314176504 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30800cd80 con 0x7ff31810a560 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.784+0000 7ff2fffff640 1 -- 172.21.15.78:0/2314176504 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff308010910 con 0x7ff31810a560 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.784+0000 7ff31d04d640 1 -- 172.21.15.78:0/2314176504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3181a1280 con 0x7ff31810a560 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.784+0000 7ff31d04d640 1 -- 172.21.15.78:0/2314176504 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff318113d20 con 0x7ff31810a560 2024-09-16T00:23:48.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.784+0000 7ff2fffff640 1 -- 172.21.15.78:0/2314176504 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308018950 con 0x7ff31810a560 2024-09-16T00:23:48.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31f426740 1 Processor -- start 2024-09-16T00:23:48.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31f426740 1 -- start start 2024-09-16T00:23:48.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff2fffff640 1 -- 172.21.15.78:0/2314176504 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff308018c10 con 0x7ff31810a560 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31f426740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae14090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31f426740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae14a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31f426740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae27b80 0x560aaae29f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31f426740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560aaad7a2f0 con 0x560aaae27b80 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31f426740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560aaad21cf0 con 0x560aaae145d0 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31f426740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560aaad762b0 con 0x560aaae15b40 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae27b80 0x560aaae29f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae27b80 0x560aaae29f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47348/0 (socket says 172.21.15.78:47348) 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31659c640 1 -- 172.21.15.78:0/887055766 learned_addr learned my addr 172.21.15.78:0/887055766 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff316d9d640 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae14a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.788+0000 7ff31759e640 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae14090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff31659c640 1 -- 172.21.15.78:0/887055766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 msgr2=0x560aaae14090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff31659c640 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae14090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff31659c640 1 -- 172.21.15.78:0/887055766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 msgr2=0x560aaae14a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff31659c640 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae14a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff31659c640 1 -- 172.21.15.78:0/887055766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aaad063a0 con 0x560aaae27b80 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff31759e640 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae14090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff31659c640 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae27b80 0x560aaae29f70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009c30 tx=0x7ff30c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535c9ca6fcf01961 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff2fd7fa640 1 -- 172.21.15.78:0/887055766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c004020 con 0x560aaae27b80 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.789+0000 7ff2fd7fa640 1 -- 172.21.15.78:0/887055766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30c0041c0 con 0x560aaae27b80 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.790+0000 7ff2fd7fa640 1 -- 172.21.15.78:0/887055766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c012780 con 0x560aaae27b80 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.790+0000 7ff2fffff640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3000a32c0 0x7ff3000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.790+0000 7ff31759e640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3000a32c0 0x7ff3000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.790+0000 7ff2fffff640 1 -- 172.21.15.78:0/2314176504 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff3080d0060 con 0x7ff31810a560 2024-09-16T00:23:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31759e640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3000a32c0 0x7ff3000a5780 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff310005e30 tx=0x7ff31003e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 -- 172.21.15.78:0/887055766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae27b80 msgr2=0x560aaae29f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae27b80 0x560aaae29f70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009c30 tx=0x7ff30c0117b0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 -- 172.21.15.78:0/887055766 shutdown_connections 2024-09-16T00:23:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae27b80 0x560aaae29f70 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae14a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 --2- 172.21.15.78:0/887055766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae14090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 -- 172.21.15.78:0/887055766 >> 172.21.15.78:0/887055766 conn(0x560aaaddd310 msgr2=0x560aaae10c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 -- 172.21.15.78:0/887055766 shutdown_connections 2024-09-16T00:23:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 -- 172.21.15.78:0/887055766 wait complete. 2024-09-16T00:23:48.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 -- start start 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.791+0000 7ff31f426740 1 Processor -- start 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31f426740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae145d0 0x560aaae27480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31f426740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae24730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31f426740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae27b80 0x560aaae24c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31f426740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560aaae2a880 con 0x560aaae145d0 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31f426740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560aaad35060 con 0x560aaae27b80 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31f426740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560aaae2aa00 con 0x560aaae15b40 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31f426740 10 client.0 _ll_register_callbacks cb 0x560aaaa888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae145d0 0x560aaae27480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae145d0 0x560aaae27480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47360/0 (socket says 172.21.15.78:47360) 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 -- 172.21.15.78:0/2400428628 learned_addr learned my addr 172.21.15.78:0/2400428628 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31659c640 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae24730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31759e640 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae27b80 0x560aaae24c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 -- 172.21.15.78:0/2400428628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 msgr2=0x560aaae24730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae24730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 -- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae27b80 msgr2=0x560aaae24c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae27b80 0x560aaae24c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 -- 172.21.15.78:0/2400428628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aaaaeb3c0 con 0x560aaae145d0 2024-09-16T00:23:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff31759e640 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae27b80 0x560aaae24c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:23:48.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff316d9d640 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae145d0 0x560aaae27480 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c6a0 tx=0x7ff3080056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6681f64ca29c0901 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.792+0000 7ff2faff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.795+0000 7ff2faff5640 1 -- 172.21.15.78:0/2400428628 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30809be60 con 0x560aaae145d0 2024-09-16T00:23:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff2faff5640 1 -- 172.21.15.78:0/2400428628 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff3080d20d0 con 0x560aaae145d0 2024-09-16T00:23:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 1 -- 172.21.15.78:0/2400428628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30c0041c0 con 0x560aaae145d0 2024-09-16T00:23:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff2faff5640 1 -- 172.21.15.78:0/2400428628 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff30809bc80 con 0x560aaae145d0 2024-09-16T00:23:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 10 client.17271 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 1 -- 172.21.15.78:0/2400428628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560aaae362c0 con 0x560aaae145d0 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff2faff5640 1 -- 172.21.15.78:0/2400428628 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30c0041c0 con 0x560aaae145d0 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff2faff5640 1 -- 172.21.15.78:0/2400428628 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff30809b770 con 0x560aaae145d0 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 10 client.17271 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 20 client.17271 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560aaae37780 0x560aaae39c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 4 client.17271 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:48.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31f426740 1 -- 172.21.15.78:0/2400428628 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aaad87860 con 0x560aaae37780 2024-09-16T00:23:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.796+0000 7ff31659c640 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560aaae37780 0x560aaae39c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.797+0000 7ff31659c640 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560aaae37780 0x560aaae39c10 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77f9071d1ca1a8b0 server_cookie=42c9535dd3dc3ca1 in_seq=0 out_seq=0 2024-09-16T00:23:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.797+0000 7ff2faff5640 10 client.17271 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff2faff5640 1 -- 172.21.15.78:0/2400428628 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560aaad87860 con 0x560aaae37780 2024-09-16T00:23:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff2faff5640 10 client.17271 handle_command_reply: tid=0 2024-09-16T00:23:48.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 client.17271 shutdown 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 -- 172.21.15.78:0/2400428628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae145d0 msgr2=0x560aaae27480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae145d0 0x560aaae27480 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c6a0 tx=0x7ff3080056d0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 -- 172.21.15.78:0/2400428628 shutdown_connections 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560aaae37780 0x560aaae39c10 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae27b80 0x560aaae24c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae15b40 0x560aaae24730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 --2- 172.21.15.78:0/2400428628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae145d0 0x560aaae27480 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 -- 172.21.15.78:0/2400428628 >> 172.21.15.78:0/2400428628 conn(0x560aaaddd310 msgr2=0x560aaae16930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff2faff5640 0 client.17271 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:48.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 -- 172.21.15.78:0/2400428628 shutdown_connections 2024-09-16T00:23:48.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 1 -- 172.21.15.78:0/2400428628 wait complete. 2024-09-16T00:23:48.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.798+0000 7ff31f426740 20 client.17271 trim_cache size 0 max 16384 2024-09-16T00:23:48.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.849+0000 7ff31f426740 1 Processor -- start 2024-09-16T00:23:48.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.853+0000 7ff31f426740 1 -- start start 2024-09-16T00:23:48.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.853+0000 7ff31f426740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae33bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.853+0000 7ff31f426740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae13e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.853+0000 7ff31f426740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae23000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.853+0000 7ff31f426740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff30400faa0 con 0x560aaae15b40 2024-09-16T00:23:48.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.853+0000 7ff31f426740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3040115e0 con 0x560aaae145d0 2024-09-16T00:23:48.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.853+0000 7ff31f426740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff304010170 con 0x560aaae27b80 2024-09-16T00:23:48.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.857+0000 7ff31759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae23000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.857+0000 7ff316d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae33bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.857+0000 7ff31659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae13e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.857+0000 7ff31659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae13e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47368/0 (socket says 172.21.15.78:47368) 2024-09-16T00:23:48.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.857+0000 7ff31759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae23000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:58134/0 (socket says 172.21.15.78:58134) 2024-09-16T00:23:48.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.857+0000 7ff31659c640 1 -- 172.21.15.78:0/1142550911 learned_addr learned my addr 172.21.15.78:0/1142550911 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.860+0000 7ff31659c640 1 -- 172.21.15.78:0/1142550911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 msgr2=0x560aaae23000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.860+0000 7ff31659c640 1 --2- 172.21.15.78:0/1142550911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae23000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.860+0000 7ff31659c640 1 -- 172.21.15.78:0/1142550911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 msgr2=0x560aaae33bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.860+0000 7ff31659c640 1 --2- 172.21.15.78:0/1142550911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae33bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.860+0000 7ff31659c640 1 -- 172.21.15.78:0/1142550911 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aaae14d50 con 0x560aaae15b40 2024-09-16T00:23:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.861+0000 7ff31659c640 1 --2- 172.21.15.78:0/1142550911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae13e00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff30c01a000 tx=0x7ff30c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a049e07e26937f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.862+0000 7ff2fcff9640 1 -- 172.21.15.78:0/1142550911 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c029020 con 0x560aaae15b40 2024-09-16T00:23:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.862+0000 7ff2fcff9640 1 -- 172.21.15.78:0/1142550911 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30c005520 con 0x560aaae15b40 2024-09-16T00:23:48.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.862+0000 7ff2fcff9640 1 -- 172.21.15.78:0/1142550911 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c003850 con 0x560aaae15b40 2024-09-16T00:23:48.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.863+0000 7ff31f426740 1 -- 172.21.15.78:0/1142550911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 msgr2=0x560aaae13e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.864+0000 7ff31f426740 1 --2- 172.21.15.78:0/1142550911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae13e00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff30c01a000 tx=0x7ff30c009580 comp rx=0 tx=0).stop 2024-09-16T00:23:48.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.870+0000 7ff31f426740 1 -- 172.21.15.78:0/1142550911 shutdown_connections 2024-09-16T00:23:48.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.870+0000 7ff31f426740 1 --2- 172.21.15.78:0/1142550911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae23000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.870+0000 7ff31f426740 1 --2- 172.21.15.78:0/1142550911 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae13e00 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.870+0000 7ff31f426740 1 --2- 172.21.15.78:0/1142550911 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae33bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.870+0000 7ff31f426740 1 -- 172.21.15.78:0/1142550911 >> 172.21.15.78:0/1142550911 conn(0x560aaaddd310 msgr2=0x560aaae246f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.871+0000 7ff31f426740 1 -- 172.21.15.78:0/1142550911 shutdown_connections 2024-09-16T00:23:48.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.871+0000 7ff31f426740 1 -- 172.21.15.78:0/1142550911 wait complete. 2024-09-16T00:23:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.871+0000 7ff31f426740 1 -- start start 2024-09-16T00:23:48.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.871+0000 7ff31f426740 1 Processor -- start 2024-09-16T00:23:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.872+0000 7ff31f426740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae2ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.872+0000 7ff31f426740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae30440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.872+0000 7ff31f426740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae20f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.873+0000 7ff31f426740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff304010f10 con 0x560aaae15b40 2024-09-16T00:23:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.873+0000 7ff31f426740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff304010840 con 0x560aaae145d0 2024-09-16T00:23:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.873+0000 7ff31f426740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff30400da80 con 0x560aaae27b80 2024-09-16T00:23:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.873+0000 7ff31759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae20f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.873+0000 7ff31759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae20f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:58140/0 (socket says 172.21.15.78:58140) 2024-09-16T00:23:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.873+0000 7ff31759e640 1 -- 172.21.15.78:0/3708396235 learned_addr learned my addr 172.21.15.78:0/3708396235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.873+0000 7ff31f426740 10 client.0 _ll_register_callbacks cb 0x7ff30800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.874+0000 7ff31759e640 1 -- 172.21.15.78:0/3708396235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 msgr2=0x560aaae2ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:48.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.874+0000 7ff31759e640 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae2ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.874+0000 7ff31759e640 1 -- 172.21.15.78:0/3708396235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 msgr2=0x560aaae30440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:23:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.874+0000 7ff31759e640 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae30440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.874+0000 7ff31759e640 1 -- 172.21.15.78:0/3708396235 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aaae216d0 con 0x560aaae27b80 2024-09-16T00:23:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.874+0000 7ff31759e640 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae20f90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff310009ca0 tx=0x7ff310005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4428da0c1b6477fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.874+0000 7ff2faff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:23:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.879+0000 7ff2faff5640 1 -- 172.21.15.78:0/3708396235 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff310058070 con 0x560aaae27b80 2024-09-16T00:23:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.879+0000 7ff31f426740 1 -- 172.21.15.78:0/3708396235 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30c005520 con 0x560aaae27b80 2024-09-16T00:23:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.879+0000 7ff2faff5640 1 -- 172.21.15.78:0/3708396235 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff310030a90 con 0x560aaae27b80 2024-09-16T00:23:48.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.879+0000 7ff2faff5640 1 -- 172.21.15.78:0/3708396235 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff310030c90 con 0x560aaae27b80 2024-09-16T00:23:48.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.883+0000 7ff31f426740 10 client.26428 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:48.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.883+0000 7ff31f426740 1 -- 172.21.15.78:0/3708396235 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560aaaea1e80 con 0x560aaae27b80 2024-09-16T00:23:48.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.884+0000 7ff2faff5640 1 -- 172.21.15.78:0/3708396235 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30c005520 con 0x560aaae27b80 2024-09-16T00:23:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.885+0000 7ff2faff5640 1 -- 172.21.15.78:0/3708396235 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff310004500 con 0x560aaae27b80 2024-09-16T00:23:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.885+0000 7ff31f426740 10 client.26428 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.885+0000 7ff31f426740 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:23:48.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.885+0000 7ff31f426740 20 client.26428 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.885+0000 7ff31f426740 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560aaaea2630 0x560aaaea4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.885+0000 7ff31f426740 4 client.26428 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:23:48.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.885+0000 7ff31f426740 1 -- 172.21.15.78:0/3708396235 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560aaad81d70 con 0x560aaaea2630 2024-09-16T00:23:48.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.889+0000 7ff316d9d640 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560aaaea2630 0x560aaaea4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.894+0000 7ff2faff5640 10 client.26428 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.894+0000 7ff316d9d640 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560aaaea2630 0x560aaaea4ad0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75946de9f128447e server_cookie=fc672a78c0920704 in_seq=0 out_seq=0 2024-09-16T00:23:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.895+0000 7ff2faff5640 1 -- 172.21.15.78:0/3708396235 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560aaad81d70 con 0x560aaaea2630 2024-09-16T00:23:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.895+0000 7ff2faff5640 10 client.26428 handle_command_reply: tid=0 2024-09-16T00:23:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.895+0000 7ff31f426740 1 client.26428 shutdown 2024-09-16T00:23:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.899+0000 7ff31f426740 1 -- 172.21.15.78:0/3708396235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 msgr2=0x560aaae20f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.899+0000 7ff31f426740 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae20f90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff310009ca0 tx=0x7ff310005fd0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.900+0000 7ff31f426740 1 -- 172.21.15.78:0/3708396235 shutdown_connections 2024-09-16T00:23:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.900+0000 7ff31f426740 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560aaaea2630 0x560aaaea4ad0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.900+0000 7ff31f426740 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560aaae27b80 0x560aaae20f90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.900+0000 7ff31f426740 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560aaae15b40 0x560aaae30440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.900+0000 7ff31f426740 1 --2- 172.21.15.78:0/3708396235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560aaae145d0 0x560aaae2ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.900+0000 7ff31f426740 1 -- 172.21.15.78:0/3708396235 >> 172.21.15.78:0/3708396235 conn(0x560aaaddd310 msgr2=0x560aaae22fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.900+0000 7ff2faff5640 0 client.26428 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.901+0000 7ff31f426740 1 -- 172.21.15.78:0/3708396235 shutdown_connections 2024-09-16T00:23:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.905+0000 7ff31f426740 1 -- 172.21.15.78:0/3708396235 wait complete. 2024-09-16T00:23:48.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.906+0000 7ff31f426740 20 client.26428 trim_cache size 0 max 16384 2024-09-16T00:23:48.910 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 -- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3000a32c0 msgr2=0x7ff3000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3000a32c0 0x7ff3000a5780 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff310005e30 tx=0x7ff31003e040 comp rx=0 tx=0).stop 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 -- 172.21.15.78:0/2314176504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff31810a560 msgr2=0x7ff3181a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff31810a560 0x7ff3181a6670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e9f0 tx=0x7ff30800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 -- 172.21.15.78:0/2314176504 shutdown_connections 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff3000a32c0 0x7ff3000a5780 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff318113940 0x7ff3181a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff31810ae80 0x7ff3181a6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 --2- 172.21.15.78:0/2314176504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff31810a560 0x7ff3181a6670 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.910+0000 7ff31d04d640 1 -- 172.21.15.78:0/2314176504 >> 172.21.15.78:0/2314176504 conn(0x7ff31806e140 msgr2=0x7ff3180716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.911+0000 7ff31d04d640 1 -- 172.21.15.78:0/2314176504 shutdown_connections 2024-09-16T00:23:48.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:48.911+0000 7ff31d04d640 1 -- 172.21.15.78:0/2314176504 wait complete. 2024-09-16T00:23:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:49 smithi193 ceph-mon[35484]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.4 MiB/s wr, 625 op/s 2024-09-16T00:23:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:49 smithi148 ceph-mon[35599]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.4 MiB/s wr, 625 op/s 2024-09-16T00:23:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:49 smithi078 ceph-mon[31157]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.4 MiB/s wr, 625 op/s 2024-09-16T00:23:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 590.0 s, 702.0 tps, lat 142.325 ms stddev 172.460 2024-09-16T00:23:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:51 smithi193 ceph-mon[35484]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.5 MiB/s wr, 635 op/s 2024-09-16T00:23:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:51 smithi148 ceph-mon[35599]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.5 MiB/s wr, 635 op/s 2024-09-16T00:23:52.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:51 smithi078 ceph-mon[31157]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.5 MiB/s wr, 635 op/s 2024-09-16T00:23:54.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:53 smithi078 ceph-mon[31157]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 MiB/s wr, 669 op/s 2024-09-16T00:23:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:53 smithi193 ceph-mon[35484]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 MiB/s wr, 669 op/s 2024-09-16T00:23:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:53 smithi148 ceph-mon[35599]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.6 MiB/s wr, 669 op/s 2024-09-16T00:23:54.681 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:23:54.681 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:23:55.206 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 595.0 s, 606.8 tps, lat 161.531 ms stddev 211.888 2024-09-16T00:23:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:55 smithi193 ceph-mon[35484]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 MiB/s wr, 640 op/s 2024-09-16T00:23:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:55 smithi148 ceph-mon[35599]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 MiB/s wr, 640 op/s 2024-09-16T00:23:56.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:55 smithi078 ceph-mon[31157]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 MiB/s wr, 640 op/s 2024-09-16T00:23:56.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.776+0000 7f4a89b41640 1 -- 172.21.15.78:0/2896408678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a8410d530 msgr2=0x7f4a841150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.776+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2896408678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a8410d530 0x7f4a841150b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009860 tx=0x7f4a7c02a380 comp rx=0 tx=0).stop 2024-09-16T00:23:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.780+0000 7f4a89b41640 1 -- 172.21.15.78:0/2896408678 shutdown_connections 2024-09-16T00:23:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.780+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2896408678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a8410d530 0x7f4a841150b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.780+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2896408678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a84108e00 0x7f4a8410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.780+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2896408678 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a84108450 0x7f4a84108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.780+0000 7f4a89b41640 1 -- 172.21.15.78:0/2896408678 >> 172.21.15.78:0/2896408678 conn(0x7f4a84077ba0 msgr2=0x7f4a84079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:56.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.785+0000 7f4a89b41640 1 -- 172.21.15.78:0/2896408678 shutdown_connections 2024-09-16T00:23:56.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.788+0000 7f4a89b41640 1 -- 172.21.15.78:0/2896408678 wait complete. 2024-09-16T00:23:56.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.790+0000 7f4a89b41640 1 Processor -- start 2024-09-16T00:23:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.792+0000 7f4a89b41640 1 -- start start 2024-09-16T00:23:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.792+0000 7f4a89b41640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a84108450 0x7f4a84082850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.792+0000 7f4a89b41640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a84108e00 0x7f4a8407b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.792+0000 7f4a89b41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a8410d530 0x7f4a8407be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.792+0000 7f4a89b41640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841185d0 con 0x7f4a8410d530 2024-09-16T00:23:56.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.792+0000 7f4a89b41640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84118450 con 0x7f4a84108450 2024-09-16T00:23:56.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.792+0000 7f4a89b41640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84118750 con 0x7f4a84108e00 2024-09-16T00:23:56.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.793+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a84108450 0x7f4a84082850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.793+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a84108450 0x7f4a84082850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48348/0 (socket says 172.21.15.78:48348) 2024-09-16T00:23:56.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.793+0000 7f4a837fe640 1 -- 172.21.15.78:0/2178837801 learned_addr learned my addr 172.21.15.78:0/2178837801 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:56.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.794+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a84108e00 0x7f4a8407b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.796+0000 7f4a83fff640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a8410d530 0x7f4a8407be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.796+0000 7f4a837fe640 1 -- 172.21.15.78:0/2178837801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a84108e00 msgr2=0x7f4a8407b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.796+0000 7f4a837fe640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a84108e00 0x7f4a8407b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.796+0000 7f4a837fe640 1 -- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a8410d530 msgr2=0x7f4a8407be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.796+0000 7f4a837fe640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a8410d530 0x7f4a8407be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.796+0000 7f4a837fe640 1 -- 172.21.15.78:0/2178837801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8407c410 con 0x7f4a84108450 2024-09-16T00:23:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.798+0000 7f4a837fe640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a84108450 0x7f4a84082850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400ba50 tx=0x7f4a7400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.799+0000 7f4a80ff9640 1 -- 172.21.15.78:0/2178837801 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74019070 con 0x7f4a84108450 2024-09-16T00:23:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.799+0000 7f4a80ff9640 1 -- 172.21.15.78:0/2178837801 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a7400cc10 con 0x7f4a84108450 2024-09-16T00:23:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.799+0000 7f4a80ff9640 1 -- 172.21.15.78:0/2178837801 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a740076f0 con 0x7f4a84108450 2024-09-16T00:23:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.799+0000 7f4a89b41640 1 -- 172.21.15.78:0/2178837801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8407c6a0 con 0x7f4a84108450 2024-09-16T00:23:56.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.802+0000 7f4a89b41640 1 -- 172.21.15.78:0/2178837801 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a84080640 con 0x7f4a84108450 2024-09-16T00:23:56.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.810+0000 7f4a80ff9640 1 -- 172.21.15.78:0/2178837801 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4a74014440 con 0x7f4a84108450 2024-09-16T00:23:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.811+0000 7f4a8bf1a740 1 Processor -- start 2024-09-16T00:23:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a8bf1a740 1 -- start start 2024-09-16T00:23:56.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a80ff9640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a700a3450 0x7f4a700a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a83fff640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a700a3450 0x7f4a700a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 0x557515510f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x55751551b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a80ff9640 1 -- 172.21.15.78:0/2178837801 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4a740cf820 con 0x7f4a84108450 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 0x557515510f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 0x557515510f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59732/0 (socket says 172.21.15.78:59732) 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.812+0000 7f4a837fe640 1 -- 172.21.15.78:0/2009620169 learned_addr learned my addr 172.21.15.78:0/2009620169 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 0x55751551e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2009620169 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557515476280 con 0x557515510b60 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2009620169 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55751541db10 con 0x55751551bd40 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2009620169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557515472240 con 0x557515511480 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x55751551b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a83fff640 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 0x55751551e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a82ffd640 1 -- 172.21.15.78:0/2009620169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 msgr2=0x55751551e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 0x55751551e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a82ffd640 1 -- 172.21.15.78:0/2009620169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 msgr2=0x557515510f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 0x557515510f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.813+0000 7f4a82ffd640 1 -- 172.21.15.78:0/2009620169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751542a1f0 con 0x557515511480 2024-09-16T00:23:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.814+0000 7f4a83fff640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a700a3450 0x7f4a700a5910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009830 tx=0x7f4a7c004160 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:56.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.815+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x55751551b800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4a8407d130 tx=0x7f4a6c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6618120e54341303 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.816+0000 7f4a79ffb640 1 -- 172.21.15.78:0/2009620169 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c012450 con 0x557515511480 2024-09-16T00:23:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.816+0000 7f4a79ffb640 1 -- 172.21.15.78:0/2009620169 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c012ad0 con 0x557515511480 2024-09-16T00:23:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.816+0000 7f4a79ffb640 1 -- 172.21.15.78:0/2009620169 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00fac0 con 0x557515511480 2024-09-16T00:23:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.817+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2009620169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 msgr2=0x55751551b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.817+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x55751551b800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4a8407d130 tx=0x7f4a6c0117b0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.821+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2009620169 shutdown_connections 2024-09-16T00:23:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.821+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 0x55751551e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.821+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x55751551b800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.821+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2009620169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 0x557515510f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.821+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2009620169 >> 172.21.15.78:0/2009620169 conn(0x55751550d0f0 msgr2=0x55751550d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.823+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2009620169 shutdown_connections 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.823+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2009620169 wait complete. 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.823+0000 7f4a8bf1a740 1 -- start start 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.823+0000 7f4a8bf1a740 1 Processor -- start 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.824+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x557515518bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.824+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x5575155190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.824+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751551bd40 0x557515519630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.824+0000 7f4a8bf1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557515540920 con 0x55751551bd40 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.824+0000 7f4a8bf1a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557515430f10 con 0x557515510b60 2024-09-16T00:23:56.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.824+0000 7f4a8bf1a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557515540aa0 con 0x557515511480 2024-09-16T00:23:56.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.824+0000 7f4a8bf1a740 10 client.0 _ll_register_callbacks cb 0x5575153f9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.825+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x557515518bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.825+0000 7f4a837fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x557515518bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48362/0 (socket says 172.21.15.78:48362) 2024-09-16T00:23:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.825+0000 7f4a837fe640 1 -- 172.21.15.78:0/2635317441 learned_addr learned my addr 172.21.15.78:0/2635317441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.825+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x5575155190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.826+0000 7f4a83fff640 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751551bd40 0x557515519630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.827+0000 7f4a83fff640 1 -- 172.21.15.78:0/2635317441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 msgr2=0x5575155190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.827+0000 7f4a83fff640 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x5575155190f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.827+0000 7f4a83fff640 1 -- 172.21.15.78:0/2635317441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 msgr2=0x557515518bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.827+0000 7f4a83fff640 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x557515518bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.827+0000 7f4a83fff640 1 -- 172.21.15.78:0/2635317441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575151e76a0 con 0x55751551bd40 2024-09-16T00:23:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.828+0000 7f4a83fff640 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751551bd40 0x557515519630 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c02a970 tx=0x7f4a7c02a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2e0226a66a680f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:56.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.828+0000 7f4a62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.830+0000 7f4a62ffd640 1 -- 172.21.15.78:0/2635317441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c037040 con 0x55751551bd40 2024-09-16T00:23:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.830+0000 7f4a62ffd640 1 -- 172.21.15.78:0/2635317441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4a7c0046c0 con 0x55751551bd40 2024-09-16T00:23:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.830+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2635317441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6c012ad0 con 0x55751551bd40 2024-09-16T00:23:56.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.835+0000 7f4a62ffd640 1 -- 172.21.15.78:0/2635317441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4a7c0048c0 con 0x55751551bd40 2024-09-16T00:23:56.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.838+0000 7f4a8bf1a740 10 client.17286 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:56.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.838+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2635317441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55751554c540 con 0x55751551bd40 2024-09-16T00:23:56.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a62ffd640 1 -- 172.21.15.78:0/2635317441 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c012ad0 con 0x55751551bd40 2024-09-16T00:23:56.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a62ffd640 1 -- 172.21.15.78:0/2635317441 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4a7c003d60 con 0x55751551bd40 2024-09-16T00:23:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a8bf1a740 10 client.17286 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a8bf1a740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:23:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a8bf1a740 20 client.17286 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55751554da50 0x55751554fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a8bf1a740 4 client.17286 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:56.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2635317441 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575154837f0 con 0x55751554da50 2024-09-16T00:23:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.841+0000 7f4a837fe640 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55751554da50 0x55751554fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.845+0000 7f4a837fe640 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55751554da50 0x55751554fe80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b4a34263c29bec server_cookie=1325136f81aa0205 in_seq=0 out_seq=0 2024-09-16T00:23:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.845+0000 7f4a62ffd640 10 client.17286 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:56.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.846+0000 7f4a62ffd640 1 -- 172.21.15.78:0/2635317441 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575154837f0 con 0x55751554da50 2024-09-16T00:23:56.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a62ffd640 10 client.17286 handle_command_reply: tid=0 2024-09-16T00:23:56.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 client.17286 shutdown 2024-09-16T00:23:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2635317441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751551bd40 msgr2=0x557515519630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751551bd40 0x557515519630 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c02a970 tx=0x7f4a7c02a380 comp rx=0 tx=0).stop 2024-09-16T00:23:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2635317441 shutdown_connections 2024-09-16T00:23:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55751554da50 0x55751554fe80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55751551bd40 0x557515519630 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x5575155190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/2635317441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x557515518bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a62ffd640 0 client.17286 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:56.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2635317441 >> 172.21.15.78:0/2635317441 conn(0x55751550d0f0 msgr2=0x557515512e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:56.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2635317441 shutdown_connections 2024-09-16T00:23:56.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.847+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/2635317441 wait complete. 2024-09-16T00:23:56.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.848+0000 7f4a8bf1a740 20 client.17286 trim_cache size 0 max 16384 2024-09-16T00:23:56.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.874+0000 7f4a8bf1a740 1 Processor -- start 2024-09-16T00:23:56.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.874+0000 7f4a8bf1a740 1 -- start start 2024-09-16T00:23:56.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.874+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x55751551aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.875+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515511480 0x5575155556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.875+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55751551bd40 0x557515555c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.875+0000 7f4a8bf1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841b5260 con 0x557515511480 2024-09-16T00:23:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.875+0000 7f4a8bf1a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841b6da0 con 0x557515510b60 2024-09-16T00:23:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.875+0000 7f4a8bf1a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841b5930 con 0x55751551bd40 2024-09-16T00:23:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.876+0000 7f4a83fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55751551bd40 0x557515555c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.876+0000 7f4a83fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55751551bd40 0x557515555c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:41616/0 (socket says 172.21.15.78:41616) 2024-09-16T00:23:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.876+0000 7f4a83fff640 1 -- 172.21.15.78:0/3300558345 learned_addr learned my addr 172.21.15.78:0/3300558345 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.876+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515511480 0x5575155556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.879+0000 7f4a837fe640 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x55751551aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.880+0000 7f4a83fff640 1 -- 172.21.15.78:0/3300558345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 msgr2=0x55751551aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:56.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.880+0000 7f4a83fff640 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x55751551aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.880+0000 7f4a83fff640 1 -- 172.21.15.78:0/3300558345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515511480 msgr2=0x5575155556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:56.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.880+0000 7f4a83fff640 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515511480 0x5575155556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.880+0000 7f4a83fff640 1 -- 172.21.15.78:0/3300558345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751551b480 con 0x55751551bd40 2024-09-16T00:23:56.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.883+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515511480 0x5575155556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:56.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.887+0000 7f4a83fff640 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55751551bd40 0x557515555c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0334a0 tx=0x7f4a7c005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650c9606a66a6146 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:56.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.896+0000 7f4a797fa640 1 -- 172.21.15.78:0/3300558345 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c037040 con 0x55751551bd40 2024-09-16T00:23:56.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.896+0000 7f4a797fa640 1 -- 172.21.15.78:0/3300558345 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c050070 con 0x55751551bd40 2024-09-16T00:23:56.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.896+0000 7f4a797fa640 1 -- 172.21.15.78:0/3300558345 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c033d90 con 0x55751551bd40 2024-09-16T00:23:56.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.898+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/3300558345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55751551bd40 msgr2=0x557515555c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.898+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55751551bd40 0x557515555c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0334a0 tx=0x7f4a7c005ad0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.901+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/3300558345 shutdown_connections 2024-09-16T00:23:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.901+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55751551bd40 0x557515555c30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.901+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515511480 0x5575155556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.901+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/3300558345 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557515510b60 0x55751551aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.901+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/3300558345 >> 172.21.15.78:0/3300558345 conn(0x55751550d0f0 msgr2=0x55751551a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:56.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.908+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/3300558345 shutdown_connections 2024-09-16T00:23:56.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.908+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/3300558345 wait complete. 2024-09-16T00:23:56.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.908+0000 7f4a8bf1a740 1 -- start start 2024-09-16T00:23:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.909+0000 7f4a8bf1a740 1 Processor -- start 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.909+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 0x5575155465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.909+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x557515541920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.909+0000 7f4a8bf1a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 0x557515541f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.910+0000 7f4a8bf1a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841b66d0 con 0x557515510b60 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.910+0000 7f4a82ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x557515541920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.910+0000 7f4a82ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x557515541920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:41626/0 (socket says 172.21.15.78:41626) 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.910+0000 7f4a82ffd640 1 -- 172.21.15.78:0/1404369841 learned_addr learned my addr 172.21.15.78:0/1404369841 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.910+0000 7f4a83fff640 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 0x557515541f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.910+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841b6000 con 0x55751551bd40 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.910+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a841b3260 con 0x557515511480 2024-09-16T00:23:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.910+0000 7f4a8bf1a740 10 client.0 _ll_register_callbacks cb 0x7f4a7c045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.914+0000 7f4a82ffd640 1 -- 172.21.15.78:0/1404369841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 msgr2=0x557515541f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:56.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.914+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 0x557515541f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.914+0000 7f4a82ffd640 1 -- 172.21.15.78:0/1404369841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 msgr2=0x5575155465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.914+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 0x5575155465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.914+0000 7f4a82ffd640 1 -- 172.21.15.78:0/1404369841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557515542740 con 0x557515511480 2024-09-16T00:23:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.914+0000 7f4a82ffd640 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x557515541920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55751550df50 tx=0x7f4a6c005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d95dc0a3ceb5a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.915+0000 7f4a62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:23:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.925+0000 7f4a62ffd640 1 -- 172.21.15.78:0/1404369841 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c029070 con 0x557515511480 2024-09-16T00:23:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.925+0000 7f4a62ffd640 1 -- 172.21.15.78:0/1404369841 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4a6c0035f0 con 0x557515511480 2024-09-16T00:23:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.928+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a7c050070 con 0x557515511480 2024-09-16T00:23:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.931+0000 7f4a62ffd640 1 -- 172.21.15.78:0/1404369841 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4a6c033050 con 0x557515511480 2024-09-16T00:23:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.931+0000 7f4a8bf1a740 10 client.26446 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.931+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5575155b87e0 con 0x557515511480 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.935+0000 7f4a62ffd640 1 -- 172.21.15.78:0/1404369841 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c050070 con 0x557515511480 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.935+0000 7f4a62ffd640 1 -- 172.21.15.78:0/1404369841 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4a6c012a60 con 0x557515511480 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.935+0000 7f4a8bf1a740 10 client.26446 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.935+0000 7f4a8bf1a740 10 client.26446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.935+0000 7f4a8bf1a740 20 client.26446 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.935+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5575155b8fe0 0x5575155bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.935+0000 7f4a8bf1a740 4 client.26446 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.935+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55751547dd00 con 0x5575155b8fe0 2024-09-16T00:23:56.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.937+0000 7f4a837fe640 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5575155b8fe0 0x5575155bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.948+0000 7f4a62ffd640 10 client.26446 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:56.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.948+0000 7f4a837fe640 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5575155b8fe0 0x5575155bb400 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846ad0a46fa6d06c server_cookie=d704015933dc9d6a in_seq=0 out_seq=0 2024-09-16T00:23:56.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.951+0000 7f4a62ffd640 1 -- 172.21.15.78:0/1404369841 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55751547dd00 con 0x5575155b8fe0 2024-09-16T00:23:56.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.951+0000 7f4a62ffd640 10 client.26446 handle_command_reply: tid=0 2024-09-16T00:23:56.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.951+0000 7f4a8bf1a740 1 client.26446 shutdown 2024-09-16T00:23:56.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 msgr2=0x557515541920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x557515541920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55751550df50 tx=0x7f4a6c005750 comp rx=0 tx=0).stop 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 shutdown_connections 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5575155b8fe0 0x5575155bb400 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55751551bd40 0x557515541f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557515511480 0x557515541920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a62ffd640 0 client.26446 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 --2- 172.21.15.78:0/1404369841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557515510b60 0x5575155465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 >> 172.21.15.78:0/1404369841 conn(0x55751550d0f0 msgr2=0x557515519390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 shutdown_connections 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.952+0000 7f4a8bf1a740 1 -- 172.21.15.78:0/1404369841 wait complete. 2024-09-16T00:23:56.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a8bf1a740 20 client.26446 trim_cache size 0 max 16384 2024-09-16T00:23:56.955 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:23:56.955 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:23:56.955 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "a8d47851-96ee-40b5-8f93-69ceb29f6b85", 2024-09-16T00:23:56.955 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:23:56.955 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 -- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a700a3450 msgr2=0x7f4a700a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a700a3450 0x7f4a700a5910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009830 tx=0x7f4a7c004160 comp rx=0 tx=0).stop 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 -- 172.21.15.78:0/2178837801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a84108450 msgr2=0x7f4a84082850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a84108450 0x7f4a84082850 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400ba50 tx=0x7f4a7400bf10 comp rx=0 tx=0).stop 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 -- 172.21.15.78:0/2178837801 shutdown_connections 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a700a3450 0x7f4a700a5910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a8410d530 0x7f4a8407be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a84108e00 0x7f4a8407b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 --2- 172.21.15.78:0/2178837801 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a84108450 0x7f4a84082850 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.953+0000 7f4a89b41640 1 -- 172.21.15.78:0/2178837801 >> 172.21.15.78:0/2178837801 conn(0x7f4a84077ba0 msgr2=0x7f4a84078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.954+0000 7f4a89b41640 1 -- 172.21.15.78:0/2178837801 shutdown_connections 2024-09-16T00:23:56.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:56.954+0000 7f4a89b41640 1 -- 172.21.15.78:0/2178837801 wait complete. 2024-09-16T00:23:57.786 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:23:58.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:57 smithi078 ceph-mon[31157]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-09-16T00:23:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:57 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:23:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:57 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:23:58.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:57 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:57 smithi193 ceph-mon[35484]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-09-16T00:23:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:57 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:23:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:57 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:23:58.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:57 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:58.180 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:57 smithi148 ceph-mon[35599]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-09-16T00:23:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:57 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:23:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:57 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:23:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:57 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.792+0000 7f0693f51640 1 -- 172.21.15.78:0/1353511551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 msgr2=0x7f068c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.792+0000 7f0693f51640 1 --2- 172.21.15.78:0/1353511551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 0x7f068c074970 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0688009860 tx=0x7f068802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.793+0000 7f0693f51640 1 -- 172.21.15.78:0/1353511551 shutdown_connections 2024-09-16T00:23:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.793+0000 7f0693f51640 1 --2- 172.21.15.78:0/1353511551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f068c112b90 0x7f068c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.793+0000 7f0693f51640 1 --2- 172.21.15.78:0/1353511551 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f068c074eb0 0x7f068c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.793+0000 7f0693f51640 1 --2- 172.21.15.78:0/1353511551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 0x7f068c074970 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.793+0000 7f0693f51640 1 -- 172.21.15.78:0/1353511551 >> 172.21.15.78:0/1353511551 conn(0x7f068c0fd880 msgr2=0x7f068c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:58.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.793+0000 7f0693f51640 1 -- 172.21.15.78:0/1353511551 shutdown_connections 2024-09-16T00:23:58.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.793+0000 7f0693f51640 1 -- 172.21.15.78:0/1353511551 wait complete. 2024-09-16T00:23:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.794+0000 7f0693f51640 1 Processor -- start 2024-09-16T00:23:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.794+0000 7f0693f51640 1 -- start start 2024-09-16T00:23:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.794+0000 7f0693f51640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 0x7f068c1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.794+0000 7f0693f51640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f068c074eb0 0x7f068c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.794+0000 7f0693f51640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f068c112b90 0x7f068c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.794+0000 7f0693f51640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f068c1185a0 con 0x7f068c112b90 2024-09-16T00:23:58.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0693f51640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f068c118420 con 0x7f068c074eb0 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0693f51640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f068c118720 con 0x7f068c074590 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0691ced640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 0x7f068c1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0691ced640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 0x7f068c1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:41650/0 (socket says 172.21.15.78:41650) 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0691ced640 1 -- 172.21.15.78:0/3838855196 learned_addr learned my addr 172.21.15.78:0/3838855196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f06924ee640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f068c112b90 0x7f068c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f06914ec640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f068c074eb0 0x7f068c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0691ced640 1 -- 172.21.15.78:0/3838855196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f068c074eb0 msgr2=0x7f068c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0691ced640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f068c074eb0 0x7f068c1a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0691ced640 1 -- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f068c112b90 msgr2=0x7f068c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0691ced640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f068c112b90 0x7f068c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f0691ced640 1 -- 172.21.15.78:0/3838855196 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c19eff0 con 0x7f068c074590 2024-09-16T00:23:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.795+0000 7f06924ee640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f068c112b90 0x7f068c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:23:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.796+0000 7f0691ced640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 0x7f068c1a4700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f068802a960 tx=0x7f06880308b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.796+0000 7f0682ffd640 1 -- 172.21.15.78:0/3838855196 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688040070 con 0x7f068c074590 2024-09-16T00:23:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.796+0000 7f0693f51640 1 -- 172.21.15.78:0/3838855196 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c19f280 con 0x7f068c074590 2024-09-16T00:23:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.796+0000 7f0693f51640 1 -- 172.21.15.78:0/3838855196 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c1ab5d0 con 0x7f068c074590 2024-09-16T00:23:58.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.796+0000 7f0682ffd640 1 -- 172.21.15.78:0/3838855196 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06880055a0 con 0x7f068c074590 2024-09-16T00:23:58.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.797+0000 7f0682ffd640 1 -- 172.21.15.78:0/3838855196 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688004920 con 0x7f068c074590 2024-09-16T00:23:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.801+0000 7f0682ffd640 1 -- 172.21.15.78:0/3838855196 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0688043070 con 0x7f068c074590 2024-09-16T00:23:58.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.802+0000 7f069632a740 1 Processor -- start 2024-09-16T00:23:58.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.802+0000 7f0682ffd640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06780a3380 0x7f06780a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.803+0000 7f0682ffd640 1 -- 172.21.15.78:0/3838855196 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f06880f0930 con 0x7f068c074590 2024-09-16T00:23:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.803+0000 7f069632a740 1 -- start start 2024-09-16T00:23:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.803+0000 7f06924ee640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06780a3380 0x7f06780a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.804+0000 7f06924ee640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06780a3380 0x7f06780a5840 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f068c076600 tx=0x7f0684009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f069632a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f069632a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f069632a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbffc680 0x55d8cbffea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f069632a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8cbf572e0 con 0x55d8cbffc680 2024-09-16T00:23:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f069632a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d8cbefece0 con 0x55d8cbff1dc0 2024-09-16T00:23:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f069632a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d8cbf532a0 con 0x55d8cbff14a0 2024-09-16T00:23:58.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f06914ec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f06914ec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48396/0 (socket says 172.21.15.78:48396) 2024-09-16T00:23:58.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.805+0000 7f06914ec640 1 -- 172.21.15.78:0/2776957950 learned_addr learned my addr 172.21.15.78:0/2776957950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.806+0000 7f0691ced640 1 --2- 172.21.15.78:0/2776957950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.806+0000 7f06914ec640 1 -- 172.21.15.78:0/2776957950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 msgr2=0x55d8cbff1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.806+0000 7f06914ec640 1 --2- 172.21.15.78:0/2776957950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.806+0000 7f06914ec640 1 -- 172.21.15.78:0/2776957950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbffc680 msgr2=0x55d8cbffea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.806+0000 7f06914ec640 1 --2- 172.21.15.78:0/2776957950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbffc680 0x55d8cbffea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.806+0000 7f06914ec640 1 -- 172.21.15.78:0/2776957950 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cbee3440 con 0x55d8cbff1dc0 2024-09-16T00:23:58.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.806+0000 7f06914ec640 1 --2- 172.21.15.78:0/2776957950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffc140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f067c00dc70 tx=0x7f067c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550fb3a59f78fc30 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:58.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.806+0000 7f0677fff640 1 -- 172.21.15.78:0/2776957950 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c015070 con 0x55d8cbff1dc0 2024-09-16T00:23:58.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.807+0000 7f0677fff640 1 -- 172.21.15.78:0/2776957950 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067c01a940 con 0x55d8cbff1dc0 2024-09-16T00:23:58.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.807+0000 7f0677fff640 1 -- 172.21.15.78:0/2776957950 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0119b0 con 0x55d8cbff1dc0 2024-09-16T00:23:58.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.808+0000 7f069632a740 1 -- 172.21.15.78:0/2776957950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 msgr2=0x55d8cbffc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.808+0000 7f069632a740 1 --2- 172.21.15.78:0/2776957950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffc140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f067c00dc70 tx=0x7f067c00f730 comp rx=0 tx=0).stop 2024-09-16T00:23:58.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.808+0000 7f069632a740 1 -- 172.21.15.78:0/2776957950 shutdown_connections 2024-09-16T00:23:58.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.808+0000 7f069632a740 1 --2- 172.21.15.78:0/2776957950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbffc680 0x55d8cbffea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.808+0000 7f069632a740 1 --2- 172.21.15.78:0/2776957950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffc140 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.808+0000 7f069632a740 1 --2- 172.21.15.78:0/2776957950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.808+0000 7f069632a740 1 -- 172.21.15.78:0/2776957950 >> 172.21.15.78:0/2776957950 conn(0x55d8cbfeda30 msgr2=0x55d8cbfede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:58.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.812+0000 7f069632a740 1 -- 172.21.15.78:0/2776957950 shutdown_connections 2024-09-16T00:23:58.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.812+0000 7f069632a740 1 -- 172.21.15.78:0/2776957950 wait complete. 2024-09-16T00:23:58.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.813+0000 7f069632a740 1 -- start start 2024-09-16T00:23:58.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.813+0000 7f069632a740 1 Processor -- start 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f069632a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff14a0 0x55d8cbffbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f069632a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff1dc0 0x55d8cbff9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f0691ced640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff14a0 0x55d8cbffbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f0691ced640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff14a0 0x55d8cbffbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59794/0 (socket says 172.21.15.78:59794) 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f0691ced640 1 -- 172.21.15.78:0/2572103740 learned_addr learned my addr 172.21.15.78:0/2572103740 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f069632a740 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbffc680 0x55d8cbff9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d8cbfff250 con 0x55d8cbff14a0 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d8cbf12050 con 0x55d8cbffc680 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d8cbfff3d0 con 0x55d8cbff1dc0 2024-09-16T00:23:58.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f06914ec640 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff1dc0 0x55d8cbff9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f069632a740 10 client.0 _ll_register_callbacks cb 0x55d8cbf52160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f06924ee640 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbffc680 0x55d8cbff9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f0691ced640 1 -- 172.21.15.78:0/2572103740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff1dc0 msgr2=0x55d8cbff9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f0691ced640 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff1dc0 0x55d8cbff9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f0691ced640 1 -- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbffc680 msgr2=0x55d8cbff9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f0691ced640 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbffc680 0x55d8cbff9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.814+0000 7f0691ced640 1 -- 172.21.15.78:0/2572103740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cbcc83c0 con 0x55d8cbff14a0 2024-09-16T00:23:58.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.815+0000 7f06914ec640 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff1dc0 0x55d8cbff9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:23:58.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.815+0000 7f0691ced640 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff14a0 0x55d8cbffbf90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f06880b8ff0 tx=0x7f0688009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd64eb1b63b4771 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:58.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.815+0000 7f06757fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.817+0000 7f06757fa640 1 -- 172.21.15.78:0/2572103740 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068804a030 con 0x55d8cbff14a0 2024-09-16T00:23:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.817+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f067c01a940 con 0x55d8cbff14a0 2024-09-16T00:23:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.817+0000 7f06757fa640 1 -- 172.21.15.78:0/2572103740 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f06880c5370 con 0x55d8cbff14a0 2024-09-16T00:23:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.817+0000 7f06757fa640 1 -- 172.21.15.78:0/2572103740 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f06880c0f40 con 0x55d8cbff14a0 2024-09-16T00:23:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.817+0000 7f069632a740 10 client.17298 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:58.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.817+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8cc00ac40 con 0x55d8cbff14a0 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f06757fa640 1 -- 172.21.15.78:0/2572103740 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067c01a940 con 0x55d8cbff14a0 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f06757fa640 1 -- 172.21.15.78:0/2572103740 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f06880bbfe0 con 0x55d8cbff14a0 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f069632a740 10 client.17298 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f069632a740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f069632a740 20 client.17298 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f069632a740 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d8cc00c0f0 0x55d8cc00e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f069632a740 4 client.17298 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8cbf64850 con 0x55d8cc00c0f0 2024-09-16T00:23:58.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.818+0000 7f06914ec640 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d8cc00c0f0 0x55d8cc00e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.819+0000 7f06914ec640 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d8cc00c0f0 0x55d8cc00e520 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e679651a3dd7460f server_cookie=ec1998e058a4d6b6 in_seq=0 out_seq=0 2024-09-16T00:23:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.820+0000 7f06757fa640 10 client.17298 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.820+0000 7f06757fa640 1 -- 172.21.15.78:0/2572103740 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8cbf64850 con 0x55d8cc00c0f0 2024-09-16T00:23:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.820+0000 7f06757fa640 10 client.17298 handle_command_reply: tid=0 2024-09-16T00:23:58.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.820+0000 7f069632a740 1 client.17298 shutdown 2024-09-16T00:23:58.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.821+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff14a0 msgr2=0x55d8cbffbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.821+0000 7f069632a740 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff14a0 0x55d8cbffbf90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f06880b8ff0 tx=0x7f0688009500 comp rx=0 tx=0).stop 2024-09-16T00:23:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.821+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 shutdown_connections 2024-09-16T00:23:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.821+0000 7f069632a740 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d8cc00c0f0 0x55d8cc00e520 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.821+0000 7f06757fa640 0 client.17298 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.822+0000 7f069632a740 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbffc680 0x55d8cbff9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.822+0000 7f069632a740 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff1dc0 0x55d8cbff9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.822+0000 7f069632a740 1 --2- 172.21.15.78:0/2572103740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff14a0 0x55d8cbffbf90 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.822+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 >> 172.21.15.78:0/2572103740 conn(0x55d8cbfeda30 msgr2=0x55d8cbff3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:58.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.822+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 shutdown_connections 2024-09-16T00:23:58.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.822+0000 7f069632a740 1 -- 172.21.15.78:0/2572103740 wait complete. 2024-09-16T00:23:58.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.822+0000 7f069632a740 20 client.17298 trim_cache size 0 max 16384 2024-09-16T00:23:58.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f069632a740 1 Processor -- start 2024-09-16T00:23:58.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f069632a740 1 -- start start 2024-09-16T00:23:58.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f069632a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbfeda30 0x55d8cc008540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f069632a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f069632a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cc013df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f0691ced640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbfeda30 0x55d8cc008540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f069632a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f067000fa50 con 0x55d8cbfeda30 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f0691ced640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbfeda30 0x55d8cc008540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59808/0 (socket says 172.21.15.78:59808) 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f069632a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0670011590 con 0x55d8cbff1dc0 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f0691ced640 1 -- 172.21.15.78:0/1964805824 learned_addr learned my addr 172.21.15.78:0/1964805824 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.849+0000 7f069632a740 1 -- 172.21.15.78:0/1964805824 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0670010120 con 0x55d8cbff14a0 2024-09-16T00:23:58.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f06914ec640 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f06924ee640 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cc013df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f0691ced640 1 -- 172.21.15.78:0/1964805824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 msgr2=0x55d8cbff47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f0691ced640 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f0691ced640 1 -- 172.21.15.78:0/1964805824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 msgr2=0x55d8cc013df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f0691ced640 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cc013df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f0691ced640 1 -- 172.21.15.78:0/1964805824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cbff7790 con 0x55d8cbfeda30 2024-09-16T00:23:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f06914ec640 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:23:58.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f0691ced640 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbfeda30 0x55d8cc008540 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f06880bae40 tx=0x7f0688048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c467782c386a9aab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f06777fe640 1 -- 172.21.15.78:0/1964805824 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068804a040 con 0x55d8cbfeda30 2024-09-16T00:23:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f06777fe640 1 -- 172.21.15.78:0/1964805824 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0688031720 con 0x55d8cbfeda30 2024-09-16T00:23:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.850+0000 7f06777fe640 1 -- 172.21.15.78:0/1964805824 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880c5370 con 0x55d8cbfeda30 2024-09-16T00:23:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 -- 172.21.15.78:0/1964805824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbfeda30 msgr2=0x55d8cc008540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbfeda30 0x55d8cc008540 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f06880bae40 tx=0x7f0688048000 comp rx=0 tx=0).stop 2024-09-16T00:23:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 -- 172.21.15.78:0/1964805824 shutdown_connections 2024-09-16T00:23:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff1dc0 0x55d8cc013df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbff14a0 0x55d8cbff47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 --2- 172.21.15.78:0/1964805824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbfeda30 0x55d8cc008540 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 -- 172.21.15.78:0/1964805824 >> 172.21.15.78:0/1964805824 conn(0x55d8cbff98e0 msgr2=0x55d8cbff40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 -- 172.21.15.78:0/1964805824 shutdown_connections 2024-09-16T00:23:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.851+0000 7f069632a740 1 -- 172.21.15.78:0/1964805824 wait complete. 2024-09-16T00:23:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.852+0000 7f069632a740 1 -- start start 2024-09-16T00:23:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.852+0000 7f069632a740 1 Processor -- start 2024-09-16T00:23:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f069632a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbfeda30 0x55d8cc004a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f069632a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff14a0 0x55d8cc004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f069632a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f069632a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0670010ec0 con 0x55d8cbff1dc0 2024-09-16T00:23:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f069632a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06700107f0 con 0x55d8cbff14a0 2024-09-16T00:23:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f069632a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f067000da30 con 0x55d8cbfeda30 2024-09-16T00:23:58.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f069632a740 10 client.0 _ll_register_callbacks cb 0x7f0688005750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f06924ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f06924ee640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59814/0 (socket says 172.21.15.78:59814) 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f06924ee640 1 -- 172.21.15.78:0/869646039 learned_addr learned my addr 172.21.15.78:0/869646039 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f0691ced640 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbfeda30 0x55d8cc004a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f06914ec640 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff14a0 0x55d8cc004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f06924ee640 1 -- 172.21.15.78:0/869646039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbfeda30 msgr2=0x55d8cc004a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.853+0000 7f06924ee640 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbfeda30 0x55d8cc004a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f06924ee640 1 -- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff14a0 msgr2=0x55d8cc004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f06924ee640 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff14a0 0x55d8cc004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f06924ee640 1 -- 172.21.15.78:0/869646039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cc0007f0 con 0x55d8cbff1dc0 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f06914ec640 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff14a0 0x55d8cc004f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:23:58.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f06924ee640 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffffc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0684012d60 tx=0x7f0684004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bd813bd9eacc42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f06757fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:23:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f06757fa640 1 -- 172.21.15.78:0/869646039 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068402d060 con 0x55d8cbff1dc0 2024-09-16T00:23:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f06757fa640 1 -- 172.21.15.78:0/869646039 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f068401e640 con 0x55d8cbff1dc0 2024-09-16T00:23:58.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.854+0000 7f069632a740 1 -- 172.21.15.78:0/869646039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0688031720 con 0x55d8cbff1dc0 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f06757fa640 1 -- 172.21.15.78:0/869646039 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f068401e880 con 0x55d8cbff1dc0 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f069632a740 10 client.17304 fetch_fsmap learned FSMap version 34 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f069632a740 1 -- 172.21.15.78:0/869646039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8cc076a90 con 0x55d8cbff1dc0 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f06757fa640 1 -- 172.21.15.78:0/869646039 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06840283f0 con 0x55d8cbff1dc0 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f06757fa640 1 -- 172.21.15.78:0/869646039 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0684027030 con 0x55d8cbff1dc0 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f069632a740 10 client.17304 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f069632a740 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f069632a740 20 client.17304 populate_metadata read hostname 'smithi078' 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f069632a740 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d8cbffc680 0x55d8cc079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f069632a740 4 client.17304 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.855+0000 7f069632a740 1 -- 172.21.15.78:0/869646039 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0688031720 con 0x55d8cbffc680 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f0691ced640 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d8cbffc680 0x55d8cc079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:58.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f06757fa640 10 client.17304 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f0691ced640 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d8cbffc680 0x55d8cc079200 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810ceecdebf27fd1 server_cookie=933d210a57964f1c in_seq=0 out_seq=0 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f06757fa640 1 -- 172.21.15.78:0/869646039 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f0688031720 con 0x55d8cbffc680 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f06757fa640 10 client.17304 handle_command_reply: tid=0 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f069632a740 1 client.17304 shutdown 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f069632a740 1 -- 172.21.15.78:0/869646039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff1dc0 msgr2=0x55d8cbffffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f069632a740 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffffc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0684012d60 tx=0x7f0684004a40 comp rx=0 tx=0).stop 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.856+0000 7f069632a740 1 -- 172.21.15.78:0/869646039 shutdown_connections 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f069632a740 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d8cbffc680 0x55d8cc079200 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f069632a740 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d8cbff1dc0 0x55d8cbffffc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f06757fa640 0 client.17304 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f069632a740 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d8cbff14a0 0x55d8cc004f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f069632a740 1 --2- 172.21.15.78:0/869646039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cbfeda30 0x55d8cc004a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f069632a740 1 -- 172.21.15.78:0/869646039 >> 172.21.15.78:0/869646039 conn(0x55d8cbff98e0 msgr2=0x55d8cbff48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f069632a740 1 -- 172.21.15.78:0/869646039 shutdown_connections 2024-09-16T00:23:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f069632a740 1 -- 172.21.15.78:0/869646039 wait complete. 2024-09-16T00:23:58.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.857+0000 7f069632a740 20 client.17304 trim_cache size 0 max 16384 2024-09-16T00:23:58.867 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:23:58.867 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:23:58.867 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:23:58.867 INFO:teuthology.orchestra.run.smithi078.stdout: "a8d47851-96ee-40b5-8f93-69ceb29f6b85": { 2024-09-16T00:23:58.867 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:23:58.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "a8d47851-96ee-40b5-8f93-69ceb29f6b85", 2024-09-16T00:23:58.868 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:23:58.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:23:58.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:23:58.868 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 -- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06780a3380 msgr2=0x7f06780a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06780a3380 0x7f06780a5840 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f068c076600 tx=0x7f0684009500 comp rx=0 tx=0).stop 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 -- 172.21.15.78:0/3838855196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 msgr2=0x7f068c1a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 0x7f068c1a4700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f068802a960 tx=0x7f06880308b0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 -- 172.21.15.78:0/3838855196 shutdown_connections 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f06780a3380 0x7f06780a5840 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f068c112b90 0x7f068c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f068c074eb0 0x7f068c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 --2- 172.21.15.78:0/3838855196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f068c074590 0x7f068c1a4700 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 -- 172.21.15.78:0/3838855196 >> 172.21.15.78:0/3838855196 conn(0x7f068c0fd880 msgr2=0x7f068c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:58.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.858+0000 7f0693f51640 1 -- 172.21.15.78:0/3838855196 shutdown_connections 2024-09-16T00:23:58.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:23:58.860+0000 7f0693f51640 1 -- 172.21.15.78:0/3838855196 wait complete. 2024-09-16T00:23:59.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:58 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:23:59.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:58 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:58 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:23:59.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:58 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:58 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:58 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:23:59.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:58 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:58 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:23:59.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:58 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:58 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:58 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:23:59.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:58 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:58 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:23:59.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:58 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:58 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:23:59.726 INFO:tasks.cephfs.filesystem:scrub status for tag:a8d47851-96ee-40b5-8f93-69ceb29f6b85 - {'path': '/', 'tag': 'a8d47851-96ee-40b5-8f93-69ceb29f6b85', 'options': 'recursive,force'} 2024-09-16T00:24:00.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:23:59 smithi078 ceph-mon[31157]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 648 op/s 2024-09-16T00:24:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:23:59 smithi193 ceph-mon[35484]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 648 op/s 2024-09-16T00:24:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:23:59 smithi148 ceph-mon[35599]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 648 op/s 2024-09-16T00:24:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 600.0 s, 538.6 tps, lat 190.203 ms stddev 238.506 2024-09-16T00:24:02.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:01 smithi193 ceph-mon[35484]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 640 op/s 2024-09-16T00:24:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:01 smithi148 ceph-mon[35599]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 640 op/s 2024-09-16T00:24:02.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:01 smithi078 ceph-mon[31157]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 MiB/s wr, 640 op/s 2024-09-16T00:24:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:03 smithi193 ceph-mon[35484]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 666 op/s 2024-09-16T00:24:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:03 smithi148 ceph-mon[35599]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 666 op/s 2024-09-16T00:24:04.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:03 smithi078 ceph-mon[31157]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 666 op/s 2024-09-16T00:24:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 605.0 s, 664.0 tps, lat 150.613 ms stddev 199.621 2024-09-16T00:24:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:05 smithi193 ceph-mon[35484]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 625 op/s 2024-09-16T00:24:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:05 smithi148 ceph-mon[35599]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 625 op/s 2024-09-16T00:24:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:05 smithi078 ceph-mon[31157]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.2 MiB/s wr, 625 op/s 2024-09-16T00:24:07.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:06 smithi193 ceph-mon[35484]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 634 op/s 2024-09-16T00:24:07.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:06 smithi148 ceph-mon[35599]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 634 op/s 2024-09-16T00:24:07.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:06 smithi078 ceph-mon[31157]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.2 MiB/s wr, 634 op/s 2024-09-16T00:24:10.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:09 smithi078 ceph-mon[31157]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 643 op/s 2024-09-16T00:24:10.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:09 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:24:10.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:09 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:24:10.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:09 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:24:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:09 smithi193 ceph-mon[35484]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 643 op/s 2024-09-16T00:24:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:09 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:24:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:09 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:24:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:09 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:24:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:09 smithi148 ceph-mon[35599]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.4 MiB/s wr, 643 op/s 2024-09-16T00:24:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:09 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:24:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:09 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:24:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:09 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:24:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 610.0 s, 854.6 tps, lat 120.482 ms stddev 162.502 2024-09-16T00:24:12.008 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:11 smithi148 ceph-mon[35599]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-09-16T00:24:12.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:11 smithi078 ceph-mon[31157]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-09-16T00:24:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:11 smithi193 ceph-mon[35484]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 652 op/s 2024-09-16T00:24:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:13 smithi078 ceph-mon[31157]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 643 op/s 2024-09-16T00:24:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:13 smithi193 ceph-mon[35484]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 643 op/s 2024-09-16T00:24:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:13 smithi148 ceph-mon[35599]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 643 op/s 2024-09-16T00:24:15.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 615.0 s, 679.4 tps, lat 143.817 ms stddev 164.788 2024-09-16T00:24:16.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:15 smithi078 ceph-mon[31157]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-09-16T00:24:16.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:15 smithi193 ceph-mon[35484]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-09-16T00:24:16.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:15 smithi148 ceph-mon[35599]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 663 op/s 2024-09-16T00:24:18.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:17 smithi193 ceph-mon[35484]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 660 op/s 2024-09-16T00:24:18.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:17 smithi148 ceph-mon[35599]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 660 op/s 2024-09-16T00:24:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:17 smithi078 ceph-mon[31157]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 660 op/s 2024-09-16T00:24:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:19 smithi193 ceph-mon[35484]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 642 op/s 2024-09-16T00:24:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:19 smithi148 ceph-mon[35599]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 642 op/s 2024-09-16T00:24:20.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:19 smithi078 ceph-mon[31157]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.9 MiB/s wr, 642 op/s 2024-09-16T00:24:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 620.0 s, 722.2 tps, lat 135.614 ms stddev 147.675 2024-09-16T00:24:21.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:20 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:21.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:20 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:21.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:20 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:21 smithi193 ceph-mon[35484]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 MiB/s wr, 632 op/s 2024-09-16T00:24:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:21 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:21 smithi148 ceph-mon[35599]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 MiB/s wr, 632 op/s 2024-09-16T00:24:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:21 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:22.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:21 smithi078 ceph-mon[31157]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 MiB/s wr, 632 op/s 2024-09-16T00:24:22.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:22.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:21 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:23.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:23.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:23.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:23.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:23.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:23.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:24:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:23 smithi193 ceph-mon[35484]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 590 op/s 2024-09-16T00:24:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:23 smithi148 ceph-mon[35599]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 590 op/s 2024-09-16T00:24:24.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:23 smithi078 ceph-mon[31157]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 590 op/s 2024-09-16T00:24:25.638 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:24:25.638 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-16T00:24:25.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:24:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 625.0 s, 615.2 tps, lat 167.960 ms stddev 238.786 2024-09-16T00:24:26.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:25 smithi078 ceph-mon[31157]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.7 MiB/s wr, 627 op/s 2024-09-16T00:24:26.057 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:26.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:25 smithi193 ceph-mon[35484]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.7 MiB/s wr, 627 op/s 2024-09-16T00:24:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:25 smithi148 ceph-mon[35599]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.7 MiB/s wr, 627 op/s 2024-09-16T00:24:26.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 -- 172.21.15.78:0/319781658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f8102d30 msgr2=0x7f40f8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/319781658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f8102d30 0x7f40f8103110 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009930 tx=0x7f40e402a420 comp rx=0 tx=0).stop 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 -- 172.21.15.78:0/319781658 shutdown_connections 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/319781658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f81082a0 0x7f40f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/319781658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40f81036e0 0x7f40f8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/319781658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f8102d30 0x7f40f8103110 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 -- 172.21.15.78:0/319781658 >> 172.21.15.78:0/319781658 conn(0x7f40f80fc7d0 msgr2=0x7f40f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 -- 172.21.15.78:0/319781658 shutdown_connections 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 -- 172.21.15.78:0/319781658 wait complete. 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.652+0000 7f40fdc6d640 1 Processor -- start 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40fdc6d640 1 -- start start 2024-09-16T00:24:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40fdc6d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40f8102d30 0x7f40f81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40fdc6d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81036e0 0x7f40f8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40fdc6d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f81082a0 0x7f40f819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40fdc6d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81120a0 con 0x7f40f81036e0 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40fdc6d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8111f20 con 0x7f40f81082a0 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40fdc6d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8112220 con 0x7f40f8102d30 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81036e0 0x7f40f8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f6ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81036e0 0x7f40f8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34280/0 (socket says 172.21.15.78:34280) 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f6ffd640 1 -- 172.21.15.78:0/4229569128 learned_addr learned my addr 172.21.15.78:0/4229569128 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f77fe640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40f8102d30 0x7f40f81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f7fff640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f81082a0 0x7f40f819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f6ffd640 1 -- 172.21.15.78:0/4229569128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40f8102d30 msgr2=0x7f40f81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f6ffd640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40f8102d30 0x7f40f81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f6ffd640 1 -- 172.21.15.78:0/4229569128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f81082a0 msgr2=0x7f40f819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f6ffd640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f81082a0 0x7f40f819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.653+0000 7f40f6ffd640 1 -- 172.21.15.78:0/4229569128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f819d550 con 0x7f40f81036e0 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.654+0000 7f40f6ffd640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81036e0 0x7f40f8198ae0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.654+0000 7f40f4ff9640 1 -- 172.21.15.78:0/4229569128 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800ccc0 con 0x7f40f81036e0 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.654+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f819d840 con 0x7f40f81036e0 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.654+0000 7f40f4ff9640 1 -- 172.21.15.78:0/4229569128 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8010910 con 0x7f40f81036e0 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.654+0000 7f40f4ff9640 1 -- 172.21.15.78:0/4229569128 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8018920 con 0x7f40f81036e0 2024-09-16T00:24:26.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.654+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f81a5120 con 0x7f40f81036e0 2024-09-16T00:24:26.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.655+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f806a850 con 0x7f40f81036e0 2024-09-16T00:24:26.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.656+0000 7f40f4ff9640 1 -- 172.21.15.78:0/4229569128 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f40e8010ac0 con 0x7f40f81036e0 2024-09-16T00:24:26.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.657+0000 7f40f4ff9640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40dc0a3240 0x7f40dc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:26.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.657+0000 7f40f77fe640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40dc0a3240 0x7f40dc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:26.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.657+0000 7f40f4ff9640 1 -- 172.21.15.78:0/4229569128 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f40e80d04e0 con 0x7f40f81036e0 2024-09-16T00:24:26.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.659+0000 7f40f77fe640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40dc0a3240 0x7f40dc0a5700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:26.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:26.661+0000 7f40f4ff9640 1 -- 172.21.15.78:0/4229569128 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40e809cea0 con 0x7f40f81036e0 2024-09-16T00:24:27.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.051+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40f810a2b0 con 0x7f40f81036e0 2024-09-16T00:24:27.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.053+0000 7f40f4ff9640 1 -- 172.21.15.78:0/4229569128 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f40e809c7a0 con 0x7f40f81036e0 2024-09-16T00:24:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:24:27.054 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:24:27.054 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.058+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40dc0a3240 msgr2=0x7f40dc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:27.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.058+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40dc0a3240 0x7f40dc0a5700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40e4000c00 tx=0x7f40e4005a20 comp rx=0 tx=0).stop 2024-09-16T00:24:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.058+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81036e0 msgr2=0x7f40f8198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:27.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.058+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81036e0 0x7f40f8198ae0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.059+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 shutdown_connections 2024-09-16T00:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.059+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f40dc0a3240 0x7f40dc0a5700 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.059+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f40f81082a0 0x7f40f819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.059+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40f81036e0 0x7f40f8198ae0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.059+0000 7f40fdc6d640 1 --2- 172.21.15.78:0/4229569128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40f8102d30 0x7f40f81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.059+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 >> 172.21.15.78:0/4229569128 conn(0x7f40f80fc7d0 msgr2=0x7f40f8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:27.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.060+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 shutdown_connections 2024-09-16T00:24:27.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:27.062+0000 7f40fdc6d640 1 -- 172.21.15.78:0/4229569128 wait complete. 2024-09-16T00:24:27.871 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-16T00:24:27.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-16T00:24:28.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:27 smithi193 ceph-mon[35484]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.5 MiB/s wr, 593 op/s 2024-09-16T00:24:28.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:27 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4229569128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:27 smithi148 ceph-mon[35599]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.5 MiB/s wr, 593 op/s 2024-09-16T00:24:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:27 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4229569128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:28.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:27 smithi078 ceph-mon[31157]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.5 MiB/s wr, 593 op/s 2024-09-16T00:24:28.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:27 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4229569128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:28.336 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:29.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.036+0000 7f583e659640 1 -- 172.21.15.78:0/3774251046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838112a20 msgr2=0x7f5838114e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:29.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.036+0000 7f583e659640 1 --2- 172.21.15.78:0/3774251046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838112a20 0x7f5838114e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f583001a910 tx=0x7f583003b430 comp rx=0 tx=0).stop 2024-09-16T00:24:29.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.038+0000 7f583e659640 1 -- 172.21.15.78:0/3774251046 shutdown_connections 2024-09-16T00:24:29.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.038+0000 7f583e659640 1 --2- 172.21.15.78:0/3774251046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838112a20 0x7f5838114e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.038+0000 7f583e659640 1 --2- 172.21.15.78:0/3774251046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5838077040 0x7f58381124e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.038+0000 7f583e659640 1 --2- 172.21.15.78:0/3774251046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5838076720 0x7f5838076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.038+0000 7f583e659640 1 -- 172.21.15.78:0/3774251046 >> 172.21.15.78:0/3774251046 conn(0x7f58380ffa60 msgr2=0x7f5838101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.042+0000 7f583e659640 1 -- 172.21.15.78:0/3774251046 shutdown_connections 2024-09-16T00:24:29.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.042+0000 7f583e659640 1 -- 172.21.15.78:0/3774251046 wait complete. 2024-09-16T00:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.043+0000 7f583e659640 1 Processor -- start 2024-09-16T00:24:29.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.043+0000 7f583e659640 1 -- start start 2024-09-16T00:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.043+0000 7f583e659640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5838076720 0x7f583810b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.043+0000 7f583e659640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838077040 0x7f583810bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.043+0000 7f583e659640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5838112a20 0x7f5838072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.043+0000 7f583e659640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5838106670 con 0x7f5838076720 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.043+0000 7f583e659640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58381064f0 con 0x7f5838112a20 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.043+0000 7f583e659640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58381067f0 con 0x7f5838077040 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838077040 0x7f583810bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838077040 0x7f583810bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47956/0 (socket says 172.21.15.78:47956) 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 -- 172.21.15.78:0/2938904221 learned_addr learned my addr 172.21.15.78:0/2938904221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 -- 172.21.15.78:0/2938904221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5838112a20 msgr2=0x7f5838072fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5838112a20 0x7f5838072fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 -- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5838076720 msgr2=0x7f583810b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5838076720 0x7f583810b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 -- 172.21.15.78:0/2938904221 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58380734e0 con 0x7f5838077040 2024-09-16T00:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58377fe640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838077040 0x7f583810bd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f582000d950 tx=0x7f582000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.044+0000 7f58357fa640 1 -- 172.21.15.78:0/2938904221 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820017070 con 0x7f5838077040 2024-09-16T00:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.045+0000 7f58357fa640 1 -- 172.21.15.78:0/2938904221 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5820009c30 con 0x7f5838077040 2024-09-16T00:24:29.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.045+0000 7f58357fa640 1 -- 172.21.15.78:0/2938904221 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582000f6b0 con 0x7f5838077040 2024-09-16T00:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.045+0000 7f583e659640 1 -- 172.21.15.78:0/2938904221 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5838073770 con 0x7f5838077040 2024-09-16T00:24:29.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.046+0000 7f583e659640 1 -- 172.21.15.78:0/2938904221 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5838073c60 con 0x7f5838077040 2024-09-16T00:24:29.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.049+0000 7f5840a32740 1 Processor -- start 2024-09-16T00:24:29.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.049+0000 7f5840a32740 1 -- start start 2024-09-16T00:24:29.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.049+0000 7f5840a32740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 0x55a5d0ffac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.049+0000 7f5840a32740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffb170 0x55a5d10054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.049+0000 7f5840a32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d1005a30 0x55a5d1007e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.049+0000 7f5840a32740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5d0f5f160 con 0x55a5d0ffb170 2024-09-16T00:24:29.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.049+0000 7f5840a32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a5d0f06720 con 0x55a5d1005a30 2024-09-16T00:24:29.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.049+0000 7f5840a32740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a5d0f5b120 con 0x55a5d0ffa850 2024-09-16T00:24:29.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d1005a30 0x55a5d1007e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d1005a30 0x55a5d1007e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:51948/0 (socket says 172.21.15.78:51948) 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 -- 172.21.15.78:0/1659916733 learned_addr learned my addr 172.21.15.78:0/1659916733 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 -- 172.21.15.78:0/1659916733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 msgr2=0x55a5d0ffac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 --2- 172.21.15.78:0/1659916733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 0x55a5d0ffac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 -- 172.21.15.78:0/1659916733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffb170 msgr2=0x55a5d10054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 --2- 172.21.15.78:0/1659916733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffb170 0x55a5d10054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 -- 172.21.15.78:0/1659916733 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d0cd0ba0 con 0x55a5d1005a30 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.050+0000 7f58377fe640 1 --2- 172.21.15.78:0/1659916733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d1005a30 0x55a5d1007e20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5820010db0 tx=0x7f5820010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=530d9a84f802153c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.052+0000 7f58357fa640 1 -- 172.21.15.78:0/2938904221 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5820051070 con 0x7f5838077040 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.052+0000 7f582e7fc640 1 -- 172.21.15.78:0/1659916733 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200d2a10 con 0x55a5d1005a30 2024-09-16T00:24:29.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.053+0000 7f582e7fc640 1 -- 172.21.15.78:0/1659916733 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58200d3280 con 0x55a5d1005a30 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.053+0000 7f58357fa640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58240a3340 0x7f58240a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f5840a32740 1 -- 172.21.15.78:0/1659916733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d1005a30 msgr2=0x55a5d1007e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f5840a32740 1 --2- 172.21.15.78:0/1659916733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d1005a30 0x55a5d1007e20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5820010db0 tx=0x7f5820010830 comp rx=0 tx=0).stop 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f58357fa640 1 -- 172.21.15.78:0/2938904221 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f58200d24a0 con 0x7f5838077040 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f583cbf6640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58240a3340 0x7f58240a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f5840a32740 1 -- 172.21.15.78:0/1659916733 shutdown_connections 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f5840a32740 1 --2- 172.21.15.78:0/1659916733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d1005a30 0x55a5d1007e20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f5840a32740 1 --2- 172.21.15.78:0/1659916733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffb170 0x55a5d10054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f5840a32740 1 --2- 172.21.15.78:0/1659916733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 0x55a5d0ffac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.054+0000 7f5840a32740 1 -- 172.21.15.78:0/1659916733 >> 172.21.15.78:0/1659916733 conn(0x55a5d0ff6de0 msgr2=0x55a5d0ff71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:29.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.055+0000 7f583cbf6640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58240a3340 0x7f58240a5800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f583001a6d0 tx=0x7f583003b430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.055+0000 7f5840a32740 1 -- 172.21.15.78:0/1659916733 shutdown_connections 2024-09-16T00:24:29.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.057+0000 7f5840a32740 1 -- 172.21.15.78:0/1659916733 wait complete. 2024-09-16T00:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.057+0000 7f5840a32740 1 -- start start 2024-09-16T00:24:29.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.057+0000 7f5840a32740 1 Processor -- start 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f5840a32740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1002890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f5840a32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d1002dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f5840a32740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 0x55a5d1003310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f5840a32740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1008600 con 0x55a5d0ffa850 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f5840a32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a5d0ee7900 con 0x55a5d0ffb170 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f5840a32740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a5d1008780 con 0x55a5d1005a30 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f5840a32740 10 client.0 _ll_register_callbacks cb 0x55a5d0ee5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f58377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d1002dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f58377fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d1002dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:51952/0 (socket says 172.21.15.78:51952) 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f58377fe640 1 -- 172.21.15.78:0/393278882 learned_addr learned my addr 172.21.15.78:0/393278882 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:29.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.058+0000 7f5837fff640 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1002890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.059+0000 7f58377fe640 1 -- 172.21.15.78:0/393278882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 msgr2=0x55a5d1003310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:24:29.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.059+0000 7f58377fe640 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 0x55a5d1003310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.059+0000 7f58377fe640 1 -- 172.21.15.78:0/393278882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 msgr2=0x55a5d1002890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:29.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.059+0000 7f58377fe640 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1002890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.059+0000 7f58377fe640 1 -- 172.21.15.78:0/393278882 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d0f27c50 con 0x55a5d0ffb170 2024-09-16T00:24:29.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.059+0000 7f58377fe640 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d1002dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f58200a6f50 tx=0x7f5820010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d911bdc6d349547 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.060+0000 7f58177fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.060+0000 7f58177fe640 1 -- 172.21.15.78:0/393278882 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582004e070 con 0x55a5d0ffb170 2024-09-16T00:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.060+0000 7f5840a32740 1 -- 172.21.15.78:0/393278882 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58200d3280 con 0x55a5d0ffb170 2024-09-16T00:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.060+0000 7f58177fe640 1 -- 172.21.15.78:0/393278882 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f58200d21a0 con 0x55a5d0ffb170 2024-09-16T00:24:29.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.064+0000 7f58177fe640 1 -- 172.21.15.78:0/393278882 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f582009e3e0 con 0x55a5d0ffb170 2024-09-16T00:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.064+0000 7f5840a32740 10 client.26654 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.064+0000 7f5840a32740 1 -- 172.21.15.78:0/393278882 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a5d1008600 con 0x55a5d0ffb170 2024-09-16T00:24:29.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.066+0000 7f58177fe640 1 -- 172.21.15.78:0/393278882 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58200d3280 con 0x55a5d0ffb170 2024-09-16T00:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.066+0000 7f58177fe640 1 -- 172.21.15.78:0/393278882 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5820023060 con 0x55a5d0ffb170 2024-09-16T00:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.066+0000 7f5840a32740 10 client.26654 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.066+0000 7f5840a32740 10 client.26654 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:24:29.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.067+0000 7f5840a32740 20 client.26654 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.067+0000 7f5840a32740 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5d1015680 0x55a5d1017ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.067+0000 7f5840a32740 4 client.26654 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.067+0000 7f5840a32740 1 -- 172.21.15.78:0/393278882 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5d0f6c890 con 0x55a5d1015680 2024-09-16T00:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.067+0000 7f5837fff640 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5d1015680 0x55a5d1017ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.068+0000 7f5837fff640 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5d1015680 0x55a5d1017ab0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223f2be9b9bbd46c server_cookie=31498de0e2dddbe5 in_seq=0 out_seq=0 2024-09-16T00:24:29.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.068+0000 7f58177fe640 10 client.26654 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.070+0000 7f58177fe640 1 -- 172.21.15.78:0/393278882 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5d0f6c890 con 0x55a5d1015680 2024-09-16T00:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.070+0000 7f58177fe640 10 client.26654 handle_command_reply: tid=0 2024-09-16T00:24:29.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.070+0000 7f5840a32740 1 client.26654 shutdown 2024-09-16T00:24:29.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.070+0000 7f5840a32740 1 -- 172.21.15.78:0/393278882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 msgr2=0x55a5d1002dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:29.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.070+0000 7f5840a32740 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d1002dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f58200a6f50 tx=0x7f5820010a90 comp rx=0 tx=0).stop 2024-09-16T00:24:29.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 1 -- 172.21.15.78:0/393278882 shutdown_connections 2024-09-16T00:24:29.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5d1015680 0x55a5d1017ab0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 0x55a5d1003310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d1002dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f58177fe640 0 client.26654 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:29.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 1 --2- 172.21.15.78:0/393278882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1002890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 1 -- 172.21.15.78:0/393278882 >> 172.21.15.78:0/393278882 conn(0x55a5d0ff6de0 msgr2=0x55a5d0ffcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:29.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 1 -- 172.21.15.78:0/393278882 shutdown_connections 2024-09-16T00:24:29.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 1 -- 172.21.15.78:0/393278882 wait complete. 2024-09-16T00:24:29.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.071+0000 7f5840a32740 20 client.26654 trim_cache size 0 max 16384 2024-09-16T00:24:29.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.093+0000 7f5840a32740 1 Processor -- start 2024-09-16T00:24:29.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.093+0000 7f5840a32740 1 -- start start 2024-09-16T00:24:29.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5840a32740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1003160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5840a32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d101d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5840a32740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 0x55a5d101d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5840a32740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58381b52c0 con 0x55a5d0ffa850 2024-09-16T00:24:29.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5840a32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58381b6e00 con 0x55a5d0ffb170 2024-09-16T00:24:29.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5840a32740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58381b5990 con 0x55a5d1005a30 2024-09-16T00:24:29.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5837fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1003160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5837fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1003160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34334/0 (socket says 172.21.15.78:34334) 2024-09-16T00:24:29.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5837fff640 1 -- 172.21.15.78:0/1696639281 learned_addr learned my addr 172.21.15.78:0/1696639281 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:29.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f58377fe640 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d101d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5837fff640 1 -- 172.21.15.78:0/1696639281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 msgr2=0x55a5d101d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:29.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f583cbf6640 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 0x55a5d101d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5837fff640 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 0x55a5d101d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5837fff640 1 -- 172.21.15.78:0/1696639281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 msgr2=0x55a5d101d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:29.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5837fff640 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d101d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f5837fff640 1 -- 172.21.15.78:0/1696639281 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5d0ffafa0 con 0x55a5d0ffa850 2024-09-16T00:24:29.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.094+0000 7f58377fe640 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d101d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:29.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.095+0000 7f5837fff640 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1003160 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5828005fd0 tx=0x7f58280093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bea74c10b98a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.095+0000 7f582dffb640 1 -- 172.21.15.78:0/1696639281 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582801d020 con 0x55a5d0ffa850 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.095+0000 7f582dffb640 1 -- 172.21.15.78:0/1696639281 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58280199b0 con 0x55a5d0ffa850 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.095+0000 7f582dffb640 1 -- 172.21.15.78:0/1696639281 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828010ac0 con 0x55a5d0ffa850 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 -- 172.21.15.78:0/1696639281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 msgr2=0x55a5d1003160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1003160 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5828005fd0 tx=0x7f58280093c0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 -- 172.21.15.78:0/1696639281 shutdown_connections 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d1005a30 0x55a5d101d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d101d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 --2- 172.21.15.78:0/1696639281 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d0ffa850 0x55a5d1003160 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 -- 172.21.15.78:0/1696639281 >> 172.21.15.78:0/1696639281 conn(0x55a5d0ff6de0 msgr2=0x55a5d1004910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:29.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 -- 172.21.15.78:0/1696639281 shutdown_connections 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 -- 172.21.15.78:0/1696639281 wait complete. 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.096+0000 7f5840a32740 1 -- start start 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5840a32740 1 Processor -- start 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5840a32740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 0x55a5d100dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5840a32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d100e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5840a32740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d1005a30 0x55a5d1008fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5840a32740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f58381b6730 con 0x55a5d1005a30 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5840a32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58381b6060 con 0x55a5d0ffb170 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5840a32740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58381b32a0 con 0x55a5d0ffa850 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5840a32740 10 client.0 _ll_register_callbacks cb 0x7f5820005630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5837fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 0x55a5d100dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5837fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 0x55a5d100dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47972/0 (socket says 172.21.15.78:47972) 2024-09-16T00:24:29.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f5837fff640 1 -- 172.21.15.78:0/3893708252 learned_addr learned my addr 172.21.15.78:0/3893708252 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f58377fe640 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d100e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.097+0000 7f583cbf6640 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d1005a30 0x55a5d1008fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58377fe640 1 -- 172.21.15.78:0/3893708252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 msgr2=0x55a5d100dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58377fe640 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 0x55a5d100dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58377fe640 1 -- 172.21.15.78:0/3893708252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d1005a30 msgr2=0x55a5d1008fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58377fe640 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d1005a30 0x55a5d1008fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58377fe640 1 -- 172.21.15.78:0/3893708252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5d1009810 con 0x55a5d0ffb170 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58377fe640 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d100e110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55a5d0ff7e40 tx=0x7f5820000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964392cf83ab0428 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58177fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58177fe640 1 -- 172.21.15.78:0/3893708252 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582004e070 con 0x55a5d0ffb170 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58177fe640 1 -- 172.21.15.78:0/3893708252 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f582009fbd0 con 0x55a5d0ffb170 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f5840a32740 1 -- 172.21.15.78:0/3893708252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58280199b0 con 0x55a5d0ffb170 2024-09-16T00:24:29.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f58177fe640 1 -- 172.21.15.78:0/3893708252 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f58200d4180 con 0x55a5d0ffb170 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f5840a32740 10 client.26657 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.098+0000 7f5840a32740 1 -- 172.21.15.78:0/3893708252 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a5d1009810 con 0x55a5d0ffb170 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f58177fe640 1 -- 172.21.15.78:0/3893708252 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582009efa0 con 0x55a5d0ffb170 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f58177fe640 1 -- 172.21.15.78:0/3893708252 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5820023070 con 0x55a5d0ffb170 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f5840a32740 10 client.26657 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f5840a32740 10 client.26657 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f5840a32740 20 client.26657 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f5840a32740 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5d1015680 0x55a5d10821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f5840a32740 4 client.26657 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f5840a32740 1 -- 172.21.15.78:0/3893708252 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x7f58280199b0 con 0x55a5d1015680 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f5837fff640 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5d1015680 0x55a5d10821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:29.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f58177fe640 10 client.26657 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:29.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.099+0000 7f5837fff640 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5d1015680 0x55a5d10821d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79a45ad4c3ff0597 server_cookie=447900b0b655d15a in_seq=0 out_seq=0 2024-09-16T00:24:29.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:28 smithi193 ceph-mon[35484]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.8 MiB/s wr, 632 op/s 2024-09-16T00:24:29.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:28 smithi148 ceph-mon[35599]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.8 MiB/s wr, 632 op/s 2024-09-16T00:24:29.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:28 smithi078 ceph-mon[31157]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.8 MiB/s wr, 632 op/s 2024-09-16T00:24:29.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f58177fe640 1 -- 172.21.15.78:0/3893708252 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f58280199b0 con 0x55a5d1015680 2024-09-16T00:24:29.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f58177fe640 10 client.26657 handle_command_reply: tid=0 2024-09-16T00:24:29.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 client.26657 shutdown 2024-09-16T00:24:29.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 -- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 msgr2=0x55a5d100e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d100e110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55a5d0ff7e40 tx=0x7f5820000bc0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 -- 172.21.15.78:0/3893708252 shutdown_connections 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5d1015680 0x55a5d10821d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5d1005a30 0x55a5d1008fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5d0ffb170 0x55a5d100e110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 --2- 172.21.15.78:0/3893708252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5d0ffa850 0x55a5d100dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f58177fe640 0 client.26657 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 -- 172.21.15.78:0/3893708252 >> 172.21.15.78:0/3893708252 conn(0x55a5d0ff6de0 msgr2=0x55a5d0ff56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 -- 172.21.15.78:0/3893708252 shutdown_connections 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 1 -- 172.21.15.78:0/3893708252 wait complete. 2024-09-16T00:24:29.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.340+0000 7f5840a32740 20 client.26657 trim_cache size 0 max 16384 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 38, 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "d73a29d3": { 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 38, 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:24:29.343 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 141.8, 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:29.344 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:24:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 -- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58240a3340 msgr2=0x7f58240a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58240a3340 0x7f58240a5800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f583001a6d0 tx=0x7f583003b430 comp rx=0 tx=0).stop 2024-09-16T00:24:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 -- 172.21.15.78:0/2938904221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838077040 msgr2=0x7f583810bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838077040 0x7f583810bd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f582000d950 tx=0x7f582000de10 comp rx=0 tx=0).stop 2024-09-16T00:24:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 -- 172.21.15.78:0/2938904221 shutdown_connections 2024-09-16T00:24:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f58240a3340 0x7f58240a5800 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5838112a20 0x7f5838072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5838077040 0x7f583810bd60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 --2- 172.21.15.78:0/2938904221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5838076720 0x7f583810b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:29.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 -- 172.21.15.78:0/2938904221 >> 172.21.15.78:0/2938904221 conn(0x7f58380ffa60 msgr2=0x7f5838101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:29.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 -- 172.21.15.78:0/2938904221 shutdown_connections 2024-09-16T00:24:29.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:29.341+0000 7f583e659640 1 -- 172.21.15.78:0/2938904221 wait complete. 2024-09-16T00:24:29.727 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:24:30.860 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 630.0 s, 509.0 tps, lat 135.187 ms stddev 154.125 2024-09-16T00:24:32.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:31 smithi078 ceph-mon[31157]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 626 op/s 2024-09-16T00:24:32.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:31 smithi193 ceph-mon[35484]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 626 op/s 2024-09-16T00:24:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:31 smithi148 ceph-mon[35599]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 626 op/s 2024-09-16T00:24:33.441 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd73a29d3', quiesce took 7.8 seconds. Will release after: 14.05 2024-09-16T00:24:33.499 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:33.977 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:33 smithi078 ceph-mon[31157]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 576 op/s 2024-09-16T00:24:34.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2091373789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102ef0 msgr2=0x7f1eb010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2091373789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102ef0 0x7f1eb010da40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009ca0 tx=0x7f1ea802a950 comp rx=0 tx=0).stop 2024-09-16T00:24:34.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2091373789 shutdown_connections 2024-09-16T00:24:34.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2091373789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102ef0 0x7f1eb010da40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2091373789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb0102570 0x7f1eb01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2091373789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1eb0100e20 0x7f1eb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2091373789 >> 172.21.15.78:0/2091373789 conn(0x7f1eb00fc7d0 msgr2=0x7f1eb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2091373789 shutdown_connections 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.058+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2091373789 wait complete. 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb6c4a640 1 Processor -- start 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb6c4a640 1 -- start start 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb6c4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb0100e20 0x7f1eb0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb6c4a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102570 0x7f1eb01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb6c4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1eb0102ef0 0x7f1eb019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb6c4a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0110f80 con 0x7f1eb0102570 2024-09-16T00:24:34.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb6c4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0110e00 con 0x7f1eb0102ef0 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb6c4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb0111100 con 0x7f1eb0100e20 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102570 0x7f1eb01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102570 0x7f1eb01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34374/0 (socket says 172.21.15.78:34374) 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1ea7fff640 1 -- 172.21.15.78:0/2387012290 learned_addr learned my addr 172.21.15.78:0/2387012290 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.059+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1eb0102ef0 0x7f1eb019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb0100e20 0x7f1eb0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea7fff640 1 -- 172.21.15.78:0/2387012290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb0100e20 msgr2=0x7f1eb0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb0100e20 0x7f1eb0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea7fff640 1 -- 172.21.15.78:0/2387012290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1eb0102ef0 msgr2=0x7f1eb019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1eb0102ef0 0x7f1eb019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea7fff640 1 -- 172.21.15.78:0/2387012290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb019d440 con 0x7f1eb0102570 2024-09-16T00:24:34.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb0100e20 0x7f1eb0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102570 0x7f1eb01989d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000c890 tx=0x7f1ea000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea5ffb640 1 -- 172.21.15.78:0/2387012290 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0004220 con 0x7f1eb0102570 2024-09-16T00:24:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2387012290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb019d730 con 0x7f1eb0102570 2024-09-16T00:24:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea5ffb640 1 -- 172.21.15.78:0/2387012290 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea00043c0 con 0x7f1eb0102570 2024-09-16T00:24:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1ea5ffb640 1 -- 172.21.15.78:0/2387012290 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00049e0 con 0x7f1eb0102570 2024-09-16T00:24:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.060+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2387012290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb01a5010 con 0x7f1eb0102570 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.062+0000 7f1ea5ffb640 1 -- 172.21.15.78:0/2387012290 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1ea00079b0 con 0x7f1eb0102570 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb9023740 1 Processor -- start 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb9023740 1 -- start start 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc17880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc22140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 0x5560cdc24a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb9023740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5560cdb7d2e0 con 0x5560cdc17dc0 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1ea5ffb640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1e980a3240 0x7f1e980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb9023740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5560cdb24ce0 con 0x5560cdc174a0 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb9023740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560cdb792a0 con 0x5560cdc22680 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb49e6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc22140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb49e6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc22140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34376/0 (socket says 172.21.15.78:34376) 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb49e6640 1 -- 172.21.15.78:0/4120600721 learned_addr learned my addr 172.21.15.78:0/4120600721 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.063+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1e980a3240 0x7f1e980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1ea5ffb640 1 -- 172.21.15.78:0/2387012290 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1ea00cf960 con 0x7f1eb0102570 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc17880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 0x5560cdc24a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1eb49e6640 1 -- 172.21.15.78:0/4120600721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 msgr2=0x5560cdc24a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 0x5560cdc24a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1eb49e6640 1 -- 172.21.15.78:0/4120600721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 msgr2=0x5560cdc17880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc17880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1eb49e6640 1 -- 172.21.15.78:0/4120600721 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560cdb09440 con 0x5560cdc17dc0 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc22140 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009c50 tx=0x7f1eac00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f056f0bdf8eb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1e980a3240 0x7f1e980a5700 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009a60 tx=0x7f1ea802fe00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1e96ffd640 1 -- 172.21.15.78:0/4120600721 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac016070 con 0x5560cdc17dc0 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1e96ffd640 1 -- 172.21.15.78:0/4120600721 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac005500 con 0x5560cdc17dc0 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.064+0000 7f1e96ffd640 1 -- 172.21.15.78:0/4120600721 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eac011670 con 0x5560cdc17dc0 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 -- 172.21.15.78:0/4120600721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 msgr2=0x5560cdc22140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc22140 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009c50 tx=0x7f1eac00e6c0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 -- 172.21.15.78:0/4120600721 shutdown_connections 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 0x5560cdc24a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc22140 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 --2- 172.21.15.78:0/4120600721 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc17880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 -- 172.21.15.78:0/4120600721 >> 172.21.15.78:0/4120600721 conn(0x5560cdc13a30 msgr2=0x5560cdc13e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 -- 172.21.15.78:0/4120600721 shutdown_connections 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 -- 172.21.15.78:0/4120600721 wait complete. 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 -- start start 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.065+0000 7f1eb9023740 1 Processor -- start 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc21f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc1f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 0x5560cdc1f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb9023740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5560cdc25250 con 0x5560cdc17dc0 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb9023740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5560cdb38050 con 0x5560cdc174a0 2024-09-16T00:24:34.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb9023740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560cdc253d0 con 0x5560cdc22680 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb9023740 10 client.0 _ll_register_callbacks cb 0x5560cdb78160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc1f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc1f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34386/0 (socket says 172.21.15.78:34386) 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb49e6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc21f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 -- 172.21.15.78:0/2874634634 learned_addr learned my addr 172.21.15.78:0/2874634634 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 0x5560cdc1f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 -- 172.21.15.78:0/2874634634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 msgr2=0x5560cdc1f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 0x5560cdc1f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 -- 172.21.15.78:0/2874634634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 msgr2=0x5560cdc21f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc21f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 -- 172.21.15.78:0/2874634634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560cd8ee3c0 con 0x5560cdc17dc0 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc1f1d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00985f0 tx=0x7f1ea009b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5a5e69046f6623 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.066+0000 7f1e8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1e8ffff640 1 -- 172.21.15.78:0/2874634634 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea001d040 con 0x5560cdc17dc0 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1e8ffff640 1 -- 172.21.15.78:0/2874634634 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1ea009a750 con 0x5560cdc17dc0 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 1 -- 172.21.15.78:0/2874634634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eac005500 con 0x5560cdc17dc0 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1e8ffff640 1 -- 172.21.15.78:0/2874634634 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ea009bc20 con 0x5560cdc17dc0 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 10 client.17331 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 1 -- 172.21.15.78:0/2874634634 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560cdc30b50 con 0x5560cdc17dc0 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1e8ffff640 1 -- 172.21.15.78:0/2874634634 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eac005500 con 0x5560cdc17dc0 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1e8ffff640 1 -- 172.21.15.78:0/2874634634 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1ea00ba430 con 0x5560cdc17dc0 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 10 client.17331 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:24:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 20 client.17331 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5560cdc31fb0 0x5560cdc34420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 4 client.17331 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.067+0000 7f1eb9023740 1 -- 172.21.15.78:0/2874634634 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560cdb8a850 con 0x5560cdc31fb0 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.068+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5560cdc31fb0 0x5560cdc34420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.068+0000 7f1e8ffff640 10 client.17331 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.068+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5560cdc31fb0 0x5560cdc34420 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218a7a3b8e96390 server_cookie=ba600745037d354f in_seq=0 out_seq=0 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1e8ffff640 1 -- 172.21.15.78:0/2874634634 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560cdb8a850 con 0x5560cdc31fb0 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1e8ffff640 10 client.17331 handle_command_reply: tid=0 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 client.17331 shutdown 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 -- 172.21.15.78:0/2874634634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 msgr2=0x5560cdc1f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc1f1d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00985f0 tx=0x7f1ea009b640 comp rx=0 tx=0).stop 2024-09-16T00:24:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 -- 172.21.15.78:0/2874634634 shutdown_connections 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5560cdc31fb0 0x5560cdc34420 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1e8ffff640 0 client.17331 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc22680 0x5560cdc1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc17dc0 0x5560cdc1f1d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 --2- 172.21.15.78:0/2874634634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc21f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 -- 172.21.15.78:0/2874634634 >> 172.21.15.78:0/2874634634 conn(0x5560cdc13a30 msgr2=0x5560cdc12b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 -- 172.21.15.78:0/2874634634 shutdown_connections 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 1 -- 172.21.15.78:0/2874634634 wait complete. 2024-09-16T00:24:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.069+0000 7f1eb9023740 20 client.17331 trim_cache size 0 max 16384 2024-09-16T00:24:34.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb9023740 1 Processor -- start 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb9023740 1 -- start start 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc2e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc17dc0 0x5560cdc39cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc3a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb9023740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01aca70 con 0x5560cdc22680 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb9023740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01ae5b0 con 0x5560cdc174a0 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb9023740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01ad140 con 0x5560cdc17dc0 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb51e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc3a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb51e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc3a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34396/0 (socket says 172.21.15.78:34396) 2024-09-16T00:24:34.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.086+0000 7f1eb51e7640 1 -- 172.21.15.78:0/1180590850 learned_addr learned my addr 172.21.15.78:0/1180590850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/1180590850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc2e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1eb51e7640 1 -- 172.21.15.78:0/1180590850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc17dc0 msgr2=0x5560cdc39cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/1180590850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc17dc0 0x5560cdc39cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1eb51e7640 1 -- 172.21.15.78:0/1180590850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 msgr2=0x5560cdc2e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/1180590850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc2e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1eb51e7640 1 -- 172.21.15.78:0/1180590850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560cdc22290 con 0x5560cdc22680 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/1180590850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc3a230 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009ca0 tx=0x7f1ea8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f78e08bb402170 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1e967fc640 1 -- 172.21.15.78:0/1180590850 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8034040 con 0x5560cdc22680 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1e967fc640 1 -- 172.21.15.78:0/1180590850 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea802f5b0 con 0x5560cdc22680 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.087+0000 7f1e967fc640 1 -- 172.21.15.78:0/1180590850 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea803b8e0 con 0x5560cdc22680 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 -- 172.21.15.78:0/1180590850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 msgr2=0x5560cdc3a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1180590850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc3a230 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009ca0 tx=0x7f1ea8005c50 comp rx=0 tx=0).stop 2024-09-16T00:24:34.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 -- 172.21.15.78:0/1180590850 shutdown_connections 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1180590850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc3a230 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1180590850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc17dc0 0x5560cdc39cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1180590850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc174a0 0x5560cdc2e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 -- 172.21.15.78:0/1180590850 >> 172.21.15.78:0/1180590850 conn(0x5560cdc13a30 msgr2=0x5560cdc1e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 -- 172.21.15.78:0/1180590850 shutdown_connections 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 -- 172.21.15.78:0/1180590850 wait complete. 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 -- start start 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.088+0000 7f1eb9023740 1 Processor -- start 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc174a0 0x5560cdc2ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc17dc0 0x5560cdc260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb9023740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc265e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb9023740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01adee0 con 0x5560cdc22680 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb9023740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01ad810 con 0x5560cdc17dc0 2024-09-16T00:24:34.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb9023740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb01aaa50 con 0x5560cdc174a0 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb9023740 10 client.0 _ll_register_callbacks cb 0x7f1ea000a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34406/0 (socket says 172.21.15.78:34406) 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 -- 172.21.15.78:0/1705768266 learned_addr learned my addr 172.21.15.78:0/1705768266 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc17dc0 0x5560cdc260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc174a0 0x5560cdc2ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 -- 172.21.15.78:0/1705768266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc174a0 msgr2=0x5560cdc2ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc174a0 0x5560cdc2ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 -- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc17dc0 msgr2=0x5560cdc260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc17dc0 0x5560cdc260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 -- 172.21.15.78:0/1705768266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560cdc26dc0 con 0x5560cdc22680 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1ea7fff640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc17dc0 0x5560cdc260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:34.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc174a0 0x5560cdc2ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1eb51e7640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc265e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea8009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6fc484c7e14c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1e8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.089+0000 7f1e8ffff640 1 -- 172.21.15.78:0/1705768266 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea802f760 con 0x5560cdc22680 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1e8ffff640 1 -- 172.21.15.78:0/1705768266 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1ea803b580 con 0x5560cdc22680 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 1 -- 172.21.15.78:0/1705768266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea802f5b0 con 0x5560cdc22680 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1e8ffff640 1 -- 172.21.15.78:0/1705768266 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ea8068050 con 0x5560cdc22680 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 10 client.17337 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 1 -- 172.21.15.78:0/1705768266 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560cdc9cc80 con 0x5560cdc22680 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1e8ffff640 1 -- 172.21.15.78:0/1705768266 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ea802f5b0 con 0x5560cdc22680 2024-09-16T00:24:34.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1e8ffff640 1 -- 172.21.15.78:0/1705768266 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1ea8030710 con 0x5560cdc22680 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 10 client.17337 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 20 client.17337 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5560cdc31fb0 0x5560cdc9f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 4 client.17337 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.090+0000 7f1eb9023740 1 -- 172.21.15.78:0/1705768266 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560cdb84d60 con 0x5560cdc31fb0 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5560cdc31fb0 0x5560cdc9f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb49e6640 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5560cdc31fb0 0x5560cdc9f360 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=405221869ee6ccf4 server_cookie=b63d49e18a8887ce in_seq=0 out_seq=0 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1e8ffff640 10 client.17337 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1e8ffff640 1 -- 172.21.15.78:0/1705768266 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5560cdb84d60 con 0x5560cdc31fb0 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1e8ffff640 10 client.17337 handle_command_reply: tid=0 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb9023740 1 client.17337 shutdown 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb9023740 1 -- 172.21.15.78:0/1705768266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 msgr2=0x5560cdc265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc265e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea8009890 comp rx=0 tx=0).stop 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb9023740 1 -- 172.21.15.78:0/1705768266 shutdown_connections 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5560cdc31fb0 0x5560cdc9f360 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5560cdc22680 0x5560cdc265e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5560cdc17dc0 0x5560cdc260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.091+0000 7f1eb9023740 1 --2- 172.21.15.78:0/1705768266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560cdc174a0 0x5560cdc2ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb9023740 1 -- 172.21.15.78:0/1705768266 >> 172.21.15.78:0/1705768266 conn(0x5560cdc13a30 msgr2=0x5560cdc19fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1e8ffff640 0 client.17337 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb9023740 1 -- 172.21.15.78:0/1705768266 shutdown_connections 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb9023740 1 -- 172.21.15.78:0/1705768266 wait complete. 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb9023740 20 client.17337 trim_cache size 0 max 16384 2024-09-16T00:24:34.099 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:24:34.100 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:24:34.100 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:24:34.100 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:24:34.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1e980a3240 msgr2=0x7f1e980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1e980a3240 0x7f1e980a5700 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009a60 tx=0x7f1ea802fe00 comp rx=0 tx=0).stop 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102570 msgr2=0x7f1eb01989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102570 0x7f1eb01989d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000c890 tx=0x7f1ea000cd50 comp rx=0 tx=0).stop 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2387012290 shutdown_connections 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1e980a3240 0x7f1e980a5700 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1eb0102ef0 0x7f1eb019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1eb0102570 0x7f1eb01989d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 --2- 172.21.15.78:0/2387012290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb0100e20 0x7f1eb0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2387012290 >> 172.21.15.78:0/2387012290 conn(0x7f1eb00fc7d0 msgr2=0x7f1eb006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2387012290 shutdown_connections 2024-09-16T00:24:34.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:34.092+0000 7f1eb6c4a640 1 -- 172.21.15.78:0/2387012290 wait complete. 2024-09-16T00:24:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:33 smithi193 ceph-mon[35484]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 576 op/s 2024-09-16T00:24:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:33 smithi148 ceph-mon[35599]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.2 MiB/s wr, 576 op/s 2024-09-16T00:24:34.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:24:34.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:24:35.180 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/2418087742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec100a80 msgr2=0x7fdcec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/2418087742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec100a80 0x7fdcec100ee0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009930 tx=0x7fdcdc02a420 comp rx=0 tx=0).stop 2024-09-16T00:24:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/2418087742 shutdown_connections 2024-09-16T00:24:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/2418087742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcec101420 0x7fdcec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/2418087742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec100a80 0x7fdcec100ee0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/2418087742 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcec106a80 0x7fdcec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/2418087742 >> 172.21.15.78:0/2418087742 conn(0x7fdcec0fc7a0 msgr2=0x7fdcec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/2418087742 shutdown_connections 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.706+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/2418087742 wait complete. 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcf0aa9640 1 Processor -- start 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcf0aa9640 1 -- start start 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcf0aa9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcec100a80 0x7fdcec1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcf0aa9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcec101420 0x7fdcec199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcf0aa9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec106a80 0x7fdcec1a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcf0aa9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec110ea0 con 0x7fdcec106a80 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcf0aa9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec110d20 con 0x7fdcec100a80 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcf0aa9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec111020 con 0x7fdcec101420 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcead9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec106a80 0x7fdcec1a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcead9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec106a80 0x7fdcec1a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34430/0 (socket says 172.21.15.78:34430) 2024-09-16T00:24:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcead9d640 1 -- 172.21.15.78:0/788876168 learned_addr learned my addr 172.21.15.78:0/788876168 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdce9d9b640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcec101420 0x7fdcec199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcea59c640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcec100a80 0x7fdcec1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcead9d640 1 -- 172.21.15.78:0/788876168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcec101420 msgr2=0x7fdcec199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcead9d640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcec101420 0x7fdcec199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcead9d640 1 -- 172.21.15.78:0/788876168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcec100a80 msgr2=0x7fdcec1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcead9d640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcec100a80 0x7fdcec1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcead9d640 1 -- 172.21.15.78:0/788876168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec1a48b0 con 0x7fdcec106a80 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdcea59c640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcec100a80 0x7fdcec1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.707+0000 7fdce9d9b640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcec101420 0x7fdcec199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.708+0000 7fdcead9d640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec106a80 0x7fdcec1a4370 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fdce400cce0 tx=0x7fdce400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.708+0000 7fdcdb7fe640 1 -- 172.21.15.78:0/788876168 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce400f450 con 0x7fdcec106a80 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.708+0000 7fdcdb7fe640 1 -- 172.21.15.78:0/788876168 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce400fad0 con 0x7fdcec106a80 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.708+0000 7fdcdb7fe640 1 -- 172.21.15.78:0/788876168 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4018c10 con 0x7fdcec106a80 2024-09-16T00:24:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.708+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec1a4a80 con 0x7fdcec106a80 2024-09-16T00:24:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.708+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1a4f40 con 0x7fdcec106a80 2024-09-16T00:24:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.709+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec100ee0 con 0x7fdcec106a80 2024-09-16T00:24:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.710+0000 7fdcdb7fe640 1 -- 172.21.15.78:0/788876168 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fdce400f5f0 con 0x7fdcec106a80 2024-09-16T00:24:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.711+0000 7fdcdb7fe640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdcd40a3240 0x7fdcd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.711+0000 7fdcea59c640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdcd40a3240 0x7fdcd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.711+0000 7fdcdb7fe640 1 -- 172.21.15.78:0/788876168 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdce40d0470 con 0x7fdcec106a80 2024-09-16T00:24:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.711+0000 7fdcea59c640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdcd40a3240 0x7fdcd40a5700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdce0006fd0 tx=0x7fdce0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.714+0000 7fdcdb7fe640 1 -- 172.21.15.78:0/788876168 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdce4014030 con 0x7fdcec106a80 2024-09-16T00:24:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 635.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:24:35.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.916+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcec06ad80 con 0x7fdcec106a80 2024-09-16T00:24:35.917 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:35 smithi078 ceph-mon[31157]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 508 op/s 2024-09-16T00:24:35.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.916+0000 7fdcdb7fe640 1 -- 172.21.15.78:0/788876168 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fdcec06ad80 con 0x7fdcec106a80 2024-09-16T00:24:35.917 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:24:35.918 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:24:35.918 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:24:35.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdcd40a3240 msgr2=0x7fdcd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:35.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdcd40a3240 0x7fdcd40a5700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdce0006fd0 tx=0x7fdce0008040 comp rx=0 tx=0).stop 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec106a80 msgr2=0x7fdcec1a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec106a80 0x7fdcec1a4370 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fdce400cce0 tx=0x7fdce400a620 comp rx=0 tx=0).stop 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 shutdown_connections 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdcd40a3240 0x7fdcd40a5700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcec106a80 0x7fdcec1a4370 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcec101420 0x7fdcec199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 --2- 172.21.15.78:0/788876168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdcec100a80 0x7fdcec1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 >> 172.21.15.78:0/788876168 conn(0x7fdcec0fc7a0 msgr2=0x7fdcec0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 shutdown_connections 2024-09-16T00:24:35.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:35.919+0000 7fdcf0aa9640 1 -- 172.21.15.78:0/788876168 wait complete. 2024-09-16T00:24:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:35 smithi193 ceph-mon[35484]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 508 op/s 2024-09-16T00:24:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:35 smithi148 ceph-mon[35599]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 MiB/s wr, 508 op/s 2024-09-16T00:24:36.562 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:24:36.811 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:36.932 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:36 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/788876168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:37.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:36 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/788876168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:37.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:36 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/788876168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:37.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 -- 172.21.15.78:0/3438343183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34105f40 msgr2=0x7fee341063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 --2- 172.21.15.78:0/3438343183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34105f40 0x7fee341063d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009900 tx=0x7fee2c02a470 comp rx=0 tx=0).stop 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 -- 172.21.15.78:0/3438343183 shutdown_connections 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 --2- 172.21.15.78:0/3438343183 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34105f40 0x7fee341063d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 --2- 172.21.15.78:0/3438343183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee34101450 0x7fee34105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 --2- 172.21.15.78:0/3438343183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee34100b30 0x7fee34100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 -- 172.21.15.78:0/3438343183 >> 172.21.15.78:0/3438343183 conn(0x7fee340fc7a0 msgr2=0x7fee340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 -- 172.21.15.78:0/3438343183 shutdown_connections 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.352+0000 7fee3bbec640 1 -- 172.21.15.78:0/3438343183 wait complete. 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3bbec640 1 Processor -- start 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3bbec640 1 -- start start 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3bbec640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34100b30 0x7fee341a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3bbec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee34101450 0x7fee341a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3bbec640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee34105f40 0x7fee3419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3bbec640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee34118580 con 0x7fee34100b30 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3bbec640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee34118400 con 0x7fee34101450 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3bbec640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee34118700 con 0x7fee34105f40 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee39988640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34100b30 0x7fee341a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee39988640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34100b30 0x7fee341a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37430/0 (socket says 172.21.15.78:37430) 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee39988640 1 -- 172.21.15.78:0/207447890 learned_addr learned my addr 172.21.15.78:0/207447890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee3a189640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee34105f40 0x7fee3419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee39187640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee34101450 0x7fee341a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee39988640 1 -- 172.21.15.78:0/207447890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee34105f40 msgr2=0x7fee3419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.353+0000 7fee39988640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee34105f40 0x7fee3419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee39988640 1 -- 172.21.15.78:0/207447890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee34101450 msgr2=0x7fee341a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee39988640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee34101450 0x7fee341a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee39988640 1 -- 172.21.15.78:0/207447890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3419eff0 con 0x7fee34100b30 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee39988640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34100b30 0x7fee341a46d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fee3000ebc0 tx=0x7fee3000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee2affd640 1 -- 172.21.15.78:0/207447890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30010630 con 0x7fee34100b30 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee3bbec640 1 -- 172.21.15.78:0/207447890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3419f2e0 con 0x7fee34100b30 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee2affd640 1 -- 172.21.15.78:0/207447890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee30010cb0 con 0x7fee34100b30 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee2affd640 1 -- 172.21.15.78:0/207447890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30018bd0 con 0x7fee34100b30 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.354+0000 7fee3bbec640 1 -- 172.21.15.78:0/207447890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee341ab530 con 0x7fee34100b30 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.356+0000 7fee2affd640 1 -- 172.21.15.78:0/207447890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fee300107d0 con 0x7fee34100b30 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.356+0000 7fee3dfc5740 1 Processor -- start 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.356+0000 7fee3dfc5740 1 -- start start 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac255e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac256b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561ac24c42f0 con 0x561ac2569370 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561ac246bcf0 con 0x561ac255fb40 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561ac24c02b0 con 0x561ac255e5d0 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac256b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac256b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37444/0 (socket says 172.21.15.78:37444) 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 -- 172.21.15.78:0/1605234639 learned_addr learned my addr 172.21.15.78:0/1605234639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee3a189640 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac255e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39988640 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee2affd640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee200a3240 0x7fee200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 -- 172.21.15.78:0/1605234639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 msgr2=0x561ac255ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 -- 172.21.15.78:0/1605234639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 msgr2=0x561ac255e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:37.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac255e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 -- 172.21.15.78:0/1605234639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac24503a0 con 0x561ac2569370 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39988640 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee3a189640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee200a3240 0x7fee200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee2affd640 1 -- 172.21.15.78:0/207447890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fee300d0350 con 0x7fee34100b30 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.357+0000 7fee39187640 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac256b760 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fee24009b60 tx=0x7fee240117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffacebf164e3180 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee1ffff640 1 -- 172.21.15.78:0/1605234639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24009e50 con 0x561ac2569370 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee1ffff640 1 -- 172.21.15.78:0/1605234639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee24005500 con 0x561ac2569370 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee1ffff640 1 -- 172.21.15.78:0/1605234639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24012690 con 0x561ac2569370 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee3dfc5740 1 -- 172.21.15.78:0/1605234639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 msgr2=0x561ac256b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee3a189640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee200a3240 0x7fee200a5700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fee2c005c40 tx=0x7fee2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac256b760 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fee24009b60 tx=0x7fee240117b0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee3dfc5740 1 -- 172.21.15.78:0/1605234639 shutdown_connections 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac256b760 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/1605234639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac255e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.358+0000 7fee3dfc5740 1 -- 172.21.15.78:0/1605234639 >> 172.21.15.78:0/1605234639 conn(0x561ac2527310 msgr2=0x561ac255ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 -- 172.21.15.78:0/1605234639 shutdown_connections 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 -- 172.21.15.78:0/1605234639 wait complete. 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 -- start start 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 Processor -- start 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255e5d0 0x561ac2568cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255fb40 0x561ac2565f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac25664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561ac256c070 con 0x561ac2569370 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561ac247f060 con 0x561ac255e5d0 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561ac256c1f0 con 0x561ac255fb40 2024-09-16T00:24:37.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.359+0000 7fee3dfc5740 10 client.0 _ll_register_callbacks cb 0x561ac21d28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:37.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac25664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac25664e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37446/0 (socket says 172.21.15.78:37446) 2024-09-16T00:24:37.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 -- 172.21.15.78:0/619687137 learned_addr learned my addr 172.21.15.78:0/619687137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:37.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee39187640 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255fb40 0x561ac2565f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee39988640 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255e5d0 0x561ac2568cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 -- 172.21.15.78:0/619687137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255fb40 msgr2=0x561ac2565f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:37.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255fb40 0x561ac2565f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 -- 172.21.15.78:0/619687137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255e5d0 msgr2=0x561ac2568cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255e5d0 0x561ac2568cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 -- 172.21.15.78:0/619687137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac22353c0 con 0x561ac2569370 2024-09-16T00:24:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee39988640 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255e5d0 0x561ac2568cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3a189640 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac25664e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fee2c033000 tx=0x7fee2c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36d95b8514f314 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee1d7fa640 1 -- 172.21.15.78:0/619687137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c04d020 con 0x561ac2569370 2024-09-16T00:24:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee1d7fa640 1 -- 172.21.15.78:0/619687137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fee2c0048c0 con 0x561ac2569370 2024-09-16T00:24:37.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3dfc5740 1 -- 172.21.15.78:0/619687137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee24005500 con 0x561ac2569370 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee1d7fa640 1 -- 172.21.15.78:0/619687137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fee2c058050 con 0x561ac2569370 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3dfc5740 10 client.17355 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.360+0000 7fee3dfc5740 1 -- 172.21.15.78:0/619687137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561ac25779f0 con 0x561ac2569370 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee1d7fa640 1 -- 172.21.15.78:0/619687137 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee24005500 con 0x561ac2569370 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee1d7fa640 1 -- 172.21.15.78:0/619687137 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fee2c049450 con 0x561ac2569370 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee3dfc5740 10 client.17355 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee3dfc5740 10 client.17355 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee3dfc5740 20 client.17355 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561ac2578eb0 0x561ac257b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee3dfc5740 4 client.17355 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee3dfc5740 1 -- 172.21.15.78:0/619687137 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ac24d1860 con 0x561ac2578eb0 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.361+0000 7fee39988640 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561ac2578eb0 0x561ac257b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.362+0000 7fee1d7fa640 10 client.17355 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:24:37.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.362+0000 7fee39988640 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561ac2578eb0 0x561ac257b320 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6fd37537885881f server_cookie=57841e3d67ab96a5 in_seq=0 out_seq=0 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee1d7fa640 1 -- 172.21.15.78:0/619687137 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ac24d1860 con 0x561ac2578eb0 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee1d7fa640 10 client.17355 handle_command_reply: tid=0 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 client.17355 shutdown 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 -- 172.21.15.78:0/619687137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 msgr2=0x561ac25664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac25664e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fee2c033000 tx=0x7fee2c0058b0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 -- 172.21.15.78:0/619687137 shutdown_connections 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac25664e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561ac2578eb0 0x561ac257b320 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255fb40 0x561ac2565f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/619687137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255e5d0 0x561ac2568cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee1d7fa640 0 client.17355 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:24:37.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 -- 172.21.15.78:0/619687137 >> 172.21.15.78:0/619687137 conn(0x561ac2527310 msgr2=0x561ac2560960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:37.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 -- 172.21.15.78:0/619687137 shutdown_connections 2024-09-16T00:24:37.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 1 -- 172.21.15.78:0/619687137 wait complete. 2024-09-16T00:24:37.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.363+0000 7fee3dfc5740 20 client.17355 trim_cache size 0 max 16384 2024-09-16T00:24:37.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3dfc5740 1 Processor -- start 2024-09-16T00:24:37.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3dfc5740 1 -- start start 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac2575310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac2580c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee341b5290 con 0x561ac2569370 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee341b6dd0 con 0x561ac255fb40 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee341b5960 con 0x561ac255e5d0 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3a189640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac2580c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3a189640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac2580c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37450/0 (socket says 172.21.15.78:37450) 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.376+0000 7fee3a189640 1 -- 172.21.15.78:0/2672417344 learned_addr learned my addr 172.21.15.78:0/2672417344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee39187640 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac2575310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee39988640 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee3a189640 1 -- 172.21.15.78:0/2672417344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 msgr2=0x561ac255ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee3a189640 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee3a189640 1 -- 172.21.15.78:0/2672417344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 msgr2=0x561ac2575310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee3a189640 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac2575310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee3a189640 1 -- 172.21.15.78:0/2672417344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac255a4f0 con 0x561ac2569370 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee39187640 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac2575310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee3a189640 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac2580c40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fee2c005940 tx=0x7fee2c02a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4b0637c5e1f91b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee1f7fe640 1 -- 172.21.15.78:0/2672417344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c04d070 con 0x561ac2569370 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee1f7fe640 1 -- 172.21.15.78:0/2672417344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2c060070 con 0x561ac2569370 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.377+0000 7fee1f7fe640 1 -- 172.21.15.78:0/2672417344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c035040 con 0x561ac2569370 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 -- 172.21.15.78:0/2672417344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 msgr2=0x561ac2580c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac2580c40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fee2c005940 tx=0x7fee2c02a470 comp rx=0 tx=0).stop 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 -- 172.21.15.78:0/2672417344 shutdown_connections 2024-09-16T00:24:37.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac2569370 0x561ac2580c40 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac255fb40 0x561ac2575310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/2672417344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac255ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 -- 172.21.15.78:0/2672417344 >> 172.21.15.78:0/2672417344 conn(0x561ac2527310 msgr2=0x561ac255a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 -- 172.21.15.78:0/2672417344 shutdown_connections 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 -- 172.21.15.78:0/2672417344 wait complete. 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 -- start start 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.378+0000 7fee3dfc5740 1 Processor -- start 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac2571990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac255fb40 0x561ac256ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3dfc5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac2569370 0x561ac256d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee341b6700 con 0x561ac255fb40 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee341b6030 con 0x561ac2569370 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3dfc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee341b3270 con 0x561ac255e5d0 2024-09-16T00:24:37.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3dfc5740 10 client.0 _ll_register_callbacks cb 0x7fee24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac255fb40 0x561ac256ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac255fb40 0x561ac256ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37464/0 (socket says 172.21.15.78:37464) 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 -- 172.21.15.78:0/3643553319 learned_addr learned my addr 172.21.15.78:0/3643553319 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39988640 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac2571990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3a189640 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac2569370 0x561ac256d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 -- 172.21.15.78:0/3643553319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 msgr2=0x561ac2571990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac2571990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 -- 172.21.15.78:0/3643553319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac2569370 msgr2=0x561ac256d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac2569370 0x561ac256d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 -- 172.21.15.78:0/3643553319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac256d7c0 con 0x561ac255fb40 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee39187640 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac255fb40 0x561ac256ccc0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fee2401a000 tx=0x7fee24004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf79cf412e473f08 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:37.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee1d7fa640 1 -- 172.21.15.78:0/3643553319 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2401f040 con 0x561ac255fb40 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee1d7fa640 1 -- 172.21.15.78:0/3643553319 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fee24003ac0 con 0x561ac255fb40 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.379+0000 7fee3dfc5740 1 -- 172.21.15.78:0/3643553319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee2c060070 con 0x561ac255fb40 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee1d7fa640 1 -- 172.21.15.78:0/3643553319 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fee24033050 con 0x561ac255fb40 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee3dfc5740 10 client.17361 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee3dfc5740 1 -- 172.21.15.78:0/3643553319 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561ac25e3a10 con 0x561ac255fb40 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee1d7fa640 1 -- 172.21.15.78:0/3643553319 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee2c060070 con 0x561ac255fb40 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee1d7fa640 1 -- 172.21.15.78:0/3643553319 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fee24004030 con 0x561ac255fb40 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee3dfc5740 10 client.17361 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee3dfc5740 10 client.17361 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee3dfc5740 20 client.17361 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561ac25e3f80 0x561ac25e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee3dfc5740 4 client.17361 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee3dfc5740 1 -- 172.21.15.78:0/3643553319 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ac24cbd70 con 0x561ac25e3f80 2024-09-16T00:24:37.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.380+0000 7fee39988640 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561ac25e3f80 0x561ac25e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee1d7fa640 10 client.17361 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee39988640 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561ac25e3f80 0x561ac25e63a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d51fa5a4ccecbbcc server_cookie=a4d5798fe2ec483f in_seq=0 out_seq=0 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee1d7fa640 1 -- 172.21.15.78:0/3643553319 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ac24cbd70 con 0x561ac25e3f80 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee1d7fa640 10 client.17361 handle_command_reply: tid=0 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 client.17361 shutdown 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 -- 172.21.15.78:0/3643553319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac255fb40 msgr2=0x561ac256ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac255fb40 0x561ac256ccc0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fee2401a000 tx=0x7fee24004b20 comp rx=0 tx=0).stop 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 -- 172.21.15.78:0/3643553319 shutdown_connections 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561ac2569370 0x561ac256d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561ac255fb40 0x561ac256ccc0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x561ac25e3f80 0x561ac25e63a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 --2- 172.21.15.78:0/3643553319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ac255e5d0 0x561ac2571990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee1d7fa640 0 client.17361 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:24:37.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 -- 172.21.15.78:0/3643553319 >> 172.21.15.78:0/3643553319 conn(0x561ac2527310 msgr2=0x561ac2559d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:37.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 -- 172.21.15.78:0/3643553319 shutdown_connections 2024-09-16T00:24:37.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 1 -- 172.21.15.78:0/3643553319 wait complete. 2024-09-16T00:24:37.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.381+0000 7fee3dfc5740 20 client.17361 trim_cache size 0 max 16384 2024-09-16T00:24:37.392 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 -- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee200a3240 msgr2=0x7fee200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee200a3240 0x7fee200a5700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fee2c005c40 tx=0x7fee2c009260 comp rx=0 tx=0).stop 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 -- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34100b30 msgr2=0x7fee341a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34100b30 0x7fee341a46d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fee3000ebc0 tx=0x7fee3000c6a0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 -- 172.21.15.78:0/207447890 shutdown_connections 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee200a3240 0x7fee200a5700 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee34105f40 0x7fee3419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee34101450 0x7fee341a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 --2- 172.21.15.78:0/207447890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee34100b30 0x7fee341a46d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 -- 172.21.15.78:0/207447890 >> 172.21.15.78:0/207447890 conn(0x7fee340fc7a0 msgr2=0x7fee341038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:37.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 -- 172.21.15.78:0/207447890 shutdown_connections 2024-09-16T00:24:37.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:37.382+0000 7fee3bbec640 1 -- 172.21.15.78:0/207447890 wait complete. 2024-09-16T00:24:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:37 smithi078 ceph-mon[31157]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-09-16T00:24:38.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:24:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:37 smithi193 ceph-mon[35484]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-09-16T00:24:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:37 smithi148 ceph-mon[35599]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-09-16T00:24:38.352 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- 172.21.15.78:0/1436275339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 msgr2=0x7f3c601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 --2- 172.21.15.78:0/1436275339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 0x7f3c601029b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009900 tx=0x7f3c5002a480 comp rx=0 tx=0).stop 2024-09-16T00:24:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- 172.21.15.78:0/1436275339 shutdown_connections 2024-09-16T00:24:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 --2- 172.21.15.78:0/1436275339 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c60102ef0 0x7f3c6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 --2- 172.21.15.78:0/1436275339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 0x7f3c601029b0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 --2- 172.21.15.78:0/1436275339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c60100e20 0x7f3c60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- 172.21.15.78:0/1436275339 >> 172.21.15.78:0/1436275339 conn(0x7f3c600fc7d0 msgr2=0x7f3c600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- 172.21.15.78:0/1436275339 shutdown_connections 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- 172.21.15.78:0/1436275339 wait complete. 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 Processor -- start 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- start start 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c60100e20 0x7f3c600fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 0x7f3c600fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c60102ef0 0x7f3c600fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60110fa0 con 0x7f3c60102570 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60110e20 con 0x7f3c60102ef0 2024-09-16T00:24:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.883+0000 7f3c6801e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60111120 con 0x7f3c60100e20 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 0x7f3c600fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 0x7f3c600fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37478/0 (socket says 172.21.15.78:37478) 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 -- 172.21.15.78:0/2188413853 learned_addr learned my addr 172.21.15.78:0/2188413853 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c65dba640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c60100e20 0x7f3c600fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c665bb640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c60102ef0 0x7f3c600fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 -- 172.21.15.78:0/2188413853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c60100e20 msgr2=0x7f3c600fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c60100e20 0x7f3c600fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 -- 172.21.15.78:0/2188413853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c60102ef0 msgr2=0x7f3c600fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c60102ef0 0x7f3c600fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 -- 172.21.15.78:0/2188413853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c601a6a30 con 0x7f3c60102570 2024-09-16T00:24:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c655b9640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 0x7f3c600fd1c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3c50006010 tx=0x7f3c50003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c56ffd640 1 -- 172.21.15.78:0/2188413853 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5003f070 con 0x7f3c60102570 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c6801e640 1 -- 172.21.15.78:0/2188413853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c601a6cc0 con 0x7f3c60102570 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c56ffd640 1 -- 172.21.15.78:0/2188413853 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c50031e30 con 0x7f3c60102570 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c56ffd640 1 -- 172.21.15.78:0/2188413853 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50035030 con 0x7f3c60102570 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.884+0000 7f3c6801e640 1 -- 172.21.15.78:0/2188413853 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c601a71a0 con 0x7f3c60102570 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c56ffd640 1 -- 172.21.15.78:0/2188413853 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3c50042070 con 0x7f3c60102570 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c6a3f7740 1 Processor -- start 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c6a3f7740 1 -- start start 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7cb40 0x55e491b7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b7ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b88760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e491ae12f0 con 0x55e491b7b5d0 2024-09-16T00:24:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e491a88cf0 con 0x55e491b86370 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.887+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e491add2b0 con 0x55e491b7cb40 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b7ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b7ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37488/0 (socket says 172.21.15.78:37488) 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 -- 172.21.15.78:0/3468158195 learned_addr learned my addr 172.21.15.78:0/3468158195 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c665bb640 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7cb40 0x55e491b7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c655b9640 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b88760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 -- 172.21.15.78:0/3468158195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7cb40 msgr2=0x55e491b7b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c56ffd640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c4c0a3240 0x7f3c4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7cb40 0x55e491b7b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 -- 172.21.15.78:0/3468158195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 msgr2=0x55e491b88760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b88760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 -- 172.21.15.78:0/3468158195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e491a6d3a0 con 0x55e491b7b5d0 2024-09-16T00:24:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c665bb640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c4c0a3240 0x7f3c4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c56ffd640 1 -- 172.21.15.78:0/2188413853 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c500f0590 con 0x7f3c60102570 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c65dba640 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b7ba10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009a70 tx=0x7f3c5c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d015535b0feba05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c4bfff640 1 -- 172.21.15.78:0/3468158195 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c017070 con 0x55e491b7b5d0 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c4bfff640 1 -- 172.21.15.78:0/3468158195 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5c013470 con 0x55e491b7b5d0 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.888+0000 7f3c4bfff640 1 -- 172.21.15.78:0/3468158195 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c012630 con 0x55e491b7b5d0 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3468158195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 msgr2=0x55e491b7ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c665bb640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c4c0a3240 0x7f3c4c0a5700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3c58008ea0 tx=0x7f3c58004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b7ba10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009a70 tx=0x7f3c5c009f30 comp rx=0 tx=0).stop 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3468158195 shutdown_connections 2024-09-16T00:24:38.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b88760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b7ba10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3468158195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7cb40 0x55e491b7b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3468158195 >> 172.21.15.78:0/3468158195 conn(0x55e491b44310 msgr2=0x55e491b77c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3468158195 shutdown_connections 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.889+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3468158195 wait complete. 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 1 -- start start 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 1 Processor -- start 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7b5d0 0x55e491b85ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b82f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e491b89070 con 0x55e491b7cb40 2024-09-16T00:24:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e491a9c060 con 0x55e491b86370 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e491b891f0 con 0x55e491b7b5d0 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c6a3f7740 10 client.0 _ll_register_callbacks cb 0x55e4917ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c655b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b82f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c655b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b82f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37500/0 (socket says 172.21.15.78:37500) 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c655b9640 1 -- 172.21.15.78:0/620743066 learned_addr learned my addr 172.21.15.78:0/620743066 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c665bb640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c65dba640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7b5d0 0x55e491b85ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c655b9640 1 -- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7b5d0 msgr2=0x55e491b85ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c655b9640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7b5d0 0x55e491b85ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c655b9640 1 -- 172.21.15.78:0/620743066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 msgr2=0x55e491b834f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.890+0000 7f3c655b9640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b834f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c655b9640 1 -- 172.21.15.78:0/620743066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4918523c0 con 0x55e491b7cb40 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c65dba640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7b5d0 0x55e491b85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c665bb640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b834f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c655b9640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b82f90 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3c500bc780 tx=0x7f3c50009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3224e143ddefa064 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c497fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c497fa640 1 -- 172.21.15.78:0/620743066 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50049050 con 0x55e491b7cb40 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c497fa640 1 -- 172.21.15.78:0/620743066 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c500f0290 con 0x55e491b7cb40 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/620743066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c5c013470 con 0x55e491b7cb40 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c497fa640 1 -- 172.21.15.78:0/620743066 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3c500bc330 con 0x55e491b7cb40 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 10 client.17373 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/620743066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e491b94a80 con 0x55e491b7cb40 2024-09-16T00:24:38.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c497fa640 1 -- 172.21.15.78:0/620743066 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5c013470 con 0x55e491b7cb40 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c497fa640 1 -- 172.21.15.78:0/620743066 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3c500b8e60 con 0x55e491b7cb40 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 10 client.17373 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 10 client.17373 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 20 client.17373 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e491b95ee0 0x55e491b98320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 4 client.17373 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.891+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/620743066 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e491aee860 con 0x55e491b95ee0 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.892+0000 7f3c65dba640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e491b95ee0 0x55e491b98320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.892+0000 7f3c65dba640 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e491b95ee0 0x55e491b98320 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=78635a7ba4075f94 server_cookie=a6099b43e93d6529 in_seq=0 out_seq=0 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.892+0000 7f3c497fa640 10 client.17373 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c497fa640 1 -- 172.21.15.78:0/620743066 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e491aee860 con 0x55e491b95ee0 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c497fa640 10 client.17373 handle_command_reply: tid=0 2024-09-16T00:24:38.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 client.17373 shutdown 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/620743066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 msgr2=0x55e491b82f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b82f90 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3c500bc780 tx=0x7f3c50009620 comp rx=0 tx=0).stop 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/620743066 shutdown_connections 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e491b95ee0 0x55e491b98320 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b86370 0x55e491b834f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c497fa640 0 client.17373 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b82f90 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/620743066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b7b5d0 0x55e491b85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/620743066 >> 172.21.15.78:0/620743066 conn(0x55e491b44310 msgr2=0x55e491b7d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/620743066 shutdown_connections 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/620743066 wait complete. 2024-09-16T00:24:38.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.893+0000 7f3c6a3f7740 20 client.17373 trim_cache size 0 max 16384 2024-09-16T00:24:38.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.906+0000 7f3c6a3f7740 1 Processor -- start 2024-09-16T00:24:38.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.906+0000 7f3c6a3f7740 1 -- start start 2024-09-16T00:24:38.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.906+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7b5d0 0x55e491b82850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.906+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b9dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.906+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b9e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.906+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601b0df0 con 0x55e491b7cb40 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.906+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601b2930 con 0x55e491b7b5d0 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.906+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601b14c0 con 0x55e491b86370 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b9dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b9dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37506/0 (socket says 172.21.15.78:37506) 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 -- 172.21.15.78:0/1732564614 learned_addr learned my addr 172.21.15.78:0/1732564614 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c665bb640 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b9e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c65dba640 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7b5d0 0x55e491b82850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 -- 172.21.15.78:0/1732564614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 msgr2=0x55e491b9e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b9e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 -- 172.21.15.78:0/1732564614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7b5d0 msgr2=0x55e491b82850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7b5d0 0x55e491b82850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 -- 172.21.15.78:0/1732564614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e491b81f40 con 0x55e491b7cb40 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c665bb640 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b9e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c65dba640 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7b5d0 0x55e491b82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c655b9640 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b9dc30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3c500c0d40 tx=0x7f3c500b8450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c82a76af9717b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c4b7fe640 1 -- 172.21.15.78:0/1732564614 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50049020 con 0x55e491b7cb40 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c4b7fe640 1 -- 172.21.15.78:0/1732564614 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c500db1e0 con 0x55e491b7cb40 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.907+0000 7f3c4b7fe640 1 -- 172.21.15.78:0/1732564614 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c500bc150 con 0x55e491b7cb40 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/1732564614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 msgr2=0x55e491b9dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b9dc30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3c500c0d40 tx=0x7f3c500b8450 comp rx=0 tx=0).stop 2024-09-16T00:24:38.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/1732564614 shutdown_connections 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b9e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7cb40 0x55e491b9dc30 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/1732564614 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7b5d0 0x55e491b82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/1732564614 >> 172.21.15.78:0/1732564614 conn(0x55e491b44310 msgr2=0x55e491b85010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/1732564614 shutdown_connections 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/1732564614 wait complete. 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 -- start start 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.908+0000 7f3c6a3f7740 1 Processor -- start 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b8c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7cb40 0x55e491b8ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c6a3f7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b8cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601b2260 con 0x55e491b7b5d0 2024-09-16T00:24:38.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601b1b90 con 0x55e491b7cb40 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c6a3f7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601aedd0 con 0x55e491b86370 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c6a3f7740 10 client.0 _ll_register_callbacks cb 0x7f3c50005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b8c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b8c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37518/0 (socket says 172.21.15.78:37518) 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 -- 172.21.15.78:0/3403193527 learned_addr learned my addr 172.21.15.78:0/3403193527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c655b9640 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7cb40 0x55e491b8ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c665bb640 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b8cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 -- 172.21.15.78:0/3403193527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 msgr2=0x55e491b8cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b8cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 -- 172.21.15.78:0/3403193527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7cb40 msgr2=0x55e491b8ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:38.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7cb40 0x55e491b8ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 -- 172.21.15.78:0/3403193527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e491b86050 con 0x55e491b7b5d0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c65dba640 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b8c4c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009a70 tx=0x7f3c5c013de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a4e7ef1a5e252a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c497fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c497fa640 1 -- 172.21.15.78:0/3403193527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c017040 con 0x55e491b7b5d0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c497fa640 1 -- 172.21.15.78:0/3403193527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c5c00ebc0 con 0x55e491b7b5d0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.909+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3403193527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c500db1e0 con 0x55e491b7b5d0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c497fa640 1 -- 172.21.15.78:0/3403193527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3c5c00ede0 con 0x55e491b7b5d0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c6a3f7740 10 client.17379 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3403193527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e491c00f50 con 0x55e491b7b5d0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c497fa640 1 -- 172.21.15.78:0/3403193527 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c500db1e0 con 0x55e491b7b5d0 2024-09-16T00:24:38.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c497fa640 1 -- 172.21.15.78:0/3403193527 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3c5c01c890 con 0x55e491b7b5d0 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c6a3f7740 10 client.17379 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c6a3f7740 10 client.17379 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c6a3f7740 20 client.17379 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e491c01390 0x55e491c03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c6a3f7740 4 client.17379 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3403193527 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e491ae8d70 con 0x55e491c01390 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.910+0000 7f3c655b9640 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e491c01390 0x55e491c03780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c497fa640 10 client.17379 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c655b9640 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e491c01390 0x55e491c03780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f1f44ca673db25f3 server_cookie=b066766bec3a415a in_seq=0 out_seq=0 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c497fa640 1 -- 172.21.15.78:0/3403193527 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e491ae8d70 con 0x55e491c01390 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c497fa640 10 client.17379 handle_command_reply: tid=0 2024-09-16T00:24:38.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 client.17379 shutdown 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3403193527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 msgr2=0x55e491b8c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b8c4c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009a70 tx=0x7f3c5c013de0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3403193527 shutdown_connections 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e491b86370 0x55e491b8cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55e491c01390 0x55e491c03780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e491b7cb40 0x55e491b8ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 --2- 172.21.15.78:0/3403193527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e491b7b5d0 0x55e491b8c4c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c497fa640 0 client.17379 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3403193527 >> 172.21.15.78:0/3403193527 conn(0x55e491b44310 msgr2=0x55e491b85010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3403193527 shutdown_connections 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 1 -- 172.21.15.78:0/3403193527 wait complete. 2024-09-16T00:24:38.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.911+0000 7f3c6a3f7740 20 client.17379 trim_cache size 0 max 16384 2024-09-16T00:24:38.930 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:24:38.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 -- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c4c0a3240 msgr2=0x7f3c4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c4c0a3240 0x7f3c4c0a5700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3c58008ea0 tx=0x7f3c58004a40 comp rx=0 tx=0).stop 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 -- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 msgr2=0x7f3c600fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 0x7f3c600fd1c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3c50006010 tx=0x7f3c50003c80 comp rx=0 tx=0).stop 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 -- 172.21.15.78:0/2188413853 shutdown_connections 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c4c0a3240 0x7f3c4c0a5700 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c60102ef0 0x7f3c600fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c60102570 0x7f3c600fd1c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 --2- 172.21.15.78:0/2188413853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c60100e20 0x7f3c600fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 -- 172.21.15.78:0/2188413853 >> 172.21.15.78:0/2188413853 conn(0x7f3c600fc7d0 msgr2=0x7f3c6006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:38.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 -- 172.21.15.78:0/2188413853 shutdown_connections 2024-09-16T00:24:38.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:38.912+0000 7f3c6801e640 1 -- 172.21.15.78:0/2188413853 wait complete. 2024-09-16T00:24:39.669 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:24:39.919 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:40.040 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:39 smithi078 ceph-mon[31157]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-09-16T00:24:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:39 smithi193 ceph-mon[35484]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-09-16T00:24:40.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:39 smithi148 ceph-mon[35599]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 3.9 MiB/s wr, 273 op/s 2024-09-16T00:24:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 -- 172.21.15.78:0/665084813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c106ed0 msgr2=0x7fae4c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 --2- 172.21.15.78:0/665084813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c106ed0 0x7fae4c10eaa0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fae44009ca0 tx=0x7fae4402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 -- 172.21.15.78:0/665084813 shutdown_connections 2024-09-16T00:24:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 --2- 172.21.15.78:0/665084813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c106ed0 0x7fae4c10eaa0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 --2- 172.21.15.78:0/665084813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae4c100eb0 0x7fae4c1067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 --2- 172.21.15.78:0/665084813 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae4c100500 0x7fae4c1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 -- 172.21.15.78:0/665084813 >> 172.21.15.78:0/665084813 conn(0x7fae4c0fc5c0 msgr2=0x7fae4c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 -- 172.21.15.78:0/665084813 shutdown_connections 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.406+0000 7fae54492640 1 -- 172.21.15.78:0/665084813 wait complete. 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae54492640 1 Processor -- start 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae54492640 1 -- start start 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae54492640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c100500 0x7fae4c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae54492640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae4c100eb0 0x7fae4c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae54492640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae4c106ed0 0x7fae4c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae54492640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c111fa0 con 0x7fae4c100500 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae54492640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c111e20 con 0x7fae4c100eb0 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae54492640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c112120 con 0x7fae4c106ed0 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae5222e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c100500 0x7fae4c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae5222e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c100500 0x7fae4c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37532/0 (socket says 172.21.15.78:37532) 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae5222e640 1 -- 172.21.15.78:0/3941521370 learned_addr learned my addr 172.21.15.78:0/3941521370 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae51a2d640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae4c100eb0 0x7fae4c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.407+0000 7fae52a2f640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae4c106ed0 0x7fae4c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae5222e640 1 -- 172.21.15.78:0/3941521370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae4c106ed0 msgr2=0x7fae4c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae5222e640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae4c106ed0 0x7fae4c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae5222e640 1 -- 172.21.15.78:0/3941521370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae4c100eb0 msgr2=0x7fae4c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae5222e640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae4c100eb0 0x7fae4c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae5222e640 1 -- 172.21.15.78:0/3941521370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c19d440 con 0x7fae4c100500 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae51a2d640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae4c100eb0 0x7fae4c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae5222e640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c100500 0x7fae4c198490 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fae4800d950 tx=0x7fae4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae437fe640 1 -- 172.21.15.78:0/3941521370 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48017070 con 0x7fae4c100500 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae437fe640 1 -- 172.21.15.78:0/3941521370 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae48009c30 con 0x7fae4c100500 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae437fe640 1 -- 172.21.15.78:0/3941521370 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4800f6b0 con 0x7fae4c100500 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae54492640 1 -- 172.21.15.78:0/3941521370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c19d730 con 0x7fae4c100500 2024-09-16T00:24:40.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.408+0000 7fae54492640 1 -- 172.21.15.78:0/3941521370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c1a5010 con 0x7fae4c100500 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.410+0000 7fae437fe640 1 -- 172.21.15.78:0/3941521370 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fae480109d0 con 0x7fae4c100500 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5686b740 1 Processor -- start 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5686b740 1 -- start start 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae437fe640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fae380a32c0 0x7fae380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5686b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x558923569090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5686b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923569a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5686b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923576760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5686b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5589234cf2f0 con 0x558923574370 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5686b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558923476cf0 con 0x55892356ab40 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5686b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5589234cb2b0 con 0x5589235695d0 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae437fe640 1 -- 172.21.15.78:0/3941521370 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fae480cf970 con 0x7fae4c100500 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae52a2f640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fae380a32c0 0x7fae380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae52a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923576760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae52a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923576760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37544/0 (socket says 172.21.15.78:37544) 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae51a2d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923569a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae52a2f640 1 -- 172.21.15.78:0/1381962139 learned_addr learned my addr 172.21.15.78:0/1381962139 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.411+0000 7fae5222e640 1 --2- 172.21.15.78:0/1381962139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x558923569090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae52a2f640 1 -- 172.21.15.78:0/1381962139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 msgr2=0x558923569a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae52a2f640 1 --2- 172.21.15.78:0/1381962139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923569a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae52a2f640 1 -- 172.21.15.78:0/1381962139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 msgr2=0x558923569090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae52a2f640 1 --2- 172.21.15.78:0/1381962139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x558923569090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae52a2f640 1 -- 172.21.15.78:0/1381962139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55892345b3a0 con 0x558923574370 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae52a2f640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fae380a32c0 0x7fae380a5780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae4402f540 tx=0x7fae4402f410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae52a2f640 1 --2- 172.21.15.78:0/1381962139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923576760 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fae44009430 tx=0x7fae44047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a13f743014d813d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae40ff9640 1 -- 172.21.15.78:0/1381962139 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4402f610 con 0x558923574370 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae40ff9640 1 -- 172.21.15.78:0/1381962139 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae44036040 con 0x558923574370 2024-09-16T00:24:40.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.412+0000 7fae40ff9640 1 -- 172.21.15.78:0/1381962139 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44030b60 con 0x558923574370 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 -- 172.21.15.78:0/1381962139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 msgr2=0x558923576760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 --2- 172.21.15.78:0/1381962139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923576760 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fae44009430 tx=0x7fae44047000 comp rx=0 tx=0).stop 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 -- 172.21.15.78:0/1381962139 shutdown_connections 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 --2- 172.21.15.78:0/1381962139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923576760 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 --2- 172.21.15.78:0/1381962139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923569a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 --2- 172.21.15.78:0/1381962139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x558923569090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 -- 172.21.15.78:0/1381962139 >> 172.21.15.78:0/1381962139 conn(0x558923532310 msgr2=0x558923565c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 -- 172.21.15.78:0/1381962139 shutdown_connections 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 -- 172.21.15.78:0/1381962139 wait complete. 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 -- start start 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 Processor -- start 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589235695d0 0x558923573d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x558923571010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.413+0000 7fae5686b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558923574370 0x558923571600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5686b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558923576e90 con 0x5589235695d0 2024-09-16T00:24:40.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5686b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55892348a060 con 0x55892356ab40 2024-09-16T00:24:40.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5686b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558923577010 con 0x558923574370 2024-09-16T00:24:40.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5686b740 10 client.0 _ll_register_callbacks cb 0x5589231dd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589235695d0 0x558923573d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589235695d0 0x558923573d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37552/0 (socket says 172.21.15.78:37552) 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 -- 172.21.15.78:0/3317490669 learned_addr learned my addr 172.21.15.78:0/3317490669 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae51a2d640 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x558923571010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae52a2f640 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558923574370 0x558923571600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 -- 172.21.15.78:0/3317490669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558923574370 msgr2=0x558923571600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558923574370 0x558923571600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 -- 172.21.15.78:0/3317490669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 msgr2=0x558923571010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x558923571010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 -- 172.21.15.78:0/3317490669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589232403c0 con 0x5589235695d0 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5222e640 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589235695d0 0x558923573d60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fae48010fd0 tx=0x7fae4809b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a38e596922bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:40.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae35ffb640 1 -- 172.21.15.78:0/3317490669 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48020040 con 0x5589235695d0 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae35ffb640 1 -- 172.21.15.78:0/3317490669 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fae48098240 con 0x5589235695d0 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5686b740 1 -- 172.21.15.78:0/3317490669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae44036040 con 0x5589235695d0 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae35ffb640 1 -- 172.21.15.78:0/3317490669 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae480b9050 con 0x5589235695d0 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5686b740 10 client.17391 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae5686b740 1 -- 172.21.15.78:0/3317490669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558923582980 con 0x5589235695d0 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.414+0000 7fae35ffb640 1 -- 172.21.15.78:0/3317490669 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae44036040 con 0x5589235695d0 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae35ffb640 1 -- 172.21.15.78:0/3317490669 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fae4806e2c0 con 0x5589235695d0 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae5686b740 10 client.17391 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae5686b740 10 client.17391 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae5686b740 20 client.17391 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae5686b740 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x558923583de0 0x558923586210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae5686b740 4 client.17391 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae5686b740 1 -- 172.21.15.78:0/3317490669 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589234dc860 con 0x558923583de0 2024-09-16T00:24:40.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae51a2d640 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x558923583de0 0x558923586210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae35ffb640 10 client.17391 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.415+0000 7fae51a2d640 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x558923583de0 0x558923586210 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5eff03ce74ab2d3b server_cookie=1dc8bf5bd5f3c4cc in_seq=0 out_seq=0 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae35ffb640 1 -- 172.21.15.78:0/3317490669 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589234dc860 con 0x558923583de0 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae35ffb640 10 client.17391 handle_command_reply: tid=0 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 client.17391 shutdown 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 -- 172.21.15.78:0/3317490669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589235695d0 msgr2=0x558923573d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589235695d0 0x558923573d60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fae48010fd0 tx=0x7fae4809b820 comp rx=0 tx=0).stop 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 -- 172.21.15.78:0/3317490669 shutdown_connections 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558923574370 0x558923571600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x558923583de0 0x558923586210 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x558923571010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae35ffb640 0 client.17391 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:24:40.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 --2- 172.21.15.78:0/3317490669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5589235695d0 0x558923573d60 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 -- 172.21.15.78:0/3317490669 >> 172.21.15.78:0/3317490669 conn(0x558923532310 msgr2=0x55892356b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 -- 172.21.15.78:0/3317490669 shutdown_connections 2024-09-16T00:24:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 1 -- 172.21.15.78:0/3317490669 wait complete. 2024-09-16T00:24:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.416+0000 7fae5686b740 20 client.17391 trim_cache size 0 max 16384 2024-09-16T00:24:40.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae5686b740 1 Processor -- start 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae5686b740 1 -- start start 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae5686b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923580180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae5686b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x5589235708d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae5686b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x55892356b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae5686b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1aebd0 con 0x558923574370 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae5686b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1b0710 con 0x55892356ab40 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae5686b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1af2a0 con 0x5589235695d0 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae52a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x55892356b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae52a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x55892356b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37566/0 (socket says 172.21.15.78:37566) 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.429+0000 7fae52a2f640 1 -- 172.21.15.78:0/585636206 learned_addr learned my addr 172.21.15.78:0/585636206 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae51a2d640 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x5589235708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae52a2f640 1 -- 172.21.15.78:0/585636206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 msgr2=0x558923580180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae5222e640 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923580180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae52a2f640 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923580180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae52a2f640 1 -- 172.21.15.78:0/585636206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 msgr2=0x5589235708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae52a2f640 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x5589235708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae52a2f640 1 -- 172.21.15.78:0/585636206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55892356ffc0 con 0x558923574370 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae51a2d640 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x5589235708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae5222e640 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923580180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae52a2f640 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x55892356b820 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fae4402ade0 tx=0x7fae44009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28788f69e43d5f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae37fff640 1 -- 172.21.15.78:0/585636206 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44057030 con 0x558923574370 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae37fff640 1 -- 172.21.15.78:0/585636206 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae44036040 con 0x558923574370 2024-09-16T00:24:40.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.430+0000 7fae37fff640 1 -- 172.21.15.78:0/585636206 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae440309b0 con 0x558923574370 2024-09-16T00:24:40.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- 172.21.15.78:0/585636206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 msgr2=0x55892356b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x55892356b820 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fae4402ade0 tx=0x7fae44009b60 comp rx=0 tx=0).stop 2024-09-16T00:24:40.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- 172.21.15.78:0/585636206 shutdown_connections 2024-09-16T00:24:40.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x55892356b820 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x5589235708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 --2- 172.21.15.78:0/585636206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x558923580180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- 172.21.15.78:0/585636206 >> 172.21.15.78:0/585636206 conn(0x558923532310 msgr2=0x55892356b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- 172.21.15.78:0/585636206 shutdown_connections 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- 172.21.15.78:0/585636206 wait complete. 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- start start 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 Processor -- start 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x5589235783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x5589235788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923578e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1b0040 con 0x558923574370 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1af970 con 0x55892356ab40 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1acbb0 con 0x5589235695d0 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae5686b740 10 client.0 _ll_register_callbacks cb 0x7fae4809b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:40.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae52a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923578e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.431+0000 7fae52a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923578e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37576/0 (socket says 172.21.15.78:37576) 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae52a2f640 1 -- 172.21.15.78:0/742342673 learned_addr learned my addr 172.21.15.78:0/742342673 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae5222e640 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x5589235783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae52a2f640 1 -- 172.21.15.78:0/742342673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 msgr2=0x5589235783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae52a2f640 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x5589235783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae52a2f640 1 -- 172.21.15.78:0/742342673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 msgr2=0x5589235788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae52a2f640 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x5589235788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae52a2f640 1 -- 172.21.15.78:0/742342673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589235edda0 con 0x558923574370 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae5222e640 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x5589235783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae52a2f640 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923578e60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fae44031fd0 tx=0x7fae4402fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1afb3c3a5cabaa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae35ffb640 1 -- 172.21.15.78:0/742342673 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44060070 con 0x558923574370 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae35ffb640 1 -- 172.21.15.78:0/742342673 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fae4402fd50 con 0x558923574370 2024-09-16T00:24:40.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae5686b740 1 -- 172.21.15.78:0/742342673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae44036040 con 0x558923574370 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae35ffb640 1 -- 172.21.15.78:0/742342673 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae44057070 con 0x558923574370 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae5686b740 10 client.17397 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae5686b740 1 -- 172.21.15.78:0/742342673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5589235ef010 con 0x558923574370 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.432+0000 7fae35ffb640 1 -- 172.21.15.78:0/742342673 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae44036040 con 0x558923574370 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae35ffb640 1 -- 172.21.15.78:0/742342673 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fae4404a070 con 0x558923574370 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae5686b740 10 client.17397 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae5686b740 10 client.17397 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae5686b740 20 client.17397 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae5686b740 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5589235ef390 0x5589235f1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae5686b740 4 client.17397 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae5686b740 1 -- 172.21.15.78:0/742342673 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589234d6d70 con 0x5589235ef390 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae5222e640 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5589235ef390 0x5589235f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae5222e640 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5589235ef390 0x5589235f1780 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be10e39215589000 server_cookie=315d2826707a0490 in_seq=0 out_seq=0 2024-09-16T00:24:40.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae35ffb640 10 client.17397 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae35ffb640 1 -- 172.21.15.78:0/742342673 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589234d6d70 con 0x5589235ef390 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.433+0000 7fae35ffb640 10 client.17397 handle_command_reply: tid=0 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 client.17397 shutdown 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 -- 172.21.15.78:0/742342673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 msgr2=0x558923578e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923578e60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fae44031fd0 tx=0x7fae4402fbb0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 -- 172.21.15.78:0/742342673 shutdown_connections 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558923574370 0x558923578e60 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5589235ef390 0x5589235f1780 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55892356ab40 0x5589235788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 --2- 172.21.15.78:0/742342673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589235695d0 0x5589235783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 -- 172.21.15.78:0/742342673 >> 172.21.15.78:0/742342673 conn(0x558923532310 msgr2=0x558923572870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae35ffb640 0 client.17397 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:24:40.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 -- 172.21.15.78:0/742342673 shutdown_connections 2024-09-16T00:24:40.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 1 -- 172.21.15.78:0/742342673 wait complete. 2024-09-16T00:24:40.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae5686b740 20 client.17397 trim_cache size 0 max 16384 2024-09-16T00:24:40.441 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae54492640 1 -- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fae380a32c0 msgr2=0x7fae380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae54492640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fae380a32c0 0x7fae380a5780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae4402f540 tx=0x7fae4402f410 comp rx=0 tx=0).stop 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae54492640 1 -- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c100500 msgr2=0x7fae4c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.434+0000 7fae54492640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c100500 0x7fae4c198490 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fae4800d950 tx=0x7fae4800de10 comp rx=0 tx=0).stop 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.435+0000 7fae54492640 1 -- 172.21.15.78:0/3941521370 shutdown_connections 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.435+0000 7fae54492640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fae380a32c0 0x7fae380a5780 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.435+0000 7fae54492640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae4c106ed0 0x7fae4c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.435+0000 7fae54492640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fae4c100eb0 0x7fae4c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.435+0000 7fae54492640 1 --2- 172.21.15.78:0/3941521370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fae4c100500 0x7fae4c198490 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.435+0000 7fae54492640 1 -- 172.21.15.78:0/3941521370 >> 172.21.15.78:0/3941521370 conn(0x7fae4c0fc5c0 msgr2=0x7fae4c105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:40.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.435+0000 7fae54492640 1 -- 172.21.15.78:0/3941521370 shutdown_connections 2024-09-16T00:24:40.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:40.435+0000 7fae54492640 1 -- 172.21.15.78:0/3941521370 wait complete. 2024-09-16T00:24:40.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 640.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:24:41.228 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:24:41.475 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:41.928 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:41 smithi078 ceph-mon[31157]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.6 MiB/s wr, 149 op/s 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/3648984290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4101240 msgr2=0x7fa7d4101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/3648984290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4101240 0x7fa7d4101680 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009900 tx=0x7fa7c402a410 comp rx=0 tx=0).stop 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/3648984290 shutdown_connections 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/3648984290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7d4101bc0 0x7fa7d410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/3648984290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4101240 0x7fa7d4101680 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/3648984290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7d4105a90 0x7fa7d4105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/3648984290 >> 172.21.15.78:0/3648984290 conn(0x7fa7d4077210 msgr2=0x7fa7d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/3648984290 shutdown_connections 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.008+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/3648984290 wait complete. 2024-09-16T00:24:42.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7dc3e2640 1 Processor -- start 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7dc3e2640 1 -- start start 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7dc3e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7d4101240 0x7fa7d4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7dc3e2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7d4101bc0 0x7fa7d4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7dc3e2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4105a90 0x7fa7d419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7dc3e2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41120b0 con 0x7fa7d4105a90 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7dc3e2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d4111f30 con 0x7fa7d4101bc0 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7dc3e2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d4112230 con 0x7fa7d4101240 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4105a90 0x7fa7d419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4105a90 0x7fa7d419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37602/0 (socket says 172.21.15.78:37602) 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 -- 172.21.15.78:0/1853812256 learned_addr learned my addr 172.21.15.78:0/1853812256 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7d4101bc0 0x7fa7d4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 -- 172.21.15.78:0/1853812256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7d4101240 msgr2=0x7fa7d4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7d4101240 0x7fa7d4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 -- 172.21.15.78:0/1853812256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7d4101bc0 msgr2=0x7fa7d4198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7d4101bc0 0x7fa7d4198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 -- 172.21.15.78:0/1853812256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d419d620 con 0x7fa7d4105a90 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7d4101bc0 0x7fa7d4198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.009+0000 7fa7da97f640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4105a90 0x7fa7d419cf40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00eaa0 tx=0x7fa7cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.010+0000 7fa7cb7fe640 1 -- 172.21.15.78:0/1853812256 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00cdb0 con 0x7fa7d4105a90 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.010+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/1853812256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d419d910 con 0x7fa7d4105a90 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.010+0000 7fa7cb7fe640 1 -- 172.21.15.78:0/1853812256 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc005020 con 0x7fa7d4105a90 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.010+0000 7fa7cb7fe640 1 -- 172.21.15.78:0/1853812256 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0116b0 con 0x7fa7d4105a90 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.010+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/1853812256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d41a51f0 con 0x7fa7d4105a90 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.012+0000 7fa7de7bb740 1 Processor -- start 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7cb7fe640 1 -- 172.21.15.78:0/1853812256 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa7cc005560 con 0x7fa7d4105a90 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7de7bb740 1 -- start start 2024-09-16T00:24:42.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563402555b40 0x563402554090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 0x563402554a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56340255f370 0x563402561760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5634024ba2f0 con 0x56340255f370 2024-09-16T00:24:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563402461cf0 con 0x563402555b40 2024-09-16T00:24:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5634024b62b0 con 0x5634025545d0 2024-09-16T00:24:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7d997d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56340255f370 0x563402561760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7d997d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56340255f370 0x563402561760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37614/0 (socket says 172.21.15.78:37614) 2024-09-16T00:24:42.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7d997d640 1 -- 172.21.15.78:0/1017864582 learned_addr learned my addr 172.21.15.78:0/1017864582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563402555b40 0x563402554090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.013+0000 7fa7cb7fe640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa7c00a3240 0x7fa7c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7da17e640 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 0x563402554a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7d997d640 1 -- 172.21.15.78:0/1017864582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 msgr2=0x563402554a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 0x563402554a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7d997d640 1 -- 172.21.15.78:0/1017864582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563402555b40 msgr2=0x563402554090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563402555b40 0x563402554090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7d997d640 1 -- 172.21.15.78:0/1017864582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634024463a0 con 0x56340255f370 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7da97f640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa7c00a3240 0x7fa7c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7cb7fe640 1 -- 172.21.15.78:0/1853812256 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa7cc0cffa0 con 0x7fa7d4105a90 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56340255f370 0x563402561760 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4005770 tx=0x7fa7c4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897b1e0fbdfaf78b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7c8dea640 1 -- 172.21.15.78:0/1017864582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4043020 con 0x56340255f370 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7c8dea640 1 -- 172.21.15.78:0/1017864582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c40046c0 con 0x56340255f370 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.014+0000 7fa7c8dea640 1 -- 172.21.15.78:0/1017864582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4035040 con 0x56340255f370 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1017864582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56340255f370 msgr2=0x563402561760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7da97f640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa7c00a3240 0x7fa7c00a5700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0cdab0 tx=0x7fa7cc0bc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56340255f370 0x563402561760 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4005770 tx=0x7fa7c4005b50 comp rx=0 tx=0).stop 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1017864582 shutdown_connections 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56340255f370 0x563402561760 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 0x563402554a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1017864582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563402555b40 0x563402554090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1017864582 >> 172.21.15.78:0/1017864582 conn(0x56340251d310 msgr2=0x563402550c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1017864582 shutdown_connections 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1017864582 wait complete. 2024-09-16T00:24:42.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 -- start start 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.015+0000 7fa7de7bb740 1 Processor -- start 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 0x56340255edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x56340255c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 0x56340255c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563402562010 con 0x563402555b40 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563402475060 con 0x56340255f370 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563402562190 con 0x5634025545d0 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 10 client.0 _ll_register_callbacks cb 0x5634021c88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:42.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x56340255c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x56340255c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37618/0 (socket says 172.21.15.78:37618) 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 -- 172.21.15.78:0/3982612171 learned_addr learned my addr 172.21.15.78:0/3982612171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7d997d640 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 0x56340255edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7d997d640 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 0x56340255c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 -- 172.21.15.78:0/3982612171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 msgr2=0x56340255edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 0x56340255edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 -- 172.21.15.78:0/3982612171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 msgr2=0x56340255c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 0x56340255c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 -- 172.21.15.78:0/3982612171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340222b3c0 con 0x563402555b40 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7d997d640 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 0x56340255c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7da17e640 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x56340255c060 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000d210 tx=0x7fa7d000d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c478ec78abf95e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7bdffb640 1 -- 172.21.15.78:0/3982612171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00125f0 con 0x563402555b40 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7bdffb640 1 -- 172.21.15.78:0/3982612171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa7d0012790 con 0x563402555b40 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3982612171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c40046c0 con 0x563402555b40 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7bdffb640 1 -- 172.21.15.78:0/3982612171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa7d0021050 con 0x563402555b40 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 10 client.17409 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.016+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3982612171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56340256dbb0 con 0x563402555b40 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7bdffb640 1 -- 172.21.15.78:0/3982612171 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c40046c0 con 0x563402555b40 2024-09-16T00:24:42.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7bdffb640 1 -- 172.21.15.78:0/3982612171 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa7d0013bc0 con 0x563402555b40 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7de7bb740 10 client.17409 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7de7bb740 10 client.17409 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7de7bb740 20 client.17409 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56340256f0c0 0x5634025714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7de7bb740 4 client.17409 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3982612171 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634024c7860 con 0x56340256f0c0 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.017+0000 7fa7d997d640 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56340256f0c0 0x5634025714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.018+0000 7fa7bdffb640 10 client.17409 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.018+0000 7fa7d997d640 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56340256f0c0 0x5634025714e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9e6a8d6a5154cf8 server_cookie=e125f1337f68cc78 in_seq=0 out_seq=0 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.018+0000 7fa7bdffb640 1 -- 172.21.15.78:0/3982612171 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634024c7860 con 0x56340256f0c0 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.018+0000 7fa7bdffb640 10 client.17409 handle_command_reply: tid=0 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.018+0000 7fa7de7bb740 1 client.17409 shutdown 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3982612171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 msgr2=0x56340255c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x56340255c060 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000d210 tx=0x7fa7d000d6d0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3982612171 shutdown_connections 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x56340256f0c0 0x5634025714e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 0x56340255c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x56340255c060 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3982612171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634025545d0 0x56340255edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3982612171 >> 172.21.15.78:0/3982612171 conn(0x56340251d310 msgr2=0x563402556960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7bdffb640 0 client.17409 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3982612171 shutdown_connections 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3982612171 wait complete. 2024-09-16T00:24:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.019+0000 7fa7de7bb740 20 client.17409 trim_cache size 0 max 16384 2024-09-16T00:24:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7de7bb740 1 Processor -- start 2024-09-16T00:24:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7de7bb740 1 -- start start 2024-09-16T00:24:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634025545d0 0x56340255c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x563402576db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56340255f370 0x5634025772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41aeb80 con 0x563402555b40 2024-09-16T00:24:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41b06c0 con 0x5634025545d0 2024-09-16T00:24:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41af250 con 0x56340255f370 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7da17e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x563402576db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7da17e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x563402576db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37628/0 (socket says 172.21.15.78:37628) 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7da17e640 1 -- 172.21.15.78:0/3439137499 learned_addr learned my addr 172.21.15.78:0/3439137499 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7d997d640 1 --2- 172.21.15.78:0/3439137499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634025545d0 0x56340255c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7da17e640 1 -- 172.21.15.78:0/3439137499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56340255f370 msgr2=0x5634025772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.032+0000 7fa7da17e640 1 --2- 172.21.15.78:0/3439137499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56340255f370 0x5634025772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.033+0000 7fa7da17e640 1 -- 172.21.15.78:0/3439137499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634025545d0 msgr2=0x56340255c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.033+0000 7fa7da17e640 1 --2- 172.21.15.78:0/3439137499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634025545d0 0x56340255c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.033+0000 7fa7da17e640 1 -- 172.21.15.78:0/3439137499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563402551960 con 0x563402555b40 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.033+0000 7fa7da17e640 1 --2- 172.21.15.78:0/3439137499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x563402576db0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0006b10 tx=0x7fa7d0014ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaef5c9065927664 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.033+0000 7fa7bffff640 1 -- 172.21.15.78:0/3439137499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0029030 con 0x563402555b40 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.033+0000 7fa7bffff640 1 -- 172.21.15.78:0/3439137499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0018070 con 0x563402555b40 2024-09-16T00:24:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.033+0000 7fa7bffff640 1 -- 172.21.15.78:0/3439137499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0012780 con 0x563402555b40 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3439137499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 msgr2=0x563402576db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3439137499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x563402576db0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0006b10 tx=0x7fa7d0014ff0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3439137499 shutdown_connections 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3439137499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56340255f370 0x5634025772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3439137499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563402555b40 0x563402576db0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/3439137499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5634025545d0 0x56340255c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3439137499 >> 172.21.15.78:0/3439137499 conn(0x56340251d310 msgr2=0x56340255b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3439137499 shutdown_connections 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- 172.21.15.78:0/3439137499 wait complete. 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- start start 2024-09-16T00:24:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 Processor -- start 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634025545d0 0x563402562980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563402555b40 0x563402562ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 0x5634025d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41afff0 con 0x5634025545d0 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41af920 con 0x56340255f370 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d41acb60 con 0x563402555b40 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7de7bb740 10 client.0 _ll_register_callbacks cb 0x7fa7d000b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7d997d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634025545d0 0x563402562980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7d997d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634025545d0 0x563402562980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37642/0 (socket says 172.21.15.78:37642) 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.034+0000 7fa7d997d640 1 -- 172.21.15.78:0/1440609193 learned_addr learned my addr 172.21.15.78:0/1440609193 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7d997d640 1 -- 172.21.15.78:0/1440609193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563402555b40 msgr2=0x563402562ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7da17e640 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563402555b40 0x563402562ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563402555b40 0x563402562ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7d997d640 1 -- 172.21.15.78:0/1440609193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 msgr2=0x5634025d7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 0x5634025d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7d997d640 1 -- 172.21.15.78:0/1440609193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634025d8380 con 0x5634025545d0 2024-09-16T00:24:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7d997d640 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634025545d0 0x563402562980 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40057c0 tx=0x7fa7c4009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85089d55dd29068 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:42.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:42.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7bdffb640 1 -- 172.21.15.78:0/1440609193 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4035040 con 0x5634025545d0 2024-09-16T00:24:42.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1440609193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d0018070 con 0x5634025545d0 2024-09-16T00:24:42.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7bdffb640 1 -- 172.21.15.78:0/1440609193 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa7c40037f0 con 0x5634025545d0 2024-09-16T00:24:42.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7bdffb640 1 -- 172.21.15.78:0/1440609193 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa7c4043040 con 0x5634025545d0 2024-09-16T00:24:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7de7bb740 10 client.17415 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1440609193 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5634025d9640 con 0x5634025545d0 2024-09-16T00:24:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.035+0000 7fa7bdffb640 1 -- 172.21.15.78:0/1440609193 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0018070 con 0x5634025545d0 2024-09-16T00:24:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7bdffb640 1 -- 172.21.15.78:0/1440609193 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa7c4051070 con 0x5634025545d0 2024-09-16T00:24:42.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7de7bb740 10 client.17415 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7de7bb740 10 client.17415 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:24:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7de7bb740 20 client.17415 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5634025d98c0 0x5634025dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7de7bb740 4 client.17415 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:24:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1440609193 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5634024c1d70 con 0x5634025d98c0 2024-09-16T00:24:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7da17e640 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5634025d98c0 0x5634025dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7bdffb640 10 client.17415 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7da17e640 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5634025d98c0 0x5634025dbcb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17d8e80d63725684 server_cookie=911a4e7fe7c89162 in_seq=0 out_seq=0 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7bdffb640 1 -- 172.21.15.78:0/1440609193 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5634024c1d70 con 0x5634025d98c0 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7bdffb640 10 client.17415 handle_command_reply: tid=0 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.036+0000 7fa7de7bb740 1 client.17415 shutdown 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1440609193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634025545d0 msgr2=0x563402562980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634025545d0 0x563402562980 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40057c0 tx=0x7fa7c4009280 comp rx=0 tx=0).stop 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1440609193 shutdown_connections 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5634025d98c0 0x5634025dbcb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56340255f370 0x5634025d7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563402555b40 0x563402562ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7bdffb640 0 client.17415 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:24:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 --2- 172.21.15.78:0/1440609193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5634025545d0 0x563402562980 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1440609193 >> 172.21.15.78:0/1440609193 conn(0x56340251d310 msgr2=0x56340255a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1440609193 shutdown_connections 2024-09-16T00:24:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 1 -- 172.21.15.78:0/1440609193 wait complete. 2024-09-16T00:24:42.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7de7bb740 20 client.17415 trim_cache size 0 max 16384 2024-09-16T00:24:42.052 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:24:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa7c00a3240 msgr2=0x7fa7c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa7c00a3240 0x7fa7c00a5700 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0cdab0 tx=0x7fa7cc0bc040 comp rx=0 tx=0).stop 2024-09-16T00:24:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4105a90 msgr2=0x7fa7d419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4105a90 0x7fa7d419cf40 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00eaa0 tx=0x7fa7cc00ef60 comp rx=0 tx=0).stop 2024-09-16T00:24:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/1853812256 shutdown_connections 2024-09-16T00:24:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa7c00a3240 0x7fa7c00a5700 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7d4105a90 0x7fa7d419cf40 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa7d4101bc0 0x7fa7d4198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 --2- 172.21.15.78:0/1853812256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7d4101240 0x7fa7d4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/1853812256 >> 172.21.15.78:0/1853812256 conn(0x7fa7d4077210 msgr2=0x7fa7d4103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/1853812256 shutdown_connections 2024-09-16T00:24:42.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:42.037+0000 7fa7dc3e2640 1 -- 172.21.15.78:0/1853812256 wait complete. 2024-09-16T00:24:42.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:41 smithi193 ceph-mon[35484]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.6 MiB/s wr, 149 op/s 2024-09-16T00:24:42.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:41 smithi148 ceph-mon[35599]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.6 MiB/s wr, 149 op/s 2024-09-16T00:24:42.775 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:24:43.025 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:43.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 -- 172.21.15.78:0/2607897970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c106f10 msgr2=0x7f4f2c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2607897970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c106f10 0x7f4f2c10ea90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009ca0 tx=0x7f4f2402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 -- 172.21.15.78:0/2607897970 shutdown_connections 2024-09-16T00:24:43.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2607897970 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c106f10 0x7f4f2c10ea90 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2607897970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c102790 0x7f4f2c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2607897970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f2c101de0 0x7f4f2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 -- 172.21.15.78:0/2607897970 >> 172.21.15.78:0/2607897970 conn(0x7f4f2c0fd8c0 msgr2=0x7f4f2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 -- 172.21.15.78:0/2607897970 shutdown_connections 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.527+0000 7f4f31d77640 1 -- 172.21.15.78:0/2607897970 wait complete. 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f31d77640 1 Processor -- start 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f31d77640 1 -- start start 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f31d77640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c101de0 0x7f4f2c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f31d77640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c102790 0x7f4f2c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f31d77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f2c106f10 0x7f4f2c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f31d77640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c111f90 con 0x7f4f2c101de0 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f31d77640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c111e10 con 0x7f4f2c102790 2024-09-16T00:24:43.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f31d77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c112110 con 0x7f4f2c106f10 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c101de0 0x7f4f2c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.528+0000 7f4f2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c101de0 0x7f4f2c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37660/0 (socket says 172.21.15.78:37660) 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/2829459767 learned_addr learned my addr 172.21.15.78:0/2829459767 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f2c106f10 0x7f4f2c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2affd640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c102790 0x7f4f2c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/2829459767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f2c106f10 msgr2=0x7f4f2c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f2c106f10 0x7f4f2c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/2829459767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c102790 msgr2=0x7f4f2c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c102790 0x7f4f2c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/2829459767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2c1a6930 con 0x7f4f2c101de0 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c101de0 0x7f4f2c0744f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b4a0 tx=0x7f4f1c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f28ff9640 1 -- 172.21.15.78:0/2829459767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c014070 con 0x7f4f2c101de0 2024-09-16T00:24:43.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f28ff9640 1 -- 172.21.15.78:0/2829459767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c010af0 con 0x7f4f2c101de0 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f28ff9640 1 -- 172.21.15.78:0/2829459767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c017d10 con 0x7f4f2c101de0 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f31d77640 1 -- 172.21.15.78:0/2829459767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2c1a6c20 con 0x7f4f2c101de0 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.529+0000 7f4f31d77640 1 -- 172.21.15.78:0/2829459767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2c1a7110 con 0x7f4f2c101de0 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.531+0000 7f4f28ff9640 1 -- 172.21.15.78:0/2829459767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4f1c017070 con 0x7f4f2c101de0 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f34150740 1 Processor -- start 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f34150740 1 -- start start 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f28ff9640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4f140a3240 0x7f4f140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f34150740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e498090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f34150740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 0x55da0e498a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f34150740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e4a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f34150740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da0e3fe2f0 con 0x55da0e499b40 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f34150740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55da0e3a5cf0 con 0x55da0e4a3370 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f34150740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da0e3fa2b0 con 0x55da0e4985d0 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4f140a3240 0x7f4f140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f28ff9640 1 -- 172.21.15.78:0/2829459767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4f1c0cf0f0 con 0x7f4f2c101de0 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e498090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e498090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37670/0 (socket says 172.21.15.78:37670) 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/209437087 learned_addr learned my addr 172.21.15.78:0/209437087 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2affd640 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 0x55da0e498a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e4a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/209437087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 msgr2=0x55da0e498a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 0x55da0e498a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/209437087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 msgr2=0x55da0e4a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e4a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.532+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/209437087 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da0e38a3a0 con 0x55da0e499b40 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e498090 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0cc830 tx=0x7f4f1c009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177a3436520b54fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4f140a3240 0x7f4f140a5700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4f24006010 tx=0x7f4f24031d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f11ffb640 1 -- 172.21.15.78:0/209437087 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c09ba50 con 0x55da0e499b40 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f11ffb640 1 -- 172.21.15.78:0/209437087 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c0adc60 con 0x55da0e499b40 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f11ffb640 1 -- 172.21.15.78:0/209437087 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0cc940 con 0x55da0e499b40 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f34150740 1 -- 172.21.15.78:0/209437087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 msgr2=0x55da0e498090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f34150740 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e498090 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0cc830 tx=0x7f4f1c009390 comp rx=0 tx=0).stop 2024-09-16T00:24:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f34150740 1 -- 172.21.15.78:0/209437087 shutdown_connections 2024-09-16T00:24:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f34150740 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e4a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f34150740 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 0x55da0e498a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f34150740 1 --2- 172.21.15.78:0/209437087 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e498090 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.533+0000 7f4f34150740 1 -- 172.21.15.78:0/209437087 >> 172.21.15.78:0/209437087 conn(0x55da0e461310 msgr2=0x55da0e494c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 -- 172.21.15.78:0/209437087 shutdown_connections 2024-09-16T00:24:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 -- 172.21.15.78:0/209437087 wait complete. 2024-09-16T00:24:43.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 -- start start 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 Processor -- start 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 0x55da0e4a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e499b40 0x55da0e4a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4a3370 0x55da0e4a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da0e4a6010 con 0x55da0e4a3370 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55da0e3b9060 con 0x55da0e499b40 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da0e4a6190 con 0x55da0e4985d0 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f34150740 10 client.0 _ll_register_callbacks cb 0x55da0e10c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f2bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4a3370 0x55da0e4a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f2bfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4a3370 0x55da0e4a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37674/0 (socket says 172.21.15.78:37674) 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f2bfff640 1 -- 172.21.15.78:0/2294784432 learned_addr learned my addr 172.21.15.78:0/2294784432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.534+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 0x55da0e4a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f2affd640 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e499b40 0x55da0e4a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f2bfff640 1 -- 172.21.15.78:0/2294784432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 msgr2=0x55da0e4a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 0x55da0e4a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f2bfff640 1 -- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e499b40 msgr2=0x55da0e4a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e499b40 0x55da0e4a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f2bfff640 1 -- 172.21.15.78:0/2294784432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0e16f3c0 con 0x55da0e4a3370 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f2affd640 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e499b40 0x55da0e4a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4a3370 0x55da0e4a07e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4f24003d40 tx=0x7f4f2402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519d5dee1e8f4b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f0affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f0affd640 1 -- 172.21.15.78:0/2294784432 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2404e020 con 0x55da0e4a3370 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f0affd640 1 -- 172.21.15.78:0/2294784432 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4f24004640 con 0x55da0e4a3370 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f34150740 1 -- 172.21.15.78:0/2294784432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f1c0adc60 con 0x55da0e4a3370 2024-09-16T00:24:43.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f0affd640 1 -- 172.21.15.78:0/2294784432 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f24059050 con 0x55da0e4a3370 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f34150740 10 client.17427 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f34150740 1 -- 172.21.15.78:0/2294784432 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da0e4b1af0 con 0x55da0e4a3370 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.535+0000 7f4f0affd640 1 -- 172.21.15.78:0/2294784432 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f1c0adc60 con 0x55da0e4a3370 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f0affd640 1 -- 172.21.15.78:0/2294784432 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f240305c0 con 0x55da0e4a3370 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f34150740 10 client.17427 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f34150740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f34150740 20 client.17427 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f34150740 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da0e4b2f50 0x55da0e4b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f34150740 4 client.17427 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f34150740 1 -- 172.21.15.78:0/2294784432 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da0e40b860 con 0x55da0e4b2f50 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da0e4b2f50 0x55da0e4b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f0affd640 10 client.17427 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:43.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.536+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da0e4b2f50 0x55da0e4b53d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea5d9e38270a26b9 server_cookie=d45bcea6dd4c9b97 in_seq=0 out_seq=0 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f0affd640 1 -- 172.21.15.78:0/2294784432 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da0e40b860 con 0x55da0e4b2f50 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f0affd640 10 client.17427 handle_command_reply: tid=0 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 client.17427 shutdown 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 -- 172.21.15.78:0/2294784432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4a3370 msgr2=0x55da0e4a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4a3370 0x55da0e4a07e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4f24003d40 tx=0x7f4f2402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 -- 172.21.15.78:0/2294784432 shutdown_connections 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da0e4b2f50 0x55da0e4b53d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4a3370 0x55da0e4a07e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e499b40 0x55da0e4a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 --2- 172.21.15.78:0/2294784432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4985d0 0x55da0e4a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f0affd640 0 client.17427 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 -- 172.21.15.78:0/2294784432 >> 172.21.15.78:0/2294784432 conn(0x55da0e461310 msgr2=0x55da0e49a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:43.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 -- 172.21.15.78:0/2294784432 shutdown_connections 2024-09-16T00:24:43.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.537+0000 7f4f34150740 1 -- 172.21.15.78:0/2294784432 wait complete. 2024-09-16T00:24:43.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.538+0000 7f4f34150740 20 client.17427 trim_cache size 0 max 16384 2024-09-16T00:24:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f34150740 1 Processor -- start 2024-09-16T00:24:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f34150740 1 -- start start 2024-09-16T00:24:43.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f34150740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4985d0 0x55da0e4a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f34150740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e49b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f34150740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4a3370 0x55da0e49b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f34150740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1b0ea0 con 0x55da0e499b40 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f34150740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1b29e0 con 0x55da0e4985d0 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f34150740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1b1570 con 0x55da0e4a3370 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e49b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e49b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37678/0 (socket says 172.21.15.78:37678) 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 -- 172.21.15.78:0/3957147411 learned_addr learned my addr 172.21.15.78:0/3957147411 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4a3370 0x55da0e49b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4985d0 0x55da0e4a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 -- 172.21.15.78:0/3957147411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4a3370 msgr2=0x55da0e49b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4a3370 0x55da0e49b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 -- 172.21.15.78:0/3957147411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4985d0 msgr2=0x55da0e4a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4985d0 0x55da0e4a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 -- 172.21.15.78:0/3957147411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da0e4af790 con 0x55da0e499b40 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4985d0 0x55da0e4a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.551+0000 7f4f2affd640 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e49b180 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c075f80 tx=0x7f4f20012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c448e63e0e1cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f117fa640 1 -- 172.21.15.78:0/3957147411 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20005220 con 0x55da0e499b40 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f117fa640 1 -- 172.21.15.78:0/3957147411 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2000ee20 con 0x55da0e499b40 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f117fa640 1 -- 172.21.15.78:0/3957147411 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20010bd0 con 0x55da0e499b40 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 -- 172.21.15.78:0/3957147411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 msgr2=0x55da0e49b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e49b180 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c075f80 tx=0x7f4f20012820 comp rx=0 tx=0).stop 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 -- 172.21.15.78:0/3957147411 shutdown_connections 2024-09-16T00:24:43.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e4a3370 0x55da0e49b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e499b40 0x55da0e49b180 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 --2- 172.21.15.78:0/3957147411 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4985d0 0x55da0e4a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 -- 172.21.15.78:0/3957147411 >> 172.21.15.78:0/3957147411 conn(0x55da0e461310 msgr2=0x55da0e4a18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 -- 172.21.15.78:0/3957147411 shutdown_connections 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.552+0000 7f4f34150740 1 -- 172.21.15.78:0/3957147411 wait complete. 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 1 -- start start 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 1 Processor -- start 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4985d0 0x55da0e4a7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e499b40 0x55da0e4a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e51ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1b2310 con 0x55da0e4985d0 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1b1c40 con 0x55da0e4a3370 2024-09-16T00:24:43.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c1aee80 con 0x55da0e499b40 2024-09-16T00:24:43.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f34150740 10 client.0 _ll_register_callbacks cb 0x7f4f24009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:43.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4985d0 0x55da0e4a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f2b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4985d0 0x55da0e4a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37684/0 (socket says 172.21.15.78:37684) 2024-09-16T00:24:43.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.553+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/2946801582 learned_addr learned my addr 172.21.15.78:0/2946801582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:43.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2affd640 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e499b40 0x55da0e4a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e51ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/2946801582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e499b40 msgr2=0x55da0e4a7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:43.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e499b40 0x55da0e4a7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 msgr2=0x55da0e51ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e51ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2b7fe640 1 -- 172.21.15.78:0/2946801582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0e51d1f0 con 0x55da0e4985d0 2024-09-16T00:24:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2bfff640 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e51ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f2b7fe640 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4985d0 0x55da0e4a7590 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0b2480 tx=0x7f4f1c096ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a606a1ad63487c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f0affd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:43.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f0affd640 1 -- 172.21.15.78:0/2946801582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c01e040 con 0x55da0e4985d0 2024-09-16T00:24:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f0affd640 1 -- 172.21.15.78:0/2946801582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4f1c0cc940 con 0x55da0e4985d0 2024-09-16T00:24:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f34150740 1 -- 172.21.15.78:0/2946801582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f2000ee20 con 0x55da0e4985d0 2024-09-16T00:24:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f0affd640 1 -- 172.21.15.78:0/2946801582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4f1c09ba50 con 0x55da0e4985d0 2024-09-16T00:24:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f34150740 10 client.17433 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f34150740 1 -- 172.21.15.78:0/2946801582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da0e51e3b0 con 0x55da0e4985d0 2024-09-16T00:24:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f0affd640 1 -- 172.21.15.78:0/2946801582 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f2000ee20 con 0x55da0e4985d0 2024-09-16T00:24:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f0affd640 1 -- 172.21.15.78:0/2946801582 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4f1c0b10b0 con 0x55da0e4985d0 2024-09-16T00:24:43.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f34150740 10 client.17433 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.554+0000 7f4f34150740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f34150740 20 client.17433 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f34150740 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da0e51e760 0x55da0e520b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f34150740 4 client.17433 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f34150740 1 -- 172.21.15.78:0/2946801582 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da0e405d70 con 0x55da0e51e760 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f2affd640 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da0e51e760 0x55da0e520b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f2affd640 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da0e51e760 0x55da0e520b50 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c876583e8173941 server_cookie=4e1754d083268c78 in_seq=0 out_seq=0 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f0affd640 10 client.17433 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f0affd640 1 -- 172.21.15.78:0/2946801582 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da0e405d70 con 0x55da0e51e760 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f0affd640 10 client.17433 handle_command_reply: tid=0 2024-09-16T00:24:43.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.555+0000 7f4f34150740 1 client.17433 shutdown 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 -- 172.21.15.78:0/2946801582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4985d0 msgr2=0x55da0e4a7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4985d0 0x55da0e4a7590 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0b2480 tx=0x7f4f1c096ff0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 -- 172.21.15.78:0/2946801582 shutdown_connections 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da0e51e760 0x55da0e520b50 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da0e4a3370 0x55da0e51ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da0e499b40 0x55da0e4a7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 --2- 172.21.15.78:0/2946801582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da0e4985d0 0x55da0e4a7590 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f0affd640 0 client.17433 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 -- 172.21.15.78:0/2946801582 >> 172.21.15.78:0/2946801582 conn(0x55da0e461310 msgr2=0x55da0e4ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 -- 172.21.15.78:0/2946801582 shutdown_connections 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 1 -- 172.21.15.78:0/2946801582 wait complete. 2024-09-16T00:24:43.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f34150740 20 client.17433 trim_cache size 0 max 16384 2024-09-16T00:24:43.571 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 -- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4f140a3240 msgr2=0x7f4f140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4f140a3240 0x7f4f140a5700 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4f24006010 tx=0x7f4f24031d20 comp rx=0 tx=0).stop 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 -- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c101de0 msgr2=0x7f4f2c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c101de0 0x7f4f2c0744f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b4a0 tx=0x7f4f1c00b960 comp rx=0 tx=0).stop 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 -- 172.21.15.78:0/2829459767 shutdown_connections 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4f140a3240 0x7f4f140a5700 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f2c106f10 0x7f4f2c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f2c102790 0x7f4f2c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 --2- 172.21.15.78:0/2829459767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f2c101de0 0x7f4f2c0744f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.556+0000 7f4f31d77640 1 -- 172.21.15.78:0/2829459767 >> 172.21.15.78:0/2829459767 conn(0x7f4f2c0fd8c0 msgr2=0x7f4f2c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.557+0000 7f4f31d77640 1 -- 172.21.15.78:0/2829459767 shutdown_connections 2024-09-16T00:24:43.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:43.557+0000 7f4f31d77640 1 -- 172.21.15.78:0/2829459767 wait complete. 2024-09-16T00:24:44.164 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:43 smithi078 ceph-mon[31157]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 727 KiB/s wr, 59 op/s 2024-09-16T00:24:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:43 smithi193 ceph-mon[35484]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 727 KiB/s wr, 59 op/s 2024-09-16T00:24:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:43 smithi148 ceph-mon[35599]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 727 KiB/s wr, 59 op/s 2024-09-16T00:24:45.279 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:24:45.279 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:24:45.529 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:45.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 645.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:24:46.006 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:45 smithi078 ceph-mon[31157]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 94 KiB/s wr, 20 op/s 2024-09-16T00:24:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 -- 172.21.15.78:0/3971255820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec102790 msgr2=0x7f8eec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 --2- 172.21.15.78:0/3971255820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec102790 0x7f8eec1067e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009900 tx=0x7f8ee002a410 comp rx=0 tx=0).stop 2024-09-16T00:24:46.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 -- 172.21.15.78:0/3971255820 shutdown_connections 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 --2- 172.21.15.78:0/3971255820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8eec106f10 0x7f8eec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 --2- 172.21.15.78:0/3971255820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec102790 0x7f8eec1067e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 --2- 172.21.15.78:0/3971255820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8eec101de0 0x7f8eec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 -- 172.21.15.78:0/3971255820 >> 172.21.15.78:0/3971255820 conn(0x7f8eec0fd8c0 msgr2=0x7f8eec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 -- 172.21.15.78:0/3971255820 shutdown_connections 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.089+0000 7f8ef2766640 1 -- 172.21.15.78:0/3971255820 wait complete. 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef2766640 1 Processor -- start 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef2766640 1 -- start start 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef2766640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8eec101de0 0x7f8eec1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef2766640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8eec102790 0x7f8eec198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef2766640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec106f10 0x7f8eec19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef2766640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec112010 con 0x7f8eec106f10 2024-09-16T00:24:46.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef2766640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec111e90 con 0x7f8eec102790 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef2766640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec112190 con 0x7f8eec101de0 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef0d03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec106f10 0x7f8eec19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef0d03640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec106f10 0x7f8eec19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37696/0 (socket says 172.21.15.78:37696) 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.090+0000 7f8ef0d03640 1 -- 172.21.15.78:0/2936854816 learned_addr learned my addr 172.21.15.78:0/2936854816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8eebfff640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8eec101de0 0x7f8eec1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8eec102790 0x7f8eec198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ef0d03640 1 -- 172.21.15.78:0/2936854816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8eec101de0 msgr2=0x7f8eec1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ef0d03640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8eec101de0 0x7f8eec1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ef0d03640 1 -- 172.21.15.78:0/2936854816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8eec102790 msgr2=0x7f8eec198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ef0d03640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8eec102790 0x7f8eec198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ef0d03640 1 -- 172.21.15.78:0/2936854816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec19d560 con 0x7f8eec106f10 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8eec102790 0x7f8eec198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:46.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ef0d03640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec106f10 0x7f8eec19ce80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b970 tx=0x7f8ee400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ee97fa640 1 -- 172.21.15.78:0/2936854816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4019070 con 0x7f8eec106f10 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ef2766640 1 -- 172.21.15.78:0/2936854816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec19d850 con 0x7f8eec106f10 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ee97fa640 1 -- 172.21.15.78:0/2936854816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee4014440 con 0x7f8eec106f10 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ee97fa640 1 -- 172.21.15.78:0/2936854816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40075d0 con 0x7f8eec106f10 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.091+0000 7f8ef2766640 1 -- 172.21.15.78:0/2936854816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec1a5130 con 0x7f8eec106f10 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.093+0000 7f8ee97fa640 1 -- 172.21.15.78:0/2936854816 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8ee401c070 con 0x7f8eec106f10 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.093+0000 7f8ef4b3f740 1 Processor -- start 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef4b3f740 1 -- start start 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 0x55e0f13f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f13f9480 0x55e0f1403800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f1403d40 0x55e0f1406130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e0f135e280 con 0x55e0f1403d40 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e0f1305b10 con 0x55e0f13f9480 2024-09-16T00:24:46.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e0f135a240 con 0x55e0f13f8b60 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ee97fa640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ed40a3240 0x7f8ed40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f1403d40 0x55e0f1406130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eebfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f13f9480 0x55e0f1403800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f1403d40 0x55e0f1406130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37710/0 (socket says 172.21.15.78:37710) 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/1277469962 learned_addr learned my addr 172.21.15.78:0/1277469962 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef0d03640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ed40a3240 0x7f8ed40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ee97fa640 1 -- 172.21.15.78:0/2936854816 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8ee40cf320 con 0x7f8eec106f10 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/1277469962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 0x55e0f13f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/1277469962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 msgr2=0x55e0f13f8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/1277469962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 0x55e0f13f8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/1277469962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f13f9480 msgr2=0x55e0f1403800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/1277469962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f13f9480 0x55e0f1403800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/1277469962 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f13121f0 con 0x55e0f1403d40 2024-09-16T00:24:46.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/1277469962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f1403d40 0x55e0f1406130 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005ec0 tx=0x7f8ee0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2039046ffdc0fb14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.094+0000 7f8ef0d03640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ed40a3240 0x7f8ed40a5700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400bfd0 tx=0x7f8ee4090a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1277469962 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0045070 con 0x55e0f1403d40 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1277469962 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee0031a00 con 0x55e0f1403d40 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ed27fc640 1 -- 172.21.15.78:0/1277469962 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0035040 con 0x55e0f1403d40 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/1277469962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f1403d40 msgr2=0x55e0f1406130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/1277469962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f1403d40 0x55e0f1406130 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005ec0 tx=0x7f8ee0000bc0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/1277469962 shutdown_connections 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/1277469962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f1403d40 0x55e0f1406130 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/1277469962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f13f9480 0x55e0f1403800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/1277469962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 0x55e0f13f8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/1277469962 >> 172.21.15.78:0/1277469962 conn(0x55e0f13f50f0 msgr2=0x55e0f13f54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/1277469962 shutdown_connections 2024-09-16T00:24:46.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.095+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/1277469962 wait complete. 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 1 -- start start 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 1 Processor -- start 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1400cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1401210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1401750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e0f1406910 con 0x55e0f13f8b60 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e0f1318f10 con 0x55e0f1403d40 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e0f1406a90 con 0x55e0f13f9480 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8ef4b3f740 10 client.0 _ll_register_callbacks cb 0x55e0f12e1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8eeb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1400cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8eeb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1400cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37712/0 (socket says 172.21.15.78:37712) 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/3597966340 learned_addr learned my addr 172.21.15.78:0/3597966340 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8eebfff640 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1401210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.096+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1401750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/3597966340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 msgr2=0x55e0f1401210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1401210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/3597966340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 msgr2=0x55e0f1401750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1401750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/3597966340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f10cf6a0 con 0x55e0f13f8b60 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1400cd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0043040 tx=0x7f8ee00055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea047794ee7f9240 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8ecb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/3597966340 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0051060 con 0x55e0f13f8b60 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/3597966340 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8ee00035f0 con 0x55e0f13f8b60 2024-09-16T00:24:46.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/3597966340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee0031a00 con 0x55e0f13f8b60 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/3597966340 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ee0058050 con 0x55e0f13f8b60 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8ef4b3f740 10 client.17445 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/3597966340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e0f14125c0 con 0x55e0f13f8b60 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.097+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/3597966340 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ee0031a00 con 0x55e0f13f8b60 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/3597966340 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8ee004b030 con 0x55e0f13f8b60 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8ef4b3f740 10 client.17445 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8ef4b3f740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8ef4b3f740 20 client.17445 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e0f1413910 0x55e0f1415d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8ef4b3f740 4 client.17445 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/3597966340 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0f136b7f0 con 0x55e0f1413910 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8eebfff640 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e0f1413910 0x55e0f1415d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8eebfff640 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e0f1413910 0x55e0f1415d00 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72861cbc3b7a81b5 server_cookie=ec3f62d4b9e91364 in_seq=0 out_seq=0 2024-09-16T00:24:46.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.098+0000 7f8ecb7fe640 10 client.17445 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/3597966340 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0f136b7f0 con 0x55e0f1413910 2024-09-16T00:24:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ecb7fe640 10 client.17445 handle_command_reply: tid=0 2024-09-16T00:24:46.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 client.17445 shutdown 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/3597966340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 msgr2=0x55e0f1400cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1400cd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0043040 tx=0x7f8ee00055d0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/3597966340 shutdown_connections 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e0f1413910 0x55e0f1415d00 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1401750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1401210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/3597966340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1400cd0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/3597966340 >> 172.21.15.78:0/3597966340 conn(0x55e0f13f50f0 msgr2=0x55e0f13fadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ecb7fe640 0 client.17445 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/3597966340 shutdown_connections 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.099+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/3597966340 wait complete. 2024-09-16T00:24:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.100+0000 7f8ef4b3f740 20 client.17445 trim_cache size 0 max 16384 2024-09-16T00:24:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ef4b3f740 1 Processor -- start 2024-09-16T00:24:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ef4b3f740 1 -- start start 2024-09-16T00:24:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 0x55e0f13fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f9480 0x55e0f13fe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1402420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1aebd0 con 0x55e0f13f9480 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1b0710 con 0x55e0f1403d40 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1af2a0 con 0x55e0f13f8b60 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f9480 0x55e0f13fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f9480 0x55e0f13fe490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37714/0 (socket says 172.21.15.78:37714) 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 -- 172.21.15.78:0/533659644 learned_addr learned my addr 172.21.15.78:0/533659644 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 -- 172.21.15.78:0/533659644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 msgr2=0x55e0f13fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 0x55e0f13fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1402420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 -- 172.21.15.78:0/533659644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 msgr2=0x55e0f1402420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1402420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 -- 172.21.15.78:0/533659644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f13ff220 con 0x55e0f13f9480 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1402420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:46.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8eebfff640 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f9480 0x55e0f13fe490 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x55e0f13f5f50 tx=0x7f8edc009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ad44b0983d7b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:46.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.117+0000 7f8ed1ffb640 1 -- 172.21.15.78:0/533659644 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc024070 con 0x55e0f13f9480 2024-09-16T00:24:46.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ed1ffb640 1 -- 172.21.15.78:0/533659644 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8edc015e00 con 0x55e0f13f9480 2024-09-16T00:24:46.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ed1ffb640 1 -- 172.21.15.78:0/533659644 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc014410 con 0x55e0f13f9480 2024-09-16T00:24:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/533659644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f9480 msgr2=0x55e0f13fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f9480 0x55e0f13fe490 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x55e0f13f5f50 tx=0x7f8edc009110 comp rx=0 tx=0).stop 2024-09-16T00:24:46.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/533659644 shutdown_connections 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f1402420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f9480 0x55e0f13fe490 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/533659644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f8b60 0x55e0f13fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/533659644 >> 172.21.15.78:0/533659644 conn(0x55e0f13f50f0 msgr2=0x55e0f13ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/533659644 shutdown_connections 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/533659644 wait complete. 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.118+0000 7f8ef4b3f740 1 -- start start 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8ef4b3f740 1 Processor -- start 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1418cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1419230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8ef4b3f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f14172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1b0040 con 0x55e0f13f8b60 2024-09-16T00:24:46.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1af970 con 0x55e0f1403d40 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8ef4b3f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1acbb0 con 0x55e0f13f9480 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8ef4b3f740 10 client.0 _ll_register_callbacks cb 0x7f8ee0043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eeb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1418cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eeb7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1418cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:37728/0 (socket says 172.21.15.78:37728) 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/4204492684 learned_addr learned my addr 172.21.15.78:0/4204492684 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eebfff640 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1419230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/4204492684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 msgr2=0x55e0f1419230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1419230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/4204492684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 msgr2=0x55e0f14172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f14172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eeb7fe640 1 -- 172.21.15.78:0/4204492684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f1417af0 con 0x55e0f13f8b60 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.119+0000 7f8eebfff640 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1419230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8eeb7fe640 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1418cf0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0043040 tx=0x7f8ee003d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c833840abb78e861 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:46.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ecb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/4204492684 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0061060 con 0x55e0f13f8b60 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/4204492684 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f8ee0005450 con 0x55e0f13f8b60 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/4204492684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8edc015e00 con 0x55e0f13f8b60 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/4204492684 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ee0068050 con 0x55e0f13f8b60 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 10 client.17451 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/4204492684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e0f147ee80 con 0x55e0f13f8b60 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/4204492684 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8edc015e00 con 0x55e0f13f8b60 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/4204492684 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f8ee004b070 con 0x55e0f13f8b60 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 10 client.17451 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 20 client.17451 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e0f147f570 0x55e0f1481990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 4 client.17451 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:24:46.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.120+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/4204492684 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e0f1365d00 con 0x55e0f147f570 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.121+0000 7f8eebfff640 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e0f147f570 0x55e0f1481990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.121+0000 7f8ecb7fe640 10 client.17451 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.121+0000 7f8eebfff640 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e0f147f570 0x55e0f1481990 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40807f6e6691c61 server_cookie=b47dce8e8d136705 in_seq=0 out_seq=0 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ecb7fe640 1 -- 172.21.15.78:0/4204492684 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e0f1365d00 con 0x55e0f147f570 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ecb7fe640 10 client.17451 handle_command_reply: tid=0 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ef4b3f740 1 client.17451 shutdown 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/4204492684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 msgr2=0x55e0f1418cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1418cf0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0043040 tx=0x7f8ee003d990 comp rx=0 tx=0).stop 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/4204492684 shutdown_connections 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e0f147f570 0x55e0f1481990 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e0f1403d40 0x55e0f14172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f13f9480 0x55e0f1419230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ef4b3f740 1 --2- 172.21.15.78:0/4204492684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e0f13f8b60 0x55e0f1418cf0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.122+0000 7f8ecb7fe640 0 client.17451 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:46.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/4204492684 >> 172.21.15.78:0/4204492684 conn(0x55e0f13f50f0 msgr2=0x55e0f13f3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:46.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/4204492684 shutdown_connections 2024-09-16T00:24:46.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef4b3f740 1 -- 172.21.15.78:0/4204492684 wait complete. 2024-09-16T00:24:46.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef4b3f740 20 client.17451 trim_cache size 0 max 16384 2024-09-16T00:24:46.128 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:24:46.128 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:24:46.128 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "15d04cc8-3532-4516-81fc-7743d0ba7321", 2024-09-16T00:24:46.128 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:24:46.128 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:24:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 -- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ed40a3240 msgr2=0x7f8ed40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ed40a3240 0x7f8ed40a5700 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400bfd0 tx=0x7f8ee4090a60 comp rx=0 tx=0).stop 2024-09-16T00:24:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 -- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec106f10 msgr2=0x7f8eec19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec106f10 0x7f8eec19ce80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b970 tx=0x7f8ee400bd70 comp rx=0 tx=0).stop 2024-09-16T00:24:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 -- 172.21.15.78:0/2936854816 shutdown_connections 2024-09-16T00:24:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ed40a3240 0x7f8ed40a5700 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8eec106f10 0x7f8eec19ce80 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8eec102790 0x7f8eec198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 --2- 172.21.15.78:0/2936854816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8eec101de0 0x7f8eec1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:46.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 -- 172.21.15.78:0/2936854816 >> 172.21.15.78:0/2936854816 conn(0x7f8eec0fd8c0 msgr2=0x7f8eec0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:46.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 -- 172.21.15.78:0/2936854816 shutdown_connections 2024-09-16T00:24:46.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:46.123+0000 7f8ef2766640 1 -- 172.21.15.78:0/2936854816 wait complete. 2024-09-16T00:24:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:45 smithi193 ceph-mon[35484]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 94 KiB/s wr, 20 op/s 2024-09-16T00:24:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:45 smithi148 ceph-mon[35599]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 94 KiB/s wr, 20 op/s 2024-09-16T00:24:46.809 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:24:47.053 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:47.174 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:46 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:24:47.174 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:46 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:24:47.175 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:46 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:47.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:46 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:24:47.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:46 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:24:47.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:46 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:46 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:24:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:46 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:24:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:46 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:47.492 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:24:47.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 -- 172.21.15.78:0/2853748753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac100100 msgr2=0x7f3aac100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 --2- 172.21.15.78:0/2853748753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac100100 0x7f3aac100560 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009900 tx=0x7f3aa002a410 comp rx=0 tx=0).stop 2024-09-16T00:24:47.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 -- 172.21.15.78:0/2853748753 shutdown_connections 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 --2- 172.21.15.78:0/2853748753 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac100aa0 0x7f3aac10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 --2- 172.21.15.78:0/2853748753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac100100 0x7f3aac100560 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 --2- 172.21.15.78:0/2853748753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3aac106b80 0x7f3aac106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 -- 172.21.15.78:0/2853748753 >> 172.21.15.78:0/2853748753 conn(0x7f3aac077230 msgr2=0x7f3aac0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 -- 172.21.15.78:0/2853748753 shutdown_connections 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.582+0000 7f3ab141f640 1 -- 172.21.15.78:0/2853748753 wait complete. 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.583+0000 7f3ab141f640 1 Processor -- start 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.583+0000 7f3ab141f640 1 -- start start 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.583+0000 7f3ab141f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac100100 0x7f3aac1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.583+0000 7f3ab141f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3aac100aa0 0x7f3aac198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.583+0000 7f3ab141f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac106b80 0x7f3aac19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.583+0000 7f3ab141f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac112050 con 0x7f3aac106b80 2024-09-16T00:24:47.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.583+0000 7f3ab141f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac111ed0 con 0x7f3aac100100 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.583+0000 7f3ab141f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1121d0 con 0x7f3aac100aa0 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac106b80 0x7f3aac19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac106b80 0x7f3aac19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39016/0 (socket says 172.21.15.78:39016) 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 -- 172.21.15.78:0/886318697 learned_addr learned my addr 172.21.15.78:0/886318697 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3aac100aa0 0x7f3aac198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac100100 0x7f3aac1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 -- 172.21.15.78:0/886318697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3aac100aa0 msgr2=0x7f3aac198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3aac100aa0 0x7f3aac198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 -- 172.21.15.78:0/886318697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac100100 msgr2=0x7f3aac1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac100100 0x7f3aac1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 -- 172.21.15.78:0/886318697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac19d470 con 0x7f3aac106b80 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac100100 0x7f3aac1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:47.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac106b80 0x7f3aac19cd90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400ccb0 tx=0x7f3aa400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3a93fff640 1 -- 172.21.15.78:0/886318697 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4003970 con 0x7f3aac106b80 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3ab141f640 1 -- 172.21.15.78:0/886318697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aac19d760 con 0x7f3aac106b80 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3a93fff640 1 -- 172.21.15.78:0/886318697 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa4004030 con 0x7f3aac106b80 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3a93fff640 1 -- 172.21.15.78:0/886318697 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400f720 con 0x7f3aac106b80 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.584+0000 7f3ab141f640 1 -- 172.21.15.78:0/886318697 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aac1a5040 con 0x7f3aac106b80 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.586+0000 7f3a93fff640 1 -- 172.21.15.78:0/886318697 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3aa40041e0 con 0x7f3aac106b80 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3ab37f8740 1 Processor -- start 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3ab37f8740 1 -- start start 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3a93fff640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a940a32c0 0x7f3a940a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 0x557a91a71880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a7c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a7ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557a919d72e0 con 0x557a91a7c680 2024-09-16T00:24:47.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557a9197ece0 con 0x557a91a714a0 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a919d32a0 con 0x557a91a71dc0 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3a93fff640 1 -- 172.21.15.78:0/886318697 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3aa40cfd80 con 0x7f3aac106b80 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a940a32c0 0x7f3a940a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.587+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a7ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a7ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39026/0 (socket says 172.21.15.78:39026) 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aab7fe640 1 -- 172.21.15.78:0/2799696446 learned_addr learned my addr 172.21.15.78:0/2799696446 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a7c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 0x557a91a71880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aab7fe640 1 -- 172.21.15.78:0/2799696446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 msgr2=0x557a91a7c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a7c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aab7fe640 1 -- 172.21.15.78:0/2799696446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 msgr2=0x557a91a71880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 0x557a91a71880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aab7fe640 1 -- 172.21.15.78:0/2799696446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a91963440 con 0x557a91a7c680 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a940a32c0 0x7f3a940a5780 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0ff8f0 tx=0x7f3aa0031690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a7ea70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40984a0 tx=0x7f3aa409bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89674c6793f764e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3a917fa640 1 -- 172.21.15.78:0/2799696446 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa401d030 con 0x557a91a7c680 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3a917fa640 1 -- 172.21.15.78:0/2799696446 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa40a03f0 con 0x557a91a7c680 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.588+0000 7f3a917fa640 1 -- 172.21.15.78:0/2799696446 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4003b10 con 0x557a91a7c680 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 -- 172.21.15.78:0/2799696446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 msgr2=0x557a91a7ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a7ea70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40984a0 tx=0x7f3aa409bab0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 -- 172.21.15.78:0/2799696446 shutdown_connections 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a7ea70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a7c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/2799696446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 0x557a91a71880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 -- 172.21.15.78:0/2799696446 >> 172.21.15.78:0/2799696446 conn(0x557a91a6da30 msgr2=0x557a91a6de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 -- 172.21.15.78:0/2799696446 shutdown_connections 2024-09-16T00:24:47.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 -- 172.21.15.78:0/2799696446 wait complete. 2024-09-16T00:24:47.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 -- start start 2024-09-16T00:24:47.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.589+0000 7f3ab37f8740 1 Processor -- start 2024-09-16T00:24:47.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 0x557a91a7bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a71dc0 0x557a91a79270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a7c680 0x557a91a797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557a91a7f250 con 0x557a91a71dc0 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557a91992050 con 0x557a91a7c680 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a91a7f3d0 con 0x557a91a714a0 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3ab37f8740 10 client.0 _ll_register_callbacks cb 0x557a919d2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3aaa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a71dc0 0x557a91a79270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3aaa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a71dc0 0x557a91a79270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39034/0 (socket says 172.21.15.78:39034) 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3aaa7fc640 1 -- 172.21.15.78:0/1705187096 learned_addr learned my addr 172.21.15.78:0/1705187096 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:47.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.590+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a7c680 0x557a91a797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 0x557a91a7bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3aaa7fc640 1 -- 172.21.15.78:0/1705187096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 msgr2=0x557a91a7bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 0x557a91a7bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3aaa7fc640 1 -- 172.21.15.78:0/1705187096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a7c680 msgr2=0x557a91a797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a7c680 0x557a91a797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3aaa7fc640 1 -- 172.21.15.78:0/1705187096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a917483c0 con 0x557a91a71dc0 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a71dc0 0x557a91a79270 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0005ab0 tx=0x7f3aa002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b53220950a7928 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3a8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:47.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1705187096 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0035050 con 0x557a91a71dc0 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1705187096 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3aa0005450 con 0x557a91a71dc0 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1705187096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa40a03f0 con 0x557a91a71dc0 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1705187096 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3aa005a050 con 0x557a91a71dc0 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3ab37f8740 10 client.17463 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.591+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1705187096 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a91a84bf0 con 0x557a91a71dc0 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1705187096 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa40a03f0 con 0x557a91a71dc0 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1705187096 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3aa0003830 con 0x557a91a71dc0 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3ab37f8740 10 client.17463 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3ab37f8740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:24:47.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3ab37f8740 20 client.17463 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:47.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557a91a860b0 0x557a91a884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3ab37f8740 4 client.17463 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:47.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1705187096 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a919e4850 con 0x557a91a860b0 2024-09-16T00:24:47.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557a91a860b0 0x557a91a884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557a91a860b0 0x557a91a884a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a93cf1531b2ee2a server_cookie=688962b285399292 in_seq=0 out_seq=0 2024-09-16T00:24:47.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.592+0000 7f3a8eff5640 10 client.17463 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1705187096 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a919e4850 con 0x557a91a860b0 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3a8eff5640 10 client.17463 handle_command_reply: tid=0 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 client.17463 shutdown 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1705187096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a71dc0 msgr2=0x557a91a79270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a71dc0 0x557a91a79270 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0005ab0 tx=0x7f3aa002a410 comp rx=0 tx=0).stop 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1705187096 shutdown_connections 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557a91a860b0 0x557a91a884a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a7c680 0x557a91a797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a71dc0 0x557a91a79270 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1705187096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 0x557a91a7bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1705187096 >> 172.21.15.78:0/1705187096 conn(0x557a91a6da30 msgr2=0x557a91a73710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3a8eff5640 0 client.17463 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:47.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1705187096 shutdown_connections 2024-09-16T00:24:47.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.593+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1705187096 wait complete. 2024-09-16T00:24:47.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.594+0000 7f3ab37f8740 20 client.17463 trim_cache size 0 max 16384 2024-09-16T00:24:47.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.610+0000 7f3ab37f8740 1 Processor -- start 2024-09-16T00:24:47.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3ab37f8740 1 -- start start 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 0x557a91a91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a71dc0 0x557a91a79b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91aa82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1ac9b0 con 0x557a91a7c680 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1ae4f0 con 0x557a91a71dc0 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1ad080 con 0x557a91a714a0 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91aa82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91aa82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39036/0 (socket says 172.21.15.78:39036) 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 -- 172.21.15.78:0/4183422852 learned_addr learned my addr 172.21.15.78:0/4183422852 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:47.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a71dc0 0x557a91a79b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 0x557a91a91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 -- 172.21.15.78:0/4183422852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 msgr2=0x557a91a91c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:47.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 0x557a91a91c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 -- 172.21.15.78:0/4183422852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a71dc0 msgr2=0x557a91a79b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:47.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a71dc0 0x557a91a79b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 -- 172.21.15.78:0/4183422852 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a91a71bf0 con 0x557a91a7c680 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.611+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91aa82f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3aa409bd40 tx=0x7f3aa4004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab98b4533e6d5632 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3a90ff9640 1 -- 172.21.15.78:0/4183422852 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40b5020 con 0x557a91a7c680 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3a90ff9640 1 -- 172.21.15.78:0/4183422852 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa409a0e0 con 0x557a91a7c680 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3a90ff9640 1 -- 172.21.15.78:0/4183422852 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa401d040 con 0x557a91a7c680 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3ab37f8740 1 -- 172.21.15.78:0/4183422852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 msgr2=0x557a91aa82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91aa82f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3aa409bd40 tx=0x7f3aa4004950 comp rx=0 tx=0).stop 2024-09-16T00:24:47.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3ab37f8740 1 -- 172.21.15.78:0/4183422852 shutdown_connections 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91aa82f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a71dc0 0x557a91a79b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/4183422852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a714a0 0x557a91a91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.612+0000 7f3ab37f8740 1 -- 172.21.15.78:0/4183422852 >> 172.21.15.78:0/4183422852 conn(0x557a91a6da30 msgr2=0x557a91a7b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 -- 172.21.15.78:0/4183422852 shutdown_connections 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 -- 172.21.15.78:0/4183422852 wait complete. 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 -- start start 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 Processor -- start 2024-09-16T00:24:47.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 0x557a91a8dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a8e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a88d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1ade20 con 0x557a91a7c680 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1ad750 con 0x557a91a714a0 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac1aa990 con 0x557a91a71dc0 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3ab37f8740 10 client.0 _ll_register_callbacks cb 0x7f3aa0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a88d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a88d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39050/0 (socket says 172.21.15.78:39050) 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3aab7fe640 1 -- 172.21.15.78:0/1351722787 learned_addr learned my addr 172.21.15.78:0/1351722787 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.613+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 0x557a91a8dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a8e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3aab7fe640 1 -- 172.21.15.78:0/1351722787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 msgr2=0x557a91a8e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a8e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3aab7fe640 1 -- 172.21.15.78:0/1351722787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 msgr2=0x557a91a8dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 0x557a91a8dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3aab7fe640 1 -- 172.21.15.78:0/1351722787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a91a893b0 con 0x557a91a7c680 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3aaa7fc640 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a8e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3aab7fe640 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a88d40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4098450 tx=0x7f3aa40986b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7b91de375dc728 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3a8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1351722787 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa407e020 con 0x557a91a7c680 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1351722787 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3aa4003b10 con 0x557a91a7c680 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1351722787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa409a0e0 con 0x557a91a7c680 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1351722787 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3aa401d040 con 0x557a91a7c680 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3ab37f8740 10 client.17469 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.614+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1351722787 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557a91b07bf0 con 0x557a91a7c680 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1351722787 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa409a0e0 con 0x557a91a7c680 2024-09-16T00:24:47.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1351722787 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3aa4004b60 con 0x557a91a7c680 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3ab37f8740 10 client.17469 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3ab37f8740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3ab37f8740 20 client.17469 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557a91b07e10 0x557a91b0a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3ab37f8740 4 client.17469 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1351722787 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557a919ded60 con 0x557a91b07e10 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557a91b07e10 0x557a91b0a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3aaaffd640 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557a91b07e10 0x557a91b0a200 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7326b48217b9381 server_cookie=e926530301924928 in_seq=0 out_seq=0 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.615+0000 7f3a8eff5640 10 client.17469 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3a8eff5640 1 -- 172.21.15.78:0/1351722787 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557a919ded60 con 0x557a91b07e10 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3a8eff5640 10 client.17469 handle_command_reply: tid=0 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 client.17469 shutdown 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1351722787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 msgr2=0x557a91a88d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a88d40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4098450 tx=0x7f3aa40986b0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1351722787 shutdown_connections 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557a91b07e10 0x557a91b0a200 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557a91a7c680 0x557a91a88d40 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a91a71dc0 0x557a91a8e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 --2- 172.21.15.78:0/1351722787 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557a91a714a0 0x557a91a8dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1351722787 >> 172.21.15.78:0/1351722787 conn(0x557a91a6da30 msgr2=0x557a91a72ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3a8eff5640 0 client.17469 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1351722787 shutdown_connections 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 1 -- 172.21.15.78:0/1351722787 wait complete. 2024-09-16T00:24:47.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.616+0000 7f3ab37f8740 20 client.17469 trim_cache size 0 max 16384 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout: "15d04cc8-3532-4516-81fc-7743d0ba7321": { 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "15d04cc8-3532-4516-81fc-7743d0ba7321", 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:47.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:47.624 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:24:47.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 -- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a940a32c0 msgr2=0x7f3a940a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a940a32c0 0x7f3a940a5780 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0ff8f0 tx=0x7f3aa0031690 comp rx=0 tx=0).stop 2024-09-16T00:24:47.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 -- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac106b80 msgr2=0x7f3aac19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:47.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac106b80 0x7f3aac19cd90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400ccb0 tx=0x7f3aa400a620 comp rx=0 tx=0).stop 2024-09-16T00:24:47.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 -- 172.21.15.78:0/886318697 shutdown_connections 2024-09-16T00:24:47.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3a940a32c0 0x7f3a940a5780 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3aac106b80 0x7f3aac19cd90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3aac100aa0 0x7f3aac198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 --2- 172.21.15.78:0/886318697 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3aac100100 0x7f3aac1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:47.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 -- 172.21.15.78:0/886318697 >> 172.21.15.78:0/886318697 conn(0x7f3aac077230 msgr2=0x7f3aac0fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:47.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 -- 172.21.15.78:0/886318697 shutdown_connections 2024-09-16T00:24:47.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:47.617+0000 7f3ab141f640 1 -- 172.21.15.78:0/886318697 wait complete. 2024-09-16T00:24:47.982 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:48.094 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:47 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:24:48.095 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:47 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.095 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:47 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:24:48.095 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:47 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.095 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:47 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.095 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:47 smithi078 ceph-mon[31157]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-16T00:24:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:47 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:24:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:47 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:47 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:24:48.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:47 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:47 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:47 smithi193 ceph-mon[35484]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-16T00:24:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:47 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:24:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:47 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:47 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:24:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:47 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:47 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:24:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:47 smithi148 ceph-mon[35599]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-16T00:24:48.542 INFO:tasks.cephfs.filesystem:scrub status for tag:15d04cc8-3532-4516-81fc-7743d0ba7321 - {'path': '/', 'tag': '15d04cc8-3532-4516-81fc-7743d0ba7321', 'options': 'recursive,force'} 2024-09-16T00:24:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 -- 172.21.15.78:0/1200530243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 msgr2=0x7f04b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 --2- 172.21.15.78:0/1200530243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 0x7f04b8106e60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009930 tx=0x7f04a802a420 comp rx=0 tx=0).stop 2024-09-16T00:24:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 -- 172.21.15.78:0/1200530243 shutdown_connections 2024-09-16T00:24:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 --2- 172.21.15.78:0/1200530243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b8101420 0x7f04b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 --2- 172.21.15.78:0/1200530243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04b8100a80 0x7f04b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 --2- 172.21.15.78:0/1200530243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 0x7f04b8106e60 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 -- 172.21.15.78:0/1200530243 >> 172.21.15.78:0/1200530243 conn(0x7f04b80fc7a0 msgr2=0x7f04b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:49.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 -- 172.21.15.78:0/1200530243 shutdown_connections 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.006+0000 7f04be040640 1 -- 172.21.15.78:0/1200530243 wait complete. 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04be040640 1 Processor -- start 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04be040640 1 -- start start 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04be040640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04b8100a80 0x7f04b8197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04be040640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b8101420 0x7f04b81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04be040640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 0x7f04b8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04be040640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8110e40 con 0x7f04b8106a80 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04be040640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8110cc0 con 0x7f04b8100a80 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04be040640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8110fc0 con 0x7f04b8101420 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 0x7f04b8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04b7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 0x7f04b8197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39064/0 (socket says 172.21.15.78:39064) 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04b7fff640 1 -- 172.21.15.78:0/3340810023 learned_addr learned my addr 172.21.15.78:0/3340810023 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04affff640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b8101420 0x7f04b81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.007+0000 7f04b77fe640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04b8100a80 0x7f04b8197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b7fff640 1 -- 172.21.15.78:0/3340810023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b8101420 msgr2=0x7f04b81975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b7fff640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b8101420 0x7f04b81975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b7fff640 1 -- 172.21.15.78:0/3340810023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04b8100a80 msgr2=0x7f04b8197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b7fff640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04b8100a80 0x7f04b8197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b7fff640 1 -- 172.21.15.78:0/3340810023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b81a68e0 con 0x7f04b8106a80 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b77fe640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04b8100a80 0x7f04b8197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b7fff640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 0x7f04b8197af0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f04b000cad0 tx=0x7f04b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b57fa640 1 -- 172.21.15.78:0/3340810023 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000ae20 con 0x7f04b8106a80 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b57fa640 1 -- 172.21.15.78:0/3340810023 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b0005020 con 0x7f04b8106a80 2024-09-16T00:24:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04b57fa640 1 -- 172.21.15.78:0/3340810023 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000f6f0 con 0x7f04b8106a80 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b81a6bd0 con 0x7f04b8106a80 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.008+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b81a70c0 con 0x7f04b8106a80 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.009+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b8100ee0 con 0x7f04b8106a80 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.010+0000 7f04b57fa640 1 -- 172.21.15.78:0/3340810023 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f04b00051d0 con 0x7f04b8106a80 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.011+0000 7f04b57fa640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.011+0000 7f04b77fe640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.011+0000 7f04b57fa640 1 -- 172.21.15.78:0/3340810023 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f04b00cfd10 con 0x7f04b8106a80 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.011+0000 7f04b77fe640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04a40a3240 0x7f04a40a5700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04a8000c00 tx=0x7f04a8032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.015+0000 7f04b57fa640 1 -- 172.21.15.78:0/3340810023 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04b0014030 con 0x7f04b8106a80 2024-09-16T00:24:49.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.217+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04b802d7c0 con 0x7f04b8106a80 2024-09-16T00:24:49.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.217+0000 7f04b57fa640 1 -- 172.21.15.78:0/3340810023 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f04b009c6d0 con 0x7f04b8106a80 2024-09-16T00:24:49.218 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:24:49.218 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:24:49.218 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:24:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.219+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04a40a3240 msgr2=0x7f04a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.219+0000 7f04be040640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04a40a3240 0x7f04a40a5700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f04a8000c00 tx=0x7f04a8032d50 comp rx=0 tx=0).stop 2024-09-16T00:24:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.219+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 msgr2=0x7f04b8197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.219+0000 7f04be040640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 0x7f04b8197af0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f04b000cad0 tx=0x7f04b000cf90 comp rx=0 tx=0).stop 2024-09-16T00:24:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.220+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 shutdown_connections 2024-09-16T00:24:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.220+0000 7f04be040640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f04a40a3240 0x7f04a40a5700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.220+0000 7f04be040640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f04b8106a80 0x7f04b8197af0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.220+0000 7f04be040640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b8101420 0x7f04b81975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.220+0000 7f04be040640 1 --2- 172.21.15.78:0/3340810023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f04b8100a80 0x7f04b8197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.220+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 >> 172.21.15.78:0/3340810023 conn(0x7f04b80fc7a0 msgr2=0x7f04b80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.220+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 shutdown_connections 2024-09-16T00:24:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:49.220+0000 7f04be040640 1 -- 172.21.15.78:0/3340810023 wait complete. 2024-09-16T00:24:49.905 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:49 smithi078 ceph-mon[31157]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-16T00:24:49.906 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:49 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3340810023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:49.933 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id d73a29d3 --release --await' 2024-09-16T00:24:49.934 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id d73a29d3 --release --await 2024-09-16T00:24:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:49 smithi193 ceph-mon[35484]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-16T00:24:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:49 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3340810023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:50.180 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:49 smithi148 ceph-mon[35599]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-16T00:24:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:49 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3340810023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 650.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:24:51.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 -- 172.21.15.78:0/66067495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0100e20 msgr2=0x7fede0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 --2- 172.21.15.78:0/66067495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0100e20 0x7fede0101200 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7feddc009900 tx=0x7feddc02a480 comp rx=0 tx=0).stop 2024-09-16T00:24:51.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 -- 172.21.15.78:0/66067495 shutdown_connections 2024-09-16T00:24:51.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 --2- 172.21.15.78:0/66067495 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fede0102ef0 0x7fede010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 --2- 172.21.15.78:0/66067495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede0102570 0x7fede01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 --2- 172.21.15.78:0/66067495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0100e20 0x7fede0101200 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 -- 172.21.15.78:0/66067495 >> 172.21.15.78:0/66067495 conn(0x7fede00fc7d0 msgr2=0x7fede00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:51.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 -- 172.21.15.78:0/66067495 shutdown_connections 2024-09-16T00:24:51.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 -- 172.21.15.78:0/66067495 wait complete. 2024-09-16T00:24:51.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 Processor -- start 2024-09-16T00:24:51.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.006+0000 7fede7505640 1 -- start start 2024-09-16T00:24:51.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede7505640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede0100e20 0x7fede01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede7505640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fede0102570 0x7fede01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede7505640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0102ef0 0x7fede01a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede7505640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fede0111020 con 0x7fede0102ef0 2024-09-16T00:24:51.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede7505640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fede0110ea0 con 0x7fede0102570 2024-09-16T00:24:51.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede7505640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fede01111a0 con 0x7fede0100e20 2024-09-16T00:24:51.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0102ef0 0x7fede01a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0102ef0 0x7fede01a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39088/0 (socket says 172.21.15.78:39088) 2024-09-16T00:24:51.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 -- 172.21.15.78:0/671006644 learned_addr learned my addr 172.21.15.78:0/671006644 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:51.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede4aa0640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fede0102570 0x7fede01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede52a1640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede0100e20 0x7fede01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 -- 172.21.15.78:0/671006644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede0100e20 msgr2=0x7fede01994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede0100e20 0x7fede01994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 -- 172.21.15.78:0/671006644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fede0102570 msgr2=0x7fede01999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fede0102570 0x7fede01999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 -- 172.21.15.78:0/671006644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede01a4a70 con 0x7fede0102ef0 2024-09-16T00:24:51.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.007+0000 7fede5aa2640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0102ef0 0x7fede01a4530 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b970 tx=0x7fedd800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:51.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.008+0000 7fedd67fc640 1 -- 172.21.15.78:0/671006644 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8019070 con 0x7fede0102ef0 2024-09-16T00:24:51.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.008+0000 7fedd67fc640 1 -- 172.21.15.78:0/671006644 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd8014440 con 0x7fede0102ef0 2024-09-16T00:24:51.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.008+0000 7fede7505640 1 -- 172.21.15.78:0/671006644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede01a4c40 con 0x7fede0102ef0 2024-09-16T00:24:51.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.008+0000 7fedd67fc640 1 -- 172.21.15.78:0/671006644 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd80075d0 con 0x7fede0102ef0 2024-09-16T00:24:51.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.008+0000 7fede7505640 1 -- 172.21.15.78:0/671006644 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede01a5150 con 0x7fede0102ef0 2024-09-16T00:24:51.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.010+0000 7fedd67fc640 1 -- 172.21.15.78:0/671006644 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fedd801c070 con 0x7fede0102ef0 2024-09-16T00:24:51.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.010+0000 7fede98de740 1 Processor -- start 2024-09-16T00:24:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.010+0000 7fede98de740 1 -- start start 2024-09-16T00:24:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede98de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65624440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede98de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec6562ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede98de740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec65631630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fedd67fc640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fedcc0a32c0 0x7fedcc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede98de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ec65589620 con 0x55ec65624060 2024-09-16T00:24:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede98de740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ec65530e50 con 0x55ec6562f240 2024-09-16T00:24:51.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede98de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec655855e0 con 0x55ec65624980 2024-09-16T00:24:51.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65624440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65624440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39100/0 (socket says 172.21.15.78:39100) 2024-09-16T00:24:51.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 -- 172.21.15.78:0/2387509367 learned_addr learned my addr 172.21.15.78:0/2387509367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:51.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede5aa2640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fedcc0a32c0 0x7fedcc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede52a1640 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec6562ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec65631630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fedd67fc640 1 -- 172.21.15.78:0/671006644 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fedd80cf190 con 0x7fede0102ef0 2024-09-16T00:24:51.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 -- 172.21.15.78:0/2387509367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 msgr2=0x55ec6562ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:51.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec6562ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 -- 172.21.15.78:0/2387509367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 msgr2=0x55ec65631630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:51.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec65631630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 -- 172.21.15.78:0/2387509367 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec65521d40 con 0x55ec65624060 2024-09-16T00:24:51.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede52a1640 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec6562ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:24:51.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede5aa2640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fedcc0a32c0 0x7fedcc0a5780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fedd8097ed0 tx=0x7fedd806a240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:51.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65624440 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fedd0006aa0 tx=0x7fedd00094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcc797b33487431 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:51.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.011+0000 7fedcb7fe640 1 -- 172.21.15.78:0/2387509367 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0017e60 con 0x55ec65624060 2024-09-16T00:24:51.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fedcb7fe640 1 -- 172.21.15.78:0/2387509367 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd001d070 con 0x55ec65624060 2024-09-16T00:24:51.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fedcb7fe640 1 -- 172.21.15.78:0/2387509367 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0018af0 con 0x55ec65624060 2024-09-16T00:24:51.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 -- 172.21.15.78:0/2387509367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 msgr2=0x55ec65624440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65624440 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fedd0006aa0 tx=0x7fedd00094c0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 -- 172.21.15.78:0/2387509367 shutdown_connections 2024-09-16T00:24:51.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec65631630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec6562ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 --2- 172.21.15.78:0/2387509367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65624440 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 -- 172.21.15.78:0/2387509367 >> 172.21.15.78:0/2387509367 conn(0x55ec656205f0 msgr2=0x55ec656209d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:51.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 -- 172.21.15.78:0/2387509367 shutdown_connections 2024-09-16T00:24:51.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.012+0000 7fede98de740 1 -- 172.21.15.78:0/2387509367 wait complete. 2024-09-16T00:24:51.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 1 -- start start 2024-09-16T00:24:51.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 1 Processor -- start 2024-09-16T00:24:51.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624060 0x55ec6562c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec65624980 0x55ec6562c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec6562f240 0x55ec6562cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ec65631e10 con 0x55ec6562f240 2024-09-16T00:24:51.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ec65511f10 con 0x55ec65624980 2024-09-16T00:24:51.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec65631f90 con 0x55ec65624060 2024-09-16T00:24:51.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede98de740 10 client.0 _ll_register_callbacks cb 0x55ec65535db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:51.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede4aa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec6562f240 0x55ec6562cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede4aa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec6562f240 0x55ec6562cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39112/0 (socket says 172.21.15.78:39112) 2024-09-16T00:24:51.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede4aa0640 1 -- 172.21.15.78:0/2066014343 learned_addr learned my addr 172.21.15.78:0/2066014343 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:51.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede52a1640 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec65624980 0x55ec6562c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.013+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624060 0x55ec6562c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede4aa0640 1 -- 172.21.15.78:0/2066014343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624060 msgr2=0x55ec6562c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624060 0x55ec6562c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede4aa0640 1 -- 172.21.15.78:0/2066014343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec65624980 msgr2=0x55ec6562c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec65624980 0x55ec6562c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede4aa0640 1 -- 172.21.15.78:0/2066014343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec65298580 con 0x55ec6562f240 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec6562f240 0x55ec6562cb30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fedd0019fd0 tx=0x7fedd00186b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb5c7515b8d21d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fedc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2066014343 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0029030 con 0x55ec6562f240 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede98de740 1 -- 172.21.15.78:0/2066014343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd001d070 con 0x55ec6562f240 2024-09-16T00:24:51.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2066014343 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fedd0018b00 con 0x55ec6562f240 2024-09-16T00:24:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2066014343 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fedd0017e60 con 0x55ec6562f240 2024-09-16T00:24:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede98de740 10 client.17487 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fede98de740 1 -- 172.21.15.78:0/2066014343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ec65362960 con 0x55ec6562f240 2024-09-16T00:24:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.014+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2066014343 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd001d070 con 0x55ec6562f240 2024-09-16T00:24:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2066014343 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fedd002e4e0 con 0x55ec6562f240 2024-09-16T00:24:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fede98de740 10 client.17487 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fede98de740 10 client.17487 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fede98de740 20 client.17487 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fede98de740 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ec6563ebc0 0x55ec65640fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fede98de740 4 client.17487 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fede98de740 1 -- 172.21.15.78:0/2066014343 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec65596b90 con 0x55ec6563ebc0 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fede52a1640 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ec6563ebc0 0x55ec65640fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fedc8ff9640 10 client.17487 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.015+0000 7fede52a1640 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ec6563ebc0 0x55ec65640fb0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d24c67ffe36a99c server_cookie=6da8a8d8812950cf in_seq=0 out_seq=0 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2066014343 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec65596b90 con 0x55ec6563ebc0 2024-09-16T00:24:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fedc8ff9640 10 client.17487 handle_command_reply: tid=0 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fede98de740 1 client.17487 shutdown 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fede98de740 1 -- 172.21.15.78:0/2066014343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec6562f240 msgr2=0x55ec6562cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fede98de740 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec6562f240 0x55ec6562cb30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fedd0019fd0 tx=0x7fedd00186b0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fede98de740 1 -- 172.21.15.78:0/2066014343 shutdown_connections 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fede98de740 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ec6563ebc0 0x55ec65640fb0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fede98de740 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec6562f240 0x55ec6562cb30 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fede98de740 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec65624980 0x55ec6562c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fede98de740 1 --2- 172.21.15.78:0/2066014343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624060 0x55ec6562c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.016+0000 7fedc8ff9640 0 client.17487 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:51.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.017+0000 7fede98de740 1 -- 172.21.15.78:0/2066014343 >> 172.21.15.78:0/2066014343 conn(0x55ec656205f0 msgr2=0x55ec656262d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:51.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.017+0000 7fede98de740 1 -- 172.21.15.78:0/2066014343 shutdown_connections 2024-09-16T00:24:51.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.017+0000 7fede98de740 1 -- 172.21.15.78:0/2066014343 wait complete. 2024-09-16T00:24:51.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.017+0000 7fede98de740 20 client.17487 trim_cache size 0 max 16384 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.033+0000 7fede98de740 1 Processor -- start 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.033+0000 7fede98de740 1 -- start start 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.033+0000 7fede98de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec6562c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.033+0000 7fede98de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec65629310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.033+0000 7fede98de740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec65629850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede98de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fede01aec90 con 0x55ec65624060 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede98de740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fede01b07d0 con 0x55ec6562f240 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede98de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fede01af360 con 0x55ec65624980 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec6562c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec6562c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39128/0 (socket says 172.21.15.78:39128) 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 -- 172.21.15.78:0/75717968 learned_addr learned my addr 172.21.15.78:0/75717968 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:51.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 -- 172.21.15.78:0/75717968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 msgr2=0x55ec65629310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede52a1640 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec65629310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec65629310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 -- 172.21.15.78:0/75717968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 msgr2=0x55ec65629850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec65629850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 -- 172.21.15.78:0/75717968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec65620420 con 0x55ec65624060 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede52a1640 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec65629310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fede4aa0640 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec6562c780 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fedd0018680 tx=0x7fedd0006e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314d762ce3dde5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fedcaffd640 1 -- 172.21.15.78:0/75717968 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0029070 con 0x55ec65624060 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fedcaffd640 1 -- 172.21.15.78:0/75717968 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd001d040 con 0x55ec65624060 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.034+0000 7fedcaffd640 1 -- 172.21.15.78:0/75717968 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd00156d0 con 0x55ec65624060 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 -- 172.21.15.78:0/75717968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 msgr2=0x55ec6562c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec6562c780 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fedd0018680 tx=0x7fedd0006e40 comp rx=0 tx=0).stop 2024-09-16T00:24:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 -- 172.21.15.78:0/75717968 shutdown_connections 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec65629850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec65629310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 --2- 172.21.15.78:0/75717968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec6562c780 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 -- 172.21.15.78:0/75717968 >> 172.21.15.78:0/75717968 conn(0x55ec656205f0 msgr2=0x55ec6562c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 -- 172.21.15.78:0/75717968 shutdown_connections 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 -- 172.21.15.78:0/75717968 wait complete. 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 -- start start 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.035+0000 7fede98de740 1 Processor -- start 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede98de740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65641f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede98de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec65642480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede98de740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec656429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede98de740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fede01b0100 con 0x55ec65624060 2024-09-16T00:24:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede98de740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fede01afa30 con 0x55ec6562f240 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede98de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fede01acc70 con 0x55ec65624980 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede98de740 10 client.0 _ll_register_callbacks cb 0x7fedd0017600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65641f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65641f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39134/0 (socket says 172.21.15.78:39134) 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 -- 172.21.15.78:0/2083030889 learned_addr learned my addr 172.21.15.78:0/2083030889 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 -- 172.21.15.78:0/2083030889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 msgr2=0x55ec65642480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec65642480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 -- 172.21.15.78:0/2083030889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 msgr2=0x55ec656429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec656429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 -- 172.21.15.78:0/2083030889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec65626ed0 con 0x55ec65624060 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fede4aa0640 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65641f40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fedd00176d0 tx=0x7fedd0017400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930715ac2025aa13 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fedc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2083030889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd003a020 con 0x55ec65624060 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.036+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2083030889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fedd0018b00 con 0x55ec65624060 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 1 -- 172.21.15.78:0/2083030889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd001d040 con 0x55ec65624060 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2083030889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fedd0045050 con 0x55ec65624060 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 10 client.17493 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 1 -- 172.21.15.78:0/2083030889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ec656a97a0 con 0x55ec65624060 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2083030889 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd001d040 con 0x55ec65624060 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2083030889 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fedd0036440 con 0x55ec65624060 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 10 client.17493 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 10 client.17493 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 20 client.17493 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ec656a9e00 0x55ec656ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 4 client.17493 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d73a29d3", "release": true, "await": true}] 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede98de740 1 -- 172.21.15.78:0/2083030889 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d73a29d3", "release": true, "await": true}) -- 0x55ec655910a0 con 0x55ec656a9e00 2024-09-16T00:24:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.037+0000 7fede52a1640 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ec656a9e00 0x55ec656ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:51.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.038+0000 7fedc8ff9640 10 client.17493 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:51.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.038+0000 7fede52a1640 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ec656a9e00 0x55ec656ac1f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f649b0ece6f4a41 server_cookie=6cd31f7a7e2480c8 in_seq=0 out_seq=0 2024-09-16T00:24:51.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fedc8ff9640 1 -- 172.21.15.78:0/2083030889 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ec655910a0 con 0x55ec656a9e00 2024-09-16T00:24:51.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fedc8ff9640 10 client.17493 handle_command_reply: tid=0 2024-09-16T00:24:51.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 client.17493 shutdown 2024-09-16T00:24:51.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 -- 172.21.15.78:0/2083030889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 msgr2=0x55ec65641f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65641f40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fedd00176d0 tx=0x7fedd0017400 comp rx=0 tx=0).stop 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 -- 172.21.15.78:0/2083030889 shutdown_connections 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ec656a9e00 0x55ec656ac1f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fedc8ff9640 0 client.17493 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ec6562f240 0x55ec656429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec65624980 0x55ec65642480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 --2- 172.21.15.78:0/2083030889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ec65624060 0x55ec65641f40 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 -- 172.21.15.78:0/2083030889 >> 172.21.15.78:0/2083030889 conn(0x55ec656205f0 msgr2=0x55ec6562c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 -- 172.21.15.78:0/2083030889 shutdown_connections 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 1 -- 172.21.15.78:0/2083030889 wait complete. 2024-09-16T00:24:51.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.137+0000 7fede98de740 20 client.17493 trim_cache size 0 max 16384 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 40, 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "d73a29d3": { 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 40, 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:24:51.140 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 141.8, 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:51.141 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:51.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:51.142 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:24:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 -- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fedcc0a32c0 msgr2=0x7fedcc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fedcc0a32c0 0x7fedcc0a5780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fedd8097ed0 tx=0x7fedd806a240 comp rx=0 tx=0).stop 2024-09-16T00:24:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 -- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0102ef0 msgr2=0x7fede01a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0102ef0 0x7fede01a4530 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b970 tx=0x7fedd800bd70 comp rx=0 tx=0).stop 2024-09-16T00:24:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 -- 172.21.15.78:0/671006644 shutdown_connections 2024-09-16T00:24:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fedcc0a32c0 0x7fedcc0a5780 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fede0102ef0 0x7fede01a4530 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fede0102570 0x7fede01999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 --2- 172.21.15.78:0/671006644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fede0100e20 0x7fede01994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.138+0000 7fede7505640 1 -- 172.21.15.78:0/671006644 >> 172.21.15.78:0/671006644 conn(0x7fede00fc7d0 msgr2=0x7fede00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.139+0000 7fede7505640 1 -- 172.21.15.78:0/671006644 shutdown_connections 2024-09-16T00:24:51.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:51.139+0000 7fede7505640 1 -- 172.21.15.78:0/671006644 wait complete. 2024-09-16T00:24:52.022 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd73a29d3', total seconds elapsed: 26.4 2024-09-16T00:24:52.022 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-16T00:24:52.022 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:24:52.055 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:51 smithi078 ceph-mon[31157]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:24:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:51 smithi193 ceph-mon[35484]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:24:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:51 smithi148 ceph-mon[35599]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-16T00:24:52.456 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:53.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.070+0000 7fee53124640 1 -- 172.21.15.78:0/2457981480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4c109450 msgr2=0x7fee4c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:53.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.070+0000 7fee53124640 1 --2- 172.21.15.78:0/2457981480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4c109450 0x7fee4c110250 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009ca0 tx=0x7fee3c02a8f0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.071+0000 7fee53124640 1 -- 172.21.15.78:0/2457981480 shutdown_connections 2024-09-16T00:24:53.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.071+0000 7fee53124640 1 --2- 172.21.15.78:0/2457981480 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee4c11b140 0x7fee4c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.071+0000 7fee53124640 1 --2- 172.21.15.78:0/2457981480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4c109830 0x7fee4c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.071+0000 7fee53124640 1 --2- 172.21.15.78:0/2457981480 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4c109450 0x7fee4c110250 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.071+0000 7fee53124640 1 -- 172.21.15.78:0/2457981480 >> 172.21.15.78:0/2457981480 conn(0x7fee4c06e140 msgr2=0x7fee4c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:53.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.077+0000 7fee53124640 1 -- 172.21.15.78:0/2457981480 shutdown_connections 2024-09-16T00:24:53.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.078+0000 7fee53124640 1 -- 172.21.15.78:0/2457981480 wait complete. 2024-09-16T00:24:53.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.078+0000 7fee53124640 1 Processor -- start 2024-09-16T00:24:53.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.079+0000 7fee53124640 1 -- start start 2024-09-16T00:24:53.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.079+0000 7fee53124640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee4c109450 0x7fee4c1184b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:53.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.079+0000 7fee53124640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4c109830 0x7fee4c1189f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:53.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.079+0000 7fee53124640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4c11b140 0x7fee4c111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:53.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.079+0000 7fee53124640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c120df0 con 0x7fee4c109450 2024-09-16T00:24:53.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.079+0000 7fee53124640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c120c70 con 0x7fee4c11b140 2024-09-16T00:24:53.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.079+0000 7fee53124640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c120f70 con 0x7fee4c109830 2024-09-16T00:24:53.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.081+0000 7fee4bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4c109830 0x7fee4c1189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:53.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.081+0000 7fee4bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4c109830 0x7fee4c1189f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:45432/0 (socket says 172.21.15.78:45432) 2024-09-16T00:24:53.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.081+0000 7fee4bfff640 1 -- 172.21.15.78:0/1741061188 learned_addr learned my addr 172.21.15.78:0/1741061188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:53.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.081+0000 7fee50ec0640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee4c109450 0x7fee4c1184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.081+0000 7fee516c1640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4c11b140 0x7fee4c111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.082+0000 7fee4bfff640 1 -- 172.21.15.78:0/1741061188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4c11b140 msgr2=0x7fee4c111540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.082+0000 7fee4bfff640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4c11b140 0x7fee4c111540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.082+0000 7fee4bfff640 1 -- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee4c109450 msgr2=0x7fee4c1184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.082+0000 7fee4bfff640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee4c109450 0x7fee4c1184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.082+0000 7fee4bfff640 1 -- 172.21.15.78:0/1741061188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c111de0 con 0x7fee4c109830 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.082+0000 7fee4bfff640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4c109830 0x7fee4c1189f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee3400d950 tx=0x7fee3400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.083+0000 7fee49ffb640 1 -- 172.21.15.78:0/1741061188 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34017070 con 0x7fee4c109830 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.083+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c1120d0 con 0x7fee4c109830 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.083+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c1161b0 con 0x7fee4c109830 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.084+0000 7fee49ffb640 1 -- 172.21.15.78:0/1741061188 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee34009c30 con 0x7fee4c109830 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.084+0000 7fee49ffb640 1 -- 172.21.15.78:0/1741061188 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3400f6b0 con 0x7fee4c109830 2024-09-16T00:24:53.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.085+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c005290 con 0x7fee4c109830 2024-09-16T00:24:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.088+0000 7fee49ffb640 1 -- 172.21.15.78:0/1741061188 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fee3400f850 con 0x7fee4c109830 2024-09-16T00:24:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.089+0000 7fee49ffb640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee380a3450 0x7fee380a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:53.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.089+0000 7fee49ffb640 1 -- 172.21.15.78:0/1741061188 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fee340cf6d0 con 0x7fee4c109830 2024-09-16T00:24:53.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.092+0000 7fee49ffb640 1 -- 172.21.15.78:0/1741061188 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee3409c130 con 0x7fee4c109830 2024-09-16T00:24:53.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.094+0000 7fee50ec0640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee380a3450 0x7fee380a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:53.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.098+0000 7fee50ec0640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee380a3450 0x7fee380a5910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fee3c02aee0 tx=0x7fee3c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:53.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.475+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee2c005580 con 0x7fee4c109830 2024-09-16T00:24:53.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.478+0000 7fee49ffb640 1 -- 172.21.15.78:0/1741061188 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fee34009de0 con 0x7fee4c109830 2024-09-16T00:24:53.479 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:24:53.479 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:24:53.479 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:24:53.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.484+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee380a3450 msgr2=0x7fee380a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:53.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.484+0000 7fee53124640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee380a3450 0x7fee380a5910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fee3c02aee0 tx=0x7fee3c000f40 comp rx=0 tx=0).stop 2024-09-16T00:24:53.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.484+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4c109830 msgr2=0x7fee4c1189f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:53.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.484+0000 7fee53124640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4c109830 0x7fee4c1189f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee3400d950 tx=0x7fee3400de10 comp rx=0 tx=0).stop 2024-09-16T00:24:53.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.487+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 shutdown_connections 2024-09-16T00:24:53.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.487+0000 7fee53124640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fee380a3450 0x7fee380a5910 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.487+0000 7fee53124640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fee4c11b140 0x7fee4c111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.487+0000 7fee53124640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4c109830 0x7fee4c1189f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.487+0000 7fee53124640 1 --2- 172.21.15.78:0/1741061188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee4c109450 0x7fee4c1184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:53.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.487+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 >> 172.21.15.78:0/1741061188 conn(0x7fee4c06e140 msgr2=0x7fee4c11c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:53.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.489+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 shutdown_connections 2024-09-16T00:24:53.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:53.490+0000 7fee53124640 1 -- 172.21.15.78:0/1741061188 wait complete. 2024-09-16T00:24:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:53 smithi193 ceph-mon[35484]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 511 KiB/s wr, 28 op/s 2024-09-16T00:24:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:53 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1741061188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:53 smithi148 ceph-mon[35599]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 511 KiB/s wr, 28 op/s 2024-09-16T00:24:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:53 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1741061188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:54.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:53 smithi078 ceph-mon[31157]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 511 KiB/s wr, 28 op/s 2024-09-16T00:24:54.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:53 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1741061188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:24:54.342 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-16T00:24:54.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-16T00:24:54.928 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:55.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 -- 172.21.15.78:0/3361660700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c100a70 msgr2=0x7f4a3c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/3361660700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c100a70 0x7f4a3c100ed0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009930 tx=0x7f4a2802a420 comp rx=0 tx=0).stop 2024-09-16T00:24:55.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 -- 172.21.15.78:0/3361660700 shutdown_connections 2024-09-16T00:24:55.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/3361660700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c101410 0x7f4a3c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/3361660700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c100a70 0x7f4a3c100ed0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/3361660700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c106a70 0x7f4a3c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 -- 172.21.15.78:0/3361660700 >> 172.21.15.78:0/3361660700 conn(0x7f4a3c0fc790 msgr2=0x7f4a3c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:55.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 -- 172.21.15.78:0/3361660700 shutdown_connections 2024-09-16T00:24:55.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.571+0000 7f4a42dd4640 1 -- 172.21.15.78:0/3361660700 wait complete. 2024-09-16T00:24:55.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.572+0000 7f4a42dd4640 1 Processor -- start 2024-09-16T00:24:55.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.572+0000 7f4a42dd4640 1 -- start start 2024-09-16T00:24:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.572+0000 7f4a42dd4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a70 0x7f4a3c1079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.572+0000 7f4a42dd4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c101410 0x7f4a3c107ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.572+0000 7f4a42dd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a70 0x7f4a3c108420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.572+0000 7f4a42dd4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c110e70 con 0x7f4a3c106a70 2024-09-16T00:24:55.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.572+0000 7f4a42dd4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c110cf0 con 0x7f4a3c101410 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.572+0000 7f4a42dd4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c110ff0 con 0x7f4a3c100a70 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.573+0000 7f4a41371640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a70 0x7f4a3c108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.573+0000 7f4a41371640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a70 0x7f4a3c108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39186/0 (socket says 172.21.15.78:39186) 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.573+0000 7f4a41371640 1 -- 172.21.15.78:0/144020937 learned_addr learned my addr 172.21.15.78:0/144020937 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.573+0000 7f4a40b70640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a70 0x7f4a3c1079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.574+0000 7f4a41371640 1 -- 172.21.15.78:0/144020937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a70 msgr2=0x7f4a3c1079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.574+0000 7f4a41371640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a70 0x7f4a3c1079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.574+0000 7f4a41371640 1 -- 172.21.15.78:0/144020937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c101410 msgr2=0x7f4a3c107ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.574+0000 7f4a41371640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c101410 0x7f4a3c107ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.574+0000 7f4a41371640 1 -- 172.21.15.78:0/144020937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c1aad20 con 0x7f4a3c106a70 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.574+0000 7f4a41371640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a70 0x7f4a3c108420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400eb40 tx=0x7f4a3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:55.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.575+0000 7f4a39ffb640 1 -- 172.21.15.78:0/144020937 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34004b20 con 0x7f4a3c106a70 2024-09-16T00:24:55.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.575+0000 7f4a42dd4640 1 -- 172.21.15.78:0/144020937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c19a440 con 0x7f4a3c106a70 2024-09-16T00:24:55.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.575+0000 7f4a42dd4640 1 -- 172.21.15.78:0/144020937 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c19a980 con 0x7f4a3c106a70 2024-09-16T00:24:55.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.575+0000 7f4a39ffb640 1 -- 172.21.15.78:0/144020937 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a34004cc0 con 0x7f4a3c106a70 2024-09-16T00:24:55.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.575+0000 7f4a39ffb640 1 -- 172.21.15.78:0/144020937 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a340217c0 con 0x7f4a3c106a70 2024-09-16T00:24:55.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.582+0000 7f4a39ffb640 1 -- 172.21.15.78:0/144020937 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4a34005020 con 0x7f4a3c106a70 2024-09-16T00:24:55.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.583+0000 7f4a39ffb640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a2c0a3410 0x7f4a2c0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.583+0000 7f4a39ffb640 1 -- 172.21.15.78:0/144020937 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4a340e11b0 con 0x7f4a3c106a70 2024-09-16T00:24:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.584+0000 7f4a451ad740 1 Processor -- start 2024-09-16T00:24:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.584+0000 7f4a451ad740 1 -- start start 2024-09-16T00:24:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.584+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e31f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.585+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.585+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3210e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.585+0000 7f4a451ad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630e3157160 con 0x5630e320ea40 2024-09-16T00:24:55.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.585+0000 7f4a451ad740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630e30fe720 con 0x5630e31f3030 2024-09-16T00:24:55.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.585+0000 7f4a451ad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5630e3153120 con 0x5630e31f2710 2024-09-16T00:24:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.589+0000 7f4a41371640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3210e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.589+0000 7f4a41371640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3210e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39196/0 (socket says 172.21.15.78:39196) 2024-09-16T00:24:55.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.589+0000 7f4a41371640 1 -- 172.21.15.78:0/1871840157 learned_addr learned my addr 172.21.15.78:0/1871840157 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.589+0000 7f4a40b70640 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e31f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.589+0000 7f4a40b70640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a2c0a3410 0x7f4a2c0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.589+0000 7f4a3bfff640 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.594+0000 7f4a41371640 1 -- 172.21.15.78:0/1871840157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 msgr2=0x5630e31f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.594+0000 7f4a41371640 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e31f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.594+0000 7f4a41371640 1 -- 172.21.15.78:0/1871840157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 msgr2=0x5630e320e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.595+0000 7f4a41371640 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.595+0000 7f4a41371640 1 -- 172.21.15.78:0/1871840157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e2ec8ba0 con 0x5630e320ea40 2024-09-16T00:24:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.596+0000 7f4a40b70640 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e31f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:24:55.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.597+0000 7f4a3bfff640 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:24:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.600+0000 7f4a41371640 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3210e30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4a340a9de0 tx=0x7f4a340ad2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28402e55fa441372 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.601+0000 7f4a40b70640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a2c0a3410 0x7f4a2c0a58d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4a30005d00 tx=0x7f4a30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.606+0000 7f4a26ffd640 1 -- 172.21.15.78:0/1871840157 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a340ad410 con 0x5630e320ea40 2024-09-16T00:24:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.606+0000 7f4a26ffd640 1 -- 172.21.15.78:0/1871840157 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a340b17c0 con 0x5630e320ea40 2024-09-16T00:24:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.606+0000 7f4a26ffd640 1 -- 172.21.15.78:0/1871840157 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a340ad670 con 0x5630e320ea40 2024-09-16T00:24:55.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.608+0000 7f4a451ad740 1 -- 172.21.15.78:0/1871840157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 msgr2=0x5630e3210e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.608+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3210e30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4a340a9de0 tx=0x7f4a340ad2a0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.613+0000 7f4a451ad740 1 -- 172.21.15.78:0/1871840157 shutdown_connections 2024-09-16T00:24:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.613+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3210e30 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.613+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.613+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1871840157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e31f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.613+0000 7f4a451ad740 1 -- 172.21.15.78:0/1871840157 >> 172.21.15.78:0/1871840157 conn(0x5630e31eede0 msgr2=0x5630e31ef1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:55.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.618+0000 7f4a451ad740 1 -- 172.21.15.78:0/1871840157 shutdown_connections 2024-09-16T00:24:55.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.618+0000 7f4a451ad740 1 -- 172.21.15.78:0/1871840157 wait complete. 2024-09-16T00:24:55.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.618+0000 7f4a451ad740 1 -- start start 2024-09-16T00:24:55.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.618+0000 7f4a451ad740 1 Processor -- start 2024-09-16T00:24:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.619+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e320b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.619+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.619+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e320c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.619+0000 7f4a451ad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630e3211610 con 0x5630e320ea40 2024-09-16T00:24:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.619+0000 7f4a451ad740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630e30df900 con 0x5630e31f3030 2024-09-16T00:24:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.619+0000 7f4a451ad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5630e3211790 con 0x5630e31f2710 2024-09-16T00:24:55.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.619+0000 7f4a451ad740 10 client.0 _ll_register_callbacks cb 0x5630e30dd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.620+0000 7f4a41371640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e320c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.621+0000 7f4a41371640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e320c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39206/0 (socket says 172.21.15.78:39206) 2024-09-16T00:24:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.621+0000 7f4a41371640 1 -- 172.21.15.78:0/1418994091 learned_addr learned my addr 172.21.15.78:0/1418994091 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:55.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.623+0000 7f4a3bfff640 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.626+0000 7f4a41371640 1 -- 172.21.15.78:0/1418994091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 msgr2=0x5630e320b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.626+0000 7f4a41371640 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e320b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.626+0000 7f4a41371640 1 -- 172.21.15.78:0/1418994091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 msgr2=0x5630e320bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.626+0000 7f4a41371640 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.626+0000 7f4a41371640 1 -- 172.21.15.78:0/1418994091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e311fc50 con 0x5630e320ea40 2024-09-16T00:24:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.626+0000 7f4a41371640 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e320c470 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4a340a9820 tx=0x7f4a340ace60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc747be395144a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:55.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.627+0000 7f4a1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:24:55.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.628+0000 7f4a1bfff640 1 -- 172.21.15.78:0/1418994091 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a340b6080 con 0x5630e320ea40 2024-09-16T00:24:55.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.628+0000 7f4a451ad740 1 -- 172.21.15.78:0/1418994091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a340b17c0 con 0x5630e320ea40 2024-09-16T00:24:55.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.628+0000 7f4a1bfff640 1 -- 172.21.15.78:0/1418994091 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4a340ad880 con 0x5630e320ea40 2024-09-16T00:24:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.632+0000 7f4a1bfff640 1 -- 172.21.15.78:0/1418994091 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4a340ad410 con 0x5630e320ea40 2024-09-16T00:24:55.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.636+0000 7f4a451ad740 10 client.17511 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:55.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.636+0000 7f4a451ad740 1 -- 172.21.15.78:0/1418994091 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5630e321d450 con 0x5630e320ea40 2024-09-16T00:24:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.643+0000 7f4a1bfff640 1 -- 172.21.15.78:0/1418994091 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a340b17c0 con 0x5630e320ea40 2024-09-16T00:24:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.643+0000 7f4a1bfff640 1 -- 172.21.15.78:0/1418994091 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4a340a91a0 con 0x5630e320ea40 2024-09-16T00:24:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.643+0000 7f4a451ad740 10 client.17511 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.643+0000 7f4a451ad740 10 client.17511 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:24:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.643+0000 7f4a451ad740 20 client.17511 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.643+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5630e321e950 0x5630e3220df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.643+0000 7f4a451ad740 4 client.17511 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:24:55.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.643+0000 7f4a451ad740 1 -- 172.21.15.78:0/1418994091 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630e3164890 con 0x5630e321e950 2024-09-16T00:24:55.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.644+0000 7f4a40b70640 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5630e321e950 0x5630e3220df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.648+0000 7f4a40b70640 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5630e321e950 0x5630e3220df0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6266f770a263c533 server_cookie=239fc60a0e8704bb in_seq=0 out_seq=0 2024-09-16T00:24:55.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.648+0000 7f4a1bfff640 10 client.17511 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:55.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.654+0000 7f4a1bfff640 1 -- 172.21.15.78:0/1418994091 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630e3164890 con 0x5630e321e950 2024-09-16T00:24:55.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.654+0000 7f4a1bfff640 10 client.17511 handle_command_reply: tid=0 2024-09-16T00:24:55.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.654+0000 7f4a451ad740 1 client.17511 shutdown 2024-09-16T00:24:55.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a451ad740 1 -- 172.21.15.78:0/1418994091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 msgr2=0x5630e320c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e320c470 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f4a340a9820 tx=0x7f4a340ace60 comp rx=0 tx=0).stop 2024-09-16T00:24:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a451ad740 1 -- 172.21.15.78:0/1418994091 shutdown_connections 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5630e321e950 0x5630e3220df0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e320c470 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e320bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a451ad740 1 --2- 172.21.15.78:0/1418994091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e320b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a451ad740 1 -- 172.21.15.78:0/1418994091 >> 172.21.15.78:0/1418994091 conn(0x5630e31eede0 msgr2=0x5630e31f4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.658+0000 7f4a1bfff640 0 client.17511 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.659+0000 7f4a451ad740 1 -- 172.21.15.78:0/1418994091 shutdown_connections 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.659+0000 7f4a451ad740 1 -- 172.21.15.78:0/1418994091 wait complete. 2024-09-16T00:24:55.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.659+0000 7f4a451ad740 20 client.17511 trim_cache size 0 max 16384 2024-09-16T00:24:55.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.693+0000 7f4a451ad740 1 Processor -- start 2024-09-16T00:24:55.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.693+0000 7f4a451ad740 1 -- start start 2024-09-16T00:24:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.693+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f2710 0x5630e31e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.694+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e31f3030 0x5630e32266c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.694+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e320ea40 0x5630e3226c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.694+0000 7f4a451ad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2000fb70 con 0x5630e31f3030 2024-09-16T00:24:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.694+0000 7f4a451ad740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a200116b0 con 0x5630e31f2710 2024-09-16T00:24:55.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.694+0000 7f4a451ad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20010240 con 0x5630e320ea40 2024-09-16T00:24:55.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.694+0000 7f4a40b70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f2710 0x5630e31e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.694+0000 7f4a40b70640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f2710 0x5630e31e9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:53142/0 (socket says 172.21.15.78:53142) 2024-09-16T00:24:55.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.694+0000 7f4a40b70640 1 -- 172.21.15.78:0/120446639 learned_addr learned my addr 172.21.15.78:0/120446639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:55.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.695+0000 7f4a3bfff640 1 --2- 172.21.15.78:0/120446639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e31f3030 0x5630e32266c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.697+0000 7f4a40b70640 1 -- 172.21.15.78:0/120446639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e320ea40 msgr2=0x5630e3226c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:24:55.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.697+0000 7f4a40b70640 1 --2- 172.21.15.78:0/120446639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e320ea40 0x5630e3226c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.697+0000 7f4a40b70640 1 -- 172.21.15.78:0/120446639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e31f3030 msgr2=0x5630e32266c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:55.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.697+0000 7f4a40b70640 1 --2- 172.21.15.78:0/120446639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e31f3030 0x5630e32266c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.697+0000 7f4a40b70640 1 -- 172.21.15.78:0/120446639 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e320a9a0 con 0x5630e31f2710 2024-09-16T00:24:55.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.699+0000 7f4a40b70640 1 --2- 172.21.15.78:0/120446639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f2710 0x5630e31e9d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5630e3221630 tx=0x7f4a300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5670af7ca692d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:55.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.700+0000 7f4a267fc640 1 -- 172.21.15.78:0/120446639 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30014200 con 0x5630e31f2710 2024-09-16T00:24:55.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.700+0000 7f4a267fc640 1 -- 172.21.15.78:0/120446639 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a300054d0 con 0x5630e31f2710 2024-09-16T00:24:55.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.700+0000 7f4a267fc640 1 -- 172.21.15.78:0/120446639 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30003a10 con 0x5630e31f2710 2024-09-16T00:24:55.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.702+0000 7f4a451ad740 1 -- 172.21.15.78:0/120446639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f2710 msgr2=0x5630e31e9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.702+0000 7f4a451ad740 1 --2- 172.21.15.78:0/120446639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f2710 0x5630e31e9d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5630e3221630 tx=0x7f4a300046c0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.702+0000 7f4a451ad740 1 -- 172.21.15.78:0/120446639 shutdown_connections 2024-09-16T00:24:55.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.702+0000 7f4a451ad740 1 --2- 172.21.15.78:0/120446639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e320ea40 0x5630e3226c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.702+0000 7f4a451ad740 1 --2- 172.21.15.78:0/120446639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e31f3030 0x5630e32266c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.702+0000 7f4a451ad740 1 --2- 172.21.15.78:0/120446639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f2710 0x5630e31e9d30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.702+0000 7f4a451ad740 1 -- 172.21.15.78:0/120446639 >> 172.21.15.78:0/120446639 conn(0x5630e31eede0 msgr2=0x5630e320c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:55.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.704+0000 7f4a451ad740 1 -- 172.21.15.78:0/120446639 shutdown_connections 2024-09-16T00:24:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.704+0000 7f4a451ad740 1 -- 172.21.15.78:0/120446639 wait complete. 2024-09-16T00:24:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.704+0000 7f4a451ad740 1 -- start start 2024-09-16T00:24:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.704+0000 7f4a451ad740 1 Processor -- start 2024-09-16T00:24:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.705+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e32129b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.705+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e3212ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.705+0000 7f4a451ad740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3213430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.705+0000 7f4a451ad740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20010fe0 con 0x5630e320ea40 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.705+0000 7f4a451ad740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a20010910 con 0x5630e31f3030 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.705+0000 7f4a451ad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2000db50 con 0x5630e31f2710 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.705+0000 7f4a451ad740 10 client.0 _ll_register_callbacks cb 0x7f4a340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.705+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e3212ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.706+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e3212ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:53156/0 (socket says 172.21.15.78:53156) 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.706+0000 7f4a3bfff640 1 -- 172.21.15.78:0/4107815461 learned_addr learned my addr 172.21.15.78:0/4107815461 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.706+0000 7f4a3bfff640 1 -- 172.21.15.78:0/4107815461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 msgr2=0x5630e32129b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.706+0000 7f4a3bfff640 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e32129b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.706+0000 7f4a3bfff640 1 -- 172.21.15.78:0/4107815461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 msgr2=0x5630e3213430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:24:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.706+0000 7f4a3bfff640 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3213430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.706+0000 7f4a3bfff640 1 -- 172.21.15.78:0/4107815461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e31f2e60 con 0x5630e31f3030 2024-09-16T00:24:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.707+0000 7f4a3bfff640 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e3212ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009230 tx=0x7f4a28004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc237341266f38d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.707+0000 7f4a1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:24:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.708+0000 7f4a1bfff640 1 -- 172.21.15.78:0/4107815461 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28035040 con 0x5630e31f3030 2024-09-16T00:24:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.708+0000 7f4a451ad740 1 -- 172.21.15.78:0/4107815461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a300054d0 con 0x5630e31f3030 2024-09-16T00:24:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.708+0000 7f4a1bfff640 1 -- 172.21.15.78:0/4107815461 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f4a28005240 con 0x5630e31f3030 2024-09-16T00:24:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.708+0000 7f4a1bfff640 1 -- 172.21.15.78:0/4107815461 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4a2804f050 con 0x5630e31f3030 2024-09-16T00:24:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.708+0000 7f4a451ad740 10 client.26783 fetch_fsmap learned FSMap version 34 2024-09-16T00:24:55.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.709+0000 7f4a451ad740 1 -- 172.21.15.78:0/4107815461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5630e32898f0 con 0x5630e31f3030 2024-09-16T00:24:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a1bfff640 1 -- 172.21.15.78:0/4107815461 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a300054d0 con 0x5630e31f3030 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a1bfff640 1 -- 172.21.15.78:0/4107815461 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4a28044020 con 0x5630e31f3030 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a451ad740 10 client.26783 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a451ad740 10 client.26783 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a451ad740 20 client.26783 populate_metadata read hostname 'smithi078' 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a451ad740 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5630e328a0c0 0x5630e328c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a451ad740 4 client.26783 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a451ad740 1 -- 172.21.15.78:0/4107815461 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x5630e315ebe0 con 0x5630e328a0c0 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.710+0000 7f4a40b70640 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5630e328a0c0 0x5630e328c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.711+0000 7f4a40b70640 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5630e328a0c0 0x5630e328c4e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cadfe02134775c4 server_cookie=f4ca2600b3e54c85 in_seq=0 out_seq=0 2024-09-16T00:24:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.711+0000 7f4a1bfff640 10 client.26783 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 655.0 s, 523.5 tps, lat 1013.701 ms stddev 4216.191 2024-09-16T00:24:55.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a1bfff640 1 -- 172.21.15.78:0/4107815461 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5630e315ebe0 con 0x5630e328a0c0 2024-09-16T00:24:55.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a1bfff640 10 client.26783 handle_command_reply: tid=0 2024-09-16T00:24:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 client.26783 shutdown 2024-09-16T00:24:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 -- 172.21.15.78:0/4107815461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 msgr2=0x5630e3212ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e3212ef0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009230 tx=0x7f4a28004250 comp rx=0 tx=0).stop 2024-09-16T00:24:55.967 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:55 smithi078 ceph-mon[31157]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 918 KiB/s wr, 77 op/s 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 -- 172.21.15.78:0/4107815461 shutdown_connections 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5630e328a0c0 0x5630e328c4e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630e320ea40 0x5630e3213430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630e31f3030 0x5630e3212ef0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a1bfff640 0 client.26783 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 --2- 172.21.15.78:0/4107815461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630e31f2710 0x5630e32129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 -- 172.21.15.78:0/4107815461 >> 172.21.15.78:0/4107815461 conn(0x5630e31eede0 msgr2=0x5630e320d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 -- 172.21.15.78:0/4107815461 shutdown_connections 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 1 -- 172.21.15.78:0/4107815461 wait complete. 2024-09-16T00:24:55.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.943+0000 7f4a451ad740 20 client.26783 trim_cache size 0 max 16384 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 42, 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "48333ca2": { 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 42, 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:24:55.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 141.8, 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:55.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:55.971 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:24:55.971 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:24:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 -- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a2c0a3410 msgr2=0x7f4a2c0a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a2c0a3410 0x7f4a2c0a58d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4a30005d00 tx=0x7f4a30005f90 comp rx=0 tx=0).stop 2024-09-16T00:24:55.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 -- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a70 msgr2=0x7f4a3c108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a70 0x7f4a3c108420 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400eb40 tx=0x7f4a3400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 -- 172.21.15.78:0/144020937 shutdown_connections 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f4a2c0a3410 0x7f4a2c0a58d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a3c106a70 0x7f4a3c108420 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a3c101410 0x7f4a3c107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 --2- 172.21.15.78:0/144020937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a3c100a70 0x7f4a3c1079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 -- 172.21.15.78:0/144020937 >> 172.21.15.78:0/144020937 conn(0x7f4a3c0fc790 msgr2=0x7f4a3c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 -- 172.21.15.78:0/144020937 shutdown_connections 2024-09-16T00:24:55.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:24:55.944+0000 7f4a42dd4640 1 -- 172.21.15.78:0/144020937 wait complete. 2024-09-16T00:24:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:55 smithi193 ceph-mon[35484]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 918 KiB/s wr, 77 op/s 2024-09-16T00:24:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:55 smithi148 ceph-mon[35599]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 918 KiB/s wr, 77 op/s 2024-09-16T00:24:57.207 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '48333ca2', quiesce took 5.2 seconds. Will release after: 16.650000000000002 2024-09-16T00:24:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:57 smithi193 ceph-mon[35484]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-09-16T00:24:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:57 smithi148 ceph-mon[35599]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-09-16T00:24:58.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:57 smithi078 ceph-mon[31157]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-09-16T00:25:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:59 smithi193 ceph-mon[35484]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.2 MiB/s wr, 270 op/s 2024-09-16T00:25:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:25:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:25:00.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:24:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:25:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:59 smithi148 ceph-mon[35599]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.2 MiB/s wr, 270 op/s 2024-09-16T00:25:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:25:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:25:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:24:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:25:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:59 smithi078 ceph-mon[31157]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.2 MiB/s wr, 270 op/s 2024-09-16T00:25:00.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:25:00.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:25:00.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:24:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:25:00.860 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 660.0 s, 3.8 tps, lat 395.056 ms stddev 289.342 2024-09-16T00:25:02.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:01 smithi193 ceph-mon[35484]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.6 MiB/s wr, 310 op/s 2024-09-16T00:25:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:01 smithi148 ceph-mon[35599]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.6 MiB/s wr, 310 op/s 2024-09-16T00:25:02.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:01 smithi078 ceph-mon[31157]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.6 MiB/s wr, 310 op/s 2024-09-16T00:25:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:03 smithi193 ceph-mon[35484]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.6 MiB/s wr, 310 op/s 2024-09-16T00:25:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:03 smithi148 ceph-mon[35599]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.6 MiB/s wr, 310 op/s 2024-09-16T00:25:04.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:03 smithi078 ceph-mon[31157]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 3.6 MiB/s wr, 310 op/s 2024-09-16T00:25:05.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 665.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:25:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:05 smithi193 ceph-mon[35484]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-09-16T00:25:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:05 smithi148 ceph-mon[35599]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-09-16T00:25:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:05 smithi078 ceph-mon[31157]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.1 MiB/s wr, 294 op/s 2024-09-16T00:25:08.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:07 smithi193 ceph-mon[35484]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.7 MiB/s wr, 244 op/s 2024-09-16T00:25:08.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:07 smithi148 ceph-mon[35599]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.7 MiB/s wr, 244 op/s 2024-09-16T00:25:08.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:07 smithi078 ceph-mon[31157]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.7 MiB/s wr, 244 op/s 2024-09-16T00:25:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:09 smithi193 ceph-mon[35484]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-16T00:25:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:09 smithi148 ceph-mon[35599]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-16T00:25:10.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:09 smithi078 ceph-mon[31157]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-16T00:25:10.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 670.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:25:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:11 smithi193 ceph-mon[35484]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 349 KiB/s wr, 51 op/s 2024-09-16T00:25:12.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:11 smithi148 ceph-mon[35599]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 349 KiB/s wr, 51 op/s 2024-09-16T00:25:12.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:11 smithi078 ceph-mon[31157]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 349 KiB/s wr, 51 op/s 2024-09-16T00:25:13.858 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:14.116 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:14.127 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:13 smithi148 ceph-mon[35599]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-16T00:25:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:13 smithi193 ceph-mon[35484]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-16T00:25:14.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:13 smithi078 ceph-mon[31157]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-16T00:25:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 -- 172.21.15.78:0/389418898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608101df0 msgr2=0x7f8608105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 --2- 172.21.15.78:0/389418898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608101df0 0x7f8608105920 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8604009900 tx=0x7f860402a410 comp rx=0 tx=0).stop 2024-09-16T00:25:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 -- 172.21.15.78:0/389418898 shutdown_connections 2024-09-16T00:25:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 --2- 172.21.15.78:0/389418898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8608106050 0x7f860810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 --2- 172.21.15.78:0/389418898 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608101df0 0x7f8608105920 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 --2- 172.21.15.78:0/389418898 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8608101440 0x7f8608101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 -- 172.21.15.78:0/389418898 >> 172.21.15.78:0/389418898 conn(0x7f8608077210 msgr2=0x7f86080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 -- 172.21.15.78:0/389418898 shutdown_connections 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.682+0000 7f860f522640 1 -- 172.21.15.78:0/389418898 wait complete. 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860f522640 1 Processor -- start 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860f522640 1 -- start start 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860f522640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8608101440 0x7f86080fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860f522640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8608101df0 0x7f86080fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860f522640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608106050 0x7f86080fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860f522640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86081110d0 con 0x7f8608106050 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860f522640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8608110f50 con 0x7f8608101440 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860f522640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8608111250 con 0x7f8608101df0 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608106050 0x7f86080fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:14.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608106050 0x7f86080fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:34620/0 (socket says 172.21.15.78:34620) 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 -- 172.21.15.78:0/2917341186 learned_addr learned my addr 172.21.15.78:0/2917341186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860d2be640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8608101440 0x7f86080fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860cabd640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8608101df0 0x7f86080fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 -- 172.21.15.78:0/2917341186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8608101df0 msgr2=0x7f86080fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8608101df0 0x7f86080fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 -- 172.21.15.78:0/2917341186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8608101440 msgr2=0x7f86080fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8608101440 0x7f86080fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 -- 172.21.15.78:0/2917341186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86081a6c00 con 0x7f8608106050 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.683+0000 7f860dabf640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608106050 0x7f86080fd110 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f860000b970 tx=0x7f860000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.684+0000 7f85fe7fc640 1 -- 172.21.15.78:0/2917341186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600019070 con 0x7f8608106050 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.684+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86081a6ef0 con 0x7f8608106050 2024-09-16T00:25:14.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.684+0000 7f85fe7fc640 1 -- 172.21.15.78:0/2917341186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860000cb40 con 0x7f8608106050 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.684+0000 7f85fe7fc640 1 -- 172.21.15.78:0/2917341186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600007610 con 0x7f8608106050 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.684+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a7450 con 0x7f8608106050 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.684+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8608101820 con 0x7f8608106050 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.686+0000 7f85fe7fc640 1 -- 172.21.15.78:0/2917341186 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f860001c070 con 0x7f8608106050 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.686+0000 7f85fe7fc640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f85f40a3240 0x7f85f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.686+0000 7f860d2be640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f85f40a3240 0x7f85f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.686+0000 7f85fe7fc640 1 -- 172.21.15.78:0/2917341186 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f86000cf680 con 0x7f8608106050 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.687+0000 7f860d2be640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f85f40a3240 0x7f85f40a5700 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f85f80058c0 tx=0x7f85f800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:14.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.690+0000 7f85fe7fc640 1 -- 172.21.15.78:0/2917341186 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f860009b9f0 con 0x7f8608106050 2024-09-16T00:25:14.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.890+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86080fdbc0 con 0x7f8608106050 2024-09-16T00:25:14.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.891+0000 7f85fe7fc640 1 -- 172.21.15.78:0/2917341186 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f86080fdbc0 con 0x7f8608106050 2024-09-16T00:25:14.891 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:25:14.891 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:25:14.892 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:25:14.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f85f40a3240 msgr2=0x7f85f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:14.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f85f40a3240 0x7f85f40a5700 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f85f80058c0 tx=0x7f85f800a2d0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608106050 msgr2=0x7f86080fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:14.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608106050 0x7f86080fd110 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f860000b970 tx=0x7f860000be30 comp rx=0 tx=0).stop 2024-09-16T00:25:14.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 shutdown_connections 2024-09-16T00:25:14.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f85f40a3240 0x7f85f40a5700 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8608106050 0x7f86080fd110 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8608101df0 0x7f86080fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 --2- 172.21.15.78:0/2917341186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8608101440 0x7f86080fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 >> 172.21.15.78:0/2917341186 conn(0x7f8608077210 msgr2=0x7f8608100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:14.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 shutdown_connections 2024-09-16T00:25:14.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:14.893+0000 7f860f522640 1 -- 172.21.15.78:0/2917341186 wait complete. 2024-09-16T00:25:15.692 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 48333ca2 --release --await' 2024-09-16T00:25:15.693 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 48333ca2 --release --await 2024-09-16T00:25:15.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 675.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:25:15.941 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:16.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:15 smithi078 ceph-mon[31157]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-16T00:25:16.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:15 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2917341186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:16.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:15 smithi193 ceph-mon[35484]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-16T00:25:16.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:15 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2917341186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:16.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:15 smithi148 ceph-mon[35599]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-09-16T00:25:16.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:15 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2917341186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 -- 172.21.15.78:0/3257766978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0106a80 msgr2=0x7f1ee0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/3257766978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0106a80 0x7f1ee0106e60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009930 tx=0x7f1ed002a420 comp rx=0 tx=0).stop 2024-09-16T00:25:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 -- 172.21.15.78:0/3257766978 shutdown_connections 2024-09-16T00:25:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/3257766978 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee0101420 0x7f1ee010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/3257766978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee0100a80 0x7f1ee0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/3257766978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0106a80 0x7f1ee0106e60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 -- 172.21.15.78:0/3257766978 >> 172.21.15.78:0/3257766978 conn(0x7f1ee00fc7a0 msgr2=0x7f1ee00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 -- 172.21.15.78:0/3257766978 shutdown_connections 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.472+0000 7f1ee66a4640 1 -- 172.21.15.78:0/3257766978 wait complete. 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.473+0000 7f1ee66a4640 1 Processor -- start 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.473+0000 7f1ee66a4640 1 -- start start 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.473+0000 7f1ee66a4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee0100a80 0x7f1ee01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.473+0000 7f1ee66a4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0101420 0x7f1ee01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.473+0000 7f1ee66a4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee0106a80 0x7f1ee01a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.473+0000 7f1ee66a4640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0110e80 con 0x7f1ee0101420 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.473+0000 7f1ee66a4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0110d00 con 0x7f1ee0100a80 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.473+0000 7f1ee66a4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0111000 con 0x7f1ee0106a80 2024-09-16T00:25:16.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0101420 0x7f1ee01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0101420 0x7f1ee01997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50606/0 (socket says 172.21.15.78:50606) 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 -- 172.21.15.78:0/1677643777 learned_addr learned my addr 172.21.15.78:0/1677643777 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee0106a80 0x7f1ee01a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 -- 172.21.15.78:0/1677643777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee0106a80 msgr2=0x7f1ee01a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee0106a80 0x7f1ee01a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edffff640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee0100a80 0x7f1ee01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 -- 172.21.15.78:0/1677643777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee0100a80 msgr2=0x7f1ee01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee0100a80 0x7f1ee01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 -- 172.21.15.78:0/1677643777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee01a4870 con 0x7f1ee0101420 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edffff640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee0100a80 0x7f1ee01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0101420 0x7f1ee01997f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400d770 tx=0x7f1ed400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edd7fa640 1 -- 172.21.15.78:0/1677643777 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4017070 con 0x7f1ee0101420 2024-09-16T00:25:16.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1ee66a4640 1 -- 172.21.15.78:0/1677643777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee01a4a40 con 0x7f1ee0101420 2024-09-16T00:25:16.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edd7fa640 1 -- 172.21.15.78:0/1677643777 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed4005500 con 0x7f1ee0101420 2024-09-16T00:25:16.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1ee66a4640 1 -- 172.21.15.78:0/1677643777 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee01a4f50 con 0x7f1ee0101420 2024-09-16T00:25:16.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.474+0000 7f1edd7fa640 1 -- 172.21.15.78:0/1677643777 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4004e00 con 0x7f1ee0101420 2024-09-16T00:25:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.476+0000 7f1edd7fa640 1 -- 172.21.15.78:0/1677643777 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1ed4004fa0 con 0x7f1ee0101420 2024-09-16T00:25:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee8a7d740 1 Processor -- start 2024-09-16T00:25:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee8a7d740 1 -- start start 2024-09-16T00:25:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019bed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a501919620 con 0x55a5019b4980 2024-09-16T00:25:16.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a5018c0e50 con 0x55a5019bf240 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a5019155e0 con 0x55a5019b4060 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1edffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1edd7fa640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ec80a3240 0x7f1ec80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1edffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019bed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50608/0 (socket says 172.21.15.78:50608) 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1edffff640 1 -- 172.21.15.78:0/3002151267 learned_addr learned my addr 172.21.15.78:0/3002151267 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1edffff640 1 -- 172.21.15.78:0/3002151267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 msgr2=0x55a5019b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.477+0000 7f1edffff640 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1edffff640 1 -- 172.21.15.78:0/3002151267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 msgr2=0x55a5019c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1edffff640 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1edffff640 1 -- 172.21.15.78:0/3002151267 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5018b1d40 con 0x55a5019b4980 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1edd7fa640 1 -- 172.21.15.78:0/1677643777 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1ed40cf700 con 0x7f1ee0101420 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ec80a3240 0x7f1ec80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1edffff640 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019bed00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0006010 tx=0x7f1ed0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56866dd44d10dc47 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1ec67fc640 1 -- 172.21.15.78:0/3002151267 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed003f070 con 0x55a5019b4980 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1ec67fc640 1 -- 172.21.15.78:0/3002151267 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed00317b0 con 0x55a5019b4980 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.478+0000 7f1ec67fc640 1 -- 172.21.15.78:0/3002151267 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0035070 con 0x55a5019b4980 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/3002151267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 msgr2=0x55a5019bed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ec80a3240 0x7f1ec80a5700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8007e90 tx=0x7f1ed8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019bed00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0006010 tx=0x7f1ed0009680 comp rx=0 tx=0).stop 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/3002151267 shutdown_connections 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019bed00 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/3002151267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/3002151267 >> 172.21.15.78:0/3002151267 conn(0x55a5019b05f0 msgr2=0x55a5019b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/3002151267 shutdown_connections 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/3002151267 wait complete. 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 -- start start 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.479+0000 7f1ee8a7d740 1 Processor -- start 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4060 0x55a5019bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4980 0x55a5019bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019bf240 0x55a5019bcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5019c1e10 con 0x55a5019b4060 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a5018a1f10 con 0x55a5019b4980 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a5019c1f90 con 0x55a5019bf240 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ee8a7d740 10 client.0 _ll_register_callbacks cb 0x55a5018c5db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4060 0x55a5019bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4060 0x55a5019bc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50610/0 (socket says 172.21.15.78:50610) 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 -- 172.21.15.78:0/2245585838 learned_addr learned my addr 172.21.15.78:0/2245585838 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edf7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4980 0x55a5019bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019bf240 0x55a5019bcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 -- 172.21.15.78:0/2245585838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019bf240 msgr2=0x55a5019bcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019bf240 0x55a5019bcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 -- 172.21.15.78:0/2245585838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4980 msgr2=0x55a5019bc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:16.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4980 0x55a5019bc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 -- 172.21.15.78:0/2245585838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a501628580 con 0x55a5019b4060 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1edffff640 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4060 0x55a5019bc0b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1ed002abd0 tx=0x7f1ed0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddb2a60480a01ac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.480+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2245585838 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed003f070 con 0x55a5019b4060 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2245585838 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1ed0004250 con 0x55a5019b4060 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2245585838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed00317b0 con 0x55a5019b4060 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2245585838 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ed0055050 con 0x55a5019b4060 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 10 client.17532 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2245585838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a5016f2960 con 0x55a5019b4060 2024-09-16T00:25:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2245585838 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed00317b0 con 0x55a5019b4060 2024-09-16T00:25:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2245585838 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1ed0004850 con 0x55a5019b4060 2024-09-16T00:25:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 10 client.17532 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 10 client.17532 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:25:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 20 client.17532 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5019ceb30 0x55a5019d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 4 client.17532 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2245585838 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a501926b90 con 0x55a5019ceb30 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.481+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5019ceb30 0x55a5019d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.482+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5019ceb30 0x55a5019d0f20 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92dd58f038812936 server_cookie=6882f996bb95dc73 in_seq=0 out_seq=0 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.482+0000 7f1ebf7fe640 10 client.17532 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2245585838 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a501926b90 con 0x55a5019ceb30 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ebf7fe640 10 client.17532 handle_command_reply: tid=0 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 client.17532 shutdown 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2245585838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4060 msgr2=0x55a5019bc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4060 0x55a5019bc0b0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1ed002abd0 tx=0x7f1ed0009260 comp rx=0 tx=0).stop 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2245585838 shutdown_connections 2024-09-16T00:25:16.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a5019ceb30 0x55a5019d0f20 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ebf7fe640 0 client.17532 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019bf240 0x55a5019bcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4980 0x55a5019bc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2245585838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4060 0x55a5019bc0b0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2245585838 >> 172.21.15.78:0/2245585838 conn(0x55a5019b05f0 msgr2=0x55a5019af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2245585838 shutdown_connections 2024-09-16T00:25:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2245585838 wait complete. 2024-09-16T00:25:16.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.483+0000 7f1ee8a7d740 20 client.17532 trim_cache size 0 max 16384 2024-09-16T00:25:16.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee8a7d740 1 Processor -- start 2024-09-16T00:25:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee8a7d740 1 -- start start 2024-09-16T00:25:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4060 0x55a5019af640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4980 0x55a5019d6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019bf240 0x55a5019d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01aeb70 con 0x55a5019bf240 2024-09-16T00:25:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01b06b0 con 0x55a5019b4060 2024-09-16T00:25:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01af240 con 0x55a5019b4980 2024-09-16T00:25:16.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee4c41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019bf240 0x55a5019d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee4c41640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019bf240 0x55a5019d6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50620/0 (socket says 172.21.15.78:50620) 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1ee4c41640 1 -- 172.21.15.78:0/798386590 learned_addr learned my addr 172.21.15.78:0/798386590 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1edffff640 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4060 0x55a5019af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.500+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4980 0x55a5019d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ee4c41640 1 -- 172.21.15.78:0/798386590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4980 msgr2=0x55a5019d6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4980 0x55a5019d6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ee4c41640 1 -- 172.21.15.78:0/798386590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4060 msgr2=0x55a5019af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4060 0x55a5019af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ee4c41640 1 -- 172.21.15.78:0/798386590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5019b0090 con 0x55a5019bf240 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4980 0x55a5019d6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019bf240 0x55a5019d6a90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8012770 tx=0x7f1ed8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2271bf1aa9ac73ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ec5ffb640 1 -- 172.21.15.78:0/798386590 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed801c740 con 0x55a5019bf240 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ec5ffb640 1 -- 172.21.15.78:0/798386590 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed801cdc0 con 0x55a5019bf240 2024-09-16T00:25:16.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.501+0000 7f1ec5ffb640 1 -- 172.21.15.78:0/798386590 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80243e0 con 0x55a5019bf240 2024-09-16T00:25:16.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/798386590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019bf240 msgr2=0x55a5019d6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019bf240 0x55a5019d6a90 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8012770 tx=0x7f1ed8013000 comp rx=0 tx=0).stop 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/798386590 shutdown_connections 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019bf240 0x55a5019d6a90 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4980 0x55a5019d6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/798386590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019b4060 0x55a5019af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/798386590 >> 172.21.15.78:0/798386590 conn(0x55a5019b05f0 msgr2=0x55a5019aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/798386590 shutdown_connections 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/798386590 wait complete. 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- start start 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 Processor -- start 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01affe0 con 0x55a5019b4980 2024-09-16T00:25:16.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01af910 con 0x55a5019bf240 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.502+0000 7f1ee8a7d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01acb50 con 0x55a5019b4060 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ee8a7d740 10 client.0 _ll_register_callbacks cb 0x7f1ed002aa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50622/0 (socket says 172.21.15.78:50622) 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 -- 172.21.15.78:0/2709019161 learned_addr learned my addr 172.21.15.78:0/2709019161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edffff640 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 -- 172.21.15.78:0/2709019161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 msgr2=0x55a5019c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 -- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 msgr2=0x55a5019c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:16.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 -- 172.21.15.78:0/2709019161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5019c2840 con 0x55a5019b4980 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ee4c41640 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1edf7fe640 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019c7910 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1ed409b950 tx=0x7f1ed406b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb6594d866c3edf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2709019161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed406a030 con 0x55a5019b4980 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2709019161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1ed4099cc0 con 0x55a5019b4980 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2709019161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed801cdc0 con 0x55a5019b4980 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2709019161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ed4020040 con 0x55a5019b4980 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ee8a7d740 10 client.17538 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.503+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2709019161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a501a39f00 con 0x55a5019b4980 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2709019161 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed801cdc0 con 0x55a5019b4980 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2709019161 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1ed407ece0 con 0x55a5019b4980 2024-09-16T00:25:16.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1ee8a7d740 10 client.17538 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:16.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1ee8a7d740 10 client.17538 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:25:16.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1ee8a7d740 20 client.17538 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:16.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a501a3a580 0x55a501a3c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:16.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1ee8a7d740 4 client.17538 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "48333ca2", "release": true, "await": true}] 2024-09-16T00:25:16.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2709019161 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "48333ca2", "release": true, "await": true}) -- 0x55a5019210a0 con 0x55a501a3a580 2024-09-16T00:25:16.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.504+0000 7f1edffff640 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a501a3a580 0x55a501a3c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:16.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.505+0000 7f1ebf7fe640 10 client.17538 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:16.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.505+0000 7f1edffff640 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a501a3a580 0x55a501a3c970 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f17edfed3b470d01 server_cookie=720f9242f05a4d9c in_seq=0 out_seq=0 2024-09-16T00:25:16.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.562+0000 7f1ebf7fe640 1 -- 172.21.15.78:0/2709019161 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a5019210a0 con 0x55a501a3a580 2024-09-16T00:25:16.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.562+0000 7f1ebf7fe640 10 client.17538 handle_command_reply: tid=0 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.562+0000 7f1ee8a7d740 1 client.17538 shutdown 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.562+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2709019161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 msgr2=0x55a5019c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.562+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019c7910 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1ed409b950 tx=0x7f1ed406b630 comp rx=0 tx=0).stop 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2709019161 shutdown_connections 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55a501a3a580 0x55a501a3c970 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a5019bf240 0x55a5019c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ebf7fe640 0 client.17538 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5019b4980 0x55a5019c7910 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 1 --2- 172.21.15.78:0/2709019161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a5019b4060 0x55a5019c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2709019161 >> 172.21.15.78:0/2709019161 conn(0x55a5019b05f0 msgr2=0x55a5019be480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:16.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2709019161 shutdown_connections 2024-09-16T00:25:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 1 -- 172.21.15.78:0/2709019161 wait complete. 2024-09-16T00:25:16.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.563+0000 7f1ee8a7d740 20 client.17538 trim_cache size 0 max 16384 2024-09-16T00:25:16.565 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:25:16.565 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:25:16.565 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:25:16.565 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 44, 2024-09-16T00:25:16.565 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "48333ca2": { 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 44, 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 141.8, 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:25:16.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:25:16.567 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:25:16.567 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:25:16.567 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:25:16.567 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:25:16.567 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:25:16.567 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:25:16.567 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 -- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ec80a3240 msgr2=0x7f1ec80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ec80a3240 0x7f1ec80a5700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8007e90 tx=0x7f1ed8004a40 comp rx=0 tx=0).stop 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 -- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0101420 msgr2=0x7f1ee01997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0101420 0x7f1ee01997f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400d770 tx=0x7f1ed400dc30 comp rx=0 tx=0).stop 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 -- 172.21.15.78:0/1677643777 shutdown_connections 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1ec80a3240 0x7f1ec80a5700 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee0106a80 0x7f1ee01a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ee0101420 0x7f1ee01997f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 --2- 172.21.15.78:0/1677643777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ee0100a80 0x7f1ee01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 -- 172.21.15.78:0/1677643777 >> 172.21.15.78:0/1677643777 conn(0x7f1ee00fc7a0 msgr2=0x7f1ee00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:16.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 -- 172.21.15.78:0/1677643777 shutdown_connections 2024-09-16T00:25:16.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:16.564+0000 7f1ee66a4640 1 -- 172.21.15.78:0/1677643777 wait complete. 2024-09-16T00:25:17.323 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '48333ca2', total seconds elapsed: 25.3 2024-09-16T00:25:18.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:17 smithi193 ceph-mon[35484]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-16T00:25:18.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:17 smithi148 ceph-mon[35599]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-16T00:25:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:17 smithi078 ceph-mon[31157]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-16T00:25:18.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:25:18.889 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:19 smithi193 ceph-mon[35484]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 516 KiB/s wr, 39 op/s 2024-09-16T00:25:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:19 smithi148 ceph-mon[35599]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 516 KiB/s wr, 39 op/s 2024-09-16T00:25:20.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:19 smithi078 ceph-mon[31157]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 516 KiB/s wr, 39 op/s 2024-09-16T00:25:20.281 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:25:20.281 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:25:20.281 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:25:20.281 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:25:20.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 -- 172.21.15.78:0/388401200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 msgr2=0x7feb0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 --2- 172.21.15.78:0/388401200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 0x7feb0410ea90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7feafc009ca0 tx=0x7feafc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 -- 172.21.15.78:0/388401200 shutdown_connections 2024-09-16T00:25:20.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 --2- 172.21.15.78:0/388401200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 0x7feb0410ea90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 --2- 172.21.15.78:0/388401200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb04102790 0x7feb041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 --2- 172.21.15.78:0/388401200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb04101de0 0x7feb041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 -- 172.21.15.78:0/388401200 >> 172.21.15.78:0/388401200 conn(0x7feb040fd8c0 msgr2=0x7feb040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 -- 172.21.15.78:0/388401200 shutdown_connections 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.451+0000 7feb094a1640 1 -- 172.21.15.78:0/388401200 wait complete. 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb094a1640 1 Processor -- start 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb094a1640 1 -- start start 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb094a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb04101de0 0x7feb041985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb094a1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb04102790 0x7feb04198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb094a1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 0x7feb0419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb094a1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb04112010 con 0x7feb04106f10 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb094a1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb04111e90 con 0x7feb04102790 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb094a1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb04112190 con 0x7feb04101de0 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 0x7feb0419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb037fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 0x7feb0419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50638/0 (socket says 172.21.15.78:50638) 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb037fe640 1 -- 172.21.15.78:0/2155856798 learned_addr learned my addr 172.21.15.78:0/2155856798 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb027fc640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb04102790 0x7feb04198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb02ffd640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb04101de0 0x7feb041985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb037fe640 1 -- 172.21.15.78:0/2155856798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb04101de0 msgr2=0x7feb041985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb037fe640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb04101de0 0x7feb041985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb037fe640 1 -- 172.21.15.78:0/2155856798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb04102790 msgr2=0x7feb04198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb037fe640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb04102790 0x7feb04198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb037fe640 1 -- 172.21.15.78:0/2155856798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0419d5a0 con 0x7feb04106f10 2024-09-16T00:25:20.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.452+0000 7feb027fc640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb04102790 0x7feb04198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.453+0000 7feb037fe640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 0x7feb0419cec0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7feafc031a60 tx=0x7feafc031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.453+0000 7feaebfff640 1 -- 172.21.15.78:0/2155856798 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc003c30 con 0x7feb04106f10 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.453+0000 7feb094a1640 1 -- 172.21.15.78:0/2155856798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0419d830 con 0x7feb04106f10 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.453+0000 7feaebfff640 1 -- 172.21.15.78:0/2155856798 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feafc003dd0 con 0x7feb04106f10 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.453+0000 7feaebfff640 1 -- 172.21.15.78:0/2155856798 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc0047e0 con 0x7feb04106f10 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.453+0000 7feb094a1640 1 -- 172.21.15.78:0/2155856798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb041a5170 con 0x7feb04106f10 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.455+0000 7feaebfff640 1 -- 172.21.15.78:0/2155856798 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7feafc02f790 con 0x7feb04106f10 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.455+0000 7feb0b87a740 1 Processor -- start 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb0b87a740 1 -- start start 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 0x556a44408880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44413140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44415a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb0b87a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a4436e2e0 con 0x556a44413680 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb0b87a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556a44315ce0 con 0x556a444084a0 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb0b87a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a4436a2a0 con 0x556a44408dc0 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feaebfff640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feaec0a3240 0x7feaec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44415a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb037fe640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feaec0a3240 0x7feaec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44415a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50640/0 (socket says 172.21.15.78:50640) 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 -- 172.21.15.78:0/3556951701 learned_addr learned my addr 172.21.15.78:0/3556951701 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb02ffd640 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44413140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 0x556a44408880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feaebfff640 1 -- 172.21.15.78:0/2155856798 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feafc0f11b0 con 0x7feb04106f10 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 -- 172.21.15.78:0/3556951701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 msgr2=0x556a44413140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:20.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44413140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 -- 172.21.15.78:0/3556951701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 msgr2=0x556a44408880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 0x556a44408880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb027fc640 1 -- 172.21.15.78:0/3556951701 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a442fa440 con 0x556a44413680 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.456+0000 7feb037fe640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feaec0a3240 0x7feaec0a5700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feafc0b9870 tx=0x7feafc0b28f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb027fc640 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44415a70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7feaf8006fd0 tx=0x7feaf800c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d286c98a541b89f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feae97fa640 1 -- 172.21.15.78:0/3556951701 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf801c070 con 0x556a44413680 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feae97fa640 1 -- 172.21.15.78:0/3556951701 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf8017af0 con 0x556a44413680 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feae97fa640 1 -- 172.21.15.78:0/3556951701 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf800f040 con 0x556a44413680 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 -- 172.21.15.78:0/3556951701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 msgr2=0x556a44415a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44415a70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7feaf8006fd0 tx=0x7feaf800c230 comp rx=0 tx=0).stop 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 -- 172.21.15.78:0/3556951701 shutdown_connections 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44415a70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44413140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 --2- 172.21.15.78:0/3556951701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 0x556a44408880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 -- 172.21.15.78:0/3556951701 >> 172.21.15.78:0/3556951701 conn(0x556a44404a30 msgr2=0x556a44404e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 -- 172.21.15.78:0/3556951701 shutdown_connections 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.457+0000 7feb0b87a740 1 -- 172.21.15.78:0/3556951701 wait complete. 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 1 -- start start 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 1 Processor -- start 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 0x556a44413090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44410340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44410950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556a44416250 con 0x556a44413680 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556a44329050 con 0x556a444084a0 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556a444163d0 con 0x556a44408dc0 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb0b87a740 10 client.0 _ll_register_callbacks cb 0x556a44369160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:20.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44410950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44410950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50648/0 (socket says 172.21.15.78:50648) 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb027fc640 1 -- 172.21.15.78:0/4244697582 learned_addr learned my addr 172.21.15.78:0/4244697582 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb027fc640 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 0x556a44413090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.458+0000 7feb02ffd640 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44410340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb027fc640 1 -- 172.21.15.78:0/4244697582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 msgr2=0x556a44410340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb027fc640 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44410340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb027fc640 1 -- 172.21.15.78:0/4244697582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 msgr2=0x556a44413090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb027fc640 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 0x556a44413090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb027fc640 1 -- 172.21.15.78:0/4244697582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a440df3c0 con 0x556a44413680 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb027fc640 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44410950 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7feaf8015480 tx=0x7feaf8016740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef812212cfc75a4d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feae6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feae6ff5640 1 -- 172.21.15.78:0/4244697582 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf801c040 con 0x556a44413680 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feae6ff5640 1 -- 172.21.15.78:0/4244697582 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feaf8006c30 con 0x556a44413680 2024-09-16T00:25:20.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb0b87a740 1 -- 172.21.15.78:0/4244697582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf8017af0 con 0x556a44413680 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feae6ff5640 1 -- 172.21.15.78:0/4244697582 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feaf8032050 con 0x556a44413680 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb0b87a740 10 client.17550 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb0b87a740 1 -- 172.21.15.78:0/4244697582 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a44421cc0 con 0x556a44413680 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feae6ff5640 1 -- 172.21.15.78:0/4244697582 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf8017af0 con 0x556a44413680 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feae6ff5640 1 -- 172.21.15.78:0/4244697582 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7feaf8022050 con 0x556a44413680 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb0b87a740 10 client.17550 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb0b87a740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.459+0000 7feb0b87a740 20 client.17550 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.460+0000 7feb0b87a740 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556a44423010 0x556a44425400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.460+0000 7feb0b87a740 4 client.17550 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.460+0000 7feb0b87a740 1 -- 172.21.15.78:0/4244697582 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a4437b850 con 0x556a44423010 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.460+0000 7feb02ffd640 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556a44423010 0x556a44425400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.460+0000 7feae6ff5640 10 client.17550 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:20.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.460+0000 7feb02ffd640 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556a44423010 0x556a44425400 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e52409ebada26d8 server_cookie=5b5788e4b25c257d in_seq=0 out_seq=0 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feae6ff5640 1 -- 172.21.15.78:0/4244697582 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a4437b850 con 0x556a44423010 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feae6ff5640 10 client.17550 handle_command_reply: tid=0 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 client.17550 shutdown 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 -- 172.21.15.78:0/4244697582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 msgr2=0x556a44410950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44410950 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7feaf8015480 tx=0x7feaf8016740 comp rx=0 tx=0).stop 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 -- 172.21.15.78:0/4244697582 shutdown_connections 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556a44423010 0x556a44425400 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a44413680 0x556a44410950 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44410340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feae6ff5640 0 client.17550 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 --2- 172.21.15.78:0/4244697582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a444084a0 0x556a44413090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.461+0000 7feb0b87a740 1 -- 172.21.15.78:0/4244697582 >> 172.21.15.78:0/4244697582 conn(0x556a44404a30 msgr2=0x556a4440a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.462+0000 7feb0b87a740 1 -- 172.21.15.78:0/4244697582 shutdown_connections 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.462+0000 7feb0b87a740 1 -- 172.21.15.78:0/4244697582 wait complete. 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.462+0000 7feb0b87a740 20 client.17550 trim_cache size 0 max 16384 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb0b87a740 1 Processor -- start 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb0b87a740 1 -- start start 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a4441f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44408dc0 0x556a4440ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44413680 0x556a4442aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb0b87a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb041aec50 con 0x556a444084a0 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb0b87a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb041b0790 con 0x556a44408dc0 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb0b87a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb041af320 con 0x556a44413680 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a4441f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a4441f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50654/0 (socket says 172.21.15.78:50654) 2024-09-16T00:25:20.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 -- 172.21.15.78:0/3770830808 learned_addr learned my addr 172.21.15.78:0/3770830808 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb02ffd640 1 --2- 172.21.15.78:0/3770830808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44408dc0 0x556a4440ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 -- 172.21.15.78:0/3770830808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44413680 msgr2=0x556a4442aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 --2- 172.21.15.78:0/3770830808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44413680 0x556a4442aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 -- 172.21.15.78:0/3770830808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44408dc0 msgr2=0x556a4440ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 --2- 172.21.15.78:0/3770830808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44408dc0 0x556a4440ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 -- 172.21.15.78:0/3770830808 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a444056c0 con 0x556a444084a0 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feb027fc640 1 --2- 172.21.15.78:0/3770830808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a4441f4b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7feaf8008040 tx=0x7feaf8004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96e968d9104a863 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.479+0000 7feae8ff9640 1 -- 172.21.15.78:0/3770830808 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf801c070 con 0x556a444084a0 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feae8ff9640 1 -- 172.21.15.78:0/3770830808 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf800a040 con 0x556a444084a0 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feae8ff9640 1 -- 172.21.15.78:0/3770830808 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf800c380 con 0x556a444084a0 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 -- 172.21.15.78:0/3770830808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 msgr2=0x556a4441f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 --2- 172.21.15.78:0/3770830808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a4441f4b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7feaf8008040 tx=0x7feaf8004680 comp rx=0 tx=0).stop 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 -- 172.21.15.78:0/3770830808 shutdown_connections 2024-09-16T00:25:20.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 --2- 172.21.15.78:0/3770830808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44413680 0x556a4442aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 --2- 172.21.15.78:0/3770830808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44408dc0 0x556a4440ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 --2- 172.21.15.78:0/3770830808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a4441f4b0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 -- 172.21.15.78:0/3770830808 >> 172.21.15.78:0/3770830808 conn(0x556a44404a30 msgr2=0x556a444123c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 -- 172.21.15.78:0/3770830808 shutdown_connections 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.480+0000 7feb0b87a740 1 -- 172.21.15.78:0/3770830808 wait complete. 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 1 -- start start 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 1 Processor -- start 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a44417400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44417940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44413680 0x556a44417e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb041b00c0 con 0x556a444084a0 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb041af9f0 con 0x556a44413680 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb041acc30 con 0x556a44408dc0 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb0b87a740 10 client.0 _ll_register_callbacks cb 0x7feaf801a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:20.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a44417400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a44417400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50666/0 (socket says 172.21.15.78:50666) 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 -- 172.21.15.78:0/762316995 learned_addr learned my addr 172.21.15.78:0/762316995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb02ffd640 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44417940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44413680 0x556a44417e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 -- 172.21.15.78:0/762316995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 msgr2=0x556a44417940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44417940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 -- 172.21.15.78:0/762316995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44413680 msgr2=0x556a44417e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44413680 0x556a44417e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.481+0000 7feb027fc640 1 -- 172.21.15.78:0/762316995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a4448ca50 con 0x556a444084a0 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb027fc640 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a44417400 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7feaf8016010 tx=0x7feaf8007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11fcc5d58c862f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feae6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feae6ff5640 1 -- 172.21.15.78:0/762316995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf801c070 con 0x556a444084a0 2024-09-16T00:25:20.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feae6ff5640 1 -- 172.21.15.78:0/762316995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7feaf80255a0 con 0x556a444084a0 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 1 -- 172.21.15.78:0/762316995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf800a040 con 0x556a444084a0 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feae6ff5640 1 -- 172.21.15.78:0/762316995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feaf8048050 con 0x556a444084a0 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 10 client.17556 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 1 -- 172.21.15.78:0/762316995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a4448dd40 con 0x556a444084a0 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feae6ff5640 1 -- 172.21.15.78:0/762316995 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf800a040 con 0x556a444084a0 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feae6ff5640 1 -- 172.21.15.78:0/762316995 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7feaf8022040 con 0x556a444084a0 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 10 client.17556 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 20 client.17556 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556a4448e3f0 0x556a444907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 4 client.17556 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.482+0000 7feb0b87a740 1 -- 172.21.15.78:0/762316995 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a44375d60 con 0x556a4448e3f0 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.483+0000 7feb02ffd640 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556a4448e3f0 0x556a444907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:20.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.483+0000 7feb02ffd640 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556a4448e3f0 0x556a444907e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467d8b921eb75d1b server_cookie=b631a17a0ffce357 in_seq=0 out_seq=0 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.483+0000 7feae6ff5640 10 client.17556 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.483+0000 7feae6ff5640 1 -- 172.21.15.78:0/762316995 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556a44375d60 con 0x556a4448e3f0 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.483+0000 7feae6ff5640 10 client.17556 handle_command_reply: tid=0 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.483+0000 7feb0b87a740 1 client.17556 shutdown 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 -- 172.21.15.78:0/762316995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 msgr2=0x556a44417400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a44417400 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7feaf8016010 tx=0x7feaf8007f80 comp rx=0 tx=0).stop 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 -- 172.21.15.78:0/762316995 shutdown_connections 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556a4448e3f0 0x556a444907e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556a44413680 0x556a44417e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a44408dc0 0x556a44417940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feae6ff5640 0 client.17556 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 --2- 172.21.15.78:0/762316995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556a444084a0 0x556a44417400 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 -- 172.21.15.78:0/762316995 >> 172.21.15.78:0/762316995 conn(0x556a44404a30 msgr2=0x556a44403350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 -- 172.21.15.78:0/762316995 shutdown_connections 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 1 -- 172.21.15.78:0/762316995 wait complete. 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb0b87a740 20 client.17556 trim_cache size 0 max 16384 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 -- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feaec0a3240 msgr2=0x7feaec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feaec0a3240 0x7feaec0a5700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feafc0b9870 tx=0x7feafc0b28f0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 -- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 msgr2=0x7feb0419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 0x7feb0419cec0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7feafc031a60 tx=0x7feafc031a90 comp rx=0 tx=0).stop 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 -- 172.21.15.78:0/2155856798 shutdown_connections 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feaec0a3240 0x7feaec0a5700 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb04106f10 0x7feb0419cec0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb04102790 0x7feb04198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 --2- 172.21.15.78:0/2155856798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb04101de0 0x7feb041985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:20.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 -- 172.21.15.78:0/2155856798 >> 172.21.15.78:0/2155856798 conn(0x7feb040fd8c0 msgr2=0x7feb040fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:20.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.484+0000 7feb094a1640 1 -- 172.21.15.78:0/2155856798 shutdown_connections 2024-09-16T00:25:20.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:19.485+0000 7feb094a1640 1 -- 172.21.15.78:0/2155856798 wait complete. 2024-09-16T00:25:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 680.0 s, 311.1 tps, lat 1528.235 ms stddev 5117.190 2024-09-16T00:25:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:21 smithi193 ceph-mon[35484]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-09-16T00:25:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:21 smithi148 ceph-mon[35599]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-09-16T00:25:22.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:21 smithi078 ceph-mon[31157]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.9 MiB/s wr, 218 op/s 2024-09-16T00:25:23.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:23.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:22 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:22 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:23.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:22 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:23.613 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:25:23.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:25:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:23 smithi193 ceph-mon[35484]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-09-16T00:25:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:23 smithi148 ceph-mon[35599]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-09-16T00:25:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:24.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:23 smithi078 ceph-mon[31157]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.9 MiB/s wr, 449 op/s 2024-09-16T00:25:24.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:24.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:24.261 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:25:24.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.745+0000 7fc5c01a6640 1 -- 172.21.15.78:0/713814575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8105db0 msgr2=0x7fc5b8106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:24.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.745+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/713814575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8105db0 0x7fc5b8106240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc5b001acb0 tx=0x7fc5b003b9b0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.747+0000 7fc5c01a6640 1 -- 172.21.15.78:0/713814575 shutdown_connections 2024-09-16T00:25:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.747+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/713814575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8105db0 0x7fc5b8106240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.747+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/713814575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b81012c0 0x7fc5b8105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.747+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/713814575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5b81009a0 0x7fc5b8100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.747+0000 7fc5c01a6640 1 -- 172.21.15.78:0/713814575 >> 172.21.15.78:0/713814575 conn(0x7fc5b80fc630 msgr2=0x7fc5b80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:24.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.747+0000 7fc5c01a6640 1 -- 172.21.15.78:0/713814575 shutdown_connections 2024-09-16T00:25:24.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.747+0000 7fc5c01a6640 1 -- 172.21.15.78:0/713814575 wait complete. 2024-09-16T00:25:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.748+0000 7fc5c01a6640 1 Processor -- start 2024-09-16T00:25:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.748+0000 7fc5c01a6640 1 -- start start 2024-09-16T00:25:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.748+0000 7fc5c01a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b81009a0 0x7fc5b810d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.748+0000 7fc5c01a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5b81012c0 0x7fc5b810d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.748+0000 7fc5c01a6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8105db0 0x7fc5b8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.748+0000 7fc5c01a6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b81096f0 con 0x7fc5b81012c0 2024-09-16T00:25:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.748+0000 7fc5c01a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8109570 con 0x7fc5b81009a0 2024-09-16T00:25:24.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.748+0000 7fc5c01a6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b8109870 con 0x7fc5b8105db0 2024-09-16T00:25:24.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.750+0000 7fc5bd741640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5b81012c0 0x7fc5b810d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:24.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.750+0000 7fc5bd741640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5b81012c0 0x7fc5b810d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50678/0 (socket says 172.21.15.78:50678) 2024-09-16T00:25:24.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.750+0000 7fc5bd741640 1 -- 172.21.15.78:0/230889987 learned_addr learned my addr 172.21.15.78:0/230889987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:24.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.751+0000 7fc5be743640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8105db0 0x7fc5b8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.753+0000 7fc5bd741640 1 -- 172.21.15.78:0/230889987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8105db0 msgr2=0x7fc5b8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.753+0000 7fc5bd741640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8105db0 0x7fc5b8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.753+0000 7fc5bd741640 1 -- 172.21.15.78:0/230889987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b81009a0 msgr2=0x7fc5b810d380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:25:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.753+0000 7fc5bd741640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b81009a0 0x7fc5b810d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:24.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.753+0000 7fc5bd741640 1 -- 172.21.15.78:0/230889987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b8073290 con 0x7fc5b81012c0 2024-09-16T00:25:24.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.760+0000 7fc5bd741640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5b81012c0 0x7fc5b810d8c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800e9f0 tx=0x7fc5a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.762+0000 7fc5aeffd640 1 -- 172.21.15.78:0/230889987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a800ccc0 con 0x7fc5b81012c0 2024-09-16T00:25:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.762+0000 7fc5aeffd640 1 -- 172.21.15.78:0/230889987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a8010910 con 0x7fc5b81012c0 2024-09-16T00:25:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.762+0000 7fc5aeffd640 1 -- 172.21.15.78:0/230889987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8018920 con 0x7fc5b81012c0 2024-09-16T00:25:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.766+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b8073580 con 0x7fc5b81012c0 2024-09-16T00:25:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.766+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b8073a40 con 0x7fc5b81012c0 2024-09-16T00:25:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.767+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b806a850 con 0x7fc5b81012c0 2024-09-16T00:25:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.773+0000 7fc5aeffd640 1 -- 172.21.15.78:0/230889987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fc5a8010430 con 0x7fc5b81012c0 2024-09-16T00:25:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.773+0000 7fc5aeffd640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5a40a3450 0x7fc5a40a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:24.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.774+0000 7fc5aeffd640 1 -- 172.21.15.78:0/230889987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fc5a80d02b0 con 0x7fc5b81012c0 2024-09-16T00:25:24.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.780+0000 7fc5bdf42640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5a40a3450 0x7fc5a40a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:24.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.783+0000 7fc5aeffd640 1 -- 172.21.15.78:0/230889987 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5a809cc10 con 0x7fc5b81012c0 2024-09-16T00:25:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:24.788+0000 7fc5bdf42640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5a40a3450 0x7fc5a40a5910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400b990 tx=0x7fc5b400b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:25.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.136+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5b80742b0 con 0x7fc5b81012c0 2024-09-16T00:25:25.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.143+0000 7fc5aeffd640 1 -- 172.21.15.78:0/230889987 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fc5b80742b0 con 0x7fc5b81012c0 2024-09-16T00:25:25.145 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:25:25.145 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:25:25.145 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:25:25.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:24 smithi078 ceph-mon[31157]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 MiB/s wr, 750 op/s 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5a40a3450 msgr2=0x7fc5a40a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5a40a3450 0x7fc5a40a5910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400b990 tx=0x7fc5b400b370 comp rx=0 tx=0).stop 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5b81012c0 msgr2=0x7fc5b810d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5b81012c0 0x7fc5b810d8c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800e9f0 tx=0x7fc5a800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 shutdown_connections 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fc5a40a3450 0x7fc5a40a5910 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5b8105db0 0x7fc5b8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5b81012c0 0x7fc5b810d8c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 --2- 172.21.15.78:0/230889987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc5b81009a0 0x7fc5b810d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 >> 172.21.15.78:0/230889987 conn(0x7fc5b80fc630 msgr2=0x7fc5b80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 shutdown_connections 2024-09-16T00:25:25.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:25.152+0000 7fc5c01a6640 1 -- 172.21.15.78:0/230889987 wait complete. 2024-09-16T00:25:25.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:24 smithi193 ceph-mon[35484]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 MiB/s wr, 750 op/s 2024-09-16T00:25:25.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:24 smithi148 ceph-mon[35599]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 MiB/s wr, 750 op/s 2024-09-16T00:25:25.839 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:25:25.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 685.0 s, 697.1 tps, lat 176.002 ms stddev 363.448 2024-09-16T00:25:26.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:25 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/230889987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:25 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/230889987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:26.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:25 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/230889987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:25:26.318 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:27.096 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:26 smithi078 ceph-mon[31157]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 12 MiB/s wr, 843 op/s 2024-09-16T00:25:27.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:26 smithi193 ceph-mon[35484]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 12 MiB/s wr, 843 op/s 2024-09-16T00:25:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:26 smithi148 ceph-mon[35599]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 12 MiB/s wr, 843 op/s 2024-09-16T00:25:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.964+0000 7fd2c94c9640 1 -- 172.21.15.78:0/1504162573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 msgr2=0x7fd2c4106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.964+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/1504162573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 0x7fd2c4106840 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009930 tx=0x7fd2ac02a420 comp rx=0 tx=0).stop 2024-09-16T00:25:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.967+0000 7fd2c94c9640 1 -- 172.21.15.78:0/1504162573 shutdown_connections 2024-09-16T00:25:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.967+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/1504162573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4106f50 0x7fd2c410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.967+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/1504162573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 0x7fd2c4106840 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.967+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/1504162573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c41002f0 0x7fd2c41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.967+0000 7fd2c94c9640 1 -- 172.21.15.78:0/1504162573 >> 172.21.15.78:0/1504162573 conn(0x7fd2c4077210 msgr2=0x7fd2c40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.975+0000 7fd2c94c9640 1 -- 172.21.15.78:0/1504162573 shutdown_connections 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.976+0000 7fd2c94c9640 1 -- 172.21.15.78:0/1504162573 wait complete. 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.976+0000 7fd2c94c9640 1 Processor -- start 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.977+0000 7fd2c94c9640 1 -- start start 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.977+0000 7fd2c94c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c41002f0 0x7fd2c419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.977+0000 7fd2c94c9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 0x7fd2c419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.977+0000 7fd2c94c9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4106f50 0x7fd2c419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.977+0000 7fd2c94c9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4111ff0 con 0x7fd2c4100ca0 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.978+0000 7fd2c94c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4111e70 con 0x7fd2c41002f0 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.978+0000 7fd2c94c9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4112170 con 0x7fd2c4106f50 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.979+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4106f50 0x7fd2c419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.981+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 0x7fd2c419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.981+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 0x7fd2c419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36188/0 (socket says 172.21.15.78:36188) 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.981+0000 7fd2c27fc640 1 -- 172.21.15.78:0/154067470 learned_addr learned my addr 172.21.15.78:0/154067470 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.981+0000 7fd2c27fc640 1 -- 172.21.15.78:0/154067470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4106f50 msgr2=0x7fd2c419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.981+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4106f50 0x7fd2c419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.981+0000 7fd2c27fc640 1 -- 172.21.15.78:0/154067470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c41002f0 msgr2=0x7fd2c419efb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.981+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c41002f0 0x7fd2c419efb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.981+0000 7fd2c27fc640 1 -- 172.21.15.78:0/154067470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c419c510 con 0x7fd2c4100ca0 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.982+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 0x7fd2c419f4f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009900 tx=0x7fd2ac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.985+0000 7fd2bbfff640 1 -- 172.21.15.78:0/154067470 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac03f030 con 0x7fd2c4100ca0 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.986+0000 7fd2bbfff640 1 -- 172.21.15.78:0/154067470 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ac003cd0 con 0x7fd2c4100ca0 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.986+0000 7fd2bbfff640 1 -- 172.21.15.78:0/154067470 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac035070 con 0x7fd2c4100ca0 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.987+0000 7fd2c94c9640 1 -- 172.21.15.78:0/154067470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c419c740 con 0x7fd2c4100ca0 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.987+0000 7fd2c94c9640 1 -- 172.21.15.78:0/154067470 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c410dc00 con 0x7fd2c4100ca0 2024-09-16T00:25:27.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.993+0000 7fd2cb8a2740 1 Processor -- start 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.995+0000 7fd2cb8a2740 1 -- start start 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.995+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbabd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.995+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabd5d0 0x55a7cbabda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.995+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbaca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.995+0000 7fd2cb8a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7cba232f0 con 0x55a7cbac8370 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.995+0000 7fd2cb8a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a7cb9cacf0 con 0x55a7cbabd5d0 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.995+0000 7fd2cb8a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a7cba1f2b0 con 0x55a7cbabeb40 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.999+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbabd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.999+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbabd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47210/0 (socket says 172.21.15.78:47210) 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.999+0000 7fd2c37fe640 1 -- 172.21.15.78:0/606884703 learned_addr learned my addr 172.21.15.78:0/606884703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:26.999+0000 7fd2c2ffd640 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabd5d0 0x55a7cbabda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.000+0000 7fd2bbfff640 1 -- 172.21.15.78:0/154067470 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd2ac042030 con 0x7fd2c4100ca0 2024-09-16T00:25:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.000+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbaca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.001+0000 7fd2bbfff640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2b00a33e0 0x7fd2b00a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.002+0000 7fd2c37fe640 1 -- 172.21.15.78:0/606884703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabd5d0 msgr2=0x55a7cbabda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.002+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabd5d0 0x55a7cbabda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.002+0000 7fd2c37fe640 1 -- 172.21.15.78:0/606884703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 msgr2=0x55a7cbaca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.002+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbaca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.002+0000 7fd2c37fe640 1 -- 172.21.15.78:0/606884703 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7cb9af3a0 con 0x55a7cbabeb40 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.002+0000 7fd2bbfff640 1 -- 172.21.15.78:0/154067470 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd2ac0f20b0 con 0x7fd2c4100ca0 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.002+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbabd090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005fd0 tx=0x7fd2bc00d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=737182a474abafb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.007+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2b00a33e0 0x7fd2b00a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.007+0000 7fd2b97fa640 1 -- 172.21.15.78:0/606884703 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc018680 con 0x55a7cbabeb40 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.007+0000 7fd2b97fa640 1 -- 172.21.15.78:0/606884703 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc00f900 con 0x55a7cbabeb40 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.007+0000 7fd2b97fa640 1 -- 172.21.15.78:0/606884703 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc017b20 con 0x55a7cbabeb40 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.008+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/606884703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 msgr2=0x55a7cbabd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.008+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbabd090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005fd0 tx=0x7fd2bc00d710 comp rx=0 tx=0).stop 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/606884703 shutdown_connections 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbaca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabd5d0 0x55a7cbabda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/606884703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbabd090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/606884703 >> 172.21.15.78:0/606884703 conn(0x55a7cba86310 msgr2=0x55a7cbab9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/606884703 shutdown_connections 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/606884703 wait complete. 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2b00a33e0 0x7fd2b00a58a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005fd0 tx=0x7fd2bc00d710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.010+0000 7fd2cb8a2740 1 -- start start 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2cb8a2740 1 Processor -- start 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbac7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabeb40 0x55a7cbac5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbac5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2cb8a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a7cbacb070 con 0x55a7cbac8370 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2cb8a2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a7cb9de060 con 0x55a7cbabeb40 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2cb8a2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a7cbacb1f0 con 0x55a7cbabd5d0 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbac5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbac5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36198/0 (socket says 172.21.15.78:36198) 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2c37fe640 1 -- 172.21.15.78:0/4113814488 learned_addr learned my addr 172.21.15.78:0/4113814488 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.011+0000 7fd2cb8a2740 10 client.0 _ll_register_callbacks cb 0x55a7cb7318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.013+0000 7fd2c37fe640 1 -- 172.21.15.78:0/4113814488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 msgr2=0x55a7cbac7dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.013+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbac7dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.013+0000 7fd2c37fe640 1 -- 172.21.15.78:0/4113814488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabeb40 msgr2=0x55a7cbac5080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:25:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.013+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabeb40 0x55a7cbac5080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.013+0000 7fd2c37fe640 1 -- 172.21.15.78:0/4113814488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7cb7943c0 con 0x55a7cbac8370 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.014+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbac5810 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc017640 tx=0x7fd2bc008630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87214bac195b39df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.014+0000 7fd2aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.018+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/4113814488 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc004dd0 con 0x55a7cbac8370 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.018+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/4113814488 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd2bc018840 con 0x55a7cbac8370 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.018+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/4113814488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2bc00f900 con 0x55a7cbac8370 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.026+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/4113814488 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc00fab0 con 0x55a7cbac8370 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.027+0000 7fd2cb8a2740 10 client.17574 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.027+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/4113814488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a7cbaccef0 con 0x55a7cbac8370 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.029+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/4113814488 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc00f900 con 0x55a7cbac8370 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.029+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/4113814488 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd2bc005540 con 0x55a7cbac8370 2024-09-16T00:25:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.035+0000 7fd2cb8a2740 10 client.17574 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.035+0000 7fd2cb8a2740 10 client.17574 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.035+0000 7fd2cb8a2740 20 client.17574 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.035+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a7cbace350 0x55a7cbad0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.035+0000 7fd2cb8a2740 4 client.17574 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.035+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/4113814488 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7cba30860 con 0x55a7cbace350 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.043+0000 7fd2c2ffd640 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a7cbace350 0x55a7cbad0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.044+0000 7fd2c2ffd640 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a7cbace350 0x55a7cbad0810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2196f50cf508e0dd server_cookie=da310f69ad1522f1 in_seq=0 out_seq=0 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.044+0000 7fd2aa7fc640 10 client.17574 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.045+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/4113814488 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7cba30860 con 0x55a7cbace350 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.045+0000 7fd2aa7fc640 10 client.17574 handle_command_reply: tid=0 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.045+0000 7fd2cb8a2740 1 client.17574 shutdown 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.048+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/4113814488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 msgr2=0x55a7cbac5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.048+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbac5810 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc017640 tx=0x7fd2bc008630 comp rx=0 tx=0).stop 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.049+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/4113814488 shutdown_connections 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.049+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbac8370 0x55a7cbac5810 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.049+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbabeb40 0x55a7cbac5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.049+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a7cbace350 0x55a7cbad0810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.049+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/4113814488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbac7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.049+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/4113814488 >> 172.21.15.78:0/4113814488 conn(0x55a7cba86310 msgr2=0x55a7cbabf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.049+0000 7fd2aa7fc640 0 client.17574 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.052+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/4113814488 shutdown_connections 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.052+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/4113814488 wait complete. 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.052+0000 7fd2cb8a2740 20 client.17574 trim_cache size 0 max 16384 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.078+0000 7fd2cb8a2740 1 Processor -- start 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.078+0000 7fd2cb8a2740 1 -- start start 2024-09-16T00:25:27.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.079+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabd5d0 0x55a7cbac4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.079+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbac00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.079+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 0x55a7cbac0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.079+0000 7fd2cb8a2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41b5210 con 0x55a7cbabd5d0 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.079+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbac00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.080+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbac00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47220/0 (socket says 172.21.15.78:47220) 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.080+0000 7fd2c27fc640 1 -- 172.21.15.78:0/690965373 learned_addr learned my addr 172.21.15.78:0/690965373 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.080+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 0x55a7cbac0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.080+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/690965373 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41b6d50 con 0x55a7cbac8370 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.080+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/690965373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41b58e0 con 0x55a7cbabeb40 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.081+0000 7fd2c2ffd640 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabd5d0 0x55a7cbac4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.082+0000 7fd2c27fc640 1 -- 172.21.15.78:0/690965373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 msgr2=0x55a7cbac0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.082+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 0x55a7cbac0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.082+0000 7fd2c27fc640 1 -- 172.21.15.78:0/690965373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabd5d0 msgr2=0x55a7cbac4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.082+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabd5d0 0x55a7cbac4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.082+0000 7fd2c27fc640 1 -- 172.21.15.78:0/690965373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7cbabdbe0 con 0x55a7cbabeb40 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.083+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbac00e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac02aa10 tx=0x7fd2ac03bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7661af8c3769ee server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.083+0000 7fd2b8ff9640 1 -- 172.21.15.78:0/690965373 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0be0e0 con 0x55a7cbabeb40 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.083+0000 7fd2b8ff9640 1 -- 172.21.15.78:0/690965373 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ac003e80 con 0x55a7cbabeb40 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.083+0000 7fd2b8ff9640 1 -- 172.21.15.78:0/690965373 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac005d50 con 0x55a7cbabeb40 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.084+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/690965373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 msgr2=0x55a7cbac00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.084+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbac00e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac02aa10 tx=0x7fd2ac03bb90 comp rx=0 tx=0).stop 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.084+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/690965373 shutdown_connections 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.084+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 0x55a7cbac0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.084+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabeb40 0x55a7cbac00e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.085+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/690965373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabd5d0 0x55a7cbac4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.085+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/690965373 >> 172.21.15.78:0/690965373 conn(0x55a7cba86310 msgr2=0x55a7cbac3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.094+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/690965373 shutdown_connections 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.094+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/690965373 wait complete. 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.094+0000 7fd2cb8a2740 1 -- start start 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.095+0000 7fd2cb8a2740 1 Processor -- start 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.095+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbad3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.095+0000 7fd2cb8a2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabeb40 0x55a7cbad41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.095+0000 7fd2c2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbad3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.095+0000 7fd2c2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbad3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47222/0 (socket says 172.21.15.78:47222) 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.096+0000 7fd2c2ffd640 1 -- 172.21.15.78:0/3678213875 learned_addr learned my addr 172.21.15.78:0/3678213875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.096+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 0x55a7cbad4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.096+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41b6680 con 0x55a7cbabeb40 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.096+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41b5fb0 con 0x55a7cbac8370 2024-09-16T00:25:27.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.096+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4072950 con 0x55a7cbabd5d0 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.096+0000 7fd2cb8a2740 10 client.0 _ll_register_callbacks cb 0x7fd2bc008bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.097+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabeb40 0x55a7cbad41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.097+0000 7fd2c37fe640 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 0x55a7cbad4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.100+0000 7fd2c2ffd640 1 -- 172.21.15.78:0/3678213875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 msgr2=0x55a7cbad4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.100+0000 7fd2c2ffd640 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 0x55a7cbad4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.100+0000 7fd2c2ffd640 1 -- 172.21.15.78:0/3678213875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabeb40 msgr2=0x55a7cbad41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.100+0000 7fd2c2ffd640 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabeb40 0x55a7cbad41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.100+0000 7fd2c2ffd640 1 -- 172.21.15.78:0/3678213875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7cbb412d0 con 0x55a7cbabd5d0 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.100+0000 7fd2c2ffd640 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbad3c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4005fd0 tx=0x7fd2b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ac3bcdfc650096 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.101+0000 7fd2aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:25:27.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.103+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/3678213875 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4022670 con 0x55a7cbabd5d0 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.103+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a7cba2ad70 con 0x55a7cbabd5d0 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.104+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/3678213875 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd2b4019230 con 0x55a7cbabd5d0 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.105+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/3678213875 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd2b4022810 con 0x55a7cbabd5d0 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.105+0000 7fd2cb8a2740 10 client.26635 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.105+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a7cbb425e0 con 0x55a7cbabd5d0 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.106+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/3678213875 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a7cba2ad70 con 0x55a7cbabd5d0 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.106+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/3678213875 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd2b40103d0 con 0x55a7cbabd5d0 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.107+0000 7fd2cb8a2740 10 client.26635 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.107+0000 7fd2cb8a2740 10 client.26635 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.107+0000 7fd2cb8a2740 20 client.26635 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.107+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a7cbb42cb0 0x55a7cbb450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.107+0000 7fd2cb8a2740 4 client.26635 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:25:27.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.107+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7cb9a7170 con 0x55a7cbb42cb0 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.108+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a7cbb42cb0 0x55a7cbb450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.110+0000 7fd2c27fc640 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a7cbb42cb0 0x55a7cbb450d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4858baf9d155096d server_cookie=36fff294df20207 in_seq=0 out_seq=0 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.110+0000 7fd2aa7fc640 10 client.26635 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.110+0000 7fd2aa7fc640 1 -- 172.21.15.78:0/3678213875 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7cb9a7170 con 0x55a7cbb42cb0 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.110+0000 7fd2aa7fc640 10 client.26635 handle_command_reply: tid=0 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 client.26635 shutdown 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 msgr2=0x55a7cbad3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbad3c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4005fd0 tx=0x7fd2b4011810 comp rx=0 tx=0).stop 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 shutdown_connections 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a7cbac8370 0x55a7cbad4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a7cbabeb40 0x55a7cbad41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55a7cbb42cb0 0x55a7cbb450d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 --2- 172.21.15.78:0/3678213875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a7cbabd5d0 0x55a7cbad3c80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2aa7fc640 0 client.26635 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:25:27.740 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:25:27.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.111+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 >> 172.21.15.78:0/3678213875 conn(0x55a7cba86310 msgr2=0x55a7cbac68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:27.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.112+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 shutdown_connections 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.112+0000 7fd2cb8a2740 1 -- 172.21.15.78:0/3678213875 wait complete. 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.113+0000 7fd2cb8a2740 20 client.26635 trim_cache size 0 max 16384 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 -- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2b00a33e0 msgr2=0x7fd2b00a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2b00a33e0 0x7fd2b00a58a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005fd0 tx=0x7fd2bc00d710 comp rx=0 tx=0).stop 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 -- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 msgr2=0x7fd2c419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 0x7fd2c419f4f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009900 tx=0x7fd2ac0038a0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 -- 172.21.15.78:0/154067470 shutdown_connections 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2b00a33e0 0x7fd2b00a58a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2c4106f50 0x7fd2c419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c4100ca0 0x7fd2c419f4f0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 --2- 172.21.15.78:0/154067470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd2c41002f0 0x7fd2c419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 -- 172.21.15.78:0/154067470 >> 172.21.15.78:0/154067470 conn(0x7fd2c4077210 msgr2=0x7fd2c41055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 -- 172.21.15.78:0/154067470 shutdown_connections 2024-09-16T00:25:27.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:27.117+0000 7fd2c94c9640 1 -- 172.21.15.78:0/154067470 wait complete. 2024-09-16T00:25:28.449 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:25:28.980 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:29.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.639+0000 7f071a302640 1 -- 172.21.15.78:0/1396158448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071411b140 msgr2=0x7f071411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:29.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.639+0000 7f071a302640 1 --2- 172.21.15.78:0/1396158448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071411b140 0x7f071411d530 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f070c009ca0 tx=0x7f070c02a8f0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.640+0000 7f071a302640 1 -- 172.21.15.78:0/1396158448 shutdown_connections 2024-09-16T00:25:29.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.640+0000 7f071a302640 1 --2- 172.21.15.78:0/1396158448 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f071411b140 0x7f071411d530 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.640+0000 7f071a302640 1 --2- 172.21.15.78:0/1396158448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0714109830 0x7f0714109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.640+0000 7f071a302640 1 --2- 172.21.15.78:0/1396158448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0714109450 0x7f0714110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.640+0000 7f071a302640 1 -- 172.21.15.78:0/1396158448 >> 172.21.15.78:0/1396158448 conn(0x7f071406e140 msgr2=0x7f071406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:29.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.645+0000 7f071a302640 1 -- 172.21.15.78:0/1396158448 shutdown_connections 2024-09-16T00:25:29.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.649+0000 7f071a302640 1 -- 172.21.15.78:0/1396158448 wait complete. 2024-09-16T00:25:29.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.650+0000 7f071a302640 1 Processor -- start 2024-09-16T00:25:29.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.650+0000 7f071a302640 1 -- start start 2024-09-16T00:25:29.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.651+0000 7f071a302640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0714109450 0x7f07141a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.651+0000 7f071a302640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0714109830 0x7f07141a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.651+0000 7f071a302640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f071411b140 0x7f07141abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.651+0000 7f071a302640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0714120ed0 con 0x7f0714109830 2024-09-16T00:25:29.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.651+0000 7f071a302640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0714120d50 con 0x7f071411b140 2024-09-16T00:25:29.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.651+0000 7f071a302640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0714121050 con 0x7f0714109450 2024-09-16T00:25:29.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.651+0000 7f0713fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0714109450 0x7f07141a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.652+0000 7f0713fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0714109450 0x7f07141a7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47236/0 (socket says 172.21.15.78:47236) 2024-09-16T00:25:29.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.652+0000 7f0713fff640 1 -- 172.21.15.78:0/864846673 learned_addr learned my addr 172.21.15.78:0/864846673 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.654+0000 7f0713fff640 1 -- 172.21.15.78:0/864846673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f071411b140 msgr2=0x7f07141abb30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:25:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.654+0000 7f0713fff640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f071411b140 0x7f07141abb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.654+0000 7f0713fff640 1 -- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0714109830 msgr2=0x7f07141a77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.654+0000 7f0713fff640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0714109830 0x7f07141a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.654+0000 7f0713fff640 1 -- 172.21.15.78:0/864846673 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07141ac2b0 con 0x7f0714109450 2024-09-16T00:25:29.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.654+0000 7f0713fff640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0714109450 0x7f07141a7260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f070400c890 tx=0x7f070400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:29.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.655+0000 7f0711ffb640 1 -- 172.21.15.78:0/864846673 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704004220 con 0x7f0714109450 2024-09-16T00:25:29.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.655+0000 7f071a302640 1 -- 172.21.15.78:0/864846673 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07141ac5a0 con 0x7f0714109450 2024-09-16T00:25:29.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.655+0000 7f071a302640 1 -- 172.21.15.78:0/864846673 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07141b3de0 con 0x7f0714109450 2024-09-16T00:25:29.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.659+0000 7f071c6db740 1 Processor -- start 2024-09-16T00:25:29.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.659+0000 7f0711ffb640 1 -- 172.21.15.78:0/864846673 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07040043c0 con 0x7f0714109450 2024-09-16T00:25:29.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.659+0000 7f0711ffb640 1 -- 172.21.15.78:0/864846673 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704004a00 con 0x7f0714109450 2024-09-16T00:25:29.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f071c6db740 1 -- start start 2024-09-16T00:25:29.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f071c6db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e20b40 0x5621d2e1f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f071c6db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f071c6db740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e2a370 0x5621d2e2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f071c6db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621d2d852f0 con 0x5621d2e20b40 2024-09-16T00:25:29.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f071c6db740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5621d2d2ccf0 con 0x5621d2e2a370 2024-09-16T00:25:29.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f071c6db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621d2d812b0 con 0x5621d2e1f5d0 2024-09-16T00:25:29.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f070bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e2a370 0x5621d2e2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f070bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e2a370 0x5621d2e2c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57042/0 (socket says 172.21.15.78:57042) 2024-09-16T00:25:29.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f070bfff640 1 -- 172.21.15.78:0/1563490694 learned_addr learned my addr 172.21.15.78:0/1563490694 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:29.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.664+0000 7f071889f640 1 --2- 172.21.15.78:0/1563490694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e20b40 0x5621d2e1f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.671+0000 7f071889f640 1 -- 172.21.15.78:0/1563490694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 msgr2=0x5621d2e1fa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:25:29.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.671+0000 7f071889f640 1 --2- 172.21.15.78:0/1563490694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e1fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.671+0000 7f071889f640 1 -- 172.21.15.78:0/1563490694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e2a370 msgr2=0x5621d2e2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:29.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.671+0000 7f071889f640 1 --2- 172.21.15.78:0/1563490694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e2a370 0x5621d2e2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.671+0000 7f071889f640 1 -- 172.21.15.78:0/1563490694 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d2d113a0 con 0x5621d2e20b40 2024-09-16T00:25:29.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.676+0000 7f0711ffb640 1 -- 172.21.15.78:0/864846673 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f07040079b0 con 0x7f0714109450 2024-09-16T00:25:29.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.677+0000 7f0711ffb640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07000a3380 0x7f07000a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.678+0000 7f0711ffb640 1 -- 172.21.15.78:0/864846673 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f07040cf930 con 0x7f0714109450 2024-09-16T00:25:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.679+0000 7f0713fff640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07000a3380 0x7f07000a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.684+0000 7f071889f640 1 --2- 172.21.15.78:0/1563490694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e20b40 0x5621d2e1f090 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f070c009ca0 tx=0x7f070c0306f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363d64501814aa0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:29.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.685+0000 7f0713fff640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07000a3380 0x7f07000a5840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f07040058c0 tx=0x7f07040046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:29.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.685+0000 7f070a7fc640 1 -- 172.21.15.78:0/1563490694 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c0307e0 con 0x5621d2e20b40 2024-09-16T00:25:29.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.685+0000 7f070a7fc640 1 -- 172.21.15.78:0/1563490694 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c036040 con 0x5621d2e20b40 2024-09-16T00:25:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.688+0000 7f071c6db740 1 -- 172.21.15.78:0/1563490694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e20b40 msgr2=0x5621d2e1f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.688+0000 7f071c6db740 1 --2- 172.21.15.78:0/1563490694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e20b40 0x5621d2e1f090 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f070c009ca0 tx=0x7f070c0306f0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.690+0000 7f071c6db740 1 -- 172.21.15.78:0/1563490694 shutdown_connections 2024-09-16T00:25:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.690+0000 7f071c6db740 1 --2- 172.21.15.78:0/1563490694 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e2a370 0x5621d2e2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.690+0000 7f071c6db740 1 --2- 172.21.15.78:0/1563490694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e1fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.690+0000 7f071c6db740 1 --2- 172.21.15.78:0/1563490694 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e20b40 0x5621d2e1f090 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.690+0000 7f071c6db740 1 -- 172.21.15.78:0/1563490694 >> 172.21.15.78:0/1563490694 conn(0x5621d2de8310 msgr2=0x5621d2e1bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:29.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.692+0000 7f071c6db740 1 -- 172.21.15.78:0/1563490694 shutdown_connections 2024-09-16T00:25:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.694+0000 7f071c6db740 1 -- 172.21.15.78:0/1563490694 wait complete. 2024-09-16T00:25:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.694+0000 7f071c6db740 1 -- start start 2024-09-16T00:25:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.695+0000 7f071c6db740 1 Processor -- start 2024-09-16T00:25:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.695+0000 7f071c6db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e29b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.695+0000 7f071c6db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e26e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.695+0000 7f071c6db740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e277d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.695+0000 7f071c6db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5621d2e3e0c0 con 0x5621d2e2a370 2024-09-16T00:25:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.695+0000 7f071c6db740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5621d2d40060 con 0x5621d2e27350 2024-09-16T00:25:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.695+0000 7f071c6db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621d2e3e240 con 0x5621d2e1f5d0 2024-09-16T00:25:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.695+0000 7f071c6db740 10 client.0 _ll_register_callbacks cb 0x5621d2a938c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.697+0000 7f071889f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.697+0000 7f071889f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e277d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57054/0 (socket says 172.21.15.78:57054) 2024-09-16T00:25:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.697+0000 7f071889f640 1 -- 172.21.15.78:0/643319213 learned_addr learned my addr 172.21.15.78:0/643319213 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.701+0000 7f070bfff640 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e26e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.701+0000 7f071889f640 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e29b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.702+0000 7f071889f640 1 -- 172.21.15.78:0/643319213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 msgr2=0x5621d2e29b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.702+0000 7f071889f640 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e29b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.702+0000 7f071889f640 1 -- 172.21.15.78:0/643319213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 msgr2=0x5621d2e26e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.702+0000 7f071889f640 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e26e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.702+0000 7f071889f640 1 -- 172.21.15.78:0/643319213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d2af63c0 con 0x5621d2e27350 2024-09-16T00:25:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.704+0000 7f071889f640 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e277d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f070c009620 tx=0x7f070c02a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b376c2a9a605789 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.704+0000 7f06f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:25:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.712+0000 7f06f37fe640 1 -- 172.21.15.78:0/643319213 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c030e90 con 0x5621d2e27350 2024-09-16T00:25:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.712+0000 7f071c6db740 1 -- 172.21.15.78:0/643319213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621d2d92860 con 0x5621d2e27350 2024-09-16T00:25:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.712+0000 7f06f37fe640 1 -- 172.21.15.78:0/643319213 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f070c02f560 con 0x5621d2e27350 2024-09-16T00:25:29.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.718+0000 7f06f37fe640 1 -- 172.21.15.78:0/643319213 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f070c05b050 con 0x5621d2e27350 2024-09-16T00:25:29.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.718+0000 7f071c6db740 10 client.26831 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:29.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.718+0000 7f071c6db740 1 -- 172.21.15.78:0/643319213 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5621d2e49eb0 con 0x5621d2e27350 2024-09-16T00:25:29.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.719+0000 7f06f37fe640 1 -- 172.21.15.78:0/643319213 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5621d2d92860 con 0x5621d2e27350 2024-09-16T00:25:29.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.719+0000 7f06f37fe640 1 -- 172.21.15.78:0/643319213 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f070c031c70 con 0x5621d2e27350 2024-09-16T00:25:29.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.719+0000 7f071c6db740 10 client.26831 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:29.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.719+0000 7f071c6db740 10 client.26831 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:25:29.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.719+0000 7f071c6db740 20 client.26831 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:29.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.719+0000 7f071c6db740 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5621d2e4b560 0x5621d2e4d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.719+0000 7f071c6db740 4 client.26831 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:29.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.719+0000 7f071c6db740 1 -- 172.21.15.78:0/643319213 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621d2d8cd70 con 0x5621d2e4b560 2024-09-16T00:25:29.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.720+0000 7f070bfff640 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5621d2e4b560 0x5621d2e4d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.722+0000 7f070bfff640 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5621d2e4b560 0x5621d2e4d980 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f81152871240c09d server_cookie=8dc79e72a1cac192 in_seq=0 out_seq=0 2024-09-16T00:25:29.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.722+0000 7f06f37fe640 10 client.26831 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f06f37fe640 1 -- 172.21.15.78:0/643319213 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621d2d8cd70 con 0x5621d2e4b560 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f06f37fe640 10 client.26831 handle_command_reply: tid=0 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 client.26831 shutdown 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 -- 172.21.15.78:0/643319213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 msgr2=0x5621d2e277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e277d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f070c009620 tx=0x7f070c02a8f0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 -- 172.21.15.78:0/643319213 shutdown_connections 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5621d2e4b560 0x5621d2e4d980 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e277d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e26e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 --2- 172.21.15.78:0/643319213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e29b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f06f37fe640 0 client.26831 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:25:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.727+0000 7f071c6db740 1 -- 172.21.15.78:0/643319213 >> 172.21.15.78:0/643319213 conn(0x5621d2de8310 msgr2=0x5621d2e219c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:29.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.732+0000 7f071c6db740 1 -- 172.21.15.78:0/643319213 shutdown_connections 2024-09-16T00:25:29.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.732+0000 7f071c6db740 1 -- 172.21.15.78:0/643319213 wait complete. 2024-09-16T00:25:29.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.734+0000 7f071c6db740 20 client.26831 trim_cache size 0 max 16384 2024-09-16T00:25:29.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.760+0000 7f071c6db740 1 Processor -- start 2024-09-16T00:25:29.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.760+0000 7f071c6db740 1 -- start start 2024-09-16T00:25:29.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.761+0000 7f071c6db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.761+0000 7f071c6db740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e26c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.761+0000 7f071c6db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.761+0000 7f071c6db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06f400fbb0 con 0x5621d2e2a370 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.761+0000 7f071c6db740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06f40116f0 con 0x5621d2e27350 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.761+0000 7f071c6db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4010280 con 0x5621d2e1f5d0 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.762+0000 7f070bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e26c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.762+0000 7f071889f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.762+0000 7f071889f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e53230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36264/0 (socket says 172.21.15.78:36264) 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.762+0000 7f071889f640 1 -- 172.21.15.78:0/1195092882 learned_addr learned my addr 172.21.15.78:0/1195092882 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.762+0000 7f071889f640 1 --2- 172.21.15.78:0/1195092882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.763+0000 7f070bfff640 1 -- 172.21.15.78:0/1195092882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 msgr2=0x5621d2e266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:29.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.763+0000 7f070bfff640 1 --2- 172.21.15.78:0/1195092882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.763+0000 7f070bfff640 1 -- 172.21.15.78:0/1195092882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 msgr2=0x5621d2e53230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:29.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.763+0000 7f070bfff640 1 --2- 172.21.15.78:0/1195092882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e53230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.763+0000 7f070bfff640 1 -- 172.21.15.78:0/1195092882 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d2e1c1a0 con 0x5621d2e27350 2024-09-16T00:25:29.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.764+0000 7f070bfff640 1 --2- 172.21.15.78:0/1195092882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e26c10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5621d2e4e1d0 tx=0x7f06fc009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b02cddb4626e3594 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:29.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.767+0000 7f0709ffb640 1 -- 172.21.15.78:0/1195092882 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc016ac0 con 0x5621d2e27350 2024-09-16T00:25:29.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.767+0000 7f0709ffb640 1 -- 172.21.15.78:0/1195092882 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc016c60 con 0x5621d2e27350 2024-09-16T00:25:29.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.767+0000 7f0709ffb640 1 -- 172.21.15.78:0/1195092882 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0145f0 con 0x5621d2e27350 2024-09-16T00:25:29.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.768+0000 7f071c6db740 1 -- 172.21.15.78:0/1195092882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 msgr2=0x5621d2e26c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:29.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.768+0000 7f071c6db740 1 --2- 172.21.15.78:0/1195092882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e26c10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5621d2e4e1d0 tx=0x7f06fc009190 comp rx=0 tx=0).stop 2024-09-16T00:25:29.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.772+0000 7f071c6db740 1 -- 172.21.15.78:0/1195092882 shutdown_connections 2024-09-16T00:25:29.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.772+0000 7f071c6db740 1 --2- 172.21.15.78:0/1195092882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e53230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.772+0000 7f071c6db740 1 --2- 172.21.15.78:0/1195092882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e26c10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.772+0000 7f071c6db740 1 --2- 172.21.15.78:0/1195092882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.772+0000 7f071c6db740 1 -- 172.21.15.78:0/1195092882 >> 172.21.15.78:0/1195092882 conn(0x5621d2de8310 msgr2=0x5621d2e256e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:29.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.773+0000 7f071c6db740 1 -- 172.21.15.78:0/1195092882 shutdown_connections 2024-09-16T00:25:29.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.776+0000 7f071c6db740 1 -- 172.21.15.78:0/1195092882 wait complete. 2024-09-16T00:25:29.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.776+0000 7f071c6db740 1 -- start start 2024-09-16T00:25:29.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.777+0000 7f071c6db740 1 Processor -- start 2024-09-16T00:25:29.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.777+0000 7f071c6db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e43b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.777+0000 7f071c6db740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e239a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.777+0000 7f071c6db740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e23ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.777+0000 7f071c6db740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4011020 con 0x5621d2e2a370 2024-09-16T00:25:29.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.777+0000 7f071c6db740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4010950 con 0x5621d2e27350 2024-09-16T00:25:29.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.777+0000 7f071c6db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06f400db90 con 0x5621d2e1f5d0 2024-09-16T00:25:29.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.777+0000 7f071c6db740 10 client.0 _ll_register_callbacks cb 0x7f070c03f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:29.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.782+0000 7f070bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.782+0000 7f071889f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e43b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.782+0000 7f071889f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e43b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47270/0 (socket says 172.21.15.78:47270) 2024-09-16T00:25:29.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.782+0000 7f071889f640 1 -- 172.21.15.78:0/1289666159 learned_addr learned my addr 172.21.15.78:0/1289666159 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:29.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.782+0000 7f071889f640 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e23ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.786+0000 7f071889f640 1 -- 172.21.15.78:0/1289666159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 msgr2=0x5621d2e239a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:29.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.786+0000 7f071889f640 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e239a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.786+0000 7f071889f640 1 -- 172.21.15.78:0/1289666159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 msgr2=0x5621d2e23ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:29.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.786+0000 7f071889f640 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e23ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.786+0000 7f071889f640 1 -- 172.21.15.78:0/1289666159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d2e28280 con 0x5621d2e1f5d0 2024-09-16T00:25:29.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.786+0000 7f071889f640 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e43b10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f070c009a60 tx=0x7f070c02f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ba6a3d4c43aa69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:29.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.786+0000 7f06f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:25:29.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.791+0000 7f06f37fe640 1 -- 172.21.15.78:0/1289666159 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c058020 con 0x5621d2e1f5d0 2024-09-16T00:25:29.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.791+0000 7f071c6db740 1 -- 172.21.15.78:0/1289666159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06fc016c60 con 0x5621d2e1f5d0 2024-09-16T00:25:29.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.791+0000 7f06f37fe640 1 -- 172.21.15.78:0/1289666159 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f070c055040 con 0x5621d2e1f5d0 2024-09-16T00:25:29.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.805+0000 7f06f37fe640 1 -- 172.21.15.78:0/1289666159 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f070c030e90 con 0x5621d2e1f5d0 2024-09-16T00:25:29.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.808+0000 7f071c6db740 10 client.26647 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:29.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.808+0000 7f071c6db740 1 -- 172.21.15.78:0/1289666159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5621d2eb5e80 con 0x5621d2e1f5d0 2024-09-16T00:25:29.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.810+0000 7f06f37fe640 1 -- 172.21.15.78:0/1289666159 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06fc016c60 con 0x5621d2e1f5d0 2024-09-16T00:25:29.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.810+0000 7f06f37fe640 1 -- 172.21.15.78:0/1289666159 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f070c05e040 con 0x5621d2e1f5d0 2024-09-16T00:25:29.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.810+0000 7f071c6db740 10 client.26647 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:29.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.810+0000 7f071c6db740 10 client.26647 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:25:29.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.810+0000 7f071c6db740 20 client.26647 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:29.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.810+0000 7f071c6db740 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5621d2eb6380 0x5621d2eb87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:29.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.810+0000 7f071c6db740 4 client.26647 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:25:29.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.810+0000 7f071c6db740 1 -- 172.21.15.78:0/1289666159 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621d2d09170 con 0x5621d2eb6380 2024-09-16T00:25:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.811+0000 7f070bfff640 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5621d2eb6380 0x5621d2eb87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.811+0000 7f06f37fe640 10 client.26647 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:25:29.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.812+0000 7f070bfff640 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5621d2eb6380 0x5621d2eb87a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=99743eb6d7b42ed9 server_cookie=be27c3b2d0a5fafc in_seq=0 out_seq=0 2024-09-16T00:25:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.812+0000 7f06f37fe640 1 -- 172.21.15.78:0/1289666159 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621d2d09170 con 0x5621d2eb6380 2024-09-16T00:25:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.812+0000 7f06f37fe640 10 client.26647 handle_command_reply: tid=0 2024-09-16T00:25:29.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.812+0000 7f071c6db740 1 client.26647 shutdown 2024-09-16T00:25:29.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.813+0000 7f071c6db740 1 -- 172.21.15.78:0/1289666159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 msgr2=0x5621d2e43b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.813+0000 7f071c6db740 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e43b10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f070c009a60 tx=0x7f070c02f440 comp rx=0 tx=0).stop 2024-09-16T00:25:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.813+0000 7f071c6db740 1 -- 172.21.15.78:0/1289666159 shutdown_connections 2024-09-16T00:25:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.813+0000 7f071c6db740 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5621d2e2a370 0x5621d2e23ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.813+0000 7f071c6db740 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5621d2eb6380 0x5621d2eb87a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.813+0000 7f071c6db740 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5621d2e27350 0x5621d2e239a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.813+0000 7f071c6db740 1 --2- 172.21.15.78:0/1289666159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621d2e1f5d0 0x5621d2e43b10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.813+0000 7f071c6db740 1 -- 172.21.15.78:0/1289666159 >> 172.21.15.78:0/1289666159 conn(0x5621d2de8310 msgr2=0x5621d2e28670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.814+0000 7f06f37fe640 0 client.26647 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:25:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.814+0000 7f071c6db740 1 -- 172.21.15.78:0/1289666159 shutdown_connections 2024-09-16T00:25:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.814+0000 7f071c6db740 1 -- 172.21.15.78:0/1289666159 wait complete. 2024-09-16T00:25:29.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.814+0000 7f071c6db740 20 client.26647 trim_cache size 0 max 16384 2024-09-16T00:25:29.818 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 -- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07000a3380 msgr2=0x7f07000a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07000a3380 0x7f07000a5840 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f07040058c0 tx=0x7f07040046c0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 -- 172.21.15.78:0/864846673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0714109450 msgr2=0x7f07141a7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0714109450 0x7f07141a7260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f070400c890 tx=0x7f070400cd50 comp rx=0 tx=0).stop 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 -- 172.21.15.78:0/864846673 shutdown_connections 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f07000a3380 0x7f07000a5840 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f071411b140 0x7f07141abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0714109830 0x7f07141a77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 --2- 172.21.15.78:0/864846673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0714109450 0x7f07141a7260 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f070400c890 tx=0x7f070400cd50 comp rx=0 tx=0).stop 2024-09-16T00:25:29.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 -- 172.21.15.78:0/864846673 >> 172.21.15.78:0/864846673 conn(0x7f071406e140 msgr2=0x7f071411b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:29.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 -- 172.21.15.78:0/864846673 shutdown_connections 2024-09-16T00:25:29.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:29.815+0000 7f071a302640 1 -- 172.21.15.78:0/864846673 wait complete. 2024-09-16T00:25:30.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:29 smithi078 ceph-mon[31157]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-16T00:25:30.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:29 smithi193 ceph-mon[35484]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-16T00:25:30.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:29 smithi148 ceph-mon[35599]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-16T00:25:30.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 690.0 s, 429.0 tps, lat 176.485 ms stddev 192.931 2024-09-16T00:25:31.904 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:25:32.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:31 smithi193 ceph-mon[35484]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-16T00:25:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:31 smithi148 ceph-mon[35599]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-16T00:25:32.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:31 smithi078 ceph-mon[31157]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-16T00:25:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.986+0000 7f2c695c0640 1 -- 172.21.15.78:0/2903067172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64100a70 msgr2=0x7f2c64100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.986+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2903067172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64100a70 0x7f2c64100ed0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009930 tx=0x7f2c5802c630 comp rx=0 tx=0).stop 2024-09-16T00:25:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.986+0000 7f2c695c0640 1 -- 172.21.15.78:0/2903067172 shutdown_connections 2024-09-16T00:25:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.986+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2903067172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c64101410 0x7f2c6410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.986+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2903067172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64100a70 0x7f2c64100ed0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.986+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2903067172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c64106a70 0x7f2c64106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.986+0000 7f2c695c0640 1 -- 172.21.15.78:0/2903067172 >> 172.21.15.78:0/2903067172 conn(0x7f2c640fc7d0 msgr2=0x7f2c640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:32.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.987+0000 7f2c695c0640 1 -- 172.21.15.78:0/2903067172 shutdown_connections 2024-09-16T00:25:32.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.988+0000 7f2c695c0640 1 -- 172.21.15.78:0/2903067172 wait complete. 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.988+0000 7f2c695c0640 1 Processor -- start 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.988+0000 7f2c695c0640 1 -- start start 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c695c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c64100a70 0x7f2c64196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c695c0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64101410 0x7f2c641967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c695c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c64106a70 0x7f2c6419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c695c0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6410fd60 con 0x7f2c64101410 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c695c0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6410fbe0 con 0x7f2c64106a70 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64101410 0x7f2c641967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c695c0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6410fee0 con 0x7f2c64100a70 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64101410 0x7f2c641967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36286/0 (socket says 172.21.15.78:36286) 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c627fc640 1 -- 172.21.15.78:0/2784922774 learned_addr learned my addr 172.21.15.78:0/2784922774 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c64100a70 0x7f2c64196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c637fe640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c64106a70 0x7f2c6419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c627fc640 1 -- 172.21.15.78:0/2784922774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c64100a70 msgr2=0x7f2c64196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c627fc640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c64100a70 0x7f2c64196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c627fc640 1 -- 172.21.15.78:0/2784922774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c64106a70 msgr2=0x7f2c6419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c627fc640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c64106a70 0x7f2c6419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.989+0000 7f2c627fc640 1 -- 172.21.15.78:0/2784922774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6419b230 con 0x7f2c64101410 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.990+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c64100a70 0x7f2c64196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.990+0000 7f2c627fc640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64101410 0x7f2c641967c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009230 tx=0x7f2c58004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:32.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.990+0000 7f2c4bfff640 1 -- 172.21.15.78:0/2784922774 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58041030 con 0x7f2c64101410 2024-09-16T00:25:32.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.990+0000 7f2c4bfff640 1 -- 172.21.15.78:0/2784922774 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c58004440 con 0x7f2c64101410 2024-09-16T00:25:32.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.990+0000 7f2c695c0640 1 -- 172.21.15.78:0/2784922774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6419b4c0 con 0x7f2c64101410 2024-09-16T00:25:32.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.990+0000 7f2c4bfff640 1 -- 172.21.15.78:0/2784922774 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58037070 con 0x7f2c64101410 2024-09-16T00:25:32.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.990+0000 7f2c695c0640 1 -- 172.21.15.78:0/2784922774 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c641a2e00 con 0x7f2c64101410 2024-09-16T00:25:32.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.993+0000 7f2c6b999740 1 Processor -- start 2024-09-16T00:25:32.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.993+0000 7f2c6b999740 1 -- start start 2024-09-16T00:25:32.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.993+0000 7f2c4bfff640 1 -- 172.21.15.78:0/2784922774 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2c58044030 con 0x7f2c64101410 2024-09-16T00:25:32.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada50c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada4f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada64f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c6b999740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad9b52f0 con 0x55e4ada50840 2024-09-16T00:25:32.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c6b999740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad95ccf0 con 0x55e4ada4f120 2024-09-16T00:25:32.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c6b999740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad9b12b0 con 0x55e4ada4fb80 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c627fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada64f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c62ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada4f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c62ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada4f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57094/0 (socket says 172.21.15.78:57094) 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c62ffd640 1 -- 172.21.15.78:0/3252615624 learned_addr learned my addr 172.21.15.78:0/3252615624 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c62ffd640 1 -- 172.21.15.78:0/3252615624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 msgr2=0x55e4ada64f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/3252615624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada64f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c62ffd640 1 -- 172.21.15.78:0/3252615624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 msgr2=0x55e4ada50c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/3252615624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada50c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.994+0000 7f2c62ffd640 1 -- 172.21.15.78:0/3252615624 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ad9413a0 con 0x55e4ada4f120 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.995+0000 7f2c4bfff640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c4c0a3240 0x7f2c4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:32.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.995+0000 7f2c4bfff640 1 -- 172.21.15.78:0/2784922774 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2c5800c2d0 con 0x7f2c64101410 2024-09-16T00:25:32.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.995+0000 7f2c637fe640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c4c0a3240 0x7f2c4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:32.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.995+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/3252615624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada4f560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c54011520 tx=0x7f2c540119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87c27bb5c3b3920 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:32.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.996+0000 7f2c497fa640 1 -- 172.21.15.78:0/3252615624 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5401e070 con 0x55e4ada4f120 2024-09-16T00:25:32.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.996+0000 7f2c497fa640 1 -- 172.21.15.78:0/3252615624 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c5400fac0 con 0x55e4ada4f120 2024-09-16T00:25:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c497fa640 1 -- 172.21.15.78:0/3252615624 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54019c20 con 0x55e4ada4f120 2024-09-16T00:25:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 -- 172.21.15.78:0/3252615624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 msgr2=0x55e4ada4f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c637fe640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c4c0a3240 0x7f2c4c0a5700 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c005fd0 tx=0x7f2c5c009900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 --2- 172.21.15.78:0/3252615624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada4f560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c54011520 tx=0x7f2c540119e0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 -- 172.21.15.78:0/3252615624 shutdown_connections 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 --2- 172.21.15.78:0/3252615624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada64f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 --2- 172.21.15.78:0/3252615624 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada4f560 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 --2- 172.21.15.78:0/3252615624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada50c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 -- 172.21.15.78:0/3252615624 >> 172.21.15.78:0/3252615624 conn(0x55e4ada18310 msgr2=0x55e4ada4bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 -- 172.21.15.78:0/3252615624 shutdown_connections 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 -- 172.21.15.78:0/3252615624 wait complete. 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 -- start start 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.997+0000 7f2c6b999740 1 Processor -- start 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 0x55e4ada64cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada4fb80 0x55e4ada61fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada50840 0x55e4ada624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c6b999740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e4ada65980 con 0x55e4ada4fb80 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c6b999740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e4ad970060 con 0x55e4ada50840 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c6b999740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4ada65b00 con 0x55e4ada4f120 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c6b999740 10 client.0 _ll_register_callbacks cb 0x55e4ad6c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada50840 0x55e4ada624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c627fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada4fb80 0x55e4ada61fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada50840 0x55e4ada624e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57102/0 (socket says 172.21.15.78:57102) 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c637fe640 1 -- 172.21.15.78:0/606241265 learned_addr learned my addr 172.21.15.78:0/606241265 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 0x55e4ada64cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c627fc640 1 -- 172.21.15.78:0/606241265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 msgr2=0x55e4ada64cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c627fc640 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 0x55e4ada64cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c627fc640 1 -- 172.21.15.78:0/606241265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada50840 msgr2=0x55e4ada624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c627fc640 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada50840 0x55e4ada624e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c627fc640 1 -- 172.21.15.78:0/606241265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ad7263c0 con 0x55e4ada4fb80 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.998+0000 7f2c637fe640 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada50840 0x55e4ada624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c627fc640 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada4fb80 0x55e4ada61fa0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2c580ef730 tx=0x7f2c5800c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e480ba76eca44646 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c46ff5640 1 -- 172.21.15.78:0/606241265 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5804b050 con 0x55e4ada4fb80 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 1 -- 172.21.15.78:0/606241265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c5400fac0 con 0x55e4ada4fb80 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c46ff5640 1 -- 172.21.15.78:0/606241265 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2c580de0b0 con 0x55e4ada4fb80 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c46ff5640 1 -- 172.21.15.78:0/606241265 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2c580bdd40 con 0x55e4ada4fb80 2024-09-16T00:25:33.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 10 client.17604 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 1 -- 172.21.15.78:0/606241265 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e4ada712e0 con 0x55e4ada4fb80 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c46ff5640 1 -- 172.21.15.78:0/606241265 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c5400fac0 con 0x55e4ada4fb80 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c46ff5640 1 -- 172.21.15.78:0/606241265 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2c580c2150 con 0x55e4ada4fb80 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 10 client.17604 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 10 client.17604 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 20 client.17604 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55e4ada726a0 0x55e4ada74ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 4 client.17604 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:32.999+0000 7f2c6b999740 1 -- 172.21.15.78:0/606241265 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4ad9c2860 con 0x55e4ada726a0 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.000+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55e4ada726a0 0x55e4ada74ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.000+0000 7f2c46ff5640 10 client.17604 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:25:33.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.000+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55e4ada726a0 0x55e4ada74ad0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=513cb3c20b899601 server_cookie=987ac76ce78cd475 in_seq=0 out_seq=0 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c46ff5640 1 -- 172.21.15.78:0/606241265 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4ad9c2860 con 0x55e4ada726a0 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c46ff5640 10 client.17604 handle_command_reply: tid=0 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 client.17604 shutdown 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 -- 172.21.15.78:0/606241265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada4fb80 msgr2=0x55e4ada61fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada4fb80 0x55e4ada61fa0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2c580ef730 tx=0x7f2c5800c5a0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 -- 172.21.15.78:0/606241265 shutdown_connections 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55e4ada726a0 0x55e4ada74ad0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c46ff5640 0 client.17604 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada50840 0x55e4ada624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada4fb80 0x55e4ada61fa0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 --2- 172.21.15.78:0/606241265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 0x55e4ada64cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 -- 172.21.15.78:0/606241265 >> 172.21.15.78:0/606241265 conn(0x55e4ada18310 msgr2=0x55e4ada519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:33.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 -- 172.21.15.78:0/606241265 shutdown_connections 2024-09-16T00:25:33.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 1 -- 172.21.15.78:0/606241265 wait complete. 2024-09-16T00:25:33.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.001+0000 7f2c6b999740 20 client.17604 trim_cache size 0 max 16384 2024-09-16T00:25:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c6b999740 1 Processor -- start 2024-09-16T00:25:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c6b999740 1 -- start start 2024-09-16T00:25:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada6ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada55e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada53190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c6b999740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641ac8f0 con 0x55e4ada50840 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c6b999740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641ae430 con 0x55e4ada4f120 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c6b999740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641acfc0 con 0x55e4ada4fb80 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada53190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c62ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada6ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c62ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada6ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57104/0 (socket says 172.21.15.78:57104) 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada53190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36318/0 (socket says 172.21.15.78:36318) 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c62ffd640 1 -- 172.21.15.78:0/3503912547 learned_addr learned my addr 172.21.15.78:0/3503912547 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.020+0000 7f2c627fc640 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada55e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c637fe640 1 -- 172.21.15.78:0/3503912547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 msgr2=0x55e4ada55e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c637fe640 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada55e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c637fe640 1 -- 172.21.15.78:0/3503912547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 msgr2=0x55e4ada6ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:33.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c637fe640 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada6ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c637fe640 1 -- 172.21.15.78:0/3503912547 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ada50f50 con 0x55e4ada50840 2024-09-16T00:25:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada6ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:25:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c627fc640 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada55e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:25:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c637fe640 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada53190 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009e40 tx=0x7f2c5c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41a97514b218a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c48ff9640 1 -- 172.21.15.78:0/3503912547 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c004b20 con 0x55e4ada50840 2024-09-16T00:25:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c48ff9640 1 -- 172.21.15.78:0/3503912547 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c5c01dda0 con 0x55e4ada50840 2024-09-16T00:25:33.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.021+0000 7f2c48ff9640 1 -- 172.21.15.78:0/3503912547 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c005750 con 0x55e4ada50840 2024-09-16T00:25:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.022+0000 7f2c6b999740 1 -- 172.21.15.78:0/3503912547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 msgr2=0x55e4ada53190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.022+0000 7f2c6b999740 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada53190 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009e40 tx=0x7f2c5c014000 comp rx=0 tx=0).stop 2024-09-16T00:25:33.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.022+0000 7f2c6b999740 1 -- 172.21.15.78:0/3503912547 shutdown_connections 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.022+0000 7f2c6b999740 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada53190 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.022+0000 7f2c6b999740 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4fb80 0x55e4ada55e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.022+0000 7f2c6b999740 1 --2- 172.21.15.78:0/3503912547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4f120 0x55e4ada6ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.022+0000 7f2c6b999740 1 -- 172.21.15.78:0/3503912547 >> 172.21.15.78:0/3503912547 conn(0x55e4ada18310 msgr2=0x55e4ada55750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.022+0000 7f2c6b999740 1 -- 172.21.15.78:0/3503912547 shutdown_connections 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.023+0000 7f2c6b999740 1 -- 172.21.15.78:0/3503912547 wait complete. 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.023+0000 7f2c6b999740 1 -- start start 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.023+0000 7f2c6b999740 1 Processor -- start 2024-09-16T00:25:33.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 0x55e4ada6b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4fb80 0x55e4ada6b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c6b999740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada66510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c6b999740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641add60 con 0x55e4ada50840 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c6b999740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641ad690 con 0x55e4ada4fb80 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c6b999740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641aa8d0 con 0x55e4ada4f120 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c6b999740 10 client.0 _ll_register_callbacks cb 0x7f2c58009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada66510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c62ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 0x55e4ada6b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada66510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36334/0 (socket says 172.21.15.78:36334) 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 -- 172.21.15.78:0/1996020245 learned_addr learned my addr 172.21.15.78:0/1996020245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c627fc640 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4fb80 0x55e4ada6b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 -- 172.21.15.78:0/1996020245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 msgr2=0x55e4ada6b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 0x55e4ada6b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 -- 172.21.15.78:0/1996020245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4fb80 msgr2=0x55e4ada6b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4fb80 0x55e4ada6b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 -- 172.21.15.78:0/1996020245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ada66d10 con 0x55e4ada50840 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c637fe640 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada66510 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009b80 tx=0x7f2c5c01dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7899c8b68e2b2c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.024+0000 7f2c46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.025+0000 7f2c46ff5640 1 -- 172.21.15.78:0/1996020245 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c034020 con 0x55e4ada50840 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.025+0000 7f2c46ff5640 1 -- 172.21.15.78:0/1996020245 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2c5c005490 con 0x55e4ada50840 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.025+0000 7f2c6b999740 1 -- 172.21.15.78:0/1996020245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c5c01dda0 con 0x55e4ada50840 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.025+0000 7f2c46ff5640 1 -- 172.21.15.78:0/1996020245 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2c5c005690 con 0x55e4ada50840 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.025+0000 7f2c6b999740 10 client.17610 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.025+0000 7f2c6b999740 1 -- 172.21.15.78:0/1996020245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e4adadcf10 con 0x55e4ada50840 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.025+0000 7f2c46ff5640 1 -- 172.21.15.78:0/1996020245 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c5c01dda0 con 0x55e4ada50840 2024-09-16T00:25:33.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c46ff5640 1 -- 172.21.15.78:0/1996020245 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2c5c025070 con 0x55e4ada50840 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c6b999740 10 client.17610 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c6b999740 10 client.17610 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c6b999740 20 client.17610 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c6b999740 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55e4ada726a0 0x55e4adadf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c6b999740 4 client.17610 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c6b999740 1 -- 172.21.15.78:0/1996020245 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4ad9bcd70 con 0x55e4ada726a0 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55e4ada726a0 0x55e4adadf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c46ff5640 10 client.17610 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.026+0000 7f2c62ffd640 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55e4ada726a0 0x55e4adadf640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a1f284b7ea2b1f5 server_cookie=f2b5075acb1b7c66 in_seq=0 out_seq=0 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c46ff5640 1 -- 172.21.15.78:0/1996020245 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4ad9bcd70 con 0x55e4ada726a0 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c46ff5640 10 client.17610 handle_command_reply: tid=0 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 client.17610 shutdown 2024-09-16T00:25:33.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 -- 172.21.15.78:0/1996020245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 msgr2=0x55e4ada66510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada66510 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009b80 tx=0x7f2c5c01dd20 comp rx=0 tx=0).stop 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 -- 172.21.15.78:0/1996020245 shutdown_connections 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e4ada50840 0x55e4ada66510 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55e4ada726a0 0x55e4adadf640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e4ada4fb80 0x55e4ada6b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 --2- 172.21.15.78:0/1996020245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4ada4f120 0x55e4ada6b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c46ff5640 0 client.17610 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 -- 172.21.15.78:0/1996020245 >> 172.21.15.78:0/1996020245 conn(0x55e4ada18310 msgr2=0x55e4ada4a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 -- 172.21.15.78:0/1996020245 shutdown_connections 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 1 -- 172.21.15.78:0/1996020245 wait complete. 2024-09-16T00:25:33.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.027+0000 7f2c6b999740 20 client.17610 trim_cache size 0 max 16384 2024-09-16T00:25:33.037 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:25:33.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 -- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c4c0a3240 msgr2=0x7f2c4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c4c0a3240 0x7f2c4c0a5700 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c005fd0 tx=0x7f2c5c009900 comp rx=0 tx=0).stop 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 -- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64101410 msgr2=0x7f2c641967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64101410 0x7f2c641967c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009230 tx=0x7f2c58004040 comp rx=0 tx=0).stop 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 -- 172.21.15.78:0/2784922774 shutdown_connections 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c4c0a3240 0x7f2c4c0a5700 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c64106a70 0x7f2c6419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c64101410 0x7f2c641967c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 --2- 172.21.15.78:0/2784922774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c64100a70 0x7f2c64196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 -- 172.21.15.78:0/2784922774 >> 172.21.15.78:0/2784922774 conn(0x7f2c640fc7d0 msgr2=0x7f2c640fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 -- 172.21.15.78:0/2784922774 shutdown_connections 2024-09-16T00:25:33.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:33.028+0000 7f2c695c0640 1 -- 172.21.15.78:0/2784922774 wait complete. 2024-09-16T00:25:33.796 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:25:33.846 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:33 smithi078 ceph-mon[31157]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-16T00:25:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:33 smithi193 ceph-mon[35484]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-16T00:25:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:33 smithi148 ceph-mon[35599]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-09-16T00:25:34.285 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.909+0000 7fa2eb99b640 1 -- 172.21.15.78:0/924065378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2e4100e20 msgr2=0x7fa2e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.909+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/924065378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2e4100e20 0x7fa2e4101200 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0009900 tx=0x7fa2e002a480 comp rx=0 tx=0).stop 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 -- 172.21.15.78:0/924065378 shutdown_connections 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/924065378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2e4102ef0 0x7fa2e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/924065378 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e4102570 0x7fa2e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/924065378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2e4100e20 0x7fa2e4101200 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 -- 172.21.15.78:0/924065378 >> 172.21.15.78:0/924065378 conn(0x7fa2e40fc7d0 msgr2=0x7fa2e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 -- 172.21.15.78:0/924065378 shutdown_connections 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 -- 172.21.15.78:0/924065378 wait complete. 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 Processor -- start 2024-09-16T00:25:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.910+0000 7fa2eb99b640 1 -- start start 2024-09-16T00:25:34.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2eb99b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2e4100e20 0x7fa2e41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2eb99b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e4102570 0x7fa2e4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2eb99b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2e4102ef0 0x7fa2e419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2eb99b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4110f80 con 0x7fa2e4100e20 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2eb99b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4110e00 con 0x7fa2e4102570 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2eb99b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4111100 con 0x7fa2e4102ef0 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e4102570 0x7fa2e4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e4102570 0x7fa2e4198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57136/0 (socket says 172.21.15.78:57136) 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e8f36640 1 -- 172.21.15.78:0/3019682432 learned_addr learned my addr 172.21.15.78:0/3019682432 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e9f38640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2e4102ef0 0x7fa2e419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e9737640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2e4100e20 0x7fa2e41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e8f36640 1 -- 172.21.15.78:0/3019682432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2e4102ef0 msgr2=0x7fa2e419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2e4102ef0 0x7fa2e419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e8f36640 1 -- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2e4100e20 msgr2=0x7fa2e41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2e4100e20 0x7fa2e41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.911+0000 7fa2e8f36640 1 -- 172.21.15.78:0/3019682432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e419d480 con 0x7fa2e4102570 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.912+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e4102570 0x7fa2e4198a10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400e9f0 tx=0x7fa2d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:34.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.912+0000 7fa2da7fc640 1 -- 172.21.15.78:0/3019682432 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400cd80 con 0x7fa2e4102570 2024-09-16T00:25:34.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.912+0000 7fa2da7fc640 1 -- 172.21.15.78:0/3019682432 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d4010910 con 0x7fa2e4102570 2024-09-16T00:25:34.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.912+0000 7fa2eb99b640 1 -- 172.21.15.78:0/3019682432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e419d770 con 0x7fa2e4102570 2024-09-16T00:25:34.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.912+0000 7fa2eb99b640 1 -- 172.21.15.78:0/3019682432 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e41a5050 con 0x7fa2e4102570 2024-09-16T00:25:34.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.912+0000 7fa2da7fc640 1 -- 172.21.15.78:0/3019682432 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4018990 con 0x7fa2e4102570 2024-09-16T00:25:34.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.915+0000 7fa2edd74740 1 Processor -- start 2024-09-16T00:25:34.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.915+0000 7fa2edd74740 1 -- start start 2024-09-16T00:25:34.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.915+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb0881ef50 0x55eb0881d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.915+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881d9e0 0x55eb0881de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.915+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08828780 0x55eb0882ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.915+0000 7fa2edd74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb0877f2f0 con 0x55eb08828780 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.915+0000 7fa2edd74740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb08726cf0 con 0x55eb0881d9e0 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.915+0000 7fa2edd74740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb0877b2b0 con 0x55eb0881ef50 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08828780 0x55eb0882ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08828780 0x55eb0882ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36374/0 (socket says 172.21.15.78:36374) 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 -- 172.21.15.78:0/2783765994 learned_addr learned my addr 172.21.15.78:0/2783765994 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 -- 172.21.15.78:0/2783765994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb0881ef50 msgr2=0x55eb0881d4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/2783765994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb0881ef50 0x55eb0881d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 -- 172.21.15.78:0/2783765994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881d9e0 msgr2=0x55eb0881de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/2783765994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881d9e0 0x55eb0881de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 -- 172.21.15.78:0/2783765994 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0870b3a0 con 0x55eb08828780 2024-09-16T00:25:34.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.916+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/2783765994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08828780 0x55eb0882ab70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40d8290 tx=0x7fa2d40d8750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34957527ae2ba5f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:34.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.917+0000 7fa2cf7fe640 1 -- 172.21.15.78:0/2783765994 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40e3070 con 0x55eb08828780 2024-09-16T00:25:34.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.918+0000 7fa2cf7fe640 1 -- 172.21.15.78:0/2783765994 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d40d09a0 con 0x55eb08828780 2024-09-16T00:25:34.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.919+0000 7fa2edd74740 1 -- 172.21.15.78:0/2783765994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08828780 msgr2=0x55eb0882ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.919+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2783765994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08828780 0x55eb0882ab70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fa2d40d8290 tx=0x7fa2d40d8750 comp rx=0 tx=0).stop 2024-09-16T00:25:34.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.920+0000 7fa2edd74740 1 -- 172.21.15.78:0/2783765994 shutdown_connections 2024-09-16T00:25:34.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.920+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2783765994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08828780 0x55eb0882ab70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.920+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2783765994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881d9e0 0x55eb0881de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.920+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2783765994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb0881ef50 0x55eb0881d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.920+0000 7fa2edd74740 1 -- 172.21.15.78:0/2783765994 >> 172.21.15.78:0/2783765994 conn(0x55eb087e2310 msgr2=0x55eb0881a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:34.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.921+0000 7fa2edd74740 1 -- 172.21.15.78:0/2783765994 shutdown_connections 2024-09-16T00:25:34.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.925+0000 7fa2edd74740 1 -- 172.21.15.78:0/2783765994 wait complete. 2024-09-16T00:25:34.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.926+0000 7fa2edd74740 1 -- start start 2024-09-16T00:25:34.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.926+0000 7fa2edd74740 1 Processor -- start 2024-09-16T00:25:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.926+0000 7fa2da7fc640 1 -- 172.21.15.78:0/3019682432 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa2d4010ac0 con 0x7fa2e4102570 2024-09-16T00:25:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.936+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb0881ef50 0x55eb08827fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.936+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.936+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08825a10 0x55eb08825e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.936+0000 7fa2edd74740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb0883c530 con 0x55eb0881ef50 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.936+0000 7fa2edd74740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55eb0873a060 con 0x55eb08825a10 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.936+0000 7fa2edd74740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eb0883c6b0 con 0x55eb08828780 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.936+0000 7fa2edd74740 10 client.0 _ll_register_callbacks cb 0x55eb0848d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.937+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08825a10 0x55eb08825e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.939+0000 7fa2e9737640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.939+0000 7fa2e9f38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb0881ef50 0x55eb08827fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.939+0000 7fa2e9f38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb0881ef50 0x55eb08827fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36378/0 (socket says 172.21.15.78:36378) 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.939+0000 7fa2e9f38640 1 -- 172.21.15.78:0/3269994149 learned_addr learned my addr 172.21.15.78:0/3269994149 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.939+0000 7fa2da7fc640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa2d00a3180 0x7fa2d00a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.939+0000 7fa2da7fc640 1 -- 172.21.15.78:0/3019682432 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa2d40d0090 con 0x7fa2e4102570 2024-09-16T00:25:34.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.940+0000 7fa2e9f38640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa2d00a3180 0x7fa2d00a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.944+0000 7fa2e8f36640 1 -- 172.21.15.78:0/3269994149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 msgr2=0x55eb088254d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.944+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088254d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.944+0000 7fa2e8f36640 1 -- 172.21.15.78:0/3269994149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb0881ef50 msgr2=0x55eb08827fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.944+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb0881ef50 0x55eb08827fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.944+0000 7fa2e8f36640 1 -- 172.21.15.78:0/3269994149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb084f03c0 con 0x55eb08825a10 2024-09-16T00:25:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.945+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08825a10 0x55eb08825e70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2d409c280 tx=0x7fa2d40de4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6531efb1ce41d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:34.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.945+0000 7fa2ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:25:34.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.945+0000 7fa2e9f38640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa2d00a3180 0x7fa2d00a5640 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009fd0 tx=0x7fa2dc009c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:34.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.945+0000 7fa2ccff9640 1 -- 172.21.15.78:0/3269994149 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4096e60 con 0x55eb08825a10 2024-09-16T00:25:34.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.946+0000 7fa2ccff9640 1 -- 172.21.15.78:0/3269994149 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa2d4098960 con 0x55eb08825a10 2024-09-16T00:25:34.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.947+0000 7fa2edd74740 1 -- 172.21.15.78:0/3269994149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d40d09a0 con 0x55eb08825a10 2024-09-16T00:25:34.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.951+0000 7fa2ccff9640 1 -- 172.21.15.78:0/3269994149 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa2d4098590 con 0x55eb08825a10 2024-09-16T00:25:34.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.951+0000 7fa2edd74740 10 client.26864 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:34.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.951+0000 7fa2edd74740 1 -- 172.21.15.78:0/3269994149 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eb0883c530 con 0x55eb08825a10 2024-09-16T00:25:34.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.955+0000 7fa2ccff9640 1 -- 172.21.15.78:0/3269994149 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d40d3d60 con 0x55eb08825a10 2024-09-16T00:25:34.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.955+0000 7fa2ccff9640 1 -- 172.21.15.78:0/3269994149 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa2d409c550 con 0x55eb08825a10 2024-09-16T00:25:34.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.955+0000 7fa2edd74740 10 client.26864 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:34.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.955+0000 7fa2edd74740 10 client.26864 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:25:34.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.955+0000 7fa2edd74740 20 client.26864 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:34.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.955+0000 7fa2edd74740 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55eb0883f200 0x55eb088415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:34.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.955+0000 7fa2edd74740 4 client.26864 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:34.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.955+0000 7fa2edd74740 1 -- 172.21.15.78:0/3269994149 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa2d40d09a0 con 0x55eb0883f200 2024-09-16T00:25:34.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.957+0000 7fa2e9737640 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55eb0883f200 0x55eb088415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:34.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.962+0000 7fa2ccff9640 10 client.26864 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:25:34.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.962+0000 7fa2e9737640 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55eb0883f200 0x55eb088415f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f18c5f0ca73aa904 server_cookie=bff5b510b8b0dc02 in_seq=0 out_seq=0 2024-09-16T00:25:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.967+0000 7fa2ccff9640 1 -- 172.21.15.78:0/3269994149 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa2d40d09a0 con 0x55eb0883f200 2024-09-16T00:25:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.967+0000 7fa2ccff9640 10 client.26864 handle_command_reply: tid=0 2024-09-16T00:25:34.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.967+0000 7fa2edd74740 1 client.26864 shutdown 2024-09-16T00:25:34.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.971+0000 7fa2edd74740 1 -- 172.21.15.78:0/3269994149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08825a10 msgr2=0x55eb08825e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.971+0000 7fa2edd74740 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08825a10 0x55eb08825e70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2d409c280 tx=0x7fa2d40de4a0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.971+0000 7fa2edd74740 1 -- 172.21.15.78:0/3269994149 shutdown_connections 2024-09-16T00:25:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.971+0000 7fa2edd74740 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55eb0883f200 0x55eb088415f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.971+0000 7fa2edd74740 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08825a10 0x55eb08825e70 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2d409c280 tx=0x7fa2d40de4a0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.971+0000 7fa2edd74740 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.971+0000 7fa2edd74740 1 --2- 172.21.15.78:0/3269994149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb0881ef50 0x55eb08827fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.971+0000 7fa2edd74740 1 -- 172.21.15.78:0/3269994149 >> 172.21.15.78:0/3269994149 conn(0x55eb087e2310 msgr2=0x55eb0881fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:34.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.972+0000 7fa2ccff9640 0 client.26864 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:25:34.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.976+0000 7fa2edd74740 1 -- 172.21.15.78:0/3269994149 shutdown_connections 2024-09-16T00:25:34.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.977+0000 7fa2edd74740 1 -- 172.21.15.78:0/3269994149 wait complete. 2024-09-16T00:25:34.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:34.977+0000 7fa2edd74740 20 client.26864 trim_cache size 0 max 16384 2024-09-16T00:25:35.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.003+0000 7fa2edd74740 1 Processor -- start 2024-09-16T00:25:35.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.007+0000 7fa2edd74740 1 -- start start 2024-09-16T00:25:35.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.007+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb0881ef50 0x55eb08820530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:35.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.007+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08828780 0x55eb08820a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.008+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 0x55eb088616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.008+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08828780 0x55eb08820a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.008+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08828780 0x55eb08820a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57156/0 (socket says 172.21.15.78:57156) 2024-09-16T00:25:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.008+0000 7fa2e8f36640 1 -- 172.21.15.78:0/1272193915 learned_addr learned my addr 172.21.15.78:0/1272193915 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.008+0000 7fa2edd74740 1 -- 172.21.15.78:0/1272193915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4073420 con 0x55eb08861250 2024-09-16T00:25:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.008+0000 7fa2edd74740 1 -- 172.21.15.78:0/1272193915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41a5d70 con 0x55eb08828780 2024-09-16T00:25:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.008+0000 7fa2edd74740 1 -- 172.21.15.78:0/1272193915 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4073af0 con 0x55eb0881ef50 2024-09-16T00:25:35.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.008+0000 7fa2e9f38640 1 --2- 172.21.15.78:0/1272193915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 0x55eb088616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.010+0000 7fa2e9f38640 1 -- 172.21.15.78:0/1272193915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb0881ef50 msgr2=0x55eb08820530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:25:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.010+0000 7fa2e9f38640 1 --2- 172.21.15.78:0/1272193915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb0881ef50 0x55eb08820530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.010+0000 7fa2e9f38640 1 -- 172.21.15.78:0/1272193915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08828780 msgr2=0x55eb08820a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:35.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.010+0000 7fa2e9f38640 1 --2- 172.21.15.78:0/1272193915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08828780 0x55eb08820a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.010+0000 7fa2e9f38640 1 -- 172.21.15.78:0/1272193915 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0882aee0 con 0x55eb08861250 2024-09-16T00:25:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.013+0000 7fa2e9f38640 1 --2- 172.21.15.78:0/1272193915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 0x55eb088616b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0137b0 tx=0x7fa2dc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12136b7fa99fa58c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:35.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.013+0000 7fa2ceffd640 1 -- 172.21.15.78:0/1272193915 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc02d070 con 0x55eb08861250 2024-09-16T00:25:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.013+0000 7fa2ceffd640 1 -- 172.21.15.78:0/1272193915 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2dc020d40 con 0x55eb08861250 2024-09-16T00:25:35.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.013+0000 7fa2ceffd640 1 -- 172.21.15.78:0/1272193915 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc010910 con 0x55eb08861250 2024-09-16T00:25:35.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.015+0000 7fa2edd74740 1 -- 172.21.15.78:0/1272193915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 msgr2=0x55eb088616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:35.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.015+0000 7fa2edd74740 1 --2- 172.21.15.78:0/1272193915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 0x55eb088616b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0137b0 tx=0x7fa2dc01e000 comp rx=0 tx=0).stop 2024-09-16T00:25:35.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.017+0000 7fa2edd74740 1 -- 172.21.15.78:0/1272193915 shutdown_connections 2024-09-16T00:25:35.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.017+0000 7fa2edd74740 1 --2- 172.21.15.78:0/1272193915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 0x55eb088616b0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.017+0000 7fa2edd74740 1 --2- 172.21.15.78:0/1272193915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb08828780 0x55eb08820a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.017+0000 7fa2edd74740 1 --2- 172.21.15.78:0/1272193915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb0881ef50 0x55eb08820530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.017+0000 7fa2edd74740 1 -- 172.21.15.78:0/1272193915 >> 172.21.15.78:0/1272193915 conn(0x55eb087e2310 msgr2=0x55eb08827550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:35.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.019+0000 7fa2edd74740 1 -- 172.21.15.78:0/1272193915 shutdown_connections 2024-09-16T00:25:35.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.019+0000 7fa2edd74740 1 -- 172.21.15.78:0/1272193915 wait complete. 2024-09-16T00:25:35.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.020+0000 7fa2edd74740 1 -- start start 2024-09-16T00:25:35.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.020+0000 7fa2edd74740 1 Processor -- start 2024-09-16T00:25:35.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.021+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881ef50 0x55eb08847550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:35.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.021+0000 7fa2edd74740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:35.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.021+0000 7fa2e9737640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881ef50 0x55eb08847550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:35.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.021+0000 7fa2e9737640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881ef50 0x55eb08847550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57172/0 (socket says 172.21.15.78:57172) 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2e8f36640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:47372/0 (socket says 172.21.15.78:47372) 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2e8f36640 1 -- 172.21.15.78:0/2676530225 learned_addr learned my addr 172.21.15.78:0/2676530225 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 0x55eb08844070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41a56a0 con 0x55eb08861250 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e40741c0 con 0x55eb0881ef50 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4071400 con 0x55eb08828780 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2edd74740 10 client.0 _ll_register_callbacks cb 0x7fa2d40e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2e8f36640 1 -- 172.21.15.78:0/2676530225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881ef50 msgr2=0x55eb08847550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881ef50 0x55eb08847550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2e8f36640 1 -- 172.21.15.78:0/2676530225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 msgr2=0x55eb08844070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 0x55eb08844070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.022+0000 7fa2e8f36640 1 -- 172.21.15.78:0/2676530225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb088c0210 con 0x55eb08828780 2024-09-16T00:25:35.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.023+0000 7fa2e8f36640 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088438e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4098750 tx=0x7fa2d409c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=117ac5c6e0b2541b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.023+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2dc020d40 con 0x55eb08828780 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.023+0000 7fa2ccff9640 10 client.26674 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.023+0000 7fa2ccff9640 1 -- 172.21.15.78:0/2676530225 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40b4060 con 0x55eb08828780 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.023+0000 7fa2ccff9640 1 -- 172.21.15.78:0/2676530225 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa2d40d3640 con 0x55eb08828780 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.023+0000 7fa2ccff9640 1 -- 172.21.15.78:0/2676530225 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa2d4096e60 con 0x55eb08828780 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.023+0000 7fa2edd74740 10 client.26674 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.024+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55eb088c1220 con 0x55eb08828780 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.024+0000 7fa2ccff9640 1 -- 172.21.15.78:0/2676530225 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa2d4098960 con 0x55eb08828780 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.024+0000 7fa2edd74740 10 client.26674 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.024+0000 7fa2edd74740 10 client.26674 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.024+0000 7fa2edd74740 20 client.26674 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.024+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55eb088c16e0 0x55eb088c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.024+0000 7fa2edd74740 4 client.26674 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:25:35.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.024+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa2dc020d40 con 0x55eb088c16e0 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.025+0000 7fa2e9737640 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55eb088c16e0 0x55eb088c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.025+0000 7fa2e9737640 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55eb088c16e0 0x55eb088c3ba0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c002e4bffbe7734c server_cookie=efff57170963b9f6 in_seq=0 out_seq=0 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.025+0000 7fa2ccff9640 10 client.26674 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.025+0000 7fa2ccff9640 1 -- 172.21.15.78:0/2676530225 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa2dc020d40 con 0x55eb088c16e0 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.025+0000 7fa2ccff9640 10 client.26674 handle_command_reply: tid=0 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.025+0000 7fa2edd74740 1 client.26674 shutdown 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 msgr2=0x55eb088438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088438e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4098750 tx=0x7fa2d409c860 comp rx=0 tx=0).stop 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 shutdown_connections 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55eb088c16e0 0x55eb088c3ba0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb08861250 0x55eb08844070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eb08828780 0x55eb088438e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2edd74740 1 --2- 172.21.15.78:0/2676530225 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55eb0881ef50 0x55eb08847550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 >> 172.21.15.78:0/2676530225 conn(0x55eb087e2310 msgr2=0x55eb08827550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:35.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.026+0000 7fa2ccff9640 0 client.26674 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:25:35.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.027+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 shutdown_connections 2024-09-16T00:25:35.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.027+0000 7fa2edd74740 1 -- 172.21.15.78:0/2676530225 wait complete. 2024-09-16T00:25:35.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.030+0000 7fa2edd74740 20 client.26674 trim_cache size 0 max 16384 2024-09-16T00:25:35.032 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.031+0000 7fa2eb99b640 1 -- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa2d00a3180 msgr2=0x7fa2d00a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.031+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa2d00a3180 0x7fa2d00a5640 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009fd0 tx=0x7fa2dc009c40 comp rx=0 tx=0).stop 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.031+0000 7fa2eb99b640 1 -- 172.21.15.78:0/3019682432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e4102570 msgr2=0x7fa2e4198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.031+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e4102570 0x7fa2e4198a10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400e9f0 tx=0x7fa2d400eeb0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.032+0000 7fa2eb99b640 1 -- 172.21.15.78:0/3019682432 shutdown_connections 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.032+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa2d00a3180 0x7fa2d00a5640 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.032+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2e4102ef0 0x7fa2e419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.032+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e4102570 0x7fa2e4198a10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.032+0000 7fa2eb99b640 1 --2- 172.21.15.78:0/3019682432 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2e4100e20 0x7fa2e41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.032+0000 7fa2eb99b640 1 -- 172.21.15.78:0/3019682432 >> 172.21.15.78:0/3019682432 conn(0x7fa2e40fc7d0 msgr2=0x7fa2e406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:35.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.033+0000 7fa2eb99b640 1 -- 172.21.15.78:0/3019682432 shutdown_connections 2024-09-16T00:25:35.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:35.033+0000 7fa2eb99b640 1 -- 172.21.15.78:0/3019682432 wait complete. 2024-09-16T00:25:35.809 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:25:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 695.0 s, 559.9 tps, lat 221.104 ms stddev 406.973 2024-09-16T00:25:36.021 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:35 smithi078 ceph-mon[31157]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 1.31k op/s 2024-09-16T00:25:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:35 smithi193 ceph-mon[35484]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 1.31k op/s 2024-09-16T00:25:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:35 smithi148 ceph-mon[35599]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 1.31k op/s 2024-09-16T00:25:36.257 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:37 smithi193 ceph-mon[35484]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T00:25:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:37 smithi148 ceph-mon[35599]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T00:25:38.206 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:37 smithi078 ceph-mon[31157]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T00:25:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:39 smithi193 ceph-mon[35484]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T00:25:40.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:39 smithi148 ceph-mon[35599]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T00:25:40.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:39 smithi078 ceph-mon[31157]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2024-09-16T00:25:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 700.0 s, 450.2 tps, lat 226.220 ms stddev 276.213 2024-09-16T00:25:42.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:41 smithi193 ceph-mon[35484]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-16T00:25:42.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:41 smithi148 ceph-mon[35599]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-16T00:25:42.230 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1859043027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 msgr2=0x7f3c38100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1859043027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 0x7f3c38100ef0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009930 tx=0x7f3c2002a420 comp rx=0 tx=0).stop 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1859043027 shutdown_connections 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1859043027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c38101430 0x7f3c3810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1859043027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 0x7f3c38100ef0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1859043027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c38106a90 0x7f3c38106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1859043027 >> 172.21.15.78:0/1859043027 conn(0x7f3c380fc7d0 msgr2=0x7f3c380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1859043027 shutdown_connections 2024-09-16T00:25:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.443+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1859043027 wait complete. 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3cd30640 1 Processor -- start 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3cd30640 1 -- start start 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3cd30640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 0x7f3c3819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3cd30640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c38101430 0x7f3c3819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3cd30640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c38106a90 0x7f3c3819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3cd30640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38110eb0 con 0x7f3c38100a90 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3cd30640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38110d30 con 0x7f3c38106a90 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3cd30640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38111030 con 0x7f3c38101430 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 0x7f3c3819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 0x7f3c3819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41964/0 (socket says 172.21.15.78:41964) 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c36d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c38106a90 0x7f3c3819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 -- 172.21.15.78:0/1744380242 learned_addr learned my addr 172.21.15.78:0/1744380242 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c35d9b640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c38101430 0x7f3c3819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 -- 172.21.15.78:0/1744380242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c38101430 msgr2=0x7f3c3819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c38101430 0x7f3c3819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 -- 172.21.15.78:0/1744380242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c38106a90 msgr2=0x7f3c3819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c38106a90 0x7f3c3819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 -- 172.21.15.78:0/1744380242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3819f640 con 0x7f3c38100a90 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c35d9b640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c38101430 0x7f3c3819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.444+0000 7f3c3659c640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 0x7f3c3819a690 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800b4a0 tx=0x7f3c2800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.445+0000 7f3c2f7fe640 1 -- 172.21.15.78:0/1744380242 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28014070 con 0x7f3c38100a90 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.445+0000 7f3c2f7fe640 1 -- 172.21.15.78:0/1744380242 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c28010af0 con 0x7f3c38100a90 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.445+0000 7f3c2f7fe640 1 -- 172.21.15.78:0/1744380242 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28017d10 con 0x7f3c38100a90 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.445+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1744380242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3819f930 con 0x7f3c38100a90 2024-09-16T00:25:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.445+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1744380242 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c381a7210 con 0x7f3c38100a90 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.447+0000 7f3c3f109740 1 Processor -- start 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.447+0000 7f3c2f7fe640 1 -- 172.21.15.78:0/1744380242 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3c28017070 con 0x7f3c38100a90 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.447+0000 7f3c3f109740 1 -- start start 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.447+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910db40 0x555b5910c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.447+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910c5d0 0x555b5910ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b59119760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3f109740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555b590722f0 con 0x555b5910c5d0 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3f109740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555b59019cf0 con 0x555b59117370 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3f109740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b5906e2b0 con 0x555b5910db40 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910c5d0 0x555b5910ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910c5d0 0x555b5910ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41974/0 (socket says 172.21.15.78:41974) 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 -- 172.21.15.78:0/3339769855 learned_addr learned my addr 172.21.15.78:0/3339769855 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910db40 0x555b5910c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c35d9b640 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b59119760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 -- 172.21.15.78:0/3339769855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910db40 msgr2=0x555b5910c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910db40 0x555b5910c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 -- 172.21.15.78:0/3339769855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 msgr2=0x555b59119760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b59119760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 -- 172.21.15.78:0/3339769855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b58ffe3a0 con 0x555b5910c5d0 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c2f7fe640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c240a3240 0x7f3c240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c240a3240 0x7f3c240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c3659c640 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910c5d0 0x555b5910ca10 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3c280d2bc0 tx=0x7f3c280d74f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991c2fe62eab34ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.448+0000 7f3c2f7fe640 1 -- 172.21.15.78:0/1744380242 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c280cea20 con 0x7f3c38100a90 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c2cff9640 1 -- 172.21.15.78:0/3339769855 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28096f80 con 0x555b5910c5d0 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c2cff9640 1 -- 172.21.15.78:0/3339769855 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2809b3e0 con 0x555b5910c5d0 2024-09-16T00:25:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c2cff9640 1 -- 172.21.15.78:0/3339769855 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2809a640 con 0x555b5910c5d0 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 -- 172.21.15.78:0/3339769855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910c5d0 msgr2=0x555b5910ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c240a3240 0x7f3c240a5700 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3c30005fd0 tx=0x7f3c30009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910c5d0 0x555b5910ca10 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3c280d2bc0 tx=0x7f3c280d74f0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 -- 172.21.15.78:0/3339769855 shutdown_connections 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b59119760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910c5d0 0x555b5910ca10 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 --2- 172.21.15.78:0/3339769855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910db40 0x555b5910c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 -- 172.21.15.78:0/3339769855 >> 172.21.15.78:0/3339769855 conn(0x555b590d5310 msgr2=0x555b59108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 -- 172.21.15.78:0/3339769855 shutdown_connections 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.449+0000 7f3c3f109740 1 -- 172.21.15.78:0/3339769855 wait complete. 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 1 -- start start 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 1 Processor -- start 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 0x555b59114630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b59114b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555b5911a010 con 0x555b59117370 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555b5902d060 con 0x555b5910db40 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555b5911a190 con 0x555b5910c5d0 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3f109740 10 client.0 _ll_register_callbacks cb 0x555b58d808c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b59114b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b59114b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:41988/0 (socket says 172.21.15.78:41988) 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c36d9d640 1 -- 172.21.15.78:0/2314382728 learned_addr learned my addr 172.21.15.78:0/2314382728 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c3659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34810/0 (socket says 172.21.15.78:34810) 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.450+0000 7f3c35d9b640 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 0x555b59114630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c36d9d640 1 -- 172.21.15.78:0/2314382728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 msgr2=0x555b591140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c36d9d640 1 -- 172.21.15.78:0/2314382728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 msgr2=0x555b59114630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 0x555b59114630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c36d9d640 1 -- 172.21.15.78:0/2314382728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b58de33c0 con 0x555b59117370 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3659c640 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b59114b70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3c30011d30 tx=0x7f3c30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee1e705a6885f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c1dffb640 1 -- 172.21.15.78:0/2314382728 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30020420 con 0x555b59117370 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c1dffb640 1 -- 172.21.15.78:0/2314382728 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c300205c0 con 0x555b59117370 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 1 -- 172.21.15.78:0/2314382728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2809b3e0 con 0x555b59117370 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c1dffb640 1 -- 172.21.15.78:0/2314382728 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3c30033050 con 0x555b59117370 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 10 client.17637 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 1 -- 172.21.15.78:0/2314382728 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555b59125c00 con 0x555b59117370 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c1dffb640 1 -- 172.21.15.78:0/2314382728 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c2809b3e0 con 0x555b59117370 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c1dffb640 1 -- 172.21.15.78:0/2314382728 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3c3000f040 con 0x555b59117370 2024-09-16T00:25:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 10 client.17637 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 20 client.17637 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555b591270c0 0x555b59129540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 4 client.17637 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.451+0000 7f3c3f109740 1 -- 172.21.15.78:0/2314382728 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b5907f860 con 0x555b591270c0 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.452+0000 7f3c3659c640 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555b591270c0 0x555b59129540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.452+0000 7f3c1dffb640 10 client.17637 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.452+0000 7f3c3659c640 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555b591270c0 0x555b59129540 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d5104f2af0220d server_cookie=1d435fb1ef74f15 in_seq=0 out_seq=0 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c1dffb640 1 -- 172.21.15.78:0/2314382728 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b5907f860 con 0x555b591270c0 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c1dffb640 10 client.17637 handle_command_reply: tid=0 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 client.17637 shutdown 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 -- 172.21.15.78:0/2314382728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 msgr2=0x555b59114b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b59114b70 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3c30011d30 tx=0x7f3c30004a40 comp rx=0 tx=0).stop 2024-09-16T00:25:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 -- 172.21.15.78:0/2314382728 shutdown_connections 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555b591270c0 0x555b59129540 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b59114b70 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 0x555b59114630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 --2- 172.21.15.78:0/2314382728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c1dffb640 0 client.17637 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 -- 172.21.15.78:0/2314382728 >> 172.21.15.78:0/2314382728 conn(0x555b590d5310 msgr2=0x555b5910e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 -- 172.21.15.78:0/2314382728 shutdown_connections 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 1 -- 172.21.15.78:0/2314382728 wait complete. 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.453+0000 7f3c3f109740 20 client.17637 trim_cache size 0 max 16384 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3f109740 1 Processor -- start 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3f109740 1 -- start start 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 0x555b5910f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b5910f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3f109740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381b0ed0 con 0x555b59117370 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3f109740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381b2a10 con 0x555b5910db40 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3f109740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381b15a0 con 0x555b5910c5d0 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b5910f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b5910f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42004/0 (socket says 172.21.15.78:42004) 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 -- 172.21.15.78:0/680766771 learned_addr learned my addr 172.21.15.78:0/680766771 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c3659c640 1 --2- 172.21.15.78:0/680766771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 -- 172.21.15.78:0/680766771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 msgr2=0x555b591129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/680766771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 -- 172.21.15.78:0/680766771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 msgr2=0x555b5910f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/680766771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 0x555b5910f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 -- 172.21.15.78:0/680766771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b59112640 con 0x555b59117370 2024-09-16T00:25:42.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/680766771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b5910f660 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3c30000f80 tx=0x7f3c3001fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cc1401133db2fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c1ffff640 1 -- 172.21.15.78:0/680766771 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3003b020 con 0x555b59117370 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c1ffff640 1 -- 172.21.15.78:0/680766771 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c30005b40 con 0x555b59117370 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.466+0000 7f3c1ffff640 1 -- 172.21.15.78:0/680766771 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30003d90 con 0x555b59117370 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 -- 172.21.15.78:0/680766771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 msgr2=0x555b5910f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 --2- 172.21.15.78:0/680766771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b5910f660 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3c30000f80 tx=0x7f3c3001fc10 comp rx=0 tx=0).stop 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 -- 172.21.15.78:0/680766771 shutdown_connections 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 --2- 172.21.15.78:0/680766771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b59117370 0x555b5910f660 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 --2- 172.21.15.78:0/680766771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b5910db40 0x555b5910f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 --2- 172.21.15.78:0/680766771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b591129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 -- 172.21.15.78:0/680766771 >> 172.21.15.78:0/680766771 conn(0x555b590d5310 msgr2=0x555b59107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 -- 172.21.15.78:0/680766771 shutdown_connections 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.467+0000 7f3c3f109740 1 -- 172.21.15.78:0/680766771 wait complete. 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.468+0000 7f3c3f109740 1 -- start start 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.468+0000 7f3c3f109740 1 Processor -- start 2024-09-16T00:25:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b5911f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910db40 0x555b5911fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3f109740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b5911aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3f109740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381b2340 con 0x555b5910db40 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3f109740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381b1c70 con 0x555b59117370 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3f109740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c381aeeb0 con 0x555b5910c5d0 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b5911f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b5911f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34830/0 (socket says 172.21.15.78:34830) 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3659c640 1 -- 172.21.15.78:0/1685933974 learned_addr learned my addr 172.21.15.78:0/1685933974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c35d9b640 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910db40 0x555b5911fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3f109740 10 client.0 _ll_register_callbacks cb 0x7f3c300119e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b5911aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3659c640 1 -- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 msgr2=0x555b5911aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:42.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3659c640 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b5911aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3659c640 1 -- 172.21.15.78:0/1685933974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910db40 msgr2=0x555b5911fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3659c640 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910db40 0x555b5911fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c3659c640 1 -- 172.21.15.78:0/1685933974 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b5911b320 con 0x555b5910c5d0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.469+0000 7f3c36d9d640 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b5911aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c3659c640 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b5911f730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c28017fd0 tx=0x7f3c280047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a75e4dfda331522 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c1dffb640 1 -- 172.21.15.78:0/1685933974 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2801e040 con 0x555b5910c5d0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c1dffb640 1 -- 172.21.15.78:0/1685933974 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c28098fe0 con 0x555b5910c5d0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c3f109740 1 -- 172.21.15.78:0/1685933974 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c30005b40 con 0x555b5910c5d0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c1dffb640 1 -- 172.21.15.78:0/1685933974 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3c2809a820 con 0x555b5910c5d0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c3f109740 10 client.26686 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c3f109740 1 -- 172.21.15.78:0/1685933974 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555b591919f0 con 0x555b5910c5d0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.470+0000 7f3c1dffb640 1 -- 172.21.15.78:0/1685933974 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c30005b40 con 0x555b5910c5d0 2024-09-16T00:25:42.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c1dffb640 1 -- 172.21.15.78:0/1685933974 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3c28097350 con 0x555b5910c5d0 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c3f109740 10 client.26686 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c3f109740 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c3f109740 20 client.26686 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c3f109740 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555b591920c0 0x555b591944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c3f109740 4 client.26686 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c3f109740 1 -- 172.21.15.78:0/1685933974 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b59079d70 con 0x555b591920c0 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c35d9b640 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555b591920c0 0x555b591944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c1dffb640 10 client.26686 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.471+0000 7f3c35d9b640 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555b591920c0 0x555b591944b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb818270196632a server_cookie=8951048526bdb5d7 in_seq=0 out_seq=0 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c1dffb640 1 -- 172.21.15.78:0/1685933974 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b59079d70 con 0x555b591920c0 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c1dffb640 10 client.26686 handle_command_reply: tid=0 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 client.26686 shutdown 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 -- 172.21.15.78:0/1685933974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 msgr2=0x555b5911f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b5911f730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c28017fd0 tx=0x7f3c280047d0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 -- 172.21.15.78:0/1685933974 shutdown_connections 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555b591920c0 0x555b591944b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555b59117370 0x555b5911aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555b5910db40 0x555b5911fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 --2- 172.21.15.78:0/1685933974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555b5910c5d0 0x555b5911f730 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 -- 172.21.15.78:0/1685933974 >> 172.21.15.78:0/1685933974 conn(0x555b590d5310 msgr2=0x555b59115950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c1dffb640 0 client.26686 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 -- 172.21.15.78:0/1685933974 shutdown_connections 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 1 -- 172.21.15.78:0/1685933974 wait complete. 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.472+0000 7f3c3f109740 20 client.26686 trim_cache size 0 max 16384 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c240a3240 msgr2=0x7f3c240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c240a3240 0x7f3c240a5700 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3c30005fd0 tx=0x7f3c30009600 comp rx=0 tx=0).stop 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 msgr2=0x7f3c3819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 0x7f3c3819a690 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800b4a0 tx=0x7f3c2800b960 comp rx=0 tx=0).stop 2024-09-16T00:25:42.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1744380242 shutdown_connections 2024-09-16T00:25:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c240a3240 0x7f3c240a5700 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c38106a90 0x7f3c3819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c38101430 0x7f3c3819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 --2- 172.21.15.78:0/1744380242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c38100a90 0x7f3c3819a690 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1744380242 >> 172.21.15.78:0/1744380242 conn(0x7f3c380fc7d0 msgr2=0x7f3c3810c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1744380242 shutdown_connections 2024-09-16T00:25:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:41.473+0000 7f3c3cd30640 1 -- 172.21.15.78:0/1744380242 wait complete. 2024-09-16T00:25:42.247 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:41 smithi078 ceph-mon[31157]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-16T00:25:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:43 smithi193 ceph-mon[35484]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-16T00:25:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:43 smithi148 ceph-mon[35599]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-16T00:25:44.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:43 smithi078 ceph-mon[31157]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 912 op/s 2024-09-16T00:25:44.272 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:25:44.273 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:25:44.745 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:45.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 -- 172.21.15.78:0/2481997955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74100e20 msgr2=0x7efe74101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/2481997955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74100e20 0x7efe74101200 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe6802a480 comp rx=0 tx=0).stop 2024-09-16T00:25:45.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 -- 172.21.15.78:0/2481997955 shutdown_connections 2024-09-16T00:25:45.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/2481997955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74102ef0 0x7efe7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/2481997955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe74102570 0x7efe741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/2481997955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74100e20 0x7efe74101200 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 -- 172.21.15.78:0/2481997955 >> 172.21.15.78:0/2481997955 conn(0x7efe740fc7d0 msgr2=0x7efe740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 -- 172.21.15.78:0/2481997955 shutdown_connections 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.374+0000 7efe7ad4b640 1 -- 172.21.15.78:0/2481997955 wait complete. 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe7ad4b640 1 Processor -- start 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe7ad4b640 1 -- start start 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe7ad4b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74100e20 0x7efe741984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe7ad4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74102570 0x7efe74198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe7ad4b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe74102ef0 0x7efe7419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe7ad4b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe74110fd0 con 0x7efe74102570 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe7ad4b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe74110e50 con 0x7efe74100e20 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe7ad4b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe74111150 con 0x7efe74102ef0 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74102570 0x7efe74198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe73fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74102570 0x7efe74198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42044/0 (socket says 172.21.15.78:42044) 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe73fff640 1 -- 172.21.15.78:0/1973828164 learned_addr learned my addr 172.21.15.78:0/1973828164 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:45.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe78ae7640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74100e20 0x7efe741984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.375+0000 7efe792e8640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe74102ef0 0x7efe7419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe73fff640 1 -- 172.21.15.78:0/1973828164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe74102ef0 msgr2=0x7efe7419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe73fff640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe74102ef0 0x7efe7419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe73fff640 1 -- 172.21.15.78:0/1973828164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74100e20 msgr2=0x7efe741984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe73fff640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74100e20 0x7efe741984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe73fff640 1 -- 172.21.15.78:0/1973828164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7419d4a0 con 0x7efe74102570 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe73fff640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74102570 0x7efe74198a30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7efe6000d770 tx=0x7efe6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe71ffb640 1 -- 172.21.15.78:0/1973828164 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60017070 con 0x7efe74102570 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe71ffb640 1 -- 172.21.15.78:0/1973828164 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60005500 con 0x7efe74102570 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe7ad4b640 1 -- 172.21.15.78:0/1973828164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7419d790 con 0x7efe74102570 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe71ffb640 1 -- 172.21.15.78:0/1973828164 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60004e00 con 0x7efe74102570 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.376+0000 7efe7ad4b640 1 -- 172.21.15.78:0/1973828164 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741a5070 con 0x7efe74102570 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.378+0000 7efe71ffb640 1 -- 172.21.15.78:0/1973828164 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7efe60004fa0 con 0x7efe74102570 2024-09-16T00:25:45.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe7d124740 1 Processor -- start 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe7d124740 1 -- start start 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe71ffb640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7efe640a3240 0x7efe640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe71ffb640 1 -- 172.21.15.78:0/1973828164 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7efe600cf640 con 0x7efe74102570 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe7d124740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6285b60 0x5611c6285f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe7d124740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 0x5611c6290800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe7d124740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c6293130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe7d124740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5611c61eb280 con 0x5611c6285b60 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe7d124740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5611c6192b10 con 0x5611c6290d40 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.379+0000 7efe7d124740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5611c61e7240 con 0x5611c6286480 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe792e8640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7efe640a3240 0x7efe640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe78ae7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6285b60 0x5611c6285f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe792e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c6293130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe792e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c6293130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:43172/0 (socket says 172.21.15.78:43172) 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe792e8640 1 -- 172.21.15.78:0/4187188608 learned_addr learned my addr 172.21.15.78:0/4187188608 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe78ae7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6285b60 0x5611c6285f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42048/0 (socket says 172.21.15.78:42048) 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe73fff640 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 0x5611c6290800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe78ae7640 1 -- 172.21.15.78:0/4187188608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 msgr2=0x5611c6290800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe78ae7640 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 0x5611c6290800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe78ae7640 1 -- 172.21.15.78:0/4187188608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 msgr2=0x5611c6293130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe78ae7640 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c6293130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.380+0000 7efe78ae7640 1 -- 172.21.15.78:0/4187188608 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c619f1f0 con 0x5611c6285b60 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.381+0000 7efe792e8640 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c6293130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:25:45.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.381+0000 7efe78ae7640 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6285b60 0x5611c6285f40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efe6802aa70 tx=0x7efe680040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19da7d68c7555786 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:45.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.382+0000 7efe5effd640 1 -- 172.21.15.78:0/4187188608 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68035040 con 0x5611c6285b60 2024-09-16T00:25:45.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.382+0000 7efe5effd640 1 -- 172.21.15.78:0/4187188608 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe68005520 con 0x5611c6285b60 2024-09-16T00:25:45.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.382+0000 7efe5effd640 1 -- 172.21.15.78:0/4187188608 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68003850 con 0x5611c6285b60 2024-09-16T00:25:45.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.383+0000 7efe7d124740 1 -- 172.21.15.78:0/4187188608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6285b60 msgr2=0x5611c6285f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.383+0000 7efe7d124740 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6285b60 0x5611c6285f40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efe6802aa70 tx=0x7efe680040f0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.388+0000 7efe792e8640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7efe640a3240 0x7efe640a5700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5611c6283470 tx=0x7efe6c009ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:45.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.389+0000 7efe7d124740 1 -- 172.21.15.78:0/4187188608 shutdown_connections 2024-09-16T00:25:45.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.389+0000 7efe7d124740 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c6293130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.389+0000 7efe7d124740 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 0x5611c6290800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.389+0000 7efe7d124740 1 --2- 172.21.15.78:0/4187188608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6285b60 0x5611c6285f40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.389+0000 7efe7d124740 1 -- 172.21.15.78:0/4187188608 >> 172.21.15.78:0/4187188608 conn(0x5611c62820f0 msgr2=0x5611c62824d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.389+0000 7efe7d124740 1 -- 172.21.15.78:0/4187188608 shutdown_connections 2024-09-16T00:25:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.389+0000 7efe7d124740 1 -- 172.21.15.78:0/4187188608 wait complete. 2024-09-16T00:25:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.390+0000 7efe7d124740 1 -- start start 2024-09-16T00:25:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.390+0000 7efe7d124740 1 Processor -- start 2024-09-16T00:25:45.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.391+0000 7efe7d124740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c628dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.391+0000 7efe7d124740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6286480 0x5611c628e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.391+0000 7efe7d124740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 0x5611c628e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.391+0000 7efe78ae7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c628dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.391+0000 7efe78ae7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c628dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34866/0 (socket says 172.21.15.78:34866) 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.391+0000 7efe78ae7640 1 -- 172.21.15.78:0/2516740934 learned_addr learned my addr 172.21.15.78:0/2516740934 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5611c6297d20 con 0x5611c6290d40 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5611c61a5f10 con 0x5611c6286480 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5611c6297ea0 con 0x5611c6285b60 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe7d124740 10 client.0 _ll_register_callbacks cb 0x5611c616ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe78ae7640 1 -- 172.21.15.78:0/2516740934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6286480 msgr2=0x5611c628e1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe78ae7640 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6286480 0x5611c628e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe78ae7640 1 -- 172.21.15.78:0/2516740934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 msgr2=0x5611c628e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe78ae7640 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 0x5611c628e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.392+0000 7efe78ae7640 1 -- 172.21.15.78:0/2516740934 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c5f5c6a0 con 0x5611c6285b60 2024-09-16T00:25:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.393+0000 7efe78ae7640 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c628dc90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe680314c0 tx=0x7efe680045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=819fff4557e26973 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.393+0000 7efe53fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:25:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.393+0000 7efe53fff640 1 -- 172.21.15.78:0/2516740934 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68035070 con 0x5611c6285b60 2024-09-16T00:25:45.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.394+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe68005520 con 0x5611c6285b60 2024-09-16T00:25:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.398+0000 7efe53fff640 1 -- 172.21.15.78:0/2516740934 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7efe680051b0 con 0x5611c6285b60 2024-09-16T00:25:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.398+0000 7efe53fff640 1 -- 172.21.15.78:0/2516740934 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efe680046c0 con 0x5611c6285b60 2024-09-16T00:25:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.400+0000 7efe7d124740 10 client.26698 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:45.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.400+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5611c6297d20 con 0x5611c6285b60 2024-09-16T00:25:45.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.401+0000 7efe53fff640 1 -- 172.21.15.78:0/2516740934 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe68004860 con 0x5611c6285b60 2024-09-16T00:25:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.404+0000 7efe53fff640 1 -- 172.21.15.78:0/2516740934 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efe68031540 con 0x5611c6285b60 2024-09-16T00:25:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.404+0000 7efe7d124740 10 client.26698 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.404+0000 7efe7d124740 10 client.26698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:25:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.404+0000 7efe7d124740 20 client.26698 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.404+0000 7efe7d124740 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5611c62a4e40 0x5611c62a7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.404+0000 7efe7d124740 4 client.26698 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.404+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efe68005520 con 0x5611c62a4e40 2024-09-16T00:25:45.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.407+0000 7efe73fff640 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5611c62a4e40 0x5611c62a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.414+0000 7efe73fff640 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5611c62a4e40 0x5611c62a7300 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c17a9533abc20a server_cookie=2a0ddc5cbfc5b6ac in_seq=0 out_seq=0 2024-09-16T00:25:45.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.414+0000 7efe53fff640 10 client.26698 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:45.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.415+0000 7efe53fff640 1 -- 172.21.15.78:0/2516740934 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efe68005520 con 0x5611c62a4e40 2024-09-16T00:25:45.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.415+0000 7efe53fff640 10 client.26698 handle_command_reply: tid=0 2024-09-16T00:25:45.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.416+0000 7efe7d124740 1 client.26698 shutdown 2024-09-16T00:25:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 msgr2=0x5611c628dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe7d124740 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c628dc90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe680314c0 tx=0x7efe680045d0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 shutdown_connections 2024-09-16T00:25:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe7d124740 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5611c62a4e40 0x5611c62a7300 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe7d124740 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 0x5611c628e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe7d124740 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6286480 0x5611c628e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe7d124740 1 --2- 172.21.15.78:0/2516740934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c628dc90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 >> 172.21.15.78:0/2516740934 conn(0x5611c62820f0 msgr2=0x5611c6287e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.419+0000 7efe53fff640 0 client.26698 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.420+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 shutdown_connections 2024-09-16T00:25:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.424+0000 7efe7d124740 1 -- 172.21.15.78:0/2516740934 wait complete. 2024-09-16T00:25:45.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.426+0000 7efe7d124740 20 client.26698 trim_cache size 0 max 16384 2024-09-16T00:25:45.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.467+0000 7efe7d124740 1 Processor -- start 2024-09-16T00:25:45.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.468+0000 7efe7d124740 1 -- start start 2024-09-16T00:25:45.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.468+0000 7efe7d124740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6285b60 0x5611c62885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.468+0000 7efe7d124740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 0x5611c62acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.468+0000 7efe7d124740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 0x5611c62ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.468+0000 7efe7d124740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe741ac8f0 con 0x5611c6290d40 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.468+0000 7efe7d124740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe741ae430 con 0x5611c6285b60 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.468+0000 7efe7d124740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe741acfc0 con 0x5611c6286480 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6285b60 0x5611c62885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe73fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 0x5611c62acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6285b60 0x5611c62885c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:43192/0 (socket says 172.21.15.78:43192) 2024-09-16T00:25:45.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 -- 172.21.15.78:0/1498501090 learned_addr learned my addr 172.21.15.78:0/1498501090 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:45.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe792e8640 1 --2- 172.21.15.78:0/1498501090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 0x5611c62ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 -- 172.21.15.78:0/1498501090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 msgr2=0x5611c62acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:45.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 --2- 172.21.15.78:0/1498501090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 0x5611c62acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 -- 172.21.15.78:0/1498501090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 msgr2=0x5611c62ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:45.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 --2- 172.21.15.78:0/1498501090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 0x5611c62ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 -- 172.21.15.78:0/1498501090 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c6281f20 con 0x5611c6285b60 2024-09-16T00:25:45.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.469+0000 7efe78ae7640 1 --2- 172.21.15.78:0/1498501090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6285b60 0x5611c62885c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe68001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a781e887e59fff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:45.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.471+0000 7efe5e7fc640 1 -- 172.21.15.78:0/1498501090 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68065070 con 0x5611c6285b60 2024-09-16T00:25:45.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.475+0000 7efe5e7fc640 1 -- 172.21.15.78:0/1498501090 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe68009520 con 0x5611c6285b60 2024-09-16T00:25:45.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.475+0000 7efe5e7fc640 1 -- 172.21.15.78:0/1498501090 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68035040 con 0x5611c6285b60 2024-09-16T00:25:45.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.476+0000 7efe7d124740 1 -- 172.21.15.78:0/1498501090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6285b60 msgr2=0x5611c62885c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.476+0000 7efe7d124740 1 --2- 172.21.15.78:0/1498501090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6285b60 0x5611c62885c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe68001e70 comp rx=0 tx=0).stop 2024-09-16T00:25:45.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.481+0000 7efe7d124740 1 -- 172.21.15.78:0/1498501090 shutdown_connections 2024-09-16T00:25:45.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.482+0000 7efe7d124740 1 --2- 172.21.15.78:0/1498501090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6290d40 0x5611c62ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.482+0000 7efe7d124740 1 --2- 172.21.15.78:0/1498501090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6286480 0x5611c62acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.482+0000 7efe7d124740 1 --2- 172.21.15.78:0/1498501090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6285b60 0x5611c62885c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.482+0000 7efe7d124740 1 -- 172.21.15.78:0/1498501090 >> 172.21.15.78:0/1498501090 conn(0x5611c62820f0 msgr2=0x5611c6281270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:45.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.490+0000 7efe7d124740 1 -- 172.21.15.78:0/1498501090 shutdown_connections 2024-09-16T00:25:45.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.490+0000 7efe7d124740 1 -- 172.21.15.78:0/1498501090 wait complete. 2024-09-16T00:25:45.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.491+0000 7efe7d124740 1 -- start start 2024-09-16T00:25:45.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.491+0000 7efe7d124740 1 Processor -- start 2024-09-16T00:25:45.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.492+0000 7efe7d124740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c629d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.493+0000 7efe7d124740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6286480 0x5611c629d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.493+0000 7efe7d124740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c62944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.493+0000 7efe7d124740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efe741add60 con 0x5611c6286480 2024-09-16T00:25:45.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.493+0000 7efe7d124740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efe741ad690 con 0x5611c6290d40 2024-09-16T00:25:45.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.493+0000 7efe7d124740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe741aa8d0 con 0x5611c6285b60 2024-09-16T00:25:45.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.493+0000 7efe7d124740 10 client.0 _ll_register_callbacks cb 0x7efe68004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.494+0000 7efe78ae7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c629d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.494+0000 7efe78ae7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c629d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34894/0 (socket says 172.21.15.78:34894) 2024-09-16T00:25:45.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.494+0000 7efe78ae7640 1 -- 172.21.15.78:0/3023330223 learned_addr learned my addr 172.21.15.78:0/3023330223 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:45.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.498+0000 7efe792e8640 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c62944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.498+0000 7efe73fff640 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6286480 0x5611c629d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.498+0000 7efe78ae7640 1 -- 172.21.15.78:0/3023330223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 msgr2=0x5611c62944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:45.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.498+0000 7efe78ae7640 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c62944d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.498+0000 7efe78ae7640 1 -- 172.21.15.78:0/3023330223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6286480 msgr2=0x5611c629d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:45.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.498+0000 7efe78ae7640 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6286480 0x5611c629d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.498+0000 7efe78ae7640 1 -- 172.21.15.78:0/3023330223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c6294cd0 con 0x5611c6285b60 2024-09-16T00:25:45.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.503+0000 7efe78ae7640 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c629d440 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe68004ac0 tx=0x7efe68004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67b3a123e563b306 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:45.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.504+0000 7efe53fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:25:45.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.504+0000 7efe53fff640 1 -- 172.21.15.78:0/3023330223 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68065070 con 0x5611c6285b60 2024-09-16T00:25:45.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.504+0000 7efe53fff640 1 -- 172.21.15.78:0/3023330223 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7efe680051b0 con 0x5611c6285b60 2024-09-16T00:25:45.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.508+0000 7efe7d124740 1 -- 172.21.15.78:0/3023330223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe68009520 con 0x5611c6285b60 2024-09-16T00:25:45.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.508+0000 7efe53fff640 1 -- 172.21.15.78:0/3023330223 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efe6806f050 con 0x5611c6285b60 2024-09-16T00:25:45.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.509+0000 7efe7d124740 10 client.26704 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:45.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.509+0000 7efe7d124740 1 -- 172.21.15.78:0/3023330223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5611c629ad20 con 0x5611c6285b60 2024-09-16T00:25:45.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.510+0000 7efe53fff640 1 -- 172.21.15.78:0/3023330223 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe68009520 con 0x5611c6285b60 2024-09-16T00:25:45.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.510+0000 7efe53fff640 1 -- 172.21.15.78:0/3023330223 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efe6803c440 con 0x5611c6285b60 2024-09-16T00:25:45.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.510+0000 7efe7d124740 10 client.26704 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:45.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.511+0000 7efe7d124740 10 client.26704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:25:45.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.511+0000 7efe7d124740 20 client.26704 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:45.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.511+0000 7efe7d124740 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5611c630bcd0 0x5611c630e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.511+0000 7efe7d124740 4 client.26704 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:25:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.511+0000 7efe7d124740 1 -- 172.21.15.78:0/3023330223 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5611c61f87f0 con 0x5611c630bcd0 2024-09-16T00:25:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.513+0000 7efe73fff640 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5611c630bcd0 0x5611c630e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:45.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.520+0000 7efe73fff640 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5611c630bcd0 0x5611c630e0c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2be179de908ac620 server_cookie=6c6bdb1f88502d10 in_seq=0 out_seq=0 2024-09-16T00:25:45.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.520+0000 7efe53fff640 10 client.26704 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:45.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.521+0000 7efe53fff640 1 -- 172.21.15.78:0/3023330223 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5611c61f87f0 con 0x5611c630bcd0 2024-09-16T00:25:45.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.521+0000 7efe53fff640 10 client.26704 handle_command_reply: tid=0 2024-09-16T00:25:45.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.522+0000 7efe7d124740 1 client.26704 shutdown 2024-09-16T00:25:45.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.526+0000 7efe7d124740 1 -- 172.21.15.78:0/3023330223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 msgr2=0x5611c629d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.526+0000 7efe7d124740 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c629d440 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe68004ac0 tx=0x7efe68004af0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.527+0000 7efe7d124740 1 -- 172.21.15.78:0/3023330223 shutdown_connections 2024-09-16T00:25:45.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.527+0000 7efe7d124740 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5611c630bcd0 0x5611c630e0c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.527+0000 7efe7d124740 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5611c6290d40 0x5611c62944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.527+0000 7efe7d124740 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5611c6286480 0x5611c629d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.527+0000 7efe7d124740 1 --2- 172.21.15.78:0/3023330223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5611c6285b60 0x5611c629d440 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.527+0000 7efe7d124740 1 -- 172.21.15.78:0/3023330223 >> 172.21.15.78:0/3023330223 conn(0x5611c62820f0 msgr2=0x5611c629ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:45.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.527+0000 7efe53fff640 0 client.26704 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:45.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.530+0000 7efe7d124740 1 -- 172.21.15.78:0/3023330223 shutdown_connections 2024-09-16T00:25:45.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.530+0000 7efe7d124740 1 -- 172.21.15.78:0/3023330223 wait complete. 2024-09-16T00:25:45.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.531+0000 7efe7d124740 20 client.26704 trim_cache size 0 max 16384 2024-09-16T00:25:45.531 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:25:45.531 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:25:45.532 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "a1102ccc-6b68-49a3-b8b9-c70bebfd3208", 2024-09-16T00:25:45.532 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:25:45.532 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:25:45.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 -- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7efe640a3240 msgr2=0x7efe640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7efe640a3240 0x7efe640a5700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x5611c6283470 tx=0x7efe6c009ae0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 -- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74102570 msgr2=0x7efe74198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:45.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74102570 0x7efe74198a30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7efe6000d770 tx=0x7efe6000dc30 comp rx=0 tx=0).stop 2024-09-16T00:25:45.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 -- 172.21.15.78:0/1973828164 shutdown_connections 2024-09-16T00:25:45.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7efe640a3240 0x7efe640a5700 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe74102ef0 0x7efe7419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efe74102570 0x7efe74198a30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 --2- 172.21.15.78:0/1973828164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efe74100e20 0x7efe741984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:45.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.532+0000 7efe7ad4b640 1 -- 172.21.15.78:0/1973828164 >> 172.21.15.78:0/1973828164 conn(0x7efe740fc7d0 msgr2=0x7efe7406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:45.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.534+0000 7efe7ad4b640 1 -- 172.21.15.78:0/1973828164 shutdown_connections 2024-09-16T00:25:45.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:45.534+0000 7efe7ad4b640 1 -- 172.21.15.78:0/1973828164 wait complete. 2024-09-16T00:25:45.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 705.0 s, 637.4 tps, lat 157.668 ms stddev 195.801 2024-09-16T00:25:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:45 smithi193 ceph-mon[35484]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 852 op/s 2024-09-16T00:25:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:45 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:25:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:45 smithi148 ceph-mon[35599]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 852 op/s 2024-09-16T00:25:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:45 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:25:46.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:45 smithi078 ceph-mon[31157]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 852 op/s 2024-09-16T00:25:46.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:45 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:25:46.572 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:25:47.106 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:47.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:46 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:25:47.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:46 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:46 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:25:47.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:46 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:25:47.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:46 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:46 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:46 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:46 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:25:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:46 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:46 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:25:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:46 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:25:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:46 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:46 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:46 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:46 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:25:47.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:46 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:46 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:25:47.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:46 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:25:47.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:46 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:46 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:46 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:25:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.721+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2987328619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc806ea30 msgr2=0x7f5cc806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.721+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2987328619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc806ea30 0x7f5cc806ee70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009be0 tx=0x7f5cc002a6c0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.730+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2987328619 shutdown_connections 2024-09-16T00:25:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.730+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2987328619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc811b140 0x7f5cc811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.730+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2987328619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc806ea30 0x7f5cc806ee70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.730+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2987328619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc8110190 0x7f5cc806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.730+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2987328619 >> 172.21.15.78:0/2987328619 conn(0x7f5cc8073180 msgr2=0x7f5cc8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.735+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2987328619 shutdown_connections 2024-09-16T00:25:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.735+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2987328619 wait complete. 2024-09-16T00:25:47.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.735+0000 7f5ccdd4b640 1 Processor -- start 2024-09-16T00:25:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.741+0000 7f5ccdd4b640 1 -- start start 2024-09-16T00:25:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5ccdd4b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc806ea30 0x7f5cc81a7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5ccdd4b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc8110190 0x7f5cc81a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5ccdd4b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc811b140 0x7f5cc81ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5ccdd4b640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8120b90 con 0x7f5cc806ea30 2024-09-16T00:25:47.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5ccdd4b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8120a10 con 0x7f5cc8110190 2024-09-16T00:25:47.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5ccdd4b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8120d10 con 0x7f5cc811b140 2024-09-16T00:25:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc8110190 0x7f5cc81a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc806ea30 0x7f5cc81a7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc806ea30 0x7f5cc81a7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39278/0 (socket says 172.21.15.78:39278) 2024-09-16T00:25:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.742+0000 7f5cccd49640 1 -- 172.21.15.78:0/2851555125 learned_addr learned my addr 172.21.15.78:0/2851555125 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.743+0000 7f5cc7fff640 1 -- 172.21.15.78:0/2851555125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc811b140 msgr2=0x7f5cc81ab9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:25:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.743+0000 7f5cc7fff640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc811b140 0x7f5cc81ab9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.743+0000 7f5cc7fff640 1 -- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc806ea30 msgr2=0x7f5cc81a7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.743+0000 7f5cc7fff640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc806ea30 0x7f5cc81a7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.743+0000 7f5cc7fff640 1 -- 172.21.15.78:0/2851555125 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc81ac0d0 con 0x7f5cc8110190 2024-09-16T00:25:47.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.744+0000 7f5cc7fff640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc8110190 0x7f5cc81a7660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5cc002acb0 tx=0x7f5cc0030620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:47.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.748+0000 7f5cc5ffb640 1 -- 172.21.15.78:0/2851555125 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0041020 con 0x7f5cc8110190 2024-09-16T00:25:47.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.748+0000 7f5cc5ffb640 1 -- 172.21.15.78:0/2851555125 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc0030850 con 0x7f5cc8110190 2024-09-16T00:25:47.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.748+0000 7f5cc5ffb640 1 -- 172.21.15.78:0/2851555125 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0036040 con 0x7f5cc8110190 2024-09-16T00:25:47.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.748+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2851555125 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc81ac360 con 0x7f5cc8110190 2024-09-16T00:25:47.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.748+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2851555125 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81ac7f0 con 0x7f5cc8110190 2024-09-16T00:25:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.752+0000 7f5cd1386740 1 Processor -- start 2024-09-16T00:25:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.752+0000 7f5cd1386740 1 -- start start 2024-09-16T00:25:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.752+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a0fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.752+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fbf90 0x55ad7a106310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.752+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a108c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.752+0000 7f5cd1386740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ad7a0612e0 con 0x55ad7a106850 2024-09-16T00:25:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.752+0000 7f5cd1386740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ad7a008ce0 con 0x55ad7a0fb670 2024-09-16T00:25:47.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.752+0000 7f5cd1386740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ad7a05d2a0 con 0x55ad7a0fbf90 2024-09-16T00:25:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.754+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fbf90 0x55ad7a106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.754+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fbf90 0x55ad7a106310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:32998/0 (socket says 172.21.15.78:32998) 2024-09-16T00:25:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.754+0000 7f5cccd49640 1 -- 172.21.15.78:0/867126107 learned_addr learned my addr 172.21.15.78:0/867126107 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:47.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.754+0000 7f5cccd49640 1 -- 172.21.15.78:0/867126107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 msgr2=0x55ad7a0fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.754+0000 7f5cccd49640 1 --2- 172.21.15.78:0/867126107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a0fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.754+0000 7f5cccd49640 1 -- 172.21.15.78:0/867126107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 msgr2=0x55ad7a108c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:25:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.754+0000 7f5cccd49640 1 --2- 172.21.15.78:0/867126107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a108c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.754+0000 7f5cccd49640 1 -- 172.21.15.78:0/867126107 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad79fed440 con 0x55ad7a0fbf90 2024-09-16T00:25:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.755+0000 7f5cccd49640 1 --2- 172.21.15.78:0/867126107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fbf90 0x55ad7a106310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009d60 tx=0x7f5cbc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a2d43de9ae81bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:47.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.755+0000 7f5cc5ffb640 1 -- 172.21.15.78:0/2851555125 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5cc0044020 con 0x7f5cc8110190 2024-09-16T00:25:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.756+0000 7f5cc5ffb640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cb00a33d0 0x7f5cb00a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.756+0000 7f5cc5ffb640 1 -- 172.21.15.78:0/2851555125 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5cc00f0500 con 0x7f5cc8110190 2024-09-16T00:25:47.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.756+0000 7f5ccd54a640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cb00a33d0 0x7f5cb00a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.758+0000 7f5caeffd640 1 -- 172.21.15.78:0/867126107 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc01e070 con 0x55ad7a0fbf90 2024-09-16T00:25:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.758+0000 7f5caeffd640 1 -- 172.21.15.78:0/867126107 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc019b10 con 0x55ad7a0fbf90 2024-09-16T00:25:47.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.758+0000 7f5caeffd640 1 -- 172.21.15.78:0/867126107 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00eb50 con 0x55ad7a0fbf90 2024-09-16T00:25:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.759+0000 7f5cd1386740 1 -- 172.21.15.78:0/867126107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fbf90 msgr2=0x55ad7a106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:47.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.759+0000 7f5cd1386740 1 --2- 172.21.15.78:0/867126107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fbf90 0x55ad7a106310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009d60 tx=0x7f5cbc011810 comp rx=0 tx=0).stop 2024-09-16T00:25:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.762+0000 7f5cd1386740 1 -- 172.21.15.78:0/867126107 shutdown_connections 2024-09-16T00:25:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.762+0000 7f5cd1386740 1 --2- 172.21.15.78:0/867126107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.762+0000 7f5cd1386740 1 --2- 172.21.15.78:0/867126107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fbf90 0x55ad7a106310 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.762+0000 7f5cd1386740 1 --2- 172.21.15.78:0/867126107 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a0fba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.762+0000 7f5cd1386740 1 -- 172.21.15.78:0/867126107 >> 172.21.15.78:0/867126107 conn(0x55ad7a0f7c00 msgr2=0x55ad7a0f7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:47.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.764+0000 7f5cd1386740 1 -- 172.21.15.78:0/867126107 shutdown_connections 2024-09-16T00:25:47.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.764+0000 7f5cd1386740 1 -- 172.21.15.78:0/867126107 wait complete. 2024-09-16T00:25:47.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.764+0000 7f5cd1386740 1 -- start start 2024-09-16T00:25:47.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.765+0000 7f5cd1386740 1 Processor -- start 2024-09-16T00:25:47.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.769+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a1061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.769+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 0x55ad7a103480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.769+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a103a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.769+0000 7f5cd1386740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ad7a109420 con 0x55ad7a106850 2024-09-16T00:25:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cd1386740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ad7a01c050 con 0x55ad7a0fbf90 2024-09-16T00:25:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cd1386740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ad7a1095a0 con 0x55ad7a0fb670 2024-09-16T00:25:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a1061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a1061d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:33010/0 (socket says 172.21.15.78:33010) 2024-09-16T00:25:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cccd49640 1 -- 172.21.15.78:0/4047975222 learned_addr learned my addr 172.21.15.78:0/4047975222 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:47.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cd1386740 10 client.0 _ll_register_callbacks cb 0x55ad7a05c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cccd49640 1 -- 172.21.15.78:0/4047975222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 msgr2=0x55ad7a103480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:25:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cccd49640 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 0x55ad7a103480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cccd49640 1 -- 172.21.15.78:0/4047975222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 msgr2=0x55ad7a103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cccd49640 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.770+0000 7f5cccd49640 1 -- 172.21.15.78:0/4047975222 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad79dd23c0 con 0x55ad7a0fb670 2024-09-16T00:25:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.771+0000 7f5ccd54a640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cb00a33d0 0x7f5cb00a5890 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5cc806b4a0 tx=0x7f5cb8005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.771+0000 7f5cccd49640 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a1061d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009a50 tx=0x7f5cbc009a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577d540fec3fdcf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:47.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.771+0000 7f5ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:25:47.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.776+0000 7f5ca7fff640 1 -- 172.21.15.78:0/4047975222 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc029060 con 0x55ad7a0fb670 2024-09-16T00:25:47.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.776+0000 7f5ca7fff640 1 -- 172.21.15.78:0/4047975222 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5cbc0197f0 con 0x55ad7a0fb670 2024-09-16T00:25:47.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.776+0000 7f5cd1386740 1 -- 172.21.15.78:0/4047975222 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc019b10 con 0x55ad7a0fb670 2024-09-16T00:25:47.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.780+0000 7f5ca7fff640 1 -- 172.21.15.78:0/4047975222 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5cbc030050 con 0x55ad7a0fb670 2024-09-16T00:25:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.781+0000 7f5cd1386740 10 client.26713 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.781+0000 7f5cd1386740 1 -- 172.21.15.78:0/4047975222 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad7a114e80 con 0x55ad7a0fb670 2024-09-16T00:25:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.782+0000 7f5ca7fff640 1 -- 172.21.15.78:0/4047975222 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc019b10 con 0x55ad7a0fb670 2024-09-16T00:25:47.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.782+0000 7f5ca7fff640 1 -- 172.21.15.78:0/4047975222 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5cbc021070 con 0x55ad7a0fb670 2024-09-16T00:25:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.785+0000 7f5cd1386740 10 client.26713 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.785+0000 7f5cd1386740 10 client.26713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:25:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.785+0000 7f5cd1386740 20 client.26713 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.785+0000 7f5cd1386740 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ad7a116290 0x55ad7a118680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.785+0000 7f5cd1386740 4 client.26713 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:25:47.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.785+0000 7f5cd1386740 1 -- 172.21.15.78:0/4047975222 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad7a06e850 con 0x55ad7a116290 2024-09-16T00:25:47.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.788+0000 7f5cc7fff640 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ad7a116290 0x55ad7a118680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.796+0000 7f5cc7fff640 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ad7a116290 0x55ad7a118680 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b032463028759bcc server_cookie=95e210c381ef484c in_seq=0 out_seq=0 2024-09-16T00:25:47.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.796+0000 7f5ca7fff640 10 client.26713 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:47.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.799+0000 7f5ca7fff640 1 -- 172.21.15.78:0/4047975222 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad7a06e850 con 0x55ad7a116290 2024-09-16T00:25:47.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.799+0000 7f5ca7fff640 10 client.26713 handle_command_reply: tid=0 2024-09-16T00:25:47.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.799+0000 7f5cd1386740 1 client.26713 shutdown 2024-09-16T00:25:47.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.799+0000 7f5cd1386740 1 -- 172.21.15.78:0/4047975222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 msgr2=0x55ad7a1061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:47.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.799+0000 7f5cd1386740 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a1061d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009a50 tx=0x7f5cbc009a80 comp rx=0 tx=0).stop 2024-09-16T00:25:47.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5cd1386740 1 -- 172.21.15.78:0/4047975222 shutdown_connections 2024-09-16T00:25:47.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5cd1386740 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ad7a116290 0x55ad7a118680 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5cd1386740 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5cd1386740 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 0x55ad7a103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5cd1386740 1 --2- 172.21.15.78:0/4047975222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a1061d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5cd1386740 1 -- 172.21.15.78:0/4047975222 >> 172.21.15.78:0/4047975222 conn(0x55ad7a0f7c00 msgr2=0x55ad7a0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:47.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5ca7fff640 0 client.26713 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:47.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5cd1386740 1 -- 172.21.15.78:0/4047975222 shutdown_connections 2024-09-16T00:25:47.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.800+0000 7f5cd1386740 1 -- 172.21.15.78:0/4047975222 wait complete. 2024-09-16T00:25:47.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.801+0000 7f5cd1386740 20 client.26713 trim_cache size 0 max 16384 2024-09-16T00:25:47.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.829+0000 7f5cd1386740 1 Processor -- start 2024-09-16T00:25:47.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cd1386740 1 -- start start 2024-09-16T00:25:47.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a112710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a0fbf90 0x55ad7a0fd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a106850 0x55ad7a11dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cd1386740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb400fbc0 con 0x55ad7a0fbf90 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cd1386740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4011700 con 0x55ad7a0fb670 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cd1386740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4010290 con 0x55ad7a106850 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a112710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a112710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48230/0 (socket says 172.21.15.78:48230) 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cccd49640 1 -- 172.21.15.78:0/1647412022 learned_addr learned my addr 172.21.15.78:0/1647412022 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.830+0000 7f5cc7fff640 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a0fbf90 0x55ad7a0fd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5ccd54a640 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a106850 0x55ad7a11dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cccd49640 1 -- 172.21.15.78:0/1647412022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a106850 msgr2=0x55ad7a11dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cccd49640 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a106850 0x55ad7a11dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cccd49640 1 -- 172.21.15.78:0/1647412022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a0fbf90 msgr2=0x55ad7a0fd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cccd49640 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a0fbf90 0x55ad7a0fd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cccd49640 1 -- 172.21.15.78:0/1647412022 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad7a0f8540 con 0x55ad7a0fb670 2024-09-16T00:25:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cccd49640 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a112710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0058c0 tx=0x7f5cbc012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1cccd5e324e4155 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cae7fc640 1 -- 172.21.15.78:0/1647412022 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc01e040 con 0x55ad7a0fb670 2024-09-16T00:25:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cae7fc640 1 -- 172.21.15.78:0/1647412022 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cbc009be0 con 0x55ad7a0fb670 2024-09-16T00:25:47.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.831+0000 7f5cae7fc640 1 -- 172.21.15.78:0/1647412022 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc019cc0 con 0x55ad7a0fb670 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.832+0000 7f5cd1386740 1 -- 172.21.15.78:0/1647412022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 msgr2=0x55ad7a112710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.832+0000 7f5cd1386740 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a112710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0058c0 tx=0x7f5cbc012f90 comp rx=0 tx=0).stop 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 -- 172.21.15.78:0/1647412022 shutdown_connections 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a106850 0x55ad7a11dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a0fbf90 0x55ad7a0fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 --2- 172.21.15.78:0/1647412022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fb670 0x55ad7a112710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 -- 172.21.15.78:0/1647412022 >> 172.21.15.78:0/1647412022 conn(0x55ad7a0f7c00 msgr2=0x55ad7a0fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 -- 172.21.15.78:0/1647412022 shutdown_connections 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 -- 172.21.15.78:0/1647412022 wait complete. 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 -- start start 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.833+0000 7f5cd1386740 1 Processor -- start 2024-09-16T00:25:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.834+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a10e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.834+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 0x55ad7a10ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.834+0000 7f5cd1386740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a109da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a10e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cccd49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a10e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:33034/0 (socket says 172.21.15.78:33034) 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 0x55ad7a10ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5ccd54a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 0x55ad7a10ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:48234/0 (socket says 172.21.15.78:48234) 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cc7fff640 1 -- 172.21.15.78:0/2967916706 learned_addr learned my addr 172.21.15.78:0/2967916706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4011030 con 0x55ad7a106850 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4010960 con 0x55ad7a0fbf90 2024-09-16T00:25:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb400dba0 con 0x55ad7a0fb670 2024-09-16T00:25:47.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.835+0000 7f5cd1386740 10 client.0 _ll_register_callbacks cb 0x7f5cbc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:25:47.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.836+0000 7f5ccd54a640 1 -- 172.21.15.78:0/2967916706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 msgr2=0x55ad7a10e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.836+0000 7f5ccd54a640 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a10e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.836+0000 7f5ccd54a640 1 -- 172.21.15.78:0/2967916706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 msgr2=0x55ad7a10ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.836+0000 7f5ccd54a640 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 0x55ad7a10ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.836+0000 7f5ccd54a640 1 -- 172.21.15.78:0/2967916706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad7a10a5d0 con 0x55ad7a106850 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.836+0000 7f5ccd54a640 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a109da0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b330 tx=0x7f5cb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1a225789227c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.836+0000 7f5ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5ca7fff640 1 -- 172.21.15.78:0/2967916706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8022070 con 0x55ad7a106850 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5ca7fff640 1 -- 172.21.15.78:0/2967916706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5cb801e620 con 0x55ad7a106850 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc009be0 con 0x55ad7a106850 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5ca7fff640 1 -- 172.21.15.78:0/2967916706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5cb802e050 con 0x55ad7a106850 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5cd1386740 10 client.17667 fetch_fsmap learned FSMap version 34 2024-09-16T00:25:47.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad7a180830 con 0x55ad7a106850 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5ca7fff640 1 -- 172.21.15.78:0/2967916706 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cb8016040 con 0x55ad7a106850 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5ca7fff640 1 -- 172.21.15.78:0/2967916706 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5cb800f900 con 0x55ad7a106850 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5cd1386740 10 client.17667 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5cd1386740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5cd1386740 20 client.17667 populate_metadata read hostname 'smithi078' 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.837+0000 7f5cd1386740 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ad7a180f50 0x55ad7a183370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.838+0000 7f5cd1386740 4 client.17667 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.838+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5cbc009be0 con 0x55ad7a180f50 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.838+0000 7f5cccd49640 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ad7a180f50 0x55ad7a183370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.838+0000 7f5ca7fff640 10 client.17667 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.838+0000 7f5cccd49640 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ad7a180f50 0x55ad7a183370 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696e06b3473d3d21 server_cookie=bf678cd95104d4e5 in_seq=0 out_seq=0 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.838+0000 7f5ca7fff640 1 -- 172.21.15.78:0/2967916706 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f5cbc009be0 con 0x55ad7a180f50 2024-09-16T00:25:47.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.838+0000 7f5ca7fff640 10 client.17667 handle_command_reply: tid=0 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.838+0000 7f5cd1386740 1 client.17667 shutdown 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 msgr2=0x55ad7a109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a109da0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b330 tx=0x7f5cb80046c0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 shutdown_connections 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ad7a180f50 0x55ad7a183370 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ad7a106850 0x55ad7a109da0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5ca7fff640 0 client.17667 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ad7a0fbf90 0x55ad7a10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 --2- 172.21.15.78:0/2967916706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ad7a0fb670 0x55ad7a10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 >> 172.21.15.78:0/2967916706 conn(0x55ad7a0f7c00 msgr2=0x55ad7a105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 shutdown_connections 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 1 -- 172.21.15.78:0/2967916706 wait complete. 2024-09-16T00:25:47.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.839+0000 7f5cd1386740 20 client.17667 trim_cache size 0 max 16384 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: "a1102ccc-6b68-49a3-b8b9-c70bebfd3208": { 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "a1102ccc-6b68-49a3-b8b9-c70bebfd3208", 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:25:47.844 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:25:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cb00a33d0 msgr2=0x7f5cb00a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cb00a33d0 0x7f5cb00a5890 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5cc806b4a0 tx=0x7f5cb8005ef0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2851555125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc8110190 msgr2=0x7f5cc81a7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc8110190 0x7f5cc81a7660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5cc002acb0 tx=0x7f5cc0030620 comp rx=0 tx=0).stop 2024-09-16T00:25:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2851555125 shutdown_connections 2024-09-16T00:25:47.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cb00a33d0 0x7f5cb00a5890 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc811b140 0x7f5cc81ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5cc8110190 0x7f5cc81a7660 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 --2- 172.21.15.78:0/2851555125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cc806ea30 0x7f5cc81a7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:47.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.840+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2851555125 >> 172.21.15.78:0/2851555125 conn(0x7f5cc8073180 msgr2=0x7f5cc81194d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:47.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.841+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2851555125 shutdown_connections 2024-09-16T00:25:47.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:25:47.841+0000 7f5ccdd4b640 1 -- 172.21.15.78:0/2851555125 wait complete. 2024-09-16T00:25:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:47 smithi193 ceph-mon[35484]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 476 op/s 2024-09-16T00:25:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:47 smithi148 ceph-mon[35599]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 476 op/s 2024-09-16T00:25:48.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:47 smithi078 ceph-mon[31157]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 MiB/s wr, 476 op/s 2024-09-16T00:25:48.607 INFO:tasks.cephfs.filesystem:scrub status for tag:a1102ccc-6b68-49a3-b8b9-c70bebfd3208 - {'path': '/', 'tag': 'a1102ccc-6b68-49a3-b8b9-c70bebfd3208', 'options': 'recursive,force'} 2024-09-16T00:25:49.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:48 smithi193 ceph-mon[35484]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 486 op/s 2024-09-16T00:25:49.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:48 smithi148 ceph-mon[35599]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 486 op/s 2024-09-16T00:25:49.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:48 smithi078 ceph-mon[31157]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.8 MiB/s wr, 486 op/s 2024-09-16T00:25:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 710.0 s, 614.8 tps, lat 161.267 ms stddev 195.551 2024-09-16T00:25:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:51 smithi193 ceph-mon[35484]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 MiB/s wr, 506 op/s 2024-09-16T00:25:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:51 smithi148 ceph-mon[35599]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 MiB/s wr, 506 op/s 2024-09-16T00:25:52.291 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:51 smithi078 ceph-mon[31157]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.0 MiB/s wr, 506 op/s 2024-09-16T00:25:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:53 smithi193 ceph-mon[35484]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-09-16T00:25:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:53 smithi148 ceph-mon[35599]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-09-16T00:25:54.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:53 smithi078 ceph-mon[31157]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.8 MiB/s wr, 482 op/s 2024-09-16T00:25:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 715.0 s, 908.8 tps, lat 111.896 ms stddev 147.070 2024-09-16T00:25:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:55 smithi193 ceph-mon[35484]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.8 MiB/s wr, 567 op/s 2024-09-16T00:25:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:55 smithi148 ceph-mon[35599]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.8 MiB/s wr, 567 op/s 2024-09-16T00:25:56.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:55 smithi078 ceph-mon[31157]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.8 MiB/s wr, 567 op/s 2024-09-16T00:25:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:57 smithi193 ceph-mon[35484]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 525 op/s 2024-09-16T00:25:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:57 smithi148 ceph-mon[35599]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 525 op/s 2024-09-16T00:25:58.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:57 smithi078 ceph-mon[31157]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 525 op/s 2024-09-16T00:26:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:59 smithi193 ceph-mon[35484]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 547 op/s 2024-09-16T00:26:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:26:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:26:00.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:25:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:26:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:59 smithi148 ceph-mon[35599]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 547 op/s 2024-09-16T00:26:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:26:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:26:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:25:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:26:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:59 smithi078 ceph-mon[31157]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.2 MiB/s wr, 547 op/s 2024-09-16T00:26:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:26:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:26:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:25:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:26:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 720.0 s, 577.4 tps, lat 168.372 ms stddev 186.315 2024-09-16T00:26:02.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:01 smithi193 ceph-mon[35484]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 584 op/s 2024-09-16T00:26:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:01 smithi148 ceph-mon[35599]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 584 op/s 2024-09-16T00:26:02.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:01 smithi078 ceph-mon[31157]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.6 MiB/s wr, 584 op/s 2024-09-16T00:26:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:03 smithi193 ceph-mon[35484]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.7 MiB/s wr, 539 op/s 2024-09-16T00:26:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:03 smithi148 ceph-mon[35599]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.7 MiB/s wr, 539 op/s 2024-09-16T00:26:04.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:03 smithi078 ceph-mon[31157]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.7 MiB/s wr, 539 op/s 2024-09-16T00:26:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 725.0 s, 538.0 tps, lat 183.630 ms stddev 258.062 2024-09-16T00:26:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:05 smithi193 ceph-mon[35484]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-09-16T00:26:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:05 smithi148 ceph-mon[35599]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-09-16T00:26:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:05 smithi078 ceph-mon[31157]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-09-16T00:26:08.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:07 smithi193 ceph-mon[35484]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.2 MiB/s wr, 716 op/s 2024-09-16T00:26:08.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:07 smithi148 ceph-mon[35599]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.2 MiB/s wr, 716 op/s 2024-09-16T00:26:08.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:07 smithi078 ceph-mon[31157]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.2 MiB/s wr, 716 op/s 2024-09-16T00:26:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:09 smithi193 ceph-mon[35484]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 713 op/s 2024-09-16T00:26:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:09 smithi148 ceph-mon[35599]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 713 op/s 2024-09-16T00:26:10.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:09 smithi078 ceph-mon[31157]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.7 MiB/s wr, 713 op/s 2024-09-16T00:26:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 730.0 s, 733.2 tps, lat 137.584 ms stddev 169.226 2024-09-16T00:26:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:11 smithi193 ceph-mon[35484]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.0 MiB/s wr, 745 op/s 2024-09-16T00:26:12.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:11 smithi148 ceph-mon[35599]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.0 MiB/s wr, 745 op/s 2024-09-16T00:26:12.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:11 smithi078 ceph-mon[31157]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.0 MiB/s wr, 745 op/s 2024-09-16T00:26:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:13 smithi193 ceph-mon[35484]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 693 op/s 2024-09-16T00:26:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:13 smithi148 ceph-mon[35599]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 693 op/s 2024-09-16T00:26:14.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:13 smithi078 ceph-mon[31157]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 693 op/s 2024-09-16T00:26:15.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 735.0 s, 680.0 tps, lat 150.733 ms stddev 191.596 2024-09-16T00:26:16.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:15 smithi193 ceph-mon[35484]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 738 op/s 2024-09-16T00:26:16.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:15 smithi148 ceph-mon[35599]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 738 op/s 2024-09-16T00:26:16.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:15 smithi078 ceph-mon[31157]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.8 MiB/s wr, 738 op/s 2024-09-16T00:26:18.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:17 smithi193 ceph-mon[35484]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-09-16T00:26:18.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:17 smithi148 ceph-mon[35599]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-09-16T00:26:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:17 smithi078 ceph-mon[31157]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-09-16T00:26:18.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:26:18.950 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:18 smithi078 ceph-mon[31157]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-09-16T00:26:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:19.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:18 smithi148 ceph-mon[35599]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-09-16T00:26:19.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:18 smithi193 ceph-mon[35484]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-09-16T00:26:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.755+0000 7fe24ad8c640 1 -- 172.21.15.78:0/3501458218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe244105580 msgr2=0x7fe24410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.755+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/3501458218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe244105580 0x7fe24410d0e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009ca0 tx=0x7fe23c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.755+0000 7fe24ad8c640 1 -- 172.21.15.78:0/3501458218 shutdown_connections 2024-09-16T00:26:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.755+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/3501458218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe244105580 0x7fe24410d0e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.755+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/3501458218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe244101080 0x7fe244104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.755+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/3501458218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2441006d0 0x7fe244100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.755+0000 7fe24ad8c640 1 -- 172.21.15.78:0/3501458218 >> 172.21.15.78:0/3501458218 conn(0x7fe2440fba90 msgr2=0x7fe2440fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:19.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.755+0000 7fe24ad8c640 1 -- 172.21.15.78:0/3501458218 shutdown_connections 2024-09-16T00:26:19.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.756+0000 7fe24ad8c640 1 -- 172.21.15.78:0/3501458218 wait complete. 2024-09-16T00:26:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.756+0000 7fe24ad8c640 1 Processor -- start 2024-09-16T00:26:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.756+0000 7fe24ad8c640 1 -- start start 2024-09-16T00:26:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe24ad8c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2441006d0 0x7fe2441983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe24ad8c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244101080 0x7fe244198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe24ad8c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe244105580 0x7fe24419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe24ad8c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe244111f50 con 0x7fe2441006d0 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe24ad8c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe244111dd0 con 0x7fe244101080 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe24ad8c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441120d0 con 0x7fe244105580 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe248b28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2441006d0 0x7fe2441983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe248b28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2441006d0 0x7fe2441983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55738/0 (socket says 172.21.15.78:55738) 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe248b28640 1 -- 172.21.15.78:0/4073933312 learned_addr learned my addr 172.21.15.78:0/4073933312 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe243fff640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244101080 0x7fe244198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe249329640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe244105580 0x7fe24419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe243fff640 1 -- 172.21.15.78:0/4073933312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe244105580 msgr2=0x7fe24419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe243fff640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe244105580 0x7fe24419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe243fff640 1 -- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2441006d0 msgr2=0x7fe2441983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe243fff640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2441006d0 0x7fe2441983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.757+0000 7fe243fff640 1 -- 172.21.15.78:0/4073933312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24419d390 con 0x7fe244101080 2024-09-16T00:26:19.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.758+0000 7fe243fff640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244101080 0x7fe244198920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe230005fd0 tx=0x7fe230009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:19.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.758+0000 7fe241ffb640 1 -- 172.21.15.78:0/4073933312 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230004030 con 0x7fe244101080 2024-09-16T00:26:19.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.758+0000 7fe24ad8c640 1 -- 172.21.15.78:0/4073933312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24419d680 con 0x7fe244101080 2024-09-16T00:26:19.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.758+0000 7fe241ffb640 1 -- 172.21.15.78:0/4073933312 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe230007dd0 con 0x7fe244101080 2024-09-16T00:26:19.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.758+0000 7fe241ffb640 1 -- 172.21.15.78:0/4073933312 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230003d60 con 0x7fe244101080 2024-09-16T00:26:19.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.758+0000 7fe24ad8c640 1 -- 172.21.15.78:0/4073933312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24410f770 con 0x7fe244101080 2024-09-16T00:26:19.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.761+0000 7fe241ffb640 1 -- 172.21.15.78:0/4073933312 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe23001a410 con 0x7fe244101080 2024-09-16T00:26:19.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.764+0000 7fe241ffb640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe2340a3340 0x7fe2340a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.764+0000 7fe241ffb640 1 -- 172.21.15.78:0/4073933312 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe2300d3250 con 0x7fe244101080 2024-09-16T00:26:19.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.764+0000 7fe24d165740 1 Processor -- start 2024-09-16T00:26:19.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.764+0000 7fe24d165740 1 -- start start 2024-09-16T00:26:19.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe24d165740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe24d165740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe24d165740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe248b28640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe2340a3340 0x7fe2340a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe24d165740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5606a08122e0 con 0x5606a08acdc0 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe24d165740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5606a07b9ce0 con 0x5606a08b7680 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe24d165740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5606a080e2a0 con 0x5606a08ac4a0 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe243fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe243fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55752/0 (socket says 172.21.15.78:55752) 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe243fff640 1 -- 172.21.15.78:0/3144954373 learned_addr learned my addr 172.21.15.78:0/3144954373 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe249329640 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe243fff640 1 -- 172.21.15.78:0/3144954373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 msgr2=0x5606a08ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.765+0000 7fe248b28640 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe243fff640 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe243fff640 1 -- 172.21.15.78:0/3144954373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 msgr2=0x5606a08b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe243fff640 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe243fff640 1 -- 172.21.15.78:0/3144954373 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606a079e440 con 0x5606a08acdc0 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe249329640 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe248b28640 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe243fff640 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b7140 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe23009bf30 tx=0x7fe23009f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a60d2bc4bf14ab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe248b28640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe2340a3340 0x7fe2340a5800 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe23800e5b0 tx=0x7fe238005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.766+0000 7fe22effd640 1 -- 172.21.15.78:0/3144954373 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2300a3860 con 0x5606a08acdc0 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.767+0000 7fe22effd640 1 -- 172.21.15.78:0/3144954373 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2300d0e50 con 0x5606a08acdc0 2024-09-16T00:26:19.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.767+0000 7fe22effd640 1 -- 172.21.15.78:0/3144954373 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23009e9b0 con 0x5606a08acdc0 2024-09-16T00:26:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.767+0000 7fe24d165740 1 -- 172.21.15.78:0/3144954373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 msgr2=0x5606a08b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.767+0000 7fe24d165740 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b7140 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe23009bf30 tx=0x7fe23009f1a0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 -- 172.21.15.78:0/3144954373 shutdown_connections 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b7140 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 --2- 172.21.15.78:0/3144954373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 -- 172.21.15.78:0/3144954373 >> 172.21.15.78:0/3144954373 conn(0x5606a08a8a30 msgr2=0x5606a08a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 -- 172.21.15.78:0/3144954373 shutdown_connections 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 -- 172.21.15.78:0/3144954373 wait complete. 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 -- start start 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.768+0000 7fe24d165740 1 Processor -- start 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe24d165740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe24d165740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe248b28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe248b28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36944/0 (socket says 172.21.15.78:36944) 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe248b28640 1 -- 172.21.15.78:0/2217336443 learned_addr learned my addr 172.21.15.78:0/2217336443 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe24d165740 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5606a08ba2a0 con 0x5606a08acdc0 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5606a07cd050 con 0x5606a08b7680 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5606a08ba420 con 0x5606a08ac4a0 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe243fff640 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe24d165740 10 client.0 _ll_register_callbacks cb 0x5606a080d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe249329640 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe248b28640 1 -- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 msgr2=0x5606a08b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe248b28640 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe248b28640 1 -- 172.21.15.78:0/2217336443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 msgr2=0x5606a08b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:19.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe248b28640 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.769+0000 7fe248b28640 1 -- 172.21.15.78:0/2217336443 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606a05833c0 con 0x5606a08ac4a0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe248b28640 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b6fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe23800efd0 tx=0x7fe238010040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea4a0f481f3e0545 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe223fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe223fff640 1 -- 172.21.15.78:0/2217336443 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23802a060 con 0x5606a08ac4a0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe223fff640 1 -- 172.21.15.78:0/2217336443 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe238017030 con 0x5606a08ac4a0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2300d0e50 con 0x5606a08ac4a0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe223fff640 1 -- 172.21.15.78:0/2217336443 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe23801ce50 con 0x5606a08ac4a0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe24d165740 10 client.26728 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5606a08ce450 con 0x5606a08ac4a0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe223fff640 1 -- 172.21.15.78:0/2217336443 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2300d0e50 con 0x5606a08ac4a0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe223fff640 1 -- 172.21.15.78:0/2217336443 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe23800b040 con 0x5606a08ac4a0 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe24d165740 10 client.26728 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.770+0000 7fe24d165740 10 client.26728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:26:19.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.771+0000 7fe24d165740 20 client.26728 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.771+0000 7fe24d165740 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5606a08cf860 0x5606a08d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.771+0000 7fe24d165740 4 client.26728 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.771+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606a081f850 con 0x5606a08cf860 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.771+0000 7fe243fff640 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5606a08cf860 0x5606a08d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.772+0000 7fe223fff640 10 client.26728 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.772+0000 7fe243fff640 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5606a08cf860 0x5606a08d1c70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5980b5ea3c4cf7b1 server_cookie=64620be8b07119eb in_seq=0 out_seq=0 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.772+0000 7fe223fff640 1 -- 172.21.15.78:0/2217336443 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606a081f850 con 0x5606a08cf860 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.772+0000 7fe223fff640 10 client.26728 handle_command_reply: tid=0 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.772+0000 7fe24d165740 1 client.26728 shutdown 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.772+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 msgr2=0x5606a08b6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b6fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe23800efd0 tx=0x7fe238010040 comp rx=0 tx=0).stop 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 shutdown_connections 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5606a08cf860 0x5606a08d1c70 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe223fff640 0 client.26728 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 --2- 172.21.15.78:0/2217336443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b6fd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 >> 172.21.15.78:0/2217336443 conn(0x5606a08a8a30 msgr2=0x5606a08ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 shutdown_connections 2024-09-16T00:26:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 1 -- 172.21.15.78:0/2217336443 wait complete. 2024-09-16T00:26:19.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.773+0000 7fe24d165740 20 client.26728 trim_cache size 0 max 16384 2024-09-16T00:26:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe24d165740 1 Processor -- start 2024-09-16T00:26:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe24d165740 1 -- start start 2024-09-16T00:26:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe24d165740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe24d165740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08d7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe24d165740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe24d165740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441acaa0 con 0x5606a08acdc0 2024-09-16T00:26:19.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe24d165740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441ae5e0 con 0x5606a08b7680 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe24d165740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441ad170 con 0x5606a08ac4a0 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe249329640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe249329640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08d7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42038/0 (socket says 172.21.15.78:42038) 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe249329640 1 -- 172.21.15.78:0/1920219743 learned_addr learned my addr 172.21.15.78:0/1920219743 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.795+0000 7fe243fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe248b28640 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe249329640 1 -- 172.21.15.78:0/1920219743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 msgr2=0x5606a08b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe249329640 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe249329640 1 -- 172.21.15.78:0/1920219743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 msgr2=0x5606a08d7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe249329640 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08d7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe249329640 1 -- 172.21.15.78:0/1920219743 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606a08b3230 con 0x5606a08b7680 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe243fff640 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08d7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe249329640 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08d7a80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009ca0 tx=0x7fe23c004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800e6b5ae51ca68e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:19.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.796+0000 7fe22e7fc640 1 -- 172.21.15.78:0/1920219743 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c003970 con 0x5606a08b7680 2024-09-16T00:26:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe22e7fc640 1 -- 172.21.15.78:0/1920219743 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c036040 con 0x5606a08b7680 2024-09-16T00:26:19.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe22e7fc640 1 -- 172.21.15.78:0/1920219743 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c004420 con 0x5606a08b7680 2024-09-16T00:26:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 -- 172.21.15.78:0/1920219743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 msgr2=0x5606a08d7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08d7a80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009ca0 tx=0x7fe23c004a80 comp rx=0 tx=0).stop 2024-09-16T00:26:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 -- 172.21.15.78:0/1920219743 shutdown_connections 2024-09-16T00:26:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08b7680 0x5606a08d7a80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08acdc0 0x5606a08d7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 --2- 172.21.15.78:0/1920219743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 -- 172.21.15.78:0/1920219743 >> 172.21.15.78:0/1920219743 conn(0x5606a08a8a30 msgr2=0x5606a08a7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:19.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 -- 172.21.15.78:0/1920219743 shutdown_connections 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.797+0000 7fe24d165740 1 -- 172.21.15.78:0/1920219743 wait complete. 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.798+0000 7fe24d165740 1 -- start start 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.798+0000 7fe24d165740 1 Processor -- start 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.798+0000 7fe24d165740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.798+0000 7fe24d165740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08acdc0 0x5606a08c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.798+0000 7fe24d165740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08b7680 0x5606a0939520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe24d165740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441adf10 con 0x5606a08b7680 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe248b28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe24d165740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441ad840 con 0x5606a08acdc0 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe24d165740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441aaa80 con 0x5606a08ac4a0 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08b7680 0x5606a0939520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08b7680 0x5606a0939520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55772/0 (socket says 172.21.15.78:55772) 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 -- 172.21.15.78:0/2278795759 learned_addr learned my addr 172.21.15.78:0/2278795759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:19.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe24d165740 10 client.0 _ll_register_callbacks cb 0x7fe23800ace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe243fff640 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08acdc0 0x5606a08c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 -- 172.21.15.78:0/2278795759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 msgr2=0x5606a08c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 -- 172.21.15.78:0/2278795759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08acdc0 msgr2=0x5606a08c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08acdc0 0x5606a08c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 -- 172.21.15.78:0/2278795759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606a08b7340 con 0x5606a08b7680 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe249329640 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08b7680 0x5606a0939520 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe23c02ade0 tx=0x7fe23c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeff074d16a2dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.799+0000 7fe223fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.800+0000 7fe223fff640 1 -- 172.21.15.78:0/2278795759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c054070 con 0x5606a08b7680 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.800+0000 7fe24d165740 1 -- 172.21.15.78:0/2278795759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23c036040 con 0x5606a08b7680 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.800+0000 7fe223fff640 1 -- 172.21.15.78:0/2278795759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe23c004320 con 0x5606a08b7680 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.800+0000 7fe223fff640 1 -- 172.21.15.78:0/2278795759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe23c02f3c0 con 0x5606a08b7680 2024-09-16T00:26:19.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.800+0000 7fe24d165740 10 client.17685 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.800+0000 7fe24d165740 1 -- 172.21.15.78:0/2278795759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5606a093aa20 con 0x5606a08b7680 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.800+0000 7fe223fff640 1 -- 172.21.15.78:0/2278795759 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c036040 con 0x5606a08b7680 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.800+0000 7fe223fff640 1 -- 172.21.15.78:0/2278795759 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe23c02fd40 con 0x5606a08b7680 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.801+0000 7fe24d165740 10 client.17685 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.801+0000 7fe24d165740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.801+0000 7fe24d165740 20 client.17685 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.801+0000 7fe24d165740 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5606a093b120 0x5606a093d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.801+0000 7fe24d165740 4 client.17685 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.801+0000 7fe24d165740 1 -- 172.21.15.78:0/2278795759 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606a0819d60 con 0x5606a093b120 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.801+0000 7fe248b28640 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5606a093b120 0x5606a093d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:19.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.801+0000 7fe248b28640 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5606a093b120 0x5606a093d540 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e89d25bbab4df8 server_cookie=e7f492d7b3da78db in_seq=0 out_seq=0 2024-09-16T00:26:19.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe223fff640 10 client.17685 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:19.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe223fff640 1 -- 172.21.15.78:0/2278795759 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5606a0819d60 con 0x5606a093b120 2024-09-16T00:26:19.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe223fff640 10 client.17685 handle_command_reply: tid=0 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 client.17685 shutdown 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 -- 172.21.15.78:0/2278795759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08b7680 msgr2=0x5606a0939520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08b7680 0x5606a0939520 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe23c02ade0 tx=0x7fe23c003c20 comp rx=0 tx=0).stop 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 -- 172.21.15.78:0/2278795759 shutdown_connections 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5606a093b120 0x5606a093d540 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5606a08b7680 0x5606a0939520 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5606a08acdc0 0x5606a08c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 --2- 172.21.15.78:0/2278795759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606a08ac4a0 0x5606a08c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe223fff640 0 client.17685 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.802+0000 7fe24d165740 1 -- 172.21.15.78:0/2278795759 >> 172.21.15.78:0/2278795759 conn(0x5606a08a8a30 msgr2=0x5606a08a7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.803+0000 7fe24d165740 1 -- 172.21.15.78:0/2278795759 shutdown_connections 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.803+0000 7fe24d165740 1 -- 172.21.15.78:0/2278795759 wait complete. 2024-09-16T00:26:19.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.803+0000 7fe24d165740 20 client.17685 trim_cache size 0 max 16384 2024-09-16T00:26:19.808 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:26:19.808 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:26:19.808 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:26:19.808 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:26:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.804+0000 7fe24ad8c640 1 -- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe2340a3340 msgr2=0x7fe2340a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.804+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe2340a3340 0x7fe2340a5800 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe23800e5b0 tx=0x7fe238005f90 comp rx=0 tx=0).stop 2024-09-16T00:26:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.804+0000 7fe24ad8c640 1 -- 172.21.15.78:0/4073933312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244101080 msgr2=0x7fe244198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.804+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244101080 0x7fe244198920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe230005fd0 tx=0x7fe230009f60 comp rx=0 tx=0).stop 2024-09-16T00:26:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.805+0000 7fe24ad8c640 1 -- 172.21.15.78:0/4073933312 shutdown_connections 2024-09-16T00:26:19.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.805+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe2340a3340 0x7fe2340a5800 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.805+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe244105580 0x7fe24419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.805+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe244101080 0x7fe244198920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.805+0000 7fe24ad8c640 1 --2- 172.21.15.78:0/4073933312 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2441006d0 0x7fe2441983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.805+0000 7fe24ad8c640 1 -- 172.21.15.78:0/4073933312 >> 172.21.15.78:0/4073933312 conn(0x7fe2440fba90 msgr2=0x7fe2440fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.805+0000 7fe24ad8c640 1 -- 172.21.15.78:0/4073933312 shutdown_connections 2024-09-16T00:26:19.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:19.805+0000 7fe24ad8c640 1 -- 172.21.15.78:0/4073933312 wait complete. 2024-09-16T00:26:20.643 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:26:20.644 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:26:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 740.0 s, 656.4 tps, lat 151.899 ms stddev 183.453 2024-09-16T00:26:21.132 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:21.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.741+0000 7f44e51da640 1 -- 172.21.15.78:0/1651724823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e011b140 msgr2=0x7f44e011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:21.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.741+0000 7f44e51da640 1 --2- 172.21.15.78:0/1651724823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e011b140 0x7f44e011d530 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009ca0 tx=0x7f44d802a8f0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.741+0000 7f44e51da640 1 -- 172.21.15.78:0/1651724823 shutdown_connections 2024-09-16T00:26:21.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.741+0000 7f44e51da640 1 --2- 172.21.15.78:0/1651724823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e011b140 0x7f44e011d530 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.741+0000 7f44e51da640 1 --2- 172.21.15.78:0/1651724823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e0109830 0x7f44e0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.741+0000 7f44e51da640 1 --2- 172.21.15.78:0/1651724823 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0109450 0x7f44e0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.741+0000 7f44e51da640 1 -- 172.21.15.78:0/1651724823 >> 172.21.15.78:0/1651724823 conn(0x7f44e006e140 msgr2=0x7f44e006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.749+0000 7f44e51da640 1 -- 172.21.15.78:0/1651724823 shutdown_connections 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.749+0000 7f44e51da640 1 -- 172.21.15.78:0/1651724823 wait complete. 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.750+0000 7f44e51da640 1 Processor -- start 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.750+0000 7f44e51da640 1 -- start start 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.751+0000 7f44e51da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e011b140 0x7f44e011a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.751+0000 7f44e51da640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e0113970 0x7f44e0113dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.751+0000 7f44e51da640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0114310 0x7f44e0118410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:21.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.751+0000 7f44e51da640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f44e0120e80 con 0x7f44e0113970 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.751+0000 7f44e51da640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f44e0120d00 con 0x7f44e0114310 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.751+0000 7f44e51da640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44e0121000 con 0x7f44e011b140 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.753+0000 7f44df59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0114310 0x7f44e0118410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.753+0000 7f44df59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0114310 0x7f44e0118410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42058/0 (socket says 172.21.15.78:42058) 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.753+0000 7f44df59e640 1 -- 172.21.15.78:0/1656165112 learned_addr learned my addr 172.21.15.78:0/1656165112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.754+0000 7f44df59e640 1 -- 172.21.15.78:0/1656165112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e011b140 msgr2=0x7f44e011a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.754+0000 7f44df59e640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e011b140 0x7f44e011a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.755+0000 7f44df59e640 1 -- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e0113970 msgr2=0x7f44e0113dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.755+0000 7f44df59e640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e0113970 0x7f44e0113dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.755+0000 7f44df59e640 1 -- 172.21.15.78:0/1656165112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e0118950 con 0x7f44e0114310 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.761+0000 7f44df59e640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0114310 0x7f44e0118410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44d802ae60 tx=0x7f44d8030c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.769+0000 7f44c7fff640 1 -- 172.21.15.78:0/1656165112 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8003970 con 0x7f44e0114310 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.769+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e0118b80 con 0x7f44e0114310 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.769+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e01190c0 con 0x7f44e0114310 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.769+0000 7f44c7fff640 1 -- 172.21.15.78:0/1656165112 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d802d070 con 0x7f44e0114310 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.769+0000 7f44c7fff640 1 -- 172.21.15.78:0/1656165112 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d80043b0 con 0x7f44e0114310 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.775+0000 7f44c7fff640 1 -- 172.21.15.78:0/1656165112 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f44d8004550 con 0x7f44e0114310 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.776+0000 7f44c7fff640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f44c80a3450 0x7f44c80a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.776+0000 7f44c7fff640 1 -- 172.21.15.78:0/1656165112 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f44d80f12e0 con 0x7f44e0114310 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.777+0000 7f44ded9d640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f44c80a3450 0x7f44c80a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.777+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44cc005290 con 0x7f44e0114310 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.783+0000 7f44ded9d640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f44c80a3450 0x7f44c80a5910 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44d0005fd0 tx=0x7f44d0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:21.787+0000 7f44c7fff640 1 -- 172.21.15.78:0/1656165112 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44d80bdc40 con 0x7f44e0114310 2024-09-16T00:26:21.885 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:21 smithi078 ceph-mon[31157]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.1 MiB/s wr, 594 op/s 2024-09-16T00:26:22.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.159+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44cc005580 con 0x7f44e0114310 2024-09-16T00:26:22.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.161+0000 7f44c7fff640 1 -- 172.21.15.78:0/1656165112 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f44d80bd540 con 0x7f44e0114310 2024-09-16T00:26:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:26:22.163 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:26:22.163 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:26:22.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f44c80a3450 msgr2=0x7f44c80a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:22.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f44c80a3450 0x7f44c80a5910 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44d0005fd0 tx=0x7f44d0005850 comp rx=0 tx=0).stop 2024-09-16T00:26:22.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0114310 msgr2=0x7f44e0118410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:22.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0114310 0x7f44e0118410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44d802ae60 tx=0x7f44d8030c60 comp rx=0 tx=0).stop 2024-09-16T00:26:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 shutdown_connections 2024-09-16T00:26:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f44c80a3450 0x7f44c80a5910 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44e0114310 0x7f44e0118410 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f44e0113970 0x7f44e0113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 --2- 172.21.15.78:0/1656165112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44e011b140 0x7f44e011a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 >> 172.21.15.78:0/1656165112 conn(0x7f44e006e140 msgr2=0x7f44e0072530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 shutdown_connections 2024-09-16T00:26:22.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:22.169+0000 7f44e51da640 1 -- 172.21.15.78:0/1656165112 wait complete. 2024-09-16T00:26:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:21 smithi193 ceph-mon[35484]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.1 MiB/s wr, 594 op/s 2024-09-16T00:26:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:21 smithi148 ceph-mon[35599]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.1 MiB/s wr, 594 op/s 2024-09-16T00:26:23.112 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:22 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1656165112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:23.161 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:26:23.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:22 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1656165112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:23.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:22 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1656165112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:24.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:23 smithi078 ceph-mon[31157]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.5 MiB/s wr, 511 op/s 2024-09-16T00:26:24.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:23 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:23 smithi193 ceph-mon[35484]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.5 MiB/s wr, 511 op/s 2024-09-16T00:26:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:23 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:24.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:23 smithi148 ceph-mon[35599]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.5 MiB/s wr, 511 op/s 2024-09-16T00:26:24.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:23 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:25.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.292+0000 7f6463576640 1 -- 172.21.15.78:0/1986844186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 msgr2=0x7f645c106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.292+0000 7f6463576640 1 --2- 172.21.15.78:0/1986844186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 0x7f645c106220 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6454009ca0 tx=0x7f645402a980 comp rx=0 tx=0).stop 2024-09-16T00:26:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.294+0000 7f6463576640 1 -- 172.21.15.78:0/1986844186 shutdown_connections 2024-09-16T00:26:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.294+0000 7f6463576640 1 --2- 172.21.15.78:0/1986844186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 0x7f645c106220 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.294+0000 7f6463576640 1 --2- 172.21.15.78:0/1986844186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f645c1012a0 0x7f645c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.294+0000 7f6463576640 1 --2- 172.21.15.78:0/1986844186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f645c100980 0x7f645c100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.294+0000 7f6463576640 1 -- 172.21.15.78:0/1986844186 >> 172.21.15.78:0/1986844186 conn(0x7f645c0fc610 msgr2=0x7f645c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:25.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.298+0000 7f6463576640 1 -- 172.21.15.78:0/1986844186 shutdown_connections 2024-09-16T00:26:25.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.300+0000 7f6463576640 1 -- 172.21.15.78:0/1986844186 wait complete. 2024-09-16T00:26:25.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.301+0000 7f6463576640 1 Processor -- start 2024-09-16T00:26:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.303+0000 7f6463576640 1 -- start start 2024-09-16T00:26:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.303+0000 7f6463576640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f645c100980 0x7f645c1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.303+0000 7f6463576640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f645c1012a0 0x7f645c1a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.303+0000 7f6463576640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 0x7f645c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.303+0000 7f6463576640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f645c118430 con 0x7f645c100980 2024-09-16T00:26:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.304+0000 7f6463576640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1182b0 con 0x7f645c1012a0 2024-09-16T00:26:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.304+0000 7f6463576640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f645c1185b0 con 0x7f645c105d90 2024-09-16T00:26:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.304+0000 7f6462574640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f645c100980 0x7f645c1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.304+0000 7f6462d75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 0x7f645c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.304+0000 7f6462d75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 0x7f645c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36996/0 (socket says 172.21.15.78:36996) 2024-09-16T00:26:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.304+0000 7f6462d75640 1 -- 172.21.15.78:0/1723673209 learned_addr learned my addr 172.21.15.78:0/1723673209 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:25.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.307+0000 7f6462574640 1 -- 172.21.15.78:0/1723673209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 msgr2=0x7f645c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.307+0000 7f6462574640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 0x7f645c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.307+0000 7f6462574640 1 -- 172.21.15.78:0/1723673209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f645c1012a0 msgr2=0x7f645c1a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.306+0000 7f6461d73640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f645c1012a0 0x7f645c1a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.310+0000 7f6462574640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f645c1012a0 0x7f645c1a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.310+0000 7f6462574640 1 -- 172.21.15.78:0/1723673209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c19ef20 con 0x7f645c100980 2024-09-16T00:26:25.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.310+0000 7f6461d73640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f645c1012a0 0x7f645c1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:25.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.315+0000 7f6462574640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f645c100980 0x7f645c1a4600 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f645800d6d0 tx=0x7f645800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:25.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.316+0000 7f64537fe640 1 -- 172.21.15.78:0/1723673209 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458017070 con 0x7f645c100980 2024-09-16T00:26:25.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.316+0000 7f64537fe640 1 -- 172.21.15.78:0/1723673209 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64580047a0 con 0x7f645c100980 2024-09-16T00:26:25.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.316+0000 7f64537fe640 1 -- 172.21.15.78:0/1723673209 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458009cd0 con 0x7f645c100980 2024-09-16T00:26:25.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.316+0000 7f6463576640 1 -- 172.21.15.78:0/1723673209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c19f210 con 0x7f645c100980 2024-09-16T00:26:25.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.316+0000 7f6463576640 1 -- 172.21.15.78:0/1723673209 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c1ab460 con 0x7f645c100980 2024-09-16T00:26:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.321+0000 7f64537fe640 1 -- 172.21.15.78:0/1723673209 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6458005220 con 0x7f645c100980 2024-09-16T00:26:25.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.322+0000 7f64537fe640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64480a3450 0x7f64480a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.323+0000 7f64537fe640 1 -- 172.21.15.78:0/1723673209 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f64580cf3e0 con 0x7f645c100980 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.324+0000 7f6466bb1740 1 Processor -- start 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.324+0000 7f6466bb1740 1 -- start start 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.324+0000 7f6461d73640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64480a3450 0x7f64480a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.324+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027288cf0 0x563027287240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.324+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x563027287bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6466bb1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5630271ed2f0 con 0x563027288cf0 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6466bb1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563027194cf0 con 0x563027287780 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6466bb1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5630271e92b0 con 0x5630272a3530 2024-09-16T00:26:25.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6462d75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6462d75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37000/0 (socket says 172.21.15.78:37000) 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6461d73640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x563027287bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6462d75640 1 -- 172.21.15.78:0/4230356950 learned_addr learned my addr 172.21.15.78:0/4230356950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.325+0000 7f6462574640 1 --2- 172.21.15.78:0/4230356950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027288cf0 0x563027287240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6462d75640 1 -- 172.21.15.78:0/4230356950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 msgr2=0x563027287bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6462d75640 1 --2- 172.21.15.78:0/4230356950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x563027287bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6462d75640 1 -- 172.21.15.78:0/4230356950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027288cf0 msgr2=0x563027287240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6462d75640 1 --2- 172.21.15.78:0/4230356950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027288cf0 0x563027287240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6462d75640 1 -- 172.21.15.78:0/4230356950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630271793a0 con 0x5630272a3530 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6461d73640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64480a3450 0x7f64480a5910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f645c0ffc20 tx=0x7f644c00ad10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6462d75640 1 --2- 172.21.15.78:0/4230356950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a5920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f645402fe90 tx=0x7f645402fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e73d4fcb37a8007 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:25.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6450ff9640 1 -- 172.21.15.78:0/4230356950 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454034040 con 0x5630272a3530 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6450ff9640 1 -- 172.21.15.78:0/4230356950 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6454003e50 con 0x5630272a3530 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.326+0000 7f6450ff9640 1 -- 172.21.15.78:0/4230356950 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64540308e0 con 0x5630272a3530 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.327+0000 7f6466bb1740 1 -- 172.21.15.78:0/4230356950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 msgr2=0x5630272a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.327+0000 7f6466bb1740 1 --2- 172.21.15.78:0/4230356950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a5920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f645402fe90 tx=0x7f645402fec0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.329+0000 7f6466bb1740 1 -- 172.21.15.78:0/4230356950 shutdown_connections 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.329+0000 7f6466bb1740 1 --2- 172.21.15.78:0/4230356950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a5920 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.329+0000 7f6466bb1740 1 --2- 172.21.15.78:0/4230356950 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x563027287bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.329+0000 7f6466bb1740 1 --2- 172.21.15.78:0/4230356950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027288cf0 0x563027287240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.329+0000 7f6466bb1740 1 -- 172.21.15.78:0/4230356950 >> 172.21.15.78:0/4230356950 conn(0x563027250310 msgr2=0x563027283e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.330+0000 7f6466bb1740 1 -- 172.21.15.78:0/4230356950 shutdown_connections 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.330+0000 7f6466bb1740 1 -- 172.21.15.78:0/4230356950 wait complete. 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.330+0000 7f6466bb1740 1 -- start start 2024-09-16T00:26:25.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.330+0000 7f6466bb1740 1 Processor -- start 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.331+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x5630272944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.331+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563027288cf0 0x563027294a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.331+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630272a3530 0x56302728f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.331+0000 7f6466bb1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56302728a640 con 0x5630272a3530 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.331+0000 7f6466bb1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630271a8060 con 0x563027287780 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.331+0000 7f6466bb1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56302728a7c0 con 0x563027288cf0 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.331+0000 7f6466bb1740 10 client.0 _ll_register_callbacks cb 0x563026efb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.331+0000 7f6462574640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x5630272944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.332+0000 7f6462574640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x5630272944c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42088/0 (socket says 172.21.15.78:42088) 2024-09-16T00:26:25.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.332+0000 7f6462574640 1 -- 172.21.15.78:0/47030376 learned_addr learned my addr 172.21.15.78:0/47030376 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.332+0000 7f6461d73640 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563027288cf0 0x563027294a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.332+0000 7f6462574640 1 -- 172.21.15.78:0/47030376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563027288cf0 msgr2=0x563027294a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.332+0000 7f6462574640 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563027288cf0 0x563027294a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.332+0000 7f6462574640 1 -- 172.21.15.78:0/47030376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630272a3530 msgr2=0x56302728f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.332+0000 7f6462574640 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630272a3530 0x56302728f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.332+0000 7f6462574640 1 -- 172.21.15.78:0/47030376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563026f5e3c0 con 0x563027287780 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.333+0000 7f6462574640 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x5630272944c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6458098010 tx=0x7f645809bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ae1748703cf1e06 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.333+0000 7f6445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.333+0000 7f6445ffb640 1 -- 172.21.15.78:0/47030376 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64580b4060 con 0x563027287780 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.333+0000 7f6466bb1740 1 -- 172.21.15.78:0/47030376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6454003e50 con 0x563027287780 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.333+0000 7f6445ffb640 1 -- 172.21.15.78:0/47030376 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6458097cb0 con 0x563027287780 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.334+0000 7f6445ffb640 1 -- 172.21.15.78:0/47030376 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f64580a42b0 con 0x563027287780 2024-09-16T00:26:25.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.334+0000 7f6466bb1740 10 client.26939 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:25.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.334+0000 7f6466bb1740 1 -- 172.21.15.78:0/47030376 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56302729e190 con 0x563027287780 2024-09-16T00:26:25.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.334+0000 7f6445ffb640 1 -- 172.21.15.78:0/47030376 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6454003e50 con 0x563027287780 2024-09-16T00:26:25.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.335+0000 7f6445ffb640 1 -- 172.21.15.78:0/47030376 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6458004950 con 0x563027287780 2024-09-16T00:26:25.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.335+0000 7f6466bb1740 10 client.26939 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:25.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.335+0000 7f6466bb1740 10 client.26939 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:26:25.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.335+0000 7f6466bb1740 20 client.26939 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:25.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.335+0000 7f6466bb1740 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56302729fb00 0x5630272a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.335+0000 7f6466bb1740 4 client.26939 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.335+0000 7f6466bb1740 1 -- 172.21.15.78:0/47030376 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630271fa860 con 0x56302729fb00 2024-09-16T00:26:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.336+0000 7f6461d73640 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56302729fb00 0x5630272a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.339+0000 7f6461d73640 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56302729fb00 0x5630272a1fc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf84de6bfa3cf492 server_cookie=22a29a4eb7f76d04 in_seq=0 out_seq=0 2024-09-16T00:26:25.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.339+0000 7f6445ffb640 10 client.26939 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.340+0000 7f6445ffb640 1 -- 172.21.15.78:0/47030376 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630271fa860 con 0x56302729fb00 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.340+0000 7f6445ffb640 10 client.26939 handle_command_reply: tid=0 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 client.26939 shutdown 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 -- 172.21.15.78:0/47030376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 msgr2=0x5630272944c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x5630272944c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6458098010 tx=0x7f645809bd40 comp rx=0 tx=0).stop 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 -- 172.21.15.78:0/47030376 shutdown_connections 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5630272a3530 0x56302728f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56302729fb00 0x5630272a1fc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563027288cf0 0x563027294a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 --2- 172.21.15.78:0/47030376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027287780 0x5630272944c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 -- 172.21.15.78:0/47030376 >> 172.21.15.78:0/47030376 conn(0x563027250310 msgr2=0x563027289ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:25.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6445ffb640 0 client.26939 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:26:25.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 -- 172.21.15.78:0/47030376 shutdown_connections 2024-09-16T00:26:25.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.341+0000 7f6466bb1740 1 -- 172.21.15.78:0/47030376 wait complete. 2024-09-16T00:26:25.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.342+0000 7f6466bb1740 20 client.26939 trim_cache size 0 max 16384 2024-09-16T00:26:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.366+0000 7f6466bb1740 1 Processor -- start 2024-09-16T00:26:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.366+0000 7f6466bb1740 1 -- start start 2024-09-16T00:26:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.366+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027287780 0x56302728f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.366+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.366+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.366+0000 7f6466bb1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f644000fa30 con 0x563027287780 2024-09-16T00:26:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.366+0000 7f6466bb1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6440011570 con 0x563027288cf0 2024-09-16T00:26:25.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.366+0000 7f6466bb1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6440010100 con 0x5630272a3530 2024-09-16T00:26:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.371+0000 7f6462d75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.371+0000 7f6462d75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:37028/0 (socket says 172.21.15.78:37028) 2024-09-16T00:26:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.371+0000 7f6462d75640 1 -- 172.21.15.78:0/1406232930 learned_addr learned my addr 172.21.15.78:0/1406232930 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.371+0000 7f6461d73640 1 --2- 172.21.15.78:0/1406232930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.373+0000 7f6462d75640 1 -- 172.21.15.78:0/1406232930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 msgr2=0x5630272a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:25.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.373+0000 7f6462d75640 1 --2- 172.21.15.78:0/1406232930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.373+0000 7f6462d75640 1 -- 172.21.15.78:0/1406232930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027287780 msgr2=0x56302728f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.373+0000 7f6462d75640 1 --2- 172.21.15.78:0/1406232930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027287780 0x56302728f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.373+0000 7f6462d75640 1 -- 172.21.15.78:0/1406232930 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563027287f00 con 0x5630272a3530 2024-09-16T00:26:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.379+0000 7f6462d75640 1 --2- 172.21.15.78:0/1406232930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a2de0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6454005e30 tx=0x7f645402ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a808ce0293f98dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:25.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.381+0000 7f6447fff640 1 -- 172.21.15.78:0/1406232930 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454072070 con 0x5630272a3530 2024-09-16T00:26:25.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.381+0000 7f6447fff640 1 -- 172.21.15.78:0/1406232930 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6454003e50 con 0x5630272a3530 2024-09-16T00:26:25.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.381+0000 7f6447fff640 1 -- 172.21.15.78:0/1406232930 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454034040 con 0x5630272a3530 2024-09-16T00:26:25.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.385+0000 7f6466bb1740 1 -- 172.21.15.78:0/1406232930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 msgr2=0x5630272a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.385+0000 7f6466bb1740 1 --2- 172.21.15.78:0/1406232930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a2de0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6454005e30 tx=0x7f645402ff70 comp rx=0 tx=0).stop 2024-09-16T00:26:25.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.385+0000 7f6466bb1740 1 -- 172.21.15.78:0/1406232930 shutdown_connections 2024-09-16T00:26:25.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.385+0000 7f6466bb1740 1 --2- 172.21.15.78:0/1406232930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a2de0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.385+0000 7f6466bb1740 1 --2- 172.21.15.78:0/1406232930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.385+0000 7f6466bb1740 1 --2- 172.21.15.78:0/1406232930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027287780 0x56302728f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.385+0000 7f6466bb1740 1 -- 172.21.15.78:0/1406232930 >> 172.21.15.78:0/1406232930 conn(0x563027250310 msgr2=0x56302728f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:25.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.391+0000 7f6466bb1740 1 -- 172.21.15.78:0/1406232930 shutdown_connections 2024-09-16T00:26:25.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.391+0000 7f6466bb1740 1 -- 172.21.15.78:0/1406232930 wait complete. 2024-09-16T00:26:25.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.391+0000 7f6466bb1740 1 -- start start 2024-09-16T00:26:25.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.391+0000 7f6466bb1740 1 Processor -- start 2024-09-16T00:26:25.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.392+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027287780 0x5630272ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.392+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.392+0000 7f6466bb1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.392+0000 7f6461d73640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6461d73640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:42102/0 (socket says 172.21.15.78:42102) 2024-09-16T00:26:25.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6461d73640 1 -- 172.21.15.78:0/3527682050 learned_addr learned my addr 172.21.15.78:0/3527682050 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:25.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6462d75640 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6440010ea0 con 0x563027287780 2024-09-16T00:26:25.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f64400107d0 con 0x563027288cf0 2024-09-16T00:26:25.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f644000dac0 con 0x5630272a3530 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6466bb1740 10 client.0 _ll_register_callbacks cb 0x7f6458005790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6461d73640 1 -- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 msgr2=0x5630272a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6461d73640 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6461d73640 1 -- 172.21.15.78:0/3527682050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027287780 msgr2=0x5630272ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6461d73640 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027287780 0x5630272ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.393+0000 7f6461d73640 1 -- 172.21.15.78:0/3527682050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630272a8730 con 0x563027288cf0 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.394+0000 7f6462d75640 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.394+0000 7f6461d73640 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a79f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x563027284c70 tx=0x7f644c0046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85502fd991ac75a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.394+0000 7f6445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:26:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.397+0000 7f6445ffb640 1 -- 172.21.15.78:0/3527682050 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c01d040 con 0x563027288cf0 2024-09-16T00:26:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.398+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6454003e50 con 0x563027288cf0 2024-09-16T00:26:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.398+0000 7f6445ffb640 1 -- 172.21.15.78:0/3527682050 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f644c015450 con 0x563027288cf0 2024-09-16T00:26:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.398+0000 7f6445ffb640 1 -- 172.21.15.78:0/3527682050 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f644c039050 con 0x563027288cf0 2024-09-16T00:26:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.400+0000 7f6466bb1740 10 client.26945 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:25.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.400+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56302730cfb0 con 0x563027288cf0 2024-09-16T00:26:25.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.401+0000 7f6445ffb640 1 -- 172.21.15.78:0/3527682050 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6454003e50 con 0x563027288cf0 2024-09-16T00:26:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.405+0000 7f6445ffb640 1 -- 172.21.15.78:0/3527682050 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f644c02a030 con 0x563027288cf0 2024-09-16T00:26:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6466bb1740 10 client.26945 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6466bb1740 10 client.26945 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:26:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6466bb1740 20 client.26945 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6466bb1740 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56302730d700 0x56302730fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6466bb1740 4 client.26945 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:26:25.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630271f4d70 con 0x56302730d700 2024-09-16T00:26:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6462574640 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56302730d700 0x56302730fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6462574640 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56302730d700 0x56302730fb20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=934c6724acdfe0af server_cookie=d4825e56a3f8f2f9 in_seq=0 out_seq=0 2024-09-16T00:26:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.407+0000 7f6445ffb640 10 client.26945 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:26:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.408+0000 7f6445ffb640 1 -- 172.21.15.78:0/3527682050 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630271f4d70 con 0x56302730d700 2024-09-16T00:26:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.408+0000 7f6445ffb640 10 client.26945 handle_command_reply: tid=0 2024-09-16T00:26:25.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.408+0000 7f6466bb1740 1 client.26945 shutdown 2024-09-16T00:26:25.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.412+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 msgr2=0x5630272a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.412+0000 7f6466bb1740 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a79f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x563027284c70 tx=0x7f644c0046e0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 shutdown_connections 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56302730d700 0x56302730fb20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5630272a3530 0x5630272a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563027288cf0 0x5630272a79f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6445ffb640 0 client.26945 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 1 --2- 172.21.15.78:0/3527682050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563027287780 0x5630272ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 >> 172.21.15.78:0/3527682050 conn(0x563027250310 msgr2=0x56302729b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 shutdown_connections 2024-09-16T00:26:25.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 1 -- 172.21.15.78:0/3527682050 wait complete. 2024-09-16T00:26:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.413+0000 7f6466bb1740 20 client.26945 trim_cache size 0 max 16384 2024-09-16T00:26:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 -- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64480a3450 msgr2=0x7f64480a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64480a3450 0x7f64480a5910 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f645c0ffc20 tx=0x7f644c00ad10 comp rx=0 tx=0).stop 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 -- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f645c100980 msgr2=0x7f645c1a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f645c100980 0x7f645c1a4600 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f645800d6d0 tx=0x7f645800db90 comp rx=0 tx=0).stop 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 -- 172.21.15.78:0/1723673209 shutdown_connections 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f64480a3450 0x7f64480a5910 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f645c105d90 0x7f645c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f645c1012a0 0x7f645c1a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 --2- 172.21.15.78:0/1723673209 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f645c100980 0x7f645c1a4600 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:25.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 -- 172.21.15.78:0/1723673209 >> 172.21.15.78:0/1723673209 conn(0x7f645c0fc610 msgr2=0x7f645c103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:25.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 -- 172.21.15.78:0/1723673209 shutdown_connections 2024-09-16T00:26:25.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:25.419+0000 7f6463576640 1 -- 172.21.15.78:0/1723673209 wait complete. 2024-09-16T00:26:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 745.0 s, 450.0 tps, lat 216.810 ms stddev 257.270 2024-09-16T00:26:26.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:25 smithi193 ceph-mon[35484]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 MiB/s wr, 608 op/s 2024-09-16T00:26:26.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:25 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:26.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:25 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:26.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:25 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:26:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:25 smithi148 ceph-mon[35599]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 MiB/s wr, 608 op/s 2024-09-16T00:26:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:25 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:25 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:25 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:26:26.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:25 smithi078 ceph-mon[31157]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 MiB/s wr, 608 op/s 2024-09-16T00:26:26.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:26.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:26.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:25 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:26:26.275 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:26:26.744 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:28.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:27 smithi193 ceph-mon[35484]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-16T00:26:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:27 smithi148 ceph-mon[35599]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-16T00:26:28.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:27 smithi078 ceph-mon[31157]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.0 MiB/s wr, 606 op/s 2024-09-16T00:26:30.166 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:29 smithi078 ceph-mon[31157]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 605 op/s 2024-09-16T00:26:30.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:29 smithi193 ceph-mon[35484]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 605 op/s 2024-09-16T00:26:30.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:29 smithi148 ceph-mon[35599]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.4 MiB/s wr, 605 op/s 2024-09-16T00:26:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 750.0 s, 688.4 tps, lat 148.223 ms stddev 179.144 2024-09-16T00:26:31.437 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:26:31.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.298+0000 7f9d76308640 1 -- 172.21.15.78:0/2368263336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 msgr2=0x7f9d70115fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.298+0000 7f9d76308640 1 --2- 172.21.15.78:0/2368263336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 0x7f9d70115fb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009ca0 tx=0x7f9d6802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.299+0000 7f9d76308640 1 -- 172.21.15.78:0/2368263336 shutdown_connections 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.299+0000 7f9d76308640 1 --2- 172.21.15.78:0/2368263336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 0x7f9d70115fb0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.299+0000 7f9d76308640 1 --2- 172.21.15.78:0/2368263336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d7006f100 0x7f9d7010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.299+0000 7f9d76308640 1 --2- 172.21.15.78:0/2368263336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d7006e750 0x7f9d7006eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.299+0000 7f9d76308640 1 -- 172.21.15.78:0/2368263336 >> 172.21.15.78:0/2368263336 conn(0x7f9d70073180 msgr2=0x7f9d70073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.299+0000 7f9d76308640 1 -- 172.21.15.78:0/2368263336 shutdown_connections 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.299+0000 7f9d76308640 1 -- 172.21.15.78:0/2368263336 wait complete. 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d76308640 1 Processor -- start 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d76308640 1 -- start start 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d76308640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d7006e750 0x7f9d701a8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d76308640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d7006f100 0x7f9d701a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d76308640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 0x7f9d701a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d76308640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701183d0 con 0x7f9d7010e3f0 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d76308640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70118250 con 0x7f9d7006e750 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d76308640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d70118550 con 0x7f9d7006f100 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d75b07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 0x7f9d701a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d75b07640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 0x7f9d701a2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40332/0 (socket says 172.21.15.78:40332) 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d75b07640 1 -- 172.21.15.78:0/1662470746 learned_addr learned my addr 172.21.15.78:0/1662470746 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d75306640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d7006e750 0x7f9d701a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.300+0000 7f9d74b05640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d7006f100 0x7f9d701a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d75b07640 1 -- 172.21.15.78:0/1662470746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d7006f100 msgr2=0x7f9d701a8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d75b07640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d7006f100 0x7f9d701a8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d75b07640 1 -- 172.21.15.78:0/1662470746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d7006e750 msgr2=0x7f9d701a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d75b07640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d7006e750 0x7f9d701a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d75b07640 1 -- 172.21.15.78:0/1662470746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d701a3160 con 0x7f9d7010e3f0 2024-09-16T00:26:31.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d75306640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d7006e750 0x7f9d701a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d75b07640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 0x7f9d701a2a30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9d68033040 tx=0x7f9d68003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d667fc640 1 -- 172.21.15.78:0/1662470746 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68031650 con 0x7f9d7010e3f0 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d667fc640 1 -- 172.21.15.78:0/1662470746 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d680317f0 con 0x7f9d7010e3f0 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d667fc640 1 -- 172.21.15.78:0/1662470746 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6802f970 con 0x7f9d7010e3f0 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d76308640 1 -- 172.21.15.78:0/1662470746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d701a33f0 con 0x7f9d7010e3f0 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.301+0000 7f9d76308640 1 -- 172.21.15.78:0/1662470746 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d701af740 con 0x7f9d7010e3f0 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.304+0000 7f9d667fc640 1 -- 172.21.15.78:0/1662470746 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9d6802fb10 con 0x7f9d7010e3f0 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.304+0000 7f9d79943740 1 Processor -- start 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.304+0000 7f9d79943740 1 -- start start 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.304+0000 7f9d79943740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db9828240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.304+0000 7f9d79943740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 0x563db9828bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d79943740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9833520 0x563db9835910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d79943740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563db978e2f0 con 0x563db9828780 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d79943740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563db9735cf0 con 0x563db9829cf0 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d79943740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563db978a2b0 con 0x563db9833520 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d667fc640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d5c0a32c0 0x7f9d5c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9833520 0x563db9835910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db9828240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db9828240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37572/0 (socket says 172.21.15.78:37572) 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 -- 172.21.15.78:0/2742241055 learned_addr learned my addr 172.21.15.78:0/2742241055 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d667fc640 1 -- 172.21.15.78:0/1662470746 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9d680f1090 con 0x7f9d7010e3f0 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 -- 172.21.15.78:0/2742241055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9833520 msgr2=0x563db9835910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 --2- 172.21.15.78:0/2742241055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9833520 0x563db9835910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 -- 172.21.15.78:0/2742241055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 msgr2=0x563db9828bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 --2- 172.21.15.78:0/2742241055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 0x563db9828bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 -- 172.21.15.78:0/2742241055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563db971a3a0 con 0x563db9829cf0 2024-09-16T00:26:31.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d75b07640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d5c0a32c0 0x7f9d5c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.305+0000 7f9d74b05640 1 --2- 172.21.15.78:0/2742241055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db9828240 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c005a70 tx=0x7f9d6c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e33fb0f8c7fc0c91 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.306+0000 7f9d75b07640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d5c0a32c0 0x7f9d5c0a5780 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9d68004af0 tx=0x7f9d680dbf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.306+0000 7f9d5b7fe640 1 -- 172.21.15.78:0/2742241055 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c028070 con 0x563db9829cf0 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.306+0000 7f9d5b7fe640 1 -- 172.21.15.78:0/2742241055 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d6c00fca0 con 0x563db9829cf0 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.306+0000 7f9d5b7fe640 1 -- 172.21.15.78:0/2742241055 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c016030 con 0x563db9829cf0 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.306+0000 7f9d79943740 1 -- 172.21.15.78:0/2742241055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 msgr2=0x563db9828240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.306+0000 7f9d79943740 1 --2- 172.21.15.78:0/2742241055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db9828240 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c005a70 tx=0x7f9d6c005f90 comp rx=0 tx=0).stop 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 -- 172.21.15.78:0/2742241055 shutdown_connections 2024-09-16T00:26:31.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 --2- 172.21.15.78:0/2742241055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9833520 0x563db9835910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 --2- 172.21.15.78:0/2742241055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 0x563db9828bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 --2- 172.21.15.78:0/2742241055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db9828240 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 -- 172.21.15.78:0/2742241055 >> 172.21.15.78:0/2742241055 conn(0x563db97f1310 msgr2=0x563db9824e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 -- 172.21.15.78:0/2742241055 shutdown_connections 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 -- 172.21.15.78:0/2742241055 wait complete. 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 -- start start 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.307+0000 7f9d79943740 1 Processor -- start 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d79943740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 0x563db9832f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d79943740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9830220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d79943740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9833520 0x563db98309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d79943740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563db9836220 con 0x563db9828780 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d79943740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563db9749060 con 0x563db9833520 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d79943740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563db98363a0 con 0x563db9829cf0 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d79943740 10 client.0 _ll_register_callbacks cb 0x563db949c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d74b05640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 0x563db9832f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d74b05640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 0x563db9832f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40354/0 (socket says 172.21.15.78:40354) 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d74b05640 1 -- 172.21.15.78:0/2885508527 learned_addr learned my addr 172.21.15.78:0/2885508527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d75306640 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9830220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d74b05640 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9833520 0x563db98309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d75306640 1 -- 172.21.15.78:0/2885508527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9833520 msgr2=0x563db98309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d75306640 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9833520 0x563db98309b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d75306640 1 -- 172.21.15.78:0/2885508527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 msgr2=0x563db9832f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d75306640 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 0x563db9832f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d75306640 1 -- 172.21.15.78:0/2885508527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563db94ff3c0 con 0x563db9829cf0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d74b05640 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9833520 0x563db98309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d75306640 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9830220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x563db9825c70 tx=0x7f9d60011e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=314ec4958f5e0997 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d58ff9640 1 -- 172.21.15.78:0/2885508527 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6001e060 con 0x563db9829cf0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.308+0000 7f9d79943740 1 -- 172.21.15.78:0/2885508527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d6c00fca0 con 0x563db9829cf0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d58ff9640 1 -- 172.21.15.78:0/2885508527 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9d6000fba0 con 0x563db9829cf0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d58ff9640 1 -- 172.21.15.78:0/2885508527 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9d6000fda0 con 0x563db9829cf0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d79943740 10 client.26767 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d79943740 1 -- 172.21.15.78:0/2885508527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563db9841de0 con 0x563db9829cf0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d58ff9640 1 -- 172.21.15.78:0/2885508527 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d6c00fca0 con 0x563db9829cf0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d58ff9640 1 -- 172.21.15.78:0/2885508527 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9d60018570 con 0x563db9829cf0 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d79943740 10 client.26767 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d79943740 10 client.26767 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:26:31.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d79943740 20 client.26767 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d79943740 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x563db98431f0 0x563db9845600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d79943740 4 client.26767 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d79943740 1 -- 172.21.15.78:0/2885508527 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563db979b860 con 0x563db98431f0 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.309+0000 7f9d74b05640 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x563db98431f0 0x563db9845600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.310+0000 7f9d58ff9640 10 client.26767 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.310+0000 7f9d74b05640 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x563db98431f0 0x563db9845600 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=37e5f69e55ad21f4 server_cookie=feb38a837fcb2be8 in_seq=0 out_seq=0 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.310+0000 7f9d58ff9640 1 -- 172.21.15.78:0/2885508527 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563db979b860 con 0x563db98431f0 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.310+0000 7f9d58ff9640 10 client.26767 handle_command_reply: tid=0 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.310+0000 7f9d79943740 1 client.26767 shutdown 2024-09-16T00:26:31.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 -- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 msgr2=0x563db9830220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9830220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x563db9825c70 tx=0x7f9d60011e40 comp rx=0 tx=0).stop 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 -- 172.21.15.78:0/2885508527 shutdown_connections 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x563db98431f0 0x563db9845600 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d58ff9640 0 client.26767 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9833520 0x563db98309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9830220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 --2- 172.21.15.78:0/2885508527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9828780 0x563db9832f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 -- 172.21.15.78:0/2885508527 >> 172.21.15.78:0/2885508527 conn(0x563db97f1310 msgr2=0x563db982aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 -- 172.21.15.78:0/2885508527 shutdown_connections 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 1 -- 172.21.15.78:0/2885508527 wait complete. 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.311+0000 7f9d79943740 20 client.26767 trim_cache size 0 max 16384 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.326+0000 7f9d79943740 1 Processor -- start 2024-09-16T00:26:31.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.326+0000 7f9d79943740 1 -- start start 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.326+0000 7f9d79943740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9828780 0x563db9835910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.326+0000 7f9d79943740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db982eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.326+0000 7f9d79943740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 0x563db982f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.326+0000 7f9d79943740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b4ff0 con 0x563db9833520 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.326+0000 7f9d79943740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b6b30 con 0x563db9829cf0 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.326+0000 7f9d79943740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b56c0 con 0x563db9828780 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db982eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d74b05640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9828780 0x563db9835910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db982eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37582/0 (socket says 172.21.15.78:37582) 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 -- 172.21.15.78:0/4141834953 learned_addr learned my addr 172.21.15.78:0/4141834953 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d74b05640 1 --2- 172.21.15.78:0/4141834953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 0x563db982f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 -- 172.21.15.78:0/4141834953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9828780 msgr2=0x563db9835910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:31.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 --2- 172.21.15.78:0/4141834953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9828780 0x563db9835910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 -- 172.21.15.78:0/4141834953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 msgr2=0x563db982f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 --2- 172.21.15.78:0/4141834953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 0x563db982f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 -- 172.21.15.78:0/4141834953 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563db9830d20 con 0x563db9829cf0 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.327+0000 7f9d75306640 1 --2- 172.21.15.78:0/4141834953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db982eaf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d600058c0 tx=0x7f9d6000fd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75449d8a0274e86c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d5affd640 1 -- 172.21.15.78:0/4141834953 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6002e070 con 0x563db9829cf0 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d5affd640 1 -- 172.21.15.78:0/4141834953 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d60018e00 con 0x563db9829cf0 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d5affd640 1 -- 172.21.15.78:0/4141834953 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60003850 con 0x563db9829cf0 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d79943740 1 -- 172.21.15.78:0/4141834953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 msgr2=0x563db982eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d79943740 1 --2- 172.21.15.78:0/4141834953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db982eaf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9d600058c0 tx=0x7f9d6000fd30 comp rx=0 tx=0).stop 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d79943740 1 -- 172.21.15.78:0/4141834953 shutdown_connections 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d79943740 1 --2- 172.21.15.78:0/4141834953 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 0x563db982f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d79943740 1 --2- 172.21.15.78:0/4141834953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9829cf0 0x563db982eaf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d79943740 1 --2- 172.21.15.78:0/4141834953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9828780 0x563db9835910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d79943740 1 -- 172.21.15.78:0/4141834953 >> 172.21.15.78:0/4141834953 conn(0x563db97f1310 msgr2=0x563db981fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.328+0000 7f9d79943740 1 -- 172.21.15.78:0/4141834953 shutdown_connections 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 -- 172.21.15.78:0/4141834953 wait complete. 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 -- start start 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 Processor -- start 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9828780 0x563db983bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9836eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 0x563db98373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b6460 con 0x563db9833520 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b5d90 con 0x563db9828780 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.329+0000 7f9d79943740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9d701b2fd0 con 0x563db9829cf0 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d79943740 10 client.0 _ll_register_callbacks cb 0x7f9d60009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9836eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d74b05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9828780 0x563db983bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9836eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:46376/0 (socket says 172.21.15.78:46376) 2024-09-16T00:26:31.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d74b05640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9828780 0x563db983bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:37588/0 (socket says 172.21.15.78:37588) 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 -- 172.21.15.78:0/3878646394 learned_addr learned my addr 172.21.15.78:0/3878646394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d74b05640 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 0x563db98373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 -- 172.21.15.78:0/3878646394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9828780 msgr2=0x563db983bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9828780 0x563db983bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 -- 172.21.15.78:0/3878646394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 msgr2=0x563db98373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 0x563db98373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 -- 172.21.15.78:0/3878646394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563db98379b0 con 0x563db9829cf0 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d75306640 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9836eb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d60018b00 tx=0x7f9d60005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e8b1a5de2e12be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d58ff9640 1 -- 172.21.15.78:0/3878646394 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6001e050 con 0x563db9829cf0 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d79943740 1 -- 172.21.15.78:0/3878646394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d60018e00 con 0x563db9829cf0 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.330+0000 7f9d58ff9640 1 -- 172.21.15.78:0/3878646394 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f9d60005310 con 0x563db9829cf0 2024-09-16T00:26:31.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d58ff9640 1 -- 172.21.15.78:0/3878646394 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9d6002e040 con 0x563db9829cf0 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d79943740 10 client.26773 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d79943740 1 -- 172.21.15.78:0/3878646394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563db98adc90 con 0x563db9829cf0 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d58ff9640 1 -- 172.21.15.78:0/3878646394 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d60018e00 con 0x563db9829cf0 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d58ff9640 1 -- 172.21.15.78:0/3878646394 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9d60031020 con 0x563db9829cf0 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d79943740 10 client.26773 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d79943740 10 client.26773 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d79943740 20 client.26773 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d79943740 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x563db98ae2c0 0x563db98b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d79943740 4 client.26773 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d79943740 1 -- 172.21.15.78:0/3878646394 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563db9795d70 con 0x563db98ae2c0 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.331+0000 7f9d74b05640 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x563db98ae2c0 0x563db98b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d58ff9640 10 client.26773 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d74b05640 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x563db98ae2c0 0x563db98b06e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce7eab9e82731b9e server_cookie=9f1f980929f13e03 in_seq=0 out_seq=0 2024-09-16T00:26:31.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d58ff9640 1 -- 172.21.15.78:0/3878646394 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563db9795d70 con 0x563db98ae2c0 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d58ff9640 10 client.26773 handle_command_reply: tid=0 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 client.26773 shutdown 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 -- 172.21.15.78:0/3878646394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 msgr2=0x563db9836eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9836eb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9d60018b00 tx=0x7f9d60005fb0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 -- 172.21.15.78:0/3878646394 shutdown_connections 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563db9833520 0x563db98373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563db9829cf0 0x563db9836eb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x563db98ae2c0 0x563db98b06e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 --2- 172.21.15.78:0/3878646394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563db9828780 0x563db983bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 -- 172.21.15.78:0/3878646394 >> 172.21.15.78:0/3878646394 conn(0x563db97f1310 msgr2=0x563db982eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d58ff9640 0 client.26773 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 -- 172.21.15.78:0/3878646394 shutdown_connections 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 1 -- 172.21.15.78:0/3878646394 wait complete. 2024-09-16T00:26:31.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.332+0000 7f9d79943740 20 client.26773 trim_cache size 0 max 16384 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 -- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d5c0a32c0 msgr2=0x7f9d5c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d5c0a32c0 0x7f9d5c0a5780 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9d68004af0 tx=0x7f9d680dbf90 comp rx=0 tx=0).stop 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 -- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 msgr2=0x7f9d701a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 0x7f9d701a2a30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9d68033040 tx=0x7f9d68003bc0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 -- 172.21.15.78:0/1662470746 shutdown_connections 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9d5c0a32c0 0x7f9d5c0a5780 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d7010e3f0 0x7f9d701a2a30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d7006f100 0x7f9d701a8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 --2- 172.21.15.78:0/1662470746 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9d7006e750 0x7f9d701a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:31.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 -- 172.21.15.78:0/1662470746 >> 172.21.15.78:0/1662470746 conn(0x7f9d70073180 msgr2=0x7f9d7006c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 -- 172.21.15.78:0/1662470746 shutdown_connections 2024-09-16T00:26:31.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:30.333+0000 7f9d76308640 1 -- 172.21.15.78:0/1662470746 wait complete. 2024-09-16T00:26:32.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:31 smithi193 ceph-mon[35484]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.1 MiB/s wr, 708 op/s 2024-09-16T00:26:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:31 smithi148 ceph-mon[35599]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.1 MiB/s wr, 708 op/s 2024-09-16T00:26:32.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:31 smithi078 ceph-mon[31157]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.1 MiB/s wr, 708 op/s 2024-09-16T00:26:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:33 smithi193 ceph-mon[35484]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 665 op/s 2024-09-16T00:26:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:33 smithi148 ceph-mon[35599]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 665 op/s 2024-09-16T00:26:34.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:33 smithi078 ceph-mon[31157]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 665 op/s 2024-09-16T00:26:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 755.0 s, 480.4 tps, lat 210.721 ms stddev 268.673 2024-09-16T00:26:36.181 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:35 smithi193 ceph-mon[35484]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 747 op/s 2024-09-16T00:26:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:35 smithi148 ceph-mon[35599]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 747 op/s 2024-09-16T00:26:36.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:35 smithi078 ceph-mon[31157]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 747 op/s 2024-09-16T00:26:36.437 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:26:36.869 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.542+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/275150602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b4102ef0 msgr2=0x7ff1b410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.542+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/275150602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b4102ef0 0x7ff1b410da40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009cd0 tx=0x7ff1ac02a9a0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.543+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/275150602 shutdown_connections 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.543+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/275150602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b4102ef0 0x7ff1b410da40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.543+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/275150602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1b4102570 0x7ff1b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.543+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/275150602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1b4100e20 0x7ff1b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.543+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/275150602 >> 172.21.15.78:0/275150602 conn(0x7ff1b40fc7d0 msgr2=0x7ff1b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.543+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/275150602 shutdown_connections 2024-09-16T00:26:37.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.543+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/275150602 wait complete. 2024-09-16T00:26:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.544+0000 7ff1ba3ef640 1 Processor -- start 2024-09-16T00:26:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.544+0000 7ff1ba3ef640 1 -- start start 2024-09-16T00:26:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.544+0000 7ff1ba3ef640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1b4100e20 0x7ff1b4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.544+0000 7ff1ba3ef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1b4102570 0x7ff1b41974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.544+0000 7ff1ba3ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b4102ef0 0x7ff1b4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.544+0000 7ff1ba3ef640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4111010 con 0x7ff1b4100e20 2024-09-16T00:26:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.544+0000 7ff1ba3ef640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4110e90 con 0x7ff1b4102ef0 2024-09-16T00:26:37.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.544+0000 7ff1ba3ef640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4111190 con 0x7ff1b4102570 2024-09-16T00:26:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1b4100e20 0x7ff1b4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1b4100e20 0x7ff1b4196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59838/0 (socket says 172.21.15.78:59838) 2024-09-16T00:26:37.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 -- 172.21.15.78:0/1595012171 learned_addr learned my addr 172.21.15.78:0/1595012171 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b898c640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b4102ef0 0x7ff1b4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 -- 172.21.15.78:0/1595012171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1b4102570 msgr2=0x7ff1b41974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1b4102570 0x7ff1b41974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 -- 172.21.15.78:0/1595012171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b4102ef0 msgr2=0x7ff1b4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b4102ef0 0x7ff1b4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 -- 172.21.15.78:0/1595012171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b41a6800 con 0x7ff1b4100e20 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.545+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1b4100e20 0x7ff1b4196f90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400d950 tx=0x7ff1a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.546+0000 7ff1b17fa640 1 -- 172.21.15.78:0/1595012171 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4017070 con 0x7ff1b4100e20 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.546+0000 7ff1b17fa640 1 -- 172.21.15.78:0/1595012171 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a4009c30 con 0x7ff1b4100e20 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.546+0000 7ff1b17fa640 1 -- 172.21.15.78:0/1595012171 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400f6b0 con 0x7ff1b4100e20 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.546+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/1595012171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b41a6af0 con 0x7ff1b4100e20 2024-09-16T00:26:37.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.546+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/1595012171 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41a6fa0 con 0x7ff1b4100e20 2024-09-16T00:26:37.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.549+0000 7ff1b17fa640 1 -- 172.21.15.78:0/1595012171 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff1a40109d0 con 0x7ff1b4100e20 2024-09-16T00:26:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.551+0000 7ff1b17fa640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff19c0a3410 0x7ff19c0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.551+0000 7ff1b17fa640 1 -- 172.21.15.78:0/1595012171 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff1a40cf830 con 0x7ff1b4100e20 2024-09-16T00:26:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.552+0000 7ff1b37fe640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff19c0a3410 0x7ff19c0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.553+0000 7ff1b37fe640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff19c0a3410 0x7ff19c0a58d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff1b41990f0 tx=0x7ff1a8006c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.553+0000 7ff1bc7c8740 1 Processor -- start 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.553+0000 7ff1bc7c8740 1 -- start start 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b6c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b6c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b6ddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56465b62e2f0 con 0x56465b6c85d0 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56465b5d5cf0 con 0x56465b6c9b40 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56465b62a2b0 con 0x56465b6dbb80 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b6c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b6c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39832/0 (socket says 172.21.15.78:39832) 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1b3fff640 1 -- 172.21.15.78:0/1052093096 learned_addr learned my addr 172.21.15.78:0/1052093096 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1b3fff640 1 -- 172.21.15.78:0/1052093096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 msgr2=0x56465b6ddf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/1052093096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b6ddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1b3fff640 1 -- 172.21.15.78:0/1052093096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 msgr2=0x56465b6c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/1052093096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b6c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.554+0000 7ff1b3fff640 1 -- 172.21.15.78:0/1052093096 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465b5ba3a0 con 0x56465b6c9b40 2024-09-16T00:26:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.555+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/1052093096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b6c8090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff1a409b790 tx=0x7ff1a4010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56d5a0856ae616c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.555+0000 7ff19a7fc640 1 -- 172.21.15.78:0/1052093096 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4020070 con 0x56465b6c9b40 2024-09-16T00:26:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.555+0000 7ff19a7fc640 1 -- 172.21.15.78:0/1052093096 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a409fe40 con 0x56465b6c9b40 2024-09-16T00:26:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.555+0000 7ff19a7fc640 1 -- 172.21.15.78:0/1052093096 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a409b1d0 con 0x56465b6c9b40 2024-09-16T00:26:37.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.557+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/1052093096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 msgr2=0x56465b6c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.557+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/1052093096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b6c8090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff1a409b790 tx=0x7ff1a4010d70 comp rx=0 tx=0).stop 2024-09-16T00:26:37.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.557+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/1052093096 shutdown_connections 2024-09-16T00:26:37.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.557+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/1052093096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b6ddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.557+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/1052093096 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b6c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.557+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/1052093096 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b6c8090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.557+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/1052093096 >> 172.21.15.78:0/1052093096 conn(0x56465b691310 msgr2=0x56465b6c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.557+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/1052093096 shutdown_connections 2024-09-16T00:26:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.558+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/1052093096 wait complete. 2024-09-16T00:26:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.558+0000 7ff1bc7c8740 1 -- start start 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.558+0000 7ff1bc7c8740 1 Processor -- start 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b6d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6c9b40 0x56465b6d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6dbb80 0x56465b6db320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56465b6cb6e0 con 0x56465b6c85d0 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56465b5e9060 con 0x56465b6dbb80 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56465b6cb860 con 0x56465b6c9b40 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1b898c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6dbb80 0x56465b6db320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1b898c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6dbb80 0x56465b6db320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39840/0 (socket says 172.21.15.78:39840) 2024-09-16T00:26:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1b898c640 1 -- 172.21.15.78:0/2585088243 learned_addr learned my addr 172.21.15.78:0/2585088243 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1b37fe640 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6c9b40 0x56465b6d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1bc7c8740 10 client.0 _ll_register_callbacks cb 0x56465b33c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1b898c640 1 -- 172.21.15.78:0/2585088243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6c9b40 msgr2=0x56465b6d5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1b898c640 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6c9b40 0x56465b6d5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1b898c640 1 -- 172.21.15.78:0/2585088243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 msgr2=0x56465b6d5610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.559+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b6d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.560+0000 7ff1b898c640 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b6d5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.560+0000 7ff1b898c640 1 -- 172.21.15.78:0/2585088243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56465b39f3c0 con 0x56465b6dbb80 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.560+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b6d5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.560+0000 7ff1b898c640 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6dbb80 0x56465b6db320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009ca0 tx=0x7ff1ac003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c5c997cf56c9dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.560+0000 7ff1937fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:26:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.560+0000 7ff1937fe640 1 -- 172.21.15.78:0/2585088243 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac049060 con 0x56465b6dbb80 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.560+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/2585088243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a409fe40 con 0x56465b6dbb80 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.560+0000 7ff1937fe640 1 -- 172.21.15.78:0/2585088243 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff1ac02f9c0 con 0x56465b6dbb80 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1937fe640 1 -- 172.21.15.78:0/2585088243 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff1ac050050 con 0x56465b6dbb80 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1bc7c8740 10 client.26975 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/2585088243 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56465b6f2c10 con 0x56465b6dbb80 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1937fe640 1 -- 172.21.15.78:0/2585088243 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a409fe40 con 0x56465b6dbb80 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1937fe640 1 -- 172.21.15.78:0/2585088243 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff1ac030ce0 con 0x56465b6dbb80 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1bc7c8740 10 client.26975 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1bc7c8740 10 client.26975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1bc7c8740 20 client.26975 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x56465b6f40d0 0x56465b6f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1bc7c8740 4 client.26975 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/2585088243 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56465b63b860 con 0x56465b6f40d0 2024-09-16T00:26:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.561+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x56465b6f40d0 0x56465b6f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.562+0000 7ff1937fe640 10 client.26975 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:26:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.562+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x56465b6f40d0 0x56465b6f64f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5696fe9e4c1deea server_cookie=49c7a2e2fc1090e5 in_seq=0 out_seq=0 2024-09-16T00:26:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.562+0000 7ff1937fe640 1 -- 172.21.15.78:0/2585088243 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56465b63b860 con 0x56465b6f40d0 2024-09-16T00:26:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.562+0000 7ff1937fe640 10 client.26975 handle_command_reply: tid=0 2024-09-16T00:26:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.562+0000 7ff1bc7c8740 1 client.26975 shutdown 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/2585088243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6dbb80 msgr2=0x56465b6db320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6dbb80 0x56465b6db320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009ca0 tx=0x7ff1ac003ba0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/2585088243 shutdown_connections 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x56465b6f40d0 0x56465b6f64f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6dbb80 0x56465b6db320 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6c9b40 0x56465b6d5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/2585088243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b6d5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1937fe640 0 client.26975 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/2585088243 >> 172.21.15.78:0/2585088243 conn(0x56465b691310 msgr2=0x56465b6ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:37.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/2585088243 shutdown_connections 2024-09-16T00:26:37.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/2585088243 wait complete. 2024-09-16T00:26:37.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.563+0000 7ff1bc7c8740 20 client.26975 trim_cache size 0 max 16384 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.579+0000 7ff1bc7c8740 1 Processor -- start 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.579+0000 7ff1bc7c8740 1 -- start start 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.579+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c85d0 0x56465b6c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.579+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c9b40 0x56465b6db290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.579+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b6fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.579+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41acab0 con 0x56465b6c9b40 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.579+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ae5f0 con 0x56465b6c85d0 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.579+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ad180 con 0x56465b6dbb80 2024-09-16T00:26:37.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c85d0 0x56465b6c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b898c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b6fbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c85d0 0x56465b6c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39852/0 (socket says 172.21.15.78:39852) 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 -- 172.21.15.78:0/378242390 learned_addr learned my addr 172.21.15.78:0/378242390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 -- 172.21.15.78:0/378242390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 msgr2=0x56465b6fbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/378242390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b6fbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 -- 172.21.15.78:0/378242390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c9b40 msgr2=0x56465b6db290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/378242390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c9b40 0x56465b6db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 -- 172.21.15.78:0/378242390 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465b6cf8e0 con 0x56465b6c85d0 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/378242390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c85d0 0x56465b6c4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff1a409ba20 tx=0x7ff1a40a4800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4299e021fe2d7abe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff199ffb640 1 -- 172.21.15.78:0/378242390 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40b9070 con 0x56465b6c85d0 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff199ffb640 1 -- 172.21.15.78:0/378242390 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a40cd080 con 0x56465b6c85d0 2024-09-16T00:26:37.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.580+0000 7ff199ffb640 1 -- 172.21.15.78:0/378242390 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4020040 con 0x56465b6c85d0 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.581+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/378242390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c85d0 msgr2=0x56465b6c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.581+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/378242390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c85d0 0x56465b6c4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff1a409ba20 tx=0x7ff1a40a4800 comp rx=0 tx=0).stop 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.581+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/378242390 shutdown_connections 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.582+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/378242390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b6fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.582+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/378242390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c9b40 0x56465b6db290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.582+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/378242390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c85d0 0x56465b6c4e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.582+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/378242390 >> 172.21.15.78:0/378242390 conn(0x56465b691310 msgr2=0x56465b6c4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.582+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/378242390 shutdown_connections 2024-09-16T00:26:37.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.582+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/378242390 wait complete. 2024-09-16T00:26:37.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.582+0000 7ff1bc7c8740 1 -- start start 2024-09-16T00:26:37.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.583+0000 7ff1bc7c8740 1 Processor -- start 2024-09-16T00:26:37.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.583+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b7082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.583+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b703610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.583+0000 7ff1bc7c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b703b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.583+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41adf20 con 0x56465b6c85d0 2024-09-16T00:26:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.583+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41ad850 con 0x56465b6c9b40 2024-09-16T00:26:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.583+0000 7ff1bc7c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b41aaad0 con 0x56465b6dbb80 2024-09-16T00:26:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.583+0000 7ff1bc7c8740 10 client.0 _ll_register_callbacks cb 0x7ff1ac005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b703610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b703610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39858/0 (socket says 172.21.15.78:39858) 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 -- 172.21.15.78:0/821957397 learned_addr learned my addr 172.21.15.78:0/821957397 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b7082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b898c640 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b703b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 -- 172.21.15.78:0/821957397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 msgr2=0x56465b703b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b703b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 -- 172.21.15.78:0/821957397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 msgr2=0x56465b7082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b7082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 -- 172.21.15.78:0/821957397 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56465b7043c0 con 0x56465b6c9b40 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1b37fe640 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b703610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8006fd0 tx=0x7ff1a800b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9c9915479b5440 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1937fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.584+0000 7ff1937fe640 1 -- 172.21.15.78:0/821957397 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8024060 con 0x56465b6c9b40 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1937fe640 1 -- 172.21.15.78:0/821957397 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff1a80105b0 con 0x56465b6c9b40 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/821957397 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a40cd080 con 0x56465b6c9b40 2024-09-16T00:26:37.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1937fe640 1 -- 172.21.15.78:0/821957397 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff1a802b050 con 0x56465b6c9b40 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 10 client.26981 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/821957397 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56465b75ed10 con 0x56465b6c9b40 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1937fe640 1 -- 172.21.15.78:0/821957397 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1a40cd080 con 0x56465b6c9b40 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1937fe640 1 -- 172.21.15.78:0/821957397 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff1a801e070 con 0x56465b6c9b40 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 10 client.26981 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 10 client.26981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 20 client.26981 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x56465b75f4f0 0x56465b761910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 4 client.26981 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/821957397 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56465b635d70 con 0x56465b75f4f0 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.585+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x56465b75f4f0 0x56465b761910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.586+0000 7ff1937fe640 10 client.26981 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.586+0000 7ff1b3fff640 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x56465b75f4f0 0x56465b761910 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f836ce5fcb863e server_cookie=95f62201cc8f3ff2 in_seq=0 out_seq=0 2024-09-16T00:26:37.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.586+0000 7ff1937fe640 1 -- 172.21.15.78:0/821957397 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56465b635d70 con 0x56465b75f4f0 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.586+0000 7ff1937fe640 10 client.26981 handle_command_reply: tid=0 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.586+0000 7ff1bc7c8740 1 client.26981 shutdown 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.586+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/821957397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 msgr2=0x56465b703610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.586+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b703610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8006fd0 tx=0x7ff1a800b800 comp rx=0 tx=0).stop 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/821957397 shutdown_connections 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56465b6dbb80 0x56465b703b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x56465b75f4f0 0x56465b761910 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56465b6c9b40 0x56465b703610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 1 --2- 172.21.15.78:0/821957397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56465b6c85d0 0x56465b7082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1937fe640 0 client.26981 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/821957397 >> 172.21.15.78:0/821957397 conn(0x56465b691310 msgr2=0x56465b6ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/821957397 shutdown_connections 2024-09-16T00:26:37.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 1 -- 172.21.15.78:0/821957397 wait complete. 2024-09-16T00:26:37.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.587+0000 7ff1bc7c8740 20 client.26981 trim_cache size 0 max 16384 2024-09-16T00:26:37.593 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:26:37.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff19c0a3410 msgr2=0x7ff19c0a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff19c0a3410 0x7ff19c0a58d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff1b41990f0 tx=0x7ff1a8006c40 comp rx=0 tx=0).stop 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1b4100e20 msgr2=0x7ff1b4196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1b4100e20 0x7ff1b4196f90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400d950 tx=0x7ff1a400de10 comp rx=0 tx=0).stop 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/1595012171 shutdown_connections 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff19c0a3410 0x7ff19c0a58d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff1b4102ef0 0x7ff1b4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1b4102570 0x7ff1b41974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 --2- 172.21.15.78:0/1595012171 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1b4100e20 0x7ff1b4196f90 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/1595012171 >> 172.21.15.78:0/1595012171 conn(0x7ff1b40fc7d0 msgr2=0x7ff1b41067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/1595012171 shutdown_connections 2024-09-16T00:26:37.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:37.588+0000 7ff1ba3ef640 1 -- 172.21.15.78:0/1595012171 wait complete. 2024-09-16T00:26:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:37 smithi193 ceph-mon[35484]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 665 op/s 2024-09-16T00:26:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:37 smithi148 ceph-mon[35599]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 665 op/s 2024-09-16T00:26:38.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:37 smithi078 ceph-mon[31157]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.8 MiB/s wr, 665 op/s 2024-09-16T00:26:38.347 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:26:38.482 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.4 2024-09-16T00:26:38.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:26:38.843 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:39.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.680+0000 7fd25aa15640 1 -- 172.21.15.78:0/4155607383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109830 msgr2=0x7fd254109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.680+0000 7fd25aa15640 1 --2- 172.21.15.78:0/4155607383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109830 0x7fd254109c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd240009900 tx=0x7fd24002a440 comp rx=0 tx=0).stop 2024-09-16T00:26:39.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.681+0000 7fd25aa15640 1 -- 172.21.15.78:0/4155607383 shutdown_connections 2024-09-16T00:26:39.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.681+0000 7fd25aa15640 1 --2- 172.21.15.78:0/4155607383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd25411b140 0x7fd25411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.681+0000 7fd25aa15640 1 --2- 172.21.15.78:0/4155607383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109830 0x7fd254109c90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.681+0000 7fd25aa15640 1 --2- 172.21.15.78:0/4155607383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd254109450 0x7fd254110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.681+0000 7fd25aa15640 1 -- 172.21.15.78:0/4155607383 >> 172.21.15.78:0/4155607383 conn(0x7fd25406e140 msgr2=0x7fd25406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.682+0000 7fd25aa15640 1 -- 172.21.15.78:0/4155607383 shutdown_connections 2024-09-16T00:26:39.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.682+0000 7fd25aa15640 1 -- 172.21.15.78:0/4155607383 wait complete. 2024-09-16T00:26:39.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.683+0000 7fd25aa15640 1 Processor -- start 2024-09-16T00:26:39.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.683+0000 7fd25aa15640 1 -- start start 2024-09-16T00:26:39.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd25aa15640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109450 0x7fd25411a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd25aa15640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd254109830 0x7fd2541148b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd25aa15640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd25411b140 0x7fd254114df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd253fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109450 0x7fd25411a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd253fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109450 0x7fd25411a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39880/0 (socket says 172.21.15.78:39880) 2024-09-16T00:26:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd253fff640 1 -- 172.21.15.78:0/2702686759 learned_addr learned my addr 172.21.15.78:0/2702686759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:39.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd254111330 con 0x7fd25411b140 2024-09-16T00:26:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541111b0 con 0x7fd254109450 2024-09-16T00:26:39.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.684+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541114b0 con 0x7fd254109830 2024-09-16T00:26:39.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.686+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd254109830 0x7fd2541148b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.688+0000 7fd258fb2640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd25411b140 0x7fd254114df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.692+0000 7fd2537fe640 1 -- 172.21.15.78:0/2702686759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109450 msgr2=0x7fd25411a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.692+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109450 0x7fd25411a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.692+0000 7fd2537fe640 1 -- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd25411b140 msgr2=0x7fd254114df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.692+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd25411b140 0x7fd254114df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.692+0000 7fd2537fe640 1 -- 172.21.15.78:0/2702686759 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd254115520 con 0x7fd254109830 2024-09-16T00:26:39.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.701+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd254109830 0x7fd2541148b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd240009850 tx=0x7fd240005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.705+0000 7fd2517fa640 1 -- 172.21.15.78:0/2702686759 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240040070 con 0x7fd254109830 2024-09-16T00:26:39.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.705+0000 7fd2517fa640 1 -- 172.21.15.78:0/2702686759 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd240004a30 con 0x7fd254109830 2024-09-16T00:26:39.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.705+0000 7fd2517fa640 1 -- 172.21.15.78:0/2702686759 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240040070 con 0x7fd254109830 2024-09-16T00:26:39.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.708+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2541bc130 con 0x7fd254109830 2024-09-16T00:26:39.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.708+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2541bc4d0 con 0x7fd254109830 2024-09-16T00:26:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.712+0000 7fd25cdee740 1 Processor -- start 2024-09-16T00:26:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.713+0000 7fd25cdee740 1 -- start start 2024-09-16T00:26:39.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.713+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc610bb40 0x55ecc610a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.713+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc610aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.713+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc6115370 0x55ecc6117760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.713+0000 7fd25cdee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ecc60702f0 con 0x55ecc610bb40 2024-09-16T00:26:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.713+0000 7fd25cdee740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6017cf0 con 0x55ecc610a5d0 2024-09-16T00:26:39.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.713+0000 7fd25cdee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ecc606c2b0 con 0x55ecc6115370 2024-09-16T00:26:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.715+0000 7fd253fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc610aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd258fb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc610bb40 0x55ecc610a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd253fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc610aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39884/0 (socket says 172.21.15.78:39884) 2024-09-16T00:26:39.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd253fff640 1 -- 172.21.15.78:0/2247792772 learned_addr learned my addr 172.21.15.78:0/2247792772 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd258fb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc610bb40 0x55ecc610a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59900/0 (socket says 172.21.15.78:59900) 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd253fff640 1 -- 172.21.15.78:0/2247792772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc6115370 msgr2=0x55ecc6117760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd253fff640 1 --2- 172.21.15.78:0/2247792772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc6115370 0x55ecc6117760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd253fff640 1 -- 172.21.15.78:0/2247792772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc610bb40 msgr2=0x55ecc610a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd253fff640 1 --2- 172.21.15.78:0/2247792772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc610bb40 0x55ecc610a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd253fff640 1 -- 172.21.15.78:0/2247792772 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc5ffc3a0 con 0x55ecc610a5d0 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.716+0000 7fd253fff640 1 --2- 172.21.15.78:0/2247792772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc610aa10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd244009dc0 tx=0x7fd244011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac4b1c063d0fe9a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.717+0000 7fd24a7fc640 1 -- 172.21.15.78:0/2247792772 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244005220 con 0x55ecc610a5d0 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.717+0000 7fd24a7fc640 1 -- 172.21.15.78:0/2247792772 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd244004c80 con 0x55ecc610a5d0 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.718+0000 7fd24a7fc640 1 -- 172.21.15.78:0/2247792772 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244019940 con 0x55ecc610a5d0 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.718+0000 7fd25cdee740 1 -- 172.21.15.78:0/2247792772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 msgr2=0x55ecc610aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.718+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2247792772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc610aa10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd244009dc0 tx=0x7fd244011810 comp rx=0 tx=0).stop 2024-09-16T00:26:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.720+0000 7fd25cdee740 1 -- 172.21.15.78:0/2247792772 shutdown_connections 2024-09-16T00:26:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.720+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2247792772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc6115370 0x55ecc6117760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.720+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2247792772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc610aa10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.720+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2247792772 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc610bb40 0x55ecc610a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.720+0000 7fd25cdee740 1 -- 172.21.15.78:0/2247792772 >> 172.21.15.78:0/2247792772 conn(0x55ecc60d3310 msgr2=0x55ecc6106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.721+0000 7fd2517fa640 1 -- 172.21.15.78:0/2702686759 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd24003a040 con 0x7fd254109830 2024-09-16T00:26:39.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.722+0000 7fd2517fa640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2380a3680 0x7fd2380a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.722+0000 7fd2517fa640 1 -- 172.21.15.78:0/2702686759 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd240003910 con 0x7fd254109830 2024-09-16T00:26:39.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.725+0000 7fd253fff640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2380a3680 0x7fd2380a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.725+0000 7fd25cdee740 1 -- 172.21.15.78:0/2247792772 shutdown_connections 2024-09-16T00:26:39.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.726+0000 7fd25cdee740 1 -- 172.21.15.78:0/2247792772 wait complete. 2024-09-16T00:26:39.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.726+0000 7fd25cdee740 1 -- start start 2024-09-16T00:26:39.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.727+0000 7fd25cdee740 1 Processor -- start 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.727+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610a5d0 0x55ecc6114cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.727+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610bb40 0x55ecc6111f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.727+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc61128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.727+0000 7fd25cdee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6118110 con 0x55ecc61124a0 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.727+0000 7fd25cdee740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ecc602b060 con 0x55ecc610bb40 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.727+0000 7fd25cdee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6118290 con 0x55ecc610a5d0 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.727+0000 7fd25cdee740 10 client.0 _ll_register_callbacks cb 0x55ecc5d7e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd253fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610a5d0 0x55ecc6114cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd253fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610a5d0 0x55ecc6114cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:50046/0 (socket says 172.21.15.78:50046) 2024-09-16T00:26:39.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd253fff640 1 -- 172.21.15.78:0/3616601478 learned_addr learned my addr 172.21.15.78:0/3616601478 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:39.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd253fff640 1 -- 172.21.15.78:0/3616601478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610bb40 msgr2=0x55ecc6111f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:26:39.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd258fb2640 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc61128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd253fff640 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610bb40 0x55ecc6111f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd253fff640 1 -- 172.21.15.78:0/3616601478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 msgr2=0x55ecc61128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd253fff640 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc61128e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd253fff640 1 -- 172.21.15.78:0/3616601478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc5de13c0 con 0x55ecc610a5d0 2024-09-16T00:26:39.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.728+0000 7fd258fb2640 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc61128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.729+0000 7fd253fff640 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610a5d0 0x55ecc6114cb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55ecc6107ac0 tx=0x7fd244011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=768413f158641a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.729+0000 7fd2377fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.729+0000 7fd2377fe640 1 -- 172.21.15.78:0/3616601478 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244036060 con 0x55ecc610a5d0 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.729+0000 7fd2377fe640 1 -- 172.21.15.78:0/3616601478 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd244004030 con 0x55ecc610a5d0 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.729+0000 7fd25cdee740 1 -- 172.21.15.78:0/3616601478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd244004c80 con 0x55ecc610a5d0 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.729+0000 7fd253fff640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2380a3680 0x7fd2380a5b40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd244011d10 tx=0x7fd2440054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.730+0000 7fd2377fe640 1 -- 172.21.15.78:0/3616601478 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd2440099d0 con 0x55ecc610a5d0 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.730+0000 7fd25cdee740 10 client.26791 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:39.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.730+0000 7fd25cdee740 1 -- 172.21.15.78:0/3616601478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ecc6123f60 con 0x55ecc610a5d0 2024-09-16T00:26:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.733+0000 7fd2377fe640 1 -- 172.21.15.78:0/3616601478 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd244004c80 con 0x55ecc610a5d0 2024-09-16T00:26:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.734+0000 7fd2377fe640 1 -- 172.21.15.78:0/3616601478 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd244026480 con 0x55ecc610a5d0 2024-09-16T00:26:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.734+0000 7fd25cdee740 10 client.26791 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.734+0000 7fd25cdee740 10 client.26791 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:26:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.734+0000 7fd25cdee740 20 client.26791 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.734+0000 7fd25cdee740 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55ecc61255f0 0x55ecc6127a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.734+0000 7fd25cdee740 4 client.26791 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.734+0000 7fd25cdee740 1 -- 172.21.15.78:0/3616601478 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecc607d860 con 0x55ecc61255f0 2024-09-16T00:26:39.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.735+0000 7fd2537fe640 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55ecc61255f0 0x55ecc6127a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.737+0000 7fd2537fe640 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55ecc61255f0 0x55ecc6127a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac3eaf465dd27bc0 server_cookie=cb6fb880f9b8b974 in_seq=0 out_seq=0 2024-09-16T00:26:39.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.737+0000 7fd2377fe640 10 client.26791 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:26:39.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.738+0000 7fd2377fe640 1 -- 172.21.15.78:0/3616601478 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecc607d860 con 0x55ecc61255f0 2024-09-16T00:26:39.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.738+0000 7fd2377fe640 10 client.26791 handle_command_reply: tid=0 2024-09-16T00:26:39.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.739+0000 7fd25cdee740 1 client.26791 shutdown 2024-09-16T00:26:39.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 -- 172.21.15.78:0/3616601478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610a5d0 msgr2=0x55ecc6114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610a5d0 0x55ecc6114cb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55ecc6107ac0 tx=0x7fd244011810 comp rx=0 tx=0).stop 2024-09-16T00:26:39.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 -- 172.21.15.78:0/3616601478 shutdown_connections 2024-09-16T00:26:39.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55ecc61255f0 0x55ecc6127a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc61128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610bb40 0x55ecc6111f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 --2- 172.21.15.78:0/3616601478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610a5d0 0x55ecc6114cb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 -- 172.21.15.78:0/3616601478 >> 172.21.15.78:0/3616601478 conn(0x55ecc60d3310 msgr2=0x55ecc610c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd2377fe640 0 client.26791 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:26:39.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 -- 172.21.15.78:0/3616601478 shutdown_connections 2024-09-16T00:26:39.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 1 -- 172.21.15.78:0/3616601478 wait complete. 2024-09-16T00:26:39.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.743+0000 7fd25cdee740 20 client.26791 trim_cache size 0 max 16384 2024-09-16T00:26:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.767+0000 7fd25cdee740 1 Processor -- start 2024-09-16T00:26:39.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.767+0000 7fd25cdee740 1 -- start start 2024-09-16T00:26:39.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6111820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc6116ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd253fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6111820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd253fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6111820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39908/0 (socket says 172.21.15.78:39908) 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:50054/0 (socket says 172.21.15.78:50054) 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd253fff640 1 -- 172.21.15.78:0/2126826566 learned_addr learned my addr 172.21.15.78:0/2126826566 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd25cdee740 1 -- 172.21.15.78:0/2126826566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541c6150 con 0x55ecc61124a0 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd25cdee740 1 -- 172.21.15.78:0/2126826566 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541c7c90 con 0x55ecc610a5d0 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd25cdee740 1 -- 172.21.15.78:0/2126826566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541c6820 con 0x55ecc610bb40 2024-09-16T00:26:39.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.768+0000 7fd258fb2640 1 --2- 172.21.15.78:0/2126826566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc6116ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.772+0000 7fd2537fe640 1 -- 172.21.15.78:0/2126826566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 msgr2=0x55ecc6111820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.772+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2126826566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6111820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.772+0000 7fd2537fe640 1 -- 172.21.15.78:0/2126826566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 msgr2=0x55ecc6116ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.772+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2126826566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc6116ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.772+0000 7fd2537fe640 1 -- 172.21.15.78:0/2126826566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc61104b0 con 0x55ecc610bb40 2024-09-16T00:26:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.772+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2126826566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd2400b6880 tx=0x7fd2400c1280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ee5dc6dce55da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.772+0000 7fd249ffb640 1 -- 172.21.15.78:0/2126826566 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400b87e0 con 0x55ecc610bb40 2024-09-16T00:26:39.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.772+0000 7fd249ffb640 1 -- 172.21.15.78:0/2126826566 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2400c4a20 con 0x55ecc610bb40 2024-09-16T00:26:39.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.773+0000 7fd25cdee740 1 -- 172.21.15.78:0/2126826566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 msgr2=0x55ecc6116760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.773+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2126826566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd2400b6880 tx=0x7fd2400c1280 comp rx=0 tx=0).stop 2024-09-16T00:26:39.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.774+0000 7fd25cdee740 1 -- 172.21.15.78:0/2126826566 shutdown_connections 2024-09-16T00:26:39.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.774+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2126826566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc6116ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.774+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2126826566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116760 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.774+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2126826566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.774+0000 7fd25cdee740 1 -- 172.21.15.78:0/2126826566 >> 172.21.15.78:0/2126826566 conn(0x55ecc60d3310 msgr2=0x55ecc6110830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.775+0000 7fd25cdee740 1 -- 172.21.15.78:0/2126826566 shutdown_connections 2024-09-16T00:26:39.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.775+0000 7fd25cdee740 1 -- 172.21.15.78:0/2126826566 wait complete. 2024-09-16T00:26:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.775+0000 7fd25cdee740 1 -- start start 2024-09-16T00:26:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd25cdee740 1 Processor -- start 2024-09-16T00:26:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6115cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd25cdee740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc6116750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd25cdee740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541c75c0 con 0x55ecc61124a0 2024-09-16T00:26:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd25cdee740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541c6ef0 con 0x55ecc610a5d0 2024-09-16T00:26:39.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd25cdee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541c41e0 con 0x55ecc610bb40 2024-09-16T00:26:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd25cdee740 10 client.0 _ll_register_callbacks cb 0x7fd244009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.776+0000 7fd253fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6115cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.777+0000 7fd253fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6115cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39914/0 (socket says 172.21.15.78:39914) 2024-09-16T00:26:39.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.777+0000 7fd253fff640 1 -- 172.21.15.78:0/2457098097 learned_addr learned my addr 172.21.15.78:0/2457098097 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.777+0000 7fd253fff640 1 -- 172.21.15.78:0/2457098097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 msgr2=0x55ecc6116210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.777+0000 7fd253fff640 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.777+0000 7fd253fff640 1 -- 172.21.15.78:0/2457098097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 msgr2=0x55ecc6116750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.777+0000 7fd253fff640 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc6116750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.777+0000 7fd253fff640 1 -- 172.21.15.78:0/2457098097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc611c5d0 con 0x55ecc610a5d0 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.778+0000 7fd253fff640 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6115cd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd2440054a0 tx=0x7fd244026730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf220b4faba4065f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.778+0000 7fd2377fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.778+0000 7fd2377fe640 1 -- 172.21.15.78:0/2457098097 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244040080 con 0x55ecc610a5d0 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.778+0000 7fd25cdee740 1 -- 172.21.15.78:0/2457098097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2400c4a20 con 0x55ecc610a5d0 2024-09-16T00:26:39.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.778+0000 7fd2377fe640 1 -- 172.21.15.78:0/2457098097 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd244026cc0 con 0x55ecc610a5d0 2024-09-16T00:26:39.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.778+0000 7fd2377fe640 1 -- 172.21.15.78:0/2457098097 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd244004270 con 0x55ecc610a5d0 2024-09-16T00:26:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.782+0000 7fd25cdee740 10 client.26996 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.782+0000 7fd25cdee740 1 -- 172.21.15.78:0/2457098097 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ecc611d8b0 con 0x55ecc610a5d0 2024-09-16T00:26:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.783+0000 7fd2377fe640 1 -- 172.21.15.78:0/2457098097 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2400c4a20 con 0x55ecc610a5d0 2024-09-16T00:26:39.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.783+0000 7fd2377fe640 1 -- 172.21.15.78:0/2457098097 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd244024cb0 con 0x55ecc610a5d0 2024-09-16T00:26:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.786+0000 7fd25cdee740 10 client.26996 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.786+0000 7fd25cdee740 10 client.26996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:26:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.786+0000 7fd25cdee740 20 client.26996 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.786+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55ecc611df20 0x55ecc6192300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.786+0000 7fd25cdee740 4 client.26996 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:26:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.786+0000 7fd25cdee740 1 -- 172.21.15.78:0/2457098097 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecc6077d70 con 0x55ecc611df20 2024-09-16T00:26:39.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.790+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55ecc611df20 0x55ecc6192300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.797+0000 7fd2377fe640 10 client.26996 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:26:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.797+0000 7fd2537fe640 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55ecc611df20 0x55ecc6192300 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5241b77167df7648 server_cookie=432f286a0d513827 in_seq=0 out_seq=0 2024-09-16T00:26:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.798+0000 7fd2377fe640 1 -- 172.21.15.78:0/2457098097 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecc6077d70 con 0x55ecc611df20 2024-09-16T00:26:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.798+0000 7fd2377fe640 10 client.26996 handle_command_reply: tid=0 2024-09-16T00:26:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.800+0000 7fd25cdee740 1 client.26996 shutdown 2024-09-16T00:26:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd25cdee740 1 -- 172.21.15.78:0/2457098097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 msgr2=0x55ecc6115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6115cd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd2440054a0 tx=0x7fd244026730 comp rx=0 tx=0).stop 2024-09-16T00:26:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd25cdee740 1 -- 172.21.15.78:0/2457098097 shutdown_connections 2024-09-16T00:26:39.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55ecc611df20 0x55ecc6192300 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ecc61124a0 0x55ecc6116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ecc610bb40 0x55ecc6116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd25cdee740 1 --2- 172.21.15.78:0/2457098097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ecc610a5d0 0x55ecc6115cd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd25cdee740 1 -- 172.21.15.78:0/2457098097 >> 172.21.15.78:0/2457098097 conn(0x55ecc60d3310 msgr2=0x55ecc61137c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.801+0000 7fd2377fe640 0 client.26996 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.805+0000 7fd25cdee740 1 -- 172.21.15.78:0/2457098097 shutdown_connections 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.805+0000 7fd25cdee740 1 -- 172.21.15.78:0/2457098097 wait complete. 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.806+0000 7fd25cdee740 20 client.26996 trim_cache size 0 max 16384 2024-09-16T00:26:39.808 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:26:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.806+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2380a3680 msgr2=0x7fd2380a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.806+0000 7fd25aa15640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2380a3680 0x7fd2380a5b40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd244011d10 tx=0x7fd2440054d0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.806+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd254109830 msgr2=0x7fd2541148b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:39.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.806+0000 7fd25aa15640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd254109830 0x7fd2541148b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd240009850 tx=0x7fd240005230 comp rx=0 tx=0).stop 2024-09-16T00:26:39.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.808+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 shutdown_connections 2024-09-16T00:26:39.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.808+0000 7fd25aa15640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd2380a3680 0x7fd2380a5b40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.808+0000 7fd25aa15640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd25411b140 0x7fd254114df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.808+0000 7fd25aa15640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd254109830 0x7fd2541148b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.808+0000 7fd25aa15640 1 --2- 172.21.15.78:0/2702686759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd254109450 0x7fd25411a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:39.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.808+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 >> 172.21.15.78:0/2702686759 conn(0x7fd25406e140 msgr2=0x7fd254070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:39.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.810+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 shutdown_connections 2024-09-16T00:26:39.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:39.810+0000 7fd25aa15640 1 -- 172.21.15.78:0/2702686759 wait complete. 2024-09-16T00:26:39.821 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:39 smithi193 ceph-mon[35484]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-16T00:26:40.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:39 smithi148 ceph-mon[35599]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-16T00:26:40.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:39 smithi078 ceph-mon[31157]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.8 MiB/s wr, 589 op/s 2024-09-16T00:26:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 760.0 s, 504.0 tps, lat 191.813 ms stddev 235.670 2024-09-16T00:26:42.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:41 smithi193 ceph-mon[35484]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 651 op/s 2024-09-16T00:26:42.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:41 smithi148 ceph-mon[35599]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 651 op/s 2024-09-16T00:26:42.251 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:41 smithi078 ceph-mon[31157]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 651 op/s 2024-09-16T00:26:42.704 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:26:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.145+0000 7f7a76389640 1 -- 172.21.15.78:0/3721101504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70109450 msgr2=0x7f7a70110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.145+0000 7f7a76389640 1 --2- 172.21.15.78:0/3721101504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70109450 0x7f7a70110250 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009900 tx=0x7f7a6002a440 comp rx=0 tx=0).stop 2024-09-16T00:26:43.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.145+0000 7f7a76389640 1 -- 172.21.15.78:0/3721101504 shutdown_connections 2024-09-16T00:26:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.145+0000 7f7a76389640 1 --2- 172.21.15.78:0/3721101504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a7011b140 0x7f7a7011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.145+0000 7f7a76389640 1 --2- 172.21.15.78:0/3721101504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a70109830 0x7f7a70109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.145+0000 7f7a76389640 1 --2- 172.21.15.78:0/3721101504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70109450 0x7f7a70110250 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.145+0000 7f7a76389640 1 -- 172.21.15.78:0/3721101504 >> 172.21.15.78:0/3721101504 conn(0x7f7a7006e140 msgr2=0x7f7a7006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:43.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.150+0000 7f7a76389640 1 -- 172.21.15.78:0/3721101504 shutdown_connections 2024-09-16T00:26:43.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.154+0000 7f7a76389640 1 -- 172.21.15.78:0/3721101504 wait complete. 2024-09-16T00:26:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.154+0000 7f7a76389640 1 Processor -- start 2024-09-16T00:26:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.157+0000 7f7a76389640 1 -- start start 2024-09-16T00:26:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.157+0000 7f7a76389640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70109450 0x7f7a7011a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.157+0000 7f7a76389640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a70109830 0x7f7a7011aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:43.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.157+0000 7f7a76389640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a7011b140 0x7f7a701145b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.157+0000 7f7a76389640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70110f30 con 0x7f7a70109450 2024-09-16T00:26:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.157+0000 7f7a76389640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70110db0 con 0x7f7a70109830 2024-09-16T00:26:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.157+0000 7f7a76389640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701110b0 con 0x7f7a7011b140 2024-09-16T00:26:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.158+0000 7f7a74926640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a7011b140 0x7f7a701145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.158+0000 7f7a74926640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a7011b140 0x7f7a701145b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:50090/0 (socket says 172.21.15.78:50090) 2024-09-16T00:26:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.158+0000 7f7a74926640 1 -- 172.21.15.78:0/1259051885 learned_addr learned my addr 172.21.15.78:0/1259051885 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:43.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.158+0000 7f7a6ffff640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70109450 0x7f7a7011a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:43.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.159+0000 7f7a74926640 1 -- 172.21.15.78:0/1259051885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a70109830 msgr2=0x7f7a7011aa50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:26:43.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.159+0000 7f7a74926640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a70109830 0x7f7a7011aa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.159+0000 7f7a74926640 1 -- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70109450 msgr2=0x7f7a7011a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:43.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.159+0000 7f7a74926640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70109450 0x7f7a7011a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.159+0000 7f7a74926640 1 -- 172.21.15.78:0/1259051885 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a70114e70 con 0x7f7a7011b140 2024-09-16T00:26:43.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.161+0000 7f7a74926640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a7011b140 0x7f7a701145b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800b9e0 tx=0x7f7a6800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:43.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.163+0000 7f7a6d7fa640 1 -- 172.21.15.78:0/1259051885 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68019070 con 0x7f7a7011b140 2024-09-16T00:26:43.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.163+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a70115160 con 0x7f7a7011b140 2024-09-16T00:26:43.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.163+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701bc7e0 con 0x7f7a7011b140 2024-09-16T00:26:43.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.164+0000 7f7a6d7fa640 1 -- 172.21.15.78:0/1259051885 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a68014440 con 0x7f7a7011b140 2024-09-16T00:26:43.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.164+0000 7f7a6d7fa640 1 -- 172.21.15.78:0/1259051885 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680075b0 con 0x7f7a7011b140 2024-09-16T00:26:43.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.166+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c005290 con 0x7f7a7011b140 2024-09-16T00:26:43.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.168+0000 7f7a6d7fa640 1 -- 172.21.15.78:0/1259051885 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7a6801c070 con 0x7f7a7011b140 2024-09-16T00:26:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.168+0000 7f7a6d7fa640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a580a3450 0x7f7a580a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:43.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.169+0000 7f7a6d7fa640 1 -- 172.21.15.78:0/1259051885 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7a680cf140 con 0x7f7a7011b140 2024-09-16T00:26:43.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.171+0000 7f7a6ffff640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a580a3450 0x7f7a580a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:43.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.177+0000 7f7a6d7fa640 1 -- 172.21.15.78:0/1259051885 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a68006e20 con 0x7f7a7011b140 2024-09-16T00:26:43.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.177+0000 7f7a6ffff640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a580a3450 0x7f7a580a5910 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009850 tx=0x7f7a60005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:43.332 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:43.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.576+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a5c005580 con 0x7f7a7011b140 2024-09-16T00:26:43.581 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:26:43.581 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:26:43.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.577+0000 7f7a6d7fa640 1 -- 172.21.15.78:0/1259051885 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f7a6809baa0 con 0x7f7a7011b140 2024-09-16T00:26:43.581 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:26:43.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a580a3450 msgr2=0x7f7a580a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a580a3450 0x7f7a580a5910 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009850 tx=0x7f7a60005fd0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a7011b140 msgr2=0x7f7a701145b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a7011b140 0x7f7a701145b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800b9e0 tx=0x7f7a6800bea0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 shutdown_connections 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7a580a3450 0x7f7a580a5910 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a7011b140 0x7f7a701145b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a70109830 0x7f7a7011aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 --2- 172.21.15.78:0/1259051885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7a70109450 0x7f7a7011a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 >> 172.21.15.78:0/1259051885 conn(0x7f7a7006e140 msgr2=0x7f7a70072390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 shutdown_connections 2024-09-16T00:26:43.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:43.584+0000 7f7a76389640 1 -- 172.21.15.78:0/1259051885 wait complete. 2024-09-16T00:26:44.074 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:43 smithi078 ceph-mon[31157]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 MiB/s wr, 500 op/s 2024-09-16T00:26:44.074 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:43 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1259051885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:43 smithi193 ceph-mon[35484]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 MiB/s wr, 500 op/s 2024-09-16T00:26:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:43 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1259051885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:43 smithi148 ceph-mon[35599]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 MiB/s wr, 500 op/s 2024-09-16T00:26:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:43 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1259051885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:44.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.695+0000 7f0905065640 1 -- 172.21.15.78:0/241288722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0900102570 msgr2=0x7f09001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.695+0000 7f0905065640 1 --2- 172.21.15.78:0/241288722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0900102570 0x7f09001029b0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009900 tx=0x7f08f402a480 comp rx=0 tx=0).stop 2024-09-16T00:26:44.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.696+0000 7f0905065640 1 -- 172.21.15.78:0/241288722 shutdown_connections 2024-09-16T00:26:44.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.696+0000 7f0905065640 1 --2- 172.21.15.78:0/241288722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0900102ef0 0x7f090010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.696+0000 7f0905065640 1 --2- 172.21.15.78:0/241288722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0900102570 0x7f09001029b0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.696+0000 7f0905065640 1 --2- 172.21.15.78:0/241288722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0900100e20 0x7f0900101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.696+0000 7f0905065640 1 -- 172.21.15.78:0/241288722 >> 172.21.15.78:0/241288722 conn(0x7f09000fc7d0 msgr2=0x7f09000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:44.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.696+0000 7f0905065640 1 -- 172.21.15.78:0/241288722 shutdown_connections 2024-09-16T00:26:44.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.696+0000 7f0905065640 1 -- 172.21.15.78:0/241288722 wait complete. 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f0905065640 1 Processor -- start 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f0905065640 1 -- start start 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f0905065640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0900100e20 0x7f090019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f0905065640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0900102570 0x7f090019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f0905065640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0900102ef0 0x7f090019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f0905065640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f090010fec0 con 0x7f0900102ef0 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f0905065640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f090010fd40 con 0x7f0900100e20 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f0905065640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0900110040 con 0x7f0900102570 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f08fe59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0900102570 0x7f090019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0900100e20 0x7f090019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0900100e20 0x7f090019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39960/0 (socket says 172.21.15.78:39960) 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f08fed9d640 1 -- 172.21.15.78:0/4236522641 learned_addr learned my addr 172.21.15.78:0/4236522641 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.697+0000 7f08ff59e640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0900102ef0 0x7f090019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08fed9d640 1 -- 172.21.15.78:0/4236522641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0900102570 msgr2=0x7f090019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08fed9d640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0900102570 0x7f090019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08fed9d640 1 -- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0900102ef0 msgr2=0x7f090019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08fed9d640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0900102ef0 0x7f090019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08fed9d640 1 -- 172.21.15.78:0/4236522641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090019f660 con 0x7f0900100e20 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08fed9d640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0900100e20 0x7f090019a6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009990 tx=0x7f08f0009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08e7fff640 1 -- 172.21.15.78:0/4236522641 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f001b070 con 0x7f0900100e20 2024-09-16T00:26:44.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f0905065640 1 -- 172.21.15.78:0/4236522641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090019f950 con 0x7f0900100e20 2024-09-16T00:26:44.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f0905065640 1 -- 172.21.15.78:0/4236522641 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09001a7230 con 0x7f0900100e20 2024-09-16T00:26:44.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08e7fff640 1 -- 172.21.15.78:0/4236522641 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f0016410 con 0x7f0900100e20 2024-09-16T00:26:44.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.698+0000 7f08e7fff640 1 -- 172.21.15.78:0/4236522641 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f001f610 con 0x7f0900100e20 2024-09-16T00:26:44.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.702+0000 7f08e7fff640 1 -- 172.21.15.78:0/4236522641 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f08f001e070 con 0x7f0900100e20 2024-09-16T00:26:44.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.702+0000 7f090743e740 1 Processor -- start 2024-09-16T00:26:44.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.702+0000 7f090743e740 1 -- start start 2024-09-16T00:26:44.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08e7fff640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08e80a3380 0x7f08e80a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f090743e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adcb40 0x561224adb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f090743e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224adba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08e7fff640 1 -- 172.21.15.78:0/4236522641 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f08f00cefe0 con 0x7f0900100e20 2024-09-16T00:26:44.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08ff59e640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08e80a3380 0x7f08e80a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f090743e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 0x561224ae8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f090743e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561224a412f0 con 0x561224adb5d0 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f090743e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5612249e8cf0 con 0x561224adcb40 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f090743e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561224a3d2b0 con 0x561224ae6370 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adcb40 0x561224adb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224adba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adcb40 0x561224adb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39962/0 (socket says 172.21.15.78:39962) 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fed9d640 1 -- 172.21.15.78:0/2718229274 learned_addr learned my addr 172.21.15.78:0/2718229274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fed9d640 1 -- 172.21.15.78:0/2718229274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 msgr2=0x561224ae8760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fed9d640 1 --2- 172.21.15.78:0/2718229274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 0x561224ae8760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fed9d640 1 -- 172.21.15.78:0/2718229274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 msgr2=0x561224adba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fed9d640 1 --2- 172.21.15.78:0/2718229274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224adba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.703+0000 7f08fed9d640 1 -- 172.21.15.78:0/2718229274 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612249cd3a0 con 0x561224adcb40 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.704+0000 7f08fed9d640 1 --2- 172.21.15.78:0/2718229274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adcb40 0x561224adb090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f08f00a3980 tx=0x7f08f009f6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba83a11e4581fad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.704+0000 7f08e57fa640 1 -- 172.21.15.78:0/2718229274 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f00cc8f0 con 0x561224adcb40 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.704+0000 7f08e57fa640 1 -- 172.21.15.78:0/2718229274 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f00ccbe0 con 0x561224adcb40 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.704+0000 7f08e57fa640 1 -- 172.21.15.78:0/2718229274 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f00a0780 con 0x561224adcb40 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.704+0000 7f08ff59e640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08e80a3380 0x7f08e80a5840 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x561224adc070 tx=0x7f08f8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 -- 172.21.15.78:0/2718229274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adcb40 msgr2=0x561224adb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 --2- 172.21.15.78:0/2718229274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adcb40 0x561224adb090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f08f00a3980 tx=0x7f08f009f6b0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 -- 172.21.15.78:0/2718229274 shutdown_connections 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 --2- 172.21.15.78:0/2718229274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 0x561224ae8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 --2- 172.21.15.78:0/2718229274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224adba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 --2- 172.21.15.78:0/2718229274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adcb40 0x561224adb090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 -- 172.21.15.78:0/2718229274 >> 172.21.15.78:0/2718229274 conn(0x561224aa4310 msgr2=0x561224ad7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 -- 172.21.15.78:0/2718229274 shutdown_connections 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.705+0000 7f090743e740 1 -- 172.21.15.78:0/2718229274 wait complete. 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 1 -- start start 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 1 Processor -- start 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224ae5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adcb40 0x561224ae3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 0x561224ae35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561224ae9070 con 0x561224adcb40 2024-09-16T00:26:44.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5612249fc060 con 0x561224adb5d0 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561224ae91f0 con 0x561224ae6370 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.706+0000 7f090743e740 10 client.0 _ll_register_callbacks cb 0x56122474f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224ae5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224ae5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:39974/0 (socket says 172.21.15.78:39974) 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fe59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adcb40 0x561224ae3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 -- 172.21.15.78:0/4072341762 learned_addr learned my addr 172.21.15.78:0/4072341762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08ff59e640 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 0x561224ae35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 -- 172.21.15.78:0/4072341762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 msgr2=0x561224ae35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 0x561224ae35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 -- 172.21.15.78:0/4072341762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adcb40 msgr2=0x561224ae3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:44.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adcb40 0x561224ae3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 -- 172.21.15.78:0/4072341762 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612247b23c0 con 0x561224adb5d0 2024-09-16T00:26:44.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08ff59e640 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 0x561224ae35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:44.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fe59c640 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adcb40 0x561224ae3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:44.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08fed9d640 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224ae5d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f08f0016de0 tx=0x7f08f009f350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8bec50765fa84f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:44.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.707+0000 7f08e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:26:44.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.708+0000 7f08e2ff5640 1 -- 172.21.15.78:0/4072341762 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f00b9060 con 0x561224adb5d0 2024-09-16T00:26:44.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.708+0000 7f08e2ff5640 1 -- 172.21.15.78:0/4072341762 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f08f0098150 con 0x561224adb5d0 2024-09-16T00:26:44.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.708+0000 7f090743e740 1 -- 172.21.15.78:0/4072341762 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f00ccbe0 con 0x561224adb5d0 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f08e2ff5640 1 -- 172.21.15.78:0/4072341762 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f08f0098350 con 0x561224adb5d0 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f090743e740 10 client.27008 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f090743e740 1 -- 172.21.15.78:0/4072341762 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561224ae9070 con 0x561224adb5d0 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f08e2ff5640 1 -- 172.21.15.78:0/4072341762 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f007f020 con 0x561224adb5d0 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f08e2ff5640 1 -- 172.21.15.78:0/4072341762 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f08f0085080 con 0x561224adb5d0 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f090743e740 10 client.27008 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f090743e740 10 client.27008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f090743e740 20 client.27008 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f090743e740 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561224afe500 0x561224b008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f090743e740 4 client.27008 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:44.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.709+0000 7f090743e740 1 -- 172.21.15.78:0/4072341762 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08f00ccbe0 con 0x561224afe500 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.710+0000 7f08fe59c640 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561224afe500 0x561224b008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.710+0000 7f08e2ff5640 10 client.27008 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.710+0000 7f08fe59c640 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561224afe500 0x561224b008f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dfc2bf7f5656091 server_cookie=6b0f95686bf75fc5 in_seq=0 out_seq=0 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f08e2ff5640 1 -- 172.21.15.78:0/4072341762 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f08f00ccbe0 con 0x561224afe500 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f08e2ff5640 10 client.27008 handle_command_reply: tid=0 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 client.27008 shutdown 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 -- 172.21.15.78:0/4072341762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 msgr2=0x561224ae5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224ae5d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f08f0016de0 tx=0x7f08f009f350 comp rx=0 tx=0).stop 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 -- 172.21.15.78:0/4072341762 shutdown_connections 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561224afe500 0x561224b008f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224ae6370 0x561224ae35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adcb40 0x561224ae3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 --2- 172.21.15.78:0/4072341762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224ae5d50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f08e2ff5640 0 client.27008 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:44.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.711+0000 7f090743e740 1 -- 172.21.15.78:0/4072341762 >> 172.21.15.78:0/4072341762 conn(0x561224aa4310 msgr2=0x561224add9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:44.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.712+0000 7f090743e740 1 -- 172.21.15.78:0/4072341762 shutdown_connections 2024-09-16T00:26:44.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.712+0000 7f090743e740 1 -- 172.21.15.78:0/4072341762 wait complete. 2024-09-16T00:26:44.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.712+0000 7f090743e740 20 client.27008 trim_cache size 0 max 16384 2024-09-16T00:26:44.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.731+0000 7f090743e740 1 Processor -- start 2024-09-16T00:26:44.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.731+0000 7f090743e740 1 -- start start 2024-09-16T00:26:44.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.731+0000 7f090743e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224afac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.731+0000 7f090743e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224add820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.731+0000 7f090743e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224ae6370 0x561224b05f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f090743e740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec00fb60 con 0x561224adb5d0 2024-09-16T00:26:44.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f090743e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec0116a0 con 0x561224ae6370 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f090743e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec010230 con 0x561224adcb40 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224afac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224afac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59978/0 (socket says 172.21.15.78:59978) 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 -- 172.21.15.78:0/164943119 learned_addr learned my addr 172.21.15.78:0/164943119 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fe59c640 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224add820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08ff59e640 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224ae6370 0x561224b05f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 -- 172.21.15.78:0/164943119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 msgr2=0x561224add820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224add820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 -- 172.21.15.78:0/164943119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224ae6370 msgr2=0x561224b05f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224ae6370 0x561224b05f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 -- 172.21.15.78:0/164943119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561224ad8960 con 0x561224adb5d0 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.732+0000 7f08fed9d640 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224afac20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f08f0016cd0 tx=0x7f08f009f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0d806df8a2cda5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:44.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.733+0000 7f08e4ff9640 1 -- 172.21.15.78:0/164943119 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f00b9070 con 0x561224adb5d0 2024-09-16T00:26:44.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.733+0000 7f08e4ff9640 1 -- 172.21.15.78:0/164943119 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f007f070 con 0x561224adb5d0 2024-09-16T00:26:44.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.733+0000 7f08e4ff9640 1 -- 172.21.15.78:0/164943119 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f00bb2d0 con 0x561224adb5d0 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 -- 172.21.15.78:0/164943119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 msgr2=0x561224afac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224afac20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f08f0016cd0 tx=0x7f08f009f6b0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 -- 172.21.15.78:0/164943119 shutdown_connections 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224ae6370 0x561224b05f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224add820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 --2- 172.21.15.78:0/164943119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224adb5d0 0x561224afac20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 -- 172.21.15.78:0/164943119 >> 172.21.15.78:0/164943119 conn(0x561224aa4310 msgr2=0x561224add110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 -- 172.21.15.78:0/164943119 shutdown_connections 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 -- 172.21.15.78:0/164943119 wait complete. 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 -- start start 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.734+0000 7f090743e740 1 Processor -- start 2024-09-16T00:26:44.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.735+0000 7f090743e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224af2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.735+0000 7f090743e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224af3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.735+0000 7f090743e740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224ae6370 0x561224af3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.735+0000 7f08fe59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224af3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.735+0000 7f08fe59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224af3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:50132/0 (socket says 172.21.15.78:50132) 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.735+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224af2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.735+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224af2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:40002/0 (socket says 172.21.15.78:40002) 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.736+0000 7f08fe59c640 1 -- 172.21.15.78:0/3908942909 learned_addr learned my addr 172.21.15.78:0/3908942909 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.736+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec010fd0 con 0x561224ae6370 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.736+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec010900 con 0x561224adb5d0 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.736+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec00db40 con 0x561224adcb40 2024-09-16T00:26:44.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.736+0000 7f090743e740 10 client.0 _ll_register_callbacks cb 0x7f08f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:44.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.737+0000 7f08ff59e640 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224ae6370 0x561224af3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.737+0000 7f08fe59c640 1 -- 172.21.15.78:0/3908942909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 msgr2=0x561224af2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.737+0000 7f08fe59c640 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224af2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.737+0000 7f08fe59c640 1 -- 172.21.15.78:0/3908942909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224ae6370 msgr2=0x561224af3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.737+0000 7f08fe59c640 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224ae6370 0x561224af3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.737+0000 7f08fe59c640 1 -- 172.21.15.78:0/3908942909 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561224ad7510 con 0x561224adcb40 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.738+0000 7f08fe59c640 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224af3240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08f402a4c0 tx=0x7f08f4009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbdcdfe3546df6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.738+0000 7f08e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.738+0000 7f08fed9d640 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224af2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.738+0000 7f08ff59e640 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224ae6370 0x561224af3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.739+0000 7f08e2ff5640 1 -- 172.21.15.78:0/3908942909 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f404a070 con 0x561224adcb40 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.739+0000 7f08e2ff5640 1 -- 172.21.15.78:0/3908942909 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f08f4005450 con 0x561224adcb40 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.739+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f007f070 con 0x561224adcb40 2024-09-16T00:26:44.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.739+0000 7f08e2ff5640 1 -- 172.21.15.78:0/3908942909 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f08f4056050 con 0x561224adcb40 2024-09-16T00:26:44.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.739+0000 7f090743e740 10 client.26812 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:44.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.739+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561224adbdd0 con 0x561224adcb40 2024-09-16T00:26:44.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.739+0000 7f08e2ff5640 1 -- 172.21.15.78:0/3908942909 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f007f070 con 0x561224adcb40 2024-09-16T00:26:44.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.739+0000 7f08e2ff5640 1 -- 172.21.15.78:0/3908942909 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f08f4003890 con 0x561224adcb40 2024-09-16T00:26:44.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.740+0000 7f090743e740 10 client.26812 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:44.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.740+0000 7f090743e740 10 client.26812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.740+0000 7f090743e740 20 client.26812 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.740+0000 7f090743e740 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561224b697d0 0x561224b6bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.740+0000 7f090743e740 4 client.26812 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.740+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561224a4e860 con 0x561224b697d0 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.740+0000 7f08fed9d640 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561224b697d0 0x561224b6bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f08fed9d640 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561224b697d0 0x561224b6bbf0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7faa43a207d57fac server_cookie=803e3a2769c1135c in_seq=0 out_seq=0 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f08e2ff5640 10 client.26812 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f08e2ff5640 1 -- 172.21.15.78:0/3908942909 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561224a4e860 con 0x561224b697d0 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f08e2ff5640 10 client.26812 handle_command_reply: tid=0 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 client.26812 shutdown 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 msgr2=0x561224af3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224af3240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08f402a4c0 tx=0x7f08f4009680 comp rx=0 tx=0).stop 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 shutdown_connections 2024-09-16T00:26:44.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x561224b697d0 0x561224b6bbf0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561224ae6370 0x561224af3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561224adcb40 0x561224af3240 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 --2- 172.21.15.78:0/3908942909 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561224adb5d0 0x561224af2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 >> 172.21.15.78:0/3908942909 conn(0x561224aa4310 msgr2=0x561224ad6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f08e2ff5640 0 client.26812 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 shutdown_connections 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 1 -- 172.21.15.78:0/3908942909 wait complete. 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.741+0000 7f090743e740 20 client.26812 trim_cache size 0 max 16384 2024-09-16T00:26:44.744 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:26:44.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 -- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08e80a3380 msgr2=0x7f08e80a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08e80a3380 0x7f08e80a5840 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x561224adc070 tx=0x7f08f8009600 comp rx=0 tx=0).stop 2024-09-16T00:26:44.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 -- 172.21.15.78:0/4236522641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0900100e20 msgr2=0x7f090019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:44.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0900100e20 0x7f090019a6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009990 tx=0x7f08f0009e50 comp rx=0 tx=0).stop 2024-09-16T00:26:44.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 -- 172.21.15.78:0/4236522641 shutdown_connections 2024-09-16T00:26:44.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f08e80a3380 0x7f08e80a5840 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0900102ef0 0x7f090019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0900102570 0x7f090019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 --2- 172.21.15.78:0/4236522641 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0900100e20 0x7f090019a6b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:44.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 -- 172.21.15.78:0/4236522641 >> 172.21.15.78:0/4236522641 conn(0x7f09000fc7d0 msgr2=0x7f0900106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:44.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 -- 172.21.15.78:0/4236522641 shutdown_connections 2024-09-16T00:26:44.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:44.742+0000 7f0905065640 1 -- 172.21.15.78:0/4236522641 wait complete. 2024-09-16T00:26:45.304 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0' 2024-09-16T00:26:45.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0 2024-09-16T00:26:45.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 765.0 s, 662.8 tps, lat 155.231 ms stddev 193.694 2024-09-16T00:26:46.111 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:45 smithi193 ceph-mon[35484]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 537 op/s 2024-09-16T00:26:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:45 smithi148 ceph-mon[35599]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 537 op/s 2024-09-16T00:26:46.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:45 smithi078 ceph-mon[31157]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 537 op/s 2024-09-16T00:26:46.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 -- 172.21.15.78:0/2057586313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0106b80 msgr2=0x7f79a0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 --2- 172.21.15.78:0/2057586313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0106b80 0x7f79a0106f60 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7990009900 tx=0x7f799002a410 comp rx=0 tx=0).stop 2024-09-16T00:26:46.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 -- 172.21.15.78:0/2057586313 shutdown_connections 2024-09-16T00:26:46.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 --2- 172.21.15.78:0/2057586313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a0100aa0 0x7f79a010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 --2- 172.21.15.78:0/2057586313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79a0100100 0x7f79a0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 --2- 172.21.15.78:0/2057586313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0106b80 0x7f79a0106f60 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 -- 172.21.15.78:0/2057586313 >> 172.21.15.78:0/2057586313 conn(0x7f79a0077230 msgr2=0x7f79a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 -- 172.21.15.78:0/2057586313 shutdown_connections 2024-09-16T00:26:46.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.785+0000 7f79a54e1640 1 -- 172.21.15.78:0/2057586313 wait complete. 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.786+0000 7f79a54e1640 1 Processor -- start 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.786+0000 7f79a54e1640 1 -- start start 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f79a54e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79a0100100 0x7f79a01162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f79a54e1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0100aa0 0x7f79a01167f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f79a54e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a0106b80 0x7f79a01113d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f79a54e1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01028a0 con 0x7f79a0100aa0 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f79a54e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0102720 con 0x7f79a0100100 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f79a54e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0102a20 con 0x7f79a0106b80 2024-09-16T00:26:46.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f7997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0100aa0 0x7f79a01167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f7997fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0100aa0 0x7f79a01167f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38416/0 (socket says 172.21.15.78:38416) 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f7997fff640 1 -- 172.21.15.78:0/3699238941 learned_addr learned my addr 172.21.15.78:0/3699238941 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f799effd640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79a0100100 0x7f79a01162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f799f7fe640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a0106b80 0x7f79a01113d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f799f7fe640 1 -- 172.21.15.78:0/3699238941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79a0100100 msgr2=0x7f79a01162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f799f7fe640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79a0100100 0x7f79a01162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f799f7fe640 1 -- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0100aa0 msgr2=0x7f79a01167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f799f7fe640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0100aa0 0x7f79a01167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f799f7fe640 1 -- 172.21.15.78:0/3699238941 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a0111c90 con 0x7f79a0106b80 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.787+0000 7f7997fff640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0100aa0 0x7f79a01167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.788+0000 7f799f7fe640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a0106b80 0x7f79a01113d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7998005fd0 tx=0x7f799800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.788+0000 7f799cff9640 1 -- 172.21.15.78:0/3699238941 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799800cdf0 con 0x7f79a0106b80 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.788+0000 7f799cff9640 1 -- 172.21.15.78:0/3699238941 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7998005020 con 0x7f79a0106b80 2024-09-16T00:26:46.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.788+0000 7f799cff9640 1 -- 172.21.15.78:0/3699238941 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79980106d0 con 0x7f79a0106b80 2024-09-16T00:26:46.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.788+0000 7f79a54e1640 1 -- 172.21.15.78:0/3699238941 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a0111f80 con 0x7f79a0106b80 2024-09-16T00:26:46.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.788+0000 7f79a54e1640 1 -- 172.21.15.78:0/3699238941 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a011c1a0 con 0x7f79a0106b80 2024-09-16T00:26:46.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.790+0000 7f799cff9640 1 -- 172.21.15.78:0/3699238941 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f79980054d0 con 0x7f79a0106b80 2024-09-16T00:26:46.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.791+0000 7f799cff9640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f798c0a3410 0x7f798c0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.791+0000 7f799effd640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f798c0a3410 0x7f798c0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.791+0000 7f799cff9640 1 -- 172.21.15.78:0/3699238941 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f79980d0180 con 0x7f79a0106b80 2024-09-16T00:26:46.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.791+0000 7f79a78ba740 1 Processor -- start 2024-09-16T00:26:46.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.791+0000 7f79a78ba740 1 -- start start 2024-09-16T00:26:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.792+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 0x55aceff48b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.792+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff49080 0x55aceff53400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.792+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.792+0000 7f79a78ba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55acefeae2a0 con 0x55aceff49080 2024-09-16T00:26:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.792+0000 7f79a78ba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acefe55b10 con 0x55aceff53940 2024-09-16T00:26:46.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.792+0000 7f79a78ba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55acefeaa260 con 0x55aceff48760 2024-09-16T00:26:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.793+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.793+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff55d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:56906/0 (socket says 172.21.15.78:56906) 2024-09-16T00:26:46.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.793+0000 7f799f7fe640 1 -- 172.21.15.78:0/3482486067 learned_addr learned my addr 172.21.15.78:0/3482486067 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.794+0000 7f7997fff640 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff49080 0x55aceff53400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.794+0000 7f799f7fe640 1 -- 172.21.15.78:0/3482486067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 msgr2=0x55aceff48b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:46.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.794+0000 7f799f7fe640 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 0x55aceff48b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.794+0000 7f799f7fe640 1 -- 172.21.15.78:0/3482486067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff49080 msgr2=0x55aceff53400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.794+0000 7f799f7fe640 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff49080 0x55aceff53400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.794+0000 7f799f7fe640 1 -- 172.21.15.78:0/3482486067 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acefc1fb90 con 0x55aceff53940 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.794+0000 7f7997fff640 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff49080 0x55aceff53400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.794+0000 7f799effd640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f798c0a3410 0x7f798c0a58d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7990009230 tx=0x7f79900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.795+0000 7f799f7fe640 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff55d30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f79980a4b20 tx=0x7f7998005440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950136ccb16722a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.795+0000 7f79957fa640 1 -- 172.21.15.78:0/3482486067 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79980b4070 con 0x55aceff53940 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.795+0000 7f79957fa640 1 -- 172.21.15.78:0/3482486067 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79980a5050 con 0x55aceff53940 2024-09-16T00:26:46.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.795+0000 7f79957fa640 1 -- 172.21.15.78:0/3482486067 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79980a4bc0 con 0x55aceff53940 2024-09-16T00:26:46.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 -- 172.21.15.78:0/3482486067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 msgr2=0x55aceff55d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff55d30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f79980a4b20 tx=0x7f7998005440 comp rx=0 tx=0).stop 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 -- 172.21.15.78:0/3482486067 shutdown_connections 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff55d30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff49080 0x55aceff53400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3482486067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 0x55aceff48b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 -- 172.21.15.78:0/3482486067 >> 172.21.15.78:0/3482486067 conn(0x55aceff44cf0 msgr2=0x55aceff450d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 -- 172.21.15.78:0/3482486067 shutdown_connections 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 -- 172.21.15.78:0/3482486067 wait complete. 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.796+0000 7f79a78ba740 1 -- start start 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f79a78ba740 1 Processor -- start 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff50850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceff50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff53940 0x55aceff512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f79a78ba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55aceff56510 con 0x55aceff48760 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f79a78ba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55acefe366e0 con 0x55aceff49080 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f79a78ba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aceff56690 con 0x55aceff53940 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f799effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff50850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff53940 0x55aceff512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f799effd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff50850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38438/0 (socket says 172.21.15.78:38438) 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f799effd640 1 -- 172.21.15.78:0/2923088689 learned_addr learned my addr 172.21.15.78:0/2923088689 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f79a78ba740 10 client.0 _ll_register_callbacks cb 0x55acefe5aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.797+0000 7f7997fff640 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceff50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f799f7fe640 1 -- 172.21.15.78:0/2923088689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 msgr2=0x55aceff50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:46.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f799f7fe640 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceff50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f799f7fe640 1 -- 172.21.15.78:0/2923088689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 msgr2=0x55aceff50850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f799f7fe640 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff50850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f799f7fe640 1 -- 172.21.15.78:0/2923088689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aceff5fa30 con 0x55aceff53940 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f799effd640 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff50850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f799f7fe640 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff53940 0x55aceff512d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f799809c5a0 tx=0x7f79980b52e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acba7fcf5d77075 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f79867fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f79867fc640 1 -- 172.21.15.78:0/2923088689 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79980b4070 con 0x55aceff53940 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f79867fc640 1 -- 172.21.15.78:0/2923088689 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f79980a4dc0 con 0x55aceff53940 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f79a78ba740 1 -- 172.21.15.78:0/2923088689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79980a5050 con 0x55aceff53940 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f79867fc640 1 -- 172.21.15.78:0/2923088689 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f799809b830 con 0x55aceff53940 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f79a78ba740 10 client.26821 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:46.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.798+0000 7f79a78ba740 1 -- 172.21.15.78:0/2923088689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55aceff56510 con 0x55aceff53940 2024-09-16T00:26:46.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f79867fc640 1 -- 172.21.15.78:0/2923088689 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79980ba070 con 0x55aceff53940 2024-09-16T00:26:46.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f79867fc640 1 -- 172.21.15.78:0/2923088689 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f79980a7060 con 0x55aceff53940 2024-09-16T00:26:46.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f79a78ba740 10 client.26821 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:46.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f79a78ba740 10 client.26821 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f79a78ba740 20 client.26821 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f79a78ba740 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55aceff63900 0x55aceff65d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f79a78ba740 4 client.26821 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f79a78ba740 1 -- 172.21.15.78:0/2923088689 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f79980a5050 con 0x55aceff63900 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.799+0000 7f799effd640 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55aceff63900 0x55aceff65d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.800+0000 7f799effd640 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55aceff63900 0x55aceff65d20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3213f4662fd58e26 server_cookie=e91c0dc43d2b07e7 in_seq=0 out_seq=0 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.800+0000 7f79867fc640 10 client.26821 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79867fc640 1 -- 172.21.15.78:0/2923088689 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f79980a5050 con 0x55aceff63900 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79867fc640 10 client.26821 handle_command_reply: tid=0 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 client.26821 shutdown 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 -- 172.21.15.78:0/2923088689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff53940 msgr2=0x55aceff512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff53940 0x55aceff512d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f799809c5a0 tx=0x7f79980b52e0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 -- 172.21.15.78:0/2923088689 shutdown_connections 2024-09-16T00:26:46.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55aceff63900 0x55aceff65d20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff53940 0x55aceff512d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceff50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 --2- 172.21.15.78:0/2923088689 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff50850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 -- 172.21.15.78:0/2923088689 >> 172.21.15.78:0/2923088689 conn(0x55aceff44cf0 msgr2=0x55aceff4aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79867fc640 0 client.26821 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.801+0000 7f79a78ba740 1 -- 172.21.15.78:0/2923088689 shutdown_connections 2024-09-16T00:26:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.802+0000 7f79a78ba740 1 -- 172.21.15.78:0/2923088689 wait complete. 2024-09-16T00:26:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.802+0000 7f79a78ba740 20 client.26821 trim_cache size 0 max 16384 2024-09-16T00:26:46.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.831+0000 7f79a78ba740 1 Processor -- start 2024-09-16T00:26:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.832+0000 7f79a78ba740 1 -- start start 2024-09-16T00:26:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.833+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff4a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.833+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff49080 0x55aceff6b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.833+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff6bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.833+0000 7f79a78ba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01bda10 con 0x55aceff48760 2024-09-16T00:26:46.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.833+0000 7f79a78ba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01bf550 con 0x55aceff53940 2024-09-16T00:26:46.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.833+0000 7f79a78ba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01be0e0 con 0x55aceff49080 2024-09-16T00:26:46.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.836+0000 7f799f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff6bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.837+0000 7f7997fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff49080 0x55aceff6b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.837+0000 7f7997fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff49080 0x55aceff6b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:39628/0 (socket says 172.21.15.78:39628) 2024-09-16T00:26:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.837+0000 7f7997fff640 1 -- 172.21.15.78:0/740546294 learned_addr learned my addr 172.21.15.78:0/740546294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:46.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.839+0000 7f799effd640 1 --2- 172.21.15.78:0/740546294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff4a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.841+0000 7f7997fff640 1 -- 172.21.15.78:0/740546294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 msgr2=0x55aceff6bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:46.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.841+0000 7f7997fff640 1 --2- 172.21.15.78:0/740546294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff6bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.841+0000 7f7997fff640 1 -- 172.21.15.78:0/740546294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 msgr2=0x55aceff4a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.841+0000 7f7997fff640 1 --2- 172.21.15.78:0/740546294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff4a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.842+0000 7f7997fff640 1 -- 172.21.15.78:0/740546294 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aceff48eb0 con 0x55aceff49080 2024-09-16T00:26:46.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.850+0000 7f7997fff640 1 --2- 172.21.15.78:0/740546294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff49080 0x55aceff6b610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f79a0112b90 tx=0x7f79880139f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc3bd43446fa6ca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.854+0000 7f7994ff9640 1 -- 172.21.15.78:0/740546294 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988020020 con 0x55aceff49080 2024-09-16T00:26:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.854+0000 7f7994ff9640 1 -- 172.21.15.78:0/740546294 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988005500 con 0x55aceff49080 2024-09-16T00:26:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.854+0000 7f7994ff9640 1 -- 172.21.15.78:0/740546294 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988011880 con 0x55aceff49080 2024-09-16T00:26:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.855+0000 7f79a78ba740 1 -- 172.21.15.78:0/740546294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff49080 msgr2=0x55aceff6b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.855+0000 7f79a78ba740 1 --2- 172.21.15.78:0/740546294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff49080 0x55aceff6b610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f79a0112b90 tx=0x7f79880139f0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.863+0000 7f79a78ba740 1 -- 172.21.15.78:0/740546294 shutdown_connections 2024-09-16T00:26:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.863+0000 7f79a78ba740 1 --2- 172.21.15.78:0/740546294 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff53940 0x55aceff6bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.863+0000 7f79a78ba740 1 --2- 172.21.15.78:0/740546294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff49080 0x55aceff6b610 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.863+0000 7f79a78ba740 1 --2- 172.21.15.78:0/740546294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff48760 0x55aceff4a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.863+0000 7f79a78ba740 1 -- 172.21.15.78:0/740546294 >> 172.21.15.78:0/740546294 conn(0x55aceff44cf0 msgr2=0x55aceff4a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:46.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.871+0000 7f79a78ba740 1 -- 172.21.15.78:0/740546294 shutdown_connections 2024-09-16T00:26:46.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.872+0000 7f79a78ba740 1 -- 172.21.15.78:0/740546294 wait complete. 2024-09-16T00:26:46.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.872+0000 7f79a78ba740 1 -- start start 2024-09-16T00:26:46.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.872+0000 7f79a78ba740 1 Processor -- start 2024-09-16T00:26:46.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.874+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 0x55aceffc6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.874+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceffc6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.874+0000 7f79a78ba740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff53940 0x55aceff56e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.874+0000 7f79a78ba740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01bee80 con 0x55aceff53940 2024-09-16T00:26:46.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.875+0000 7f79a78ba740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01be7b0 con 0x55aceff49080 2024-09-16T00:26:46.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.875+0000 7f79a78ba740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01bb9d0 con 0x55aceff48760 2024-09-16T00:26:46.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.875+0000 7f79a78ba740 10 client.0 _ll_register_callbacks cb 0x7f79980a0850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f7997fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceffc6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f7997fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceffc6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:56948/0 (socket says 172.21.15.78:56948) 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f7997fff640 1 -- 172.21.15.78:0/3220652733 learned_addr learned my addr 172.21.15.78:0/3220652733 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f799effd640 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 0x55aceffc6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f799f7fe640 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff53940 0x55aceff56e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f799effd640 1 -- 172.21.15.78:0/3220652733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 msgr2=0x55aceffc6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f799effd640 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceffc6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f799effd640 1 -- 172.21.15.78:0/3220652733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff53940 msgr2=0x55aceff56e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f799effd640 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff53940 0x55aceff56e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:46.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.876+0000 7f799effd640 1 -- 172.21.15.78:0/3220652733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aceff575f0 con 0x55aceff48760 2024-09-16T00:26:46.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.879+0000 7f799effd640 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 0x55aceffc6960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55aceff66550 tx=0x7f7990009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e717b26eed2c2044 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:46.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.879+0000 7f79867fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:26:46.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.883+0000 7f79867fc640 1 -- 172.21.15.78:0/3220652733 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990035040 con 0x55aceff48760 2024-09-16T00:26:46.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.883+0000 7f79867fc640 1 -- 172.21.15.78:0/3220652733 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f79900035f0 con 0x55aceff48760 2024-09-16T00:26:46.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.884+0000 7f79a78ba740 1 -- 172.21.15.78:0/3220652733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7988005500 con 0x55aceff48760 2024-09-16T00:26:46.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.891+0000 7f79867fc640 1 -- 172.21.15.78:0/3220652733 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f79900037f0 con 0x55aceff48760 2024-09-16T00:26:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.891+0000 7f79a78ba740 10 client.26827 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:46.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.891+0000 7f79a78ba740 1 -- 172.21.15.78:0/3220652733 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55aceffdf190 con 0x55aceff48760 2024-09-16T00:26:46.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.895+0000 7f79867fc640 1 -- 172.21.15.78:0/3220652733 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7988005500 con 0x55aceff48760 2024-09-16T00:26:46.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.896+0000 7f79867fc640 1 -- 172.21.15.78:0/3220652733 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f79900538f0 con 0x55aceff48760 2024-09-16T00:26:46.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.897+0000 7f79a78ba740 10 client.26827 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:46.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.897+0000 7f79a78ba740 10 client.26827 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:26:46.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.897+0000 7f79a78ba740 20 client.26827 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:46.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.897+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55aceffdf920 0x55aceffe1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:46.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.897+0000 7f79a78ba740 4 client.26827 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}] 2024-09-16T00:26:46.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.897+0000 7f79a78ba740 1 -- 172.21.15.78:0/3220652733 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}) -- 0x55acefebb9d0 con 0x55aceffdf920 2024-09-16T00:26:46.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.902+0000 7f7997fff640 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55aceffdf920 0x55aceffe1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:46.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.907+0000 7f7997fff640 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55aceffdf920 0x55aceffe1d70 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf4b3b26cb0f97c server_cookie=d391d0a0c5ed21b in_seq=0 out_seq=0 2024-09-16T00:26:46.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:46.907+0000 7f79867fc640 10 client.26827 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:46.975 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:26:46.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:26:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79867fc640 1 -- 172.21.15.78:0/3220652733 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55acefebb9d0 con 0x55aceffdf920 2024-09-16T00:26:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79867fc640 10 client.26827 handle_command_reply: tid=0 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 client.26827 shutdown 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 -- 172.21.15.78:0/3220652733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 msgr2=0x55aceffc6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 0x55aceffc6960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x55aceff66550 tx=0x7f7990009260 comp rx=0 tx=0).stop 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 -- 172.21.15.78:0/3220652733 shutdown_connections 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55aceffdf920 0x55aceffe1d70 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55aceff53940 0x55aceff56e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55aceff49080 0x55aceffc6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 --2- 172.21.15.78:0/3220652733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aceff48760 0x55aceffc6960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 -- 172.21.15.78:0/3220652733 >> 172.21.15.78:0/3220652733 conn(0x55aceff44cf0 msgr2=0x55aceff528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79867fc640 0 client.26827 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 -- 172.21.15.78:0/3220652733 shutdown_connections 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 1 -- 172.21.15.78:0/3220652733 wait complete. 2024-09-16T00:26:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.197+0000 7f79a78ba740 20 client.26827 trim_cache size 0 max 16384 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 46, 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "f9bfcfb": { 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 46, 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:26:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 131.4, 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:47.202 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:26:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 -- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f798c0a3410 msgr2=0x7f798c0a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f798c0a3410 0x7f798c0a58d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7990009230 tx=0x7f79900058d0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 -- 172.21.15.78:0/3699238941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a0106b80 msgr2=0x7f79a01113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a0106b80 0x7f79a01113d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7998005fd0 tx=0x7f799800ef10 comp rx=0 tx=0).stop 2024-09-16T00:26:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 -- 172.21.15.78:0/3699238941 shutdown_connections 2024-09-16T00:26:47.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f798c0a3410 0x7f798c0a58d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79a0106b80 0x7f79a01113d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f79a0100aa0 0x7f79a01167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 --2- 172.21.15.78:0/3699238941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f79a0100100 0x7f79a01162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 -- 172.21.15.78:0/3699238941 >> 172.21.15.78:0/3699238941 conn(0x7f79a0077230 msgr2=0x7f79a0104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 -- 172.21.15.78:0/3699238941 shutdown_connections 2024-09-16T00:26:47.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:47.198+0000 7f79a54e1640 1 -- 172.21.15.78:0/3699238941 wait complete. 2024-09-16T00:26:47.455 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:48.033 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f9bfcfb', quiesce took 9.6 seconds. Will release after: 1.8000000000000007 2024-09-16T00:26:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:47 smithi193 ceph-mon[35484]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-09-16T00:26:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:47 smithi148 ceph-mon[35599]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-09-16T00:26:48.230 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:47 smithi078 ceph-mon[31157]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-09-16T00:26:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.471+0000 7f7153fdd640 1 -- 172.21.15.78:0/3906961455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101de0 msgr2=0x7f714c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.471+0000 7f7153fdd640 1 --2- 172.21.15.78:0/3906961455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101de0 0x7f714c1021c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7148009900 tx=0x7f714802a410 comp rx=0 tx=0).stop 2024-09-16T00:26:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 -- 172.21.15.78:0/3906961455 shutdown_connections 2024-09-16T00:26:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 --2- 172.21.15.78:0/3906961455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c106f10 0x7f714c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 --2- 172.21.15.78:0/3906961455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c102790 0x7f714c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 --2- 172.21.15.78:0/3906961455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c101de0 0x7f714c1021c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 -- 172.21.15.78:0/3906961455 >> 172.21.15.78:0/3906961455 conn(0x7f714c0fd8c0 msgr2=0x7f714c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 -- 172.21.15.78:0/3906961455 shutdown_connections 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 -- 172.21.15.78:0/3906961455 wait complete. 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 Processor -- start 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 -- start start 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c101de0 0x7f714c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c102790 0x7f714c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c106f10 0x7f714c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f714c111fb0 con 0x7f714c106f10 2024-09-16T00:26:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f714c111e30 con 0x7f714c102790 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.472+0000 7f7153fdd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f714c112130 con 0x7f714c101de0 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c106f10 0x7f714c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c106f10 0x7f714c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38480/0 (socket says 172.21.15.78:38480) 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 -- 172.21.15.78:0/4134142554 learned_addr learned my addr 172.21.15.78:0/4134142554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f7151578640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c102790 0x7f714c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f7151d79640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c101de0 0x7f714c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 -- 172.21.15.78:0/4134142554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c101de0 msgr2=0x7f714c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c101de0 0x7f714c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 -- 172.21.15.78:0/4134142554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c102790 msgr2=0x7f714c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c102790 0x7f714c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 -- 172.21.15.78:0/4134142554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c19d4a0 con 0x7f714c106f10 2024-09-16T00:26:48.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f715257a640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c106f10 0x7f714c19cdc0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f714400b970 tx=0x7f714400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f7142ffd640 1 -- 172.21.15.78:0/4134142554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144019070 con 0x7f714c106f10 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f7153fdd640 1 -- 172.21.15.78:0/4134142554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c19d790 con 0x7f714c106f10 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f7142ffd640 1 -- 172.21.15.78:0/4134142554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714400cb40 con 0x7f714c106f10 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f7142ffd640 1 -- 172.21.15.78:0/4134142554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144007650 con 0x7f714c106f10 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.473+0000 7f7153fdd640 1 -- 172.21.15.78:0/4134142554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c1a5070 con 0x7f714c106f10 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.475+0000 7f7142ffd640 1 -- 172.21.15.78:0/4134142554 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f714401c070 con 0x7f714c106f10 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f71563b6740 1 Processor -- start 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f71563b6740 1 -- start start 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f71563b6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 0x55ba220c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f7142ffd640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f71380a3240 0x7f71380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f71563b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f71563b6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 0x55ba220d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f71563b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ba2202a280 con 0x55ba220c5480 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f71563b6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ba21fd1b10 con 0x55ba220c4b60 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f71563b6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ba22026240 con 0x55ba220cfd40 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f7151d79640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f71380a3240 0x7f71380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f7151578640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f7142ffd640 1 -- 172.21.15.78:0/4134142554 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f71440cf2c0 con 0x7f714c106f10 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f7151578640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38482/0 (socket says 172.21.15.78:38482) 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f7151578640 1 -- 172.21.15.78:0/1851606446 learned_addr learned my addr 172.21.15.78:0/1851606446 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.476+0000 7f7151578640 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 0x55ba220c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f715257a640 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 0x55ba220d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7151578640 1 -- 172.21.15.78:0/1851606446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 msgr2=0x55ba220d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7151578640 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 0x55ba220d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7151578640 1 -- 172.21.15.78:0/1851606446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 msgr2=0x55ba220c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7151578640 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 0x55ba220c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7151578640 1 -- 172.21.15.78:0/1851606446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba21fde1f0 con 0x55ba220c5480 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f715257a640 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 0x55ba220d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7151d79640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f71380a3240 0x7f71380a5700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f71480096c0 tx=0x7f7148031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7151578640 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220cf800 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f713c00aab0 tx=0x7f713c0138b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3614929e298b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7137fff640 1 -- 172.21.15.78:0/1851606446 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c004020 con 0x55ba220c5480 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7137fff640 1 -- 172.21.15.78:0/1851606446 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c005500 con 0x55ba220c5480 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.477+0000 7f7137fff640 1 -- 172.21.15.78:0/1851606446 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c00dbf0 con 0x55ba220c5480 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 -- 172.21.15.78:0/1851606446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 msgr2=0x55ba220cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220cf800 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f713c00aab0 tx=0x7f713c0138b0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 -- 172.21.15.78:0/1851606446 shutdown_connections 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 0x55ba220d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220cf800 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 --2- 172.21.15.78:0/1851606446 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 0x55ba220c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 -- 172.21.15.78:0/1851606446 >> 172.21.15.78:0/1851606446 conn(0x55ba220c10f0 msgr2=0x55ba220c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 -- 172.21.15.78:0/1851606446 shutdown_connections 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 -- 172.21.15.78:0/1851606446 wait complete. 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 -- start start 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.478+0000 7f71563b6740 1 Processor -- start 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f71563b6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c4b60 0x55ba220cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f71563b6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c5480 0x55ba220cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f71563b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220cfd40 0x55ba220cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f71563b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ba220d2910 con 0x55ba220cfd40 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f71563b6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ba21fe4f10 con 0x55ba220c5480 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f71563b6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ba220d2a90 con 0x55ba220c4b60 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f71563b6740 10 client.0 _ll_register_callbacks cb 0x55ba21fada40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220cfd40 0x55ba220cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220cfd40 0x55ba220cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38496/0 (socket says 172.21.15.78:38496) 2024-09-16T00:26:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 -- 172.21.15.78:0/3499874112 learned_addr learned my addr 172.21.15.78:0/3499874112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f7151578640 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c5480 0x55ba220cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 -- 172.21.15.78:0/3499874112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c4b60 msgr2=0x55ba220cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c4b60 0x55ba220cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 -- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c5480 msgr2=0x55ba220cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c5480 0x55ba220cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 -- 172.21.15.78:0/3499874112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba21d9b6a0 con 0x55ba220cfd40 2024-09-16T00:26:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f7151578640 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c5480 0x55ba220cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f715257a640 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220cfd40 0x55ba220cd730 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7144097fa0 tx=0x7f7144099ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f1f1b1be9ea9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.479+0000 7f71357fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71357fa640 1 -- 172.21.15.78:0/3499874112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71440b4720 con 0x55ba220cfd40 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71357fa640 1 -- 172.21.15.78:0/3499874112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f71440b4ae0 con 0x55ba220cfd40 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 1 -- 172.21.15.78:0/3499874112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713c005500 con 0x55ba220cfd40 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71357fa640 1 -- 172.21.15.78:0/3499874112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f71440b4e60 con 0x55ba220cfd40 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 10 client.17781 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 1 -- 172.21.15.78:0/3499874112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ba220de600 con 0x55ba220cfd40 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71357fa640 1 -- 172.21.15.78:0/3499874112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c005500 con 0x55ba220cfd40 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71357fa640 1 -- 172.21.15.78:0/3499874112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f71440a3d00 con 0x55ba220cfd40 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 10 client.17781 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:26:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 20 client.17781 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ba220dfa10 0x55ba220e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 4 client.17781 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.480+0000 7f71563b6740 1 -- 172.21.15.78:0/3499874112 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba220377f0 con 0x55ba220dfa10 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.481+0000 7f7151d79640 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ba220dfa10 0x55ba220e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.481+0000 7f71357fa640 10 client.17781 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.481+0000 7f7151d79640 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ba220dfa10 0x55ba220e1ed0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6210c35d6cd2bcb7 server_cookie=516bfb2e4ac0c10f in_seq=0 out_seq=0 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71357fa640 1 -- 172.21.15.78:0/3499874112 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba220377f0 con 0x55ba220dfa10 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71357fa640 10 client.17781 handle_command_reply: tid=0 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 client.17781 shutdown 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 -- 172.21.15.78:0/3499874112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220cfd40 msgr2=0x55ba220cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220cfd40 0x55ba220cd730 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7144097fa0 tx=0x7f7144099ae0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 -- 172.21.15.78:0/3499874112 shutdown_connections 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ba220dfa10 0x55ba220e1ed0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220cfd40 0x55ba220cd730 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c5480 0x55ba220cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 --2- 172.21.15.78:0/3499874112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c4b60 0x55ba220cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71357fa640 0 client.17781 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 -- 172.21.15.78:0/3499874112 >> 172.21.15.78:0/3499874112 conn(0x55ba220c10f0 msgr2=0x55ba220c6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 -- 172.21.15.78:0/3499874112 shutdown_connections 2024-09-16T00:26:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 1 -- 172.21.15.78:0/3499874112 wait complete. 2024-09-16T00:26:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.482+0000 7f71563b6740 20 client.17781 trim_cache size 0 max 16384 2024-09-16T00:26:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.499+0000 7f71563b6740 1 Processor -- start 2024-09-16T00:26:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.499+0000 7f71563b6740 1 -- start start 2024-09-16T00:26:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.499+0000 7f71563b6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 0x55ba220c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.499+0000 7f71563b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.499+0000 7f71563b6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 0x55ba220e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.499+0000 7f71563b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ac990 con 0x55ba220c5480 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.499+0000 7f71563b6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ae4d0 con 0x55ba220c4b60 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.499+0000 7f71563b6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ad060 con 0x55ba220cfd40 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220e77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38508/0 (socket says 172.21.15.78:38508) 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 -- 172.21.15.78:0/3759338828 learned_addr learned my addr 172.21.15.78:0/3759338828 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151d79640 1 --2- 172.21.15.78:0/3759338828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 0x55ba220c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 -- 172.21.15.78:0/3759338828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 msgr2=0x55ba220e7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 --2- 172.21.15.78:0/3759338828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 0x55ba220e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 -- 172.21.15.78:0/3759338828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 msgr2=0x55ba220c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 --2- 172.21.15.78:0/3759338828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 0x55ba220c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 -- 172.21.15.78:0/3759338828 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba220cb200 con 0x55ba220c5480 2024-09-16T00:26:48.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f7151578640 1 --2- 172.21.15.78:0/3759338828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220e77f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f713c0058c0 tx=0x7f713c00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a0132c7618a984 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f71377fe640 1 -- 172.21.15.78:0/3759338828 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c005030 con 0x55ba220c5480 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f71377fe640 1 -- 172.21.15.78:0/3759338828 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c0127c0 con 0x55ba220c5480 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.500+0000 7f71377fe640 1 -- 172.21.15.78:0/3759338828 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c013140 con 0x55ba220c5480 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 -- 172.21.15.78:0/3759338828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 msgr2=0x55ba220e77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 --2- 172.21.15.78:0/3759338828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220e77f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f713c0058c0 tx=0x7f713c00af70 comp rx=0 tx=0).stop 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 -- 172.21.15.78:0/3759338828 shutdown_connections 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 --2- 172.21.15.78:0/3759338828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220cfd40 0x55ba220e7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 --2- 172.21.15.78:0/3759338828 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c5480 0x55ba220e77f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 --2- 172.21.15.78:0/3759338828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220c4b60 0x55ba220c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 -- 172.21.15.78:0/3759338828 >> 172.21.15.78:0/3759338828 conn(0x55ba220c10f0 msgr2=0x55ba220cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 -- 172.21.15.78:0/3759338828 shutdown_connections 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 -- 172.21.15.78:0/3759338828 wait complete. 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.501+0000 7f71563b6740 1 -- start start 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f71563b6740 1 Processor -- start 2024-09-16T00:26:48.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f71563b6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c4b60 0x55ba220d44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f71563b6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c5480 0x55ba220d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f71563b6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220cfd40 0x55ba220d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f71563b6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ade00 con 0x55ba220c4b60 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f71563b6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1ad730 con 0x55ba220cfd40 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f71563b6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f714c1aa970 con 0x55ba220c5480 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f7151d79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c4b60 0x55ba220d44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f71563b6740 10 client.0 _ll_register_callbacks cb 0x7f71440971b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f7151d79640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c4b60 0x55ba220d44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38518/0 (socket says 172.21.15.78:38518) 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f7151d79640 1 -- 172.21.15.78:0/2932574024 learned_addr learned my addr 172.21.15.78:0/2932574024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f715257a640 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220cfd40 0x55ba220d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.502+0000 7f7151578640 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c5480 0x55ba220d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f7151d79640 1 -- 172.21.15.78:0/2932574024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c5480 msgr2=0x55ba220d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:48.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f7151d79640 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c5480 0x55ba220d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f7151d79640 1 -- 172.21.15.78:0/2932574024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220cfd40 msgr2=0x55ba220d7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f7151d79640 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220cfd40 0x55ba220d7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f7151d79640 1 -- 172.21.15.78:0/2932574024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba220d8100 con 0x55ba220c4b60 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f7151d79640 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c4b60 0x55ba220d44c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f714802a930 tx=0x7f7148009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5456f41ce046abe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f71357fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f71357fa640 1 -- 172.21.15.78:0/2932574024 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148035050 con 0x55ba220c4b60 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f71563b6740 1 -- 172.21.15.78:0/2932574024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713c0127c0 con 0x55ba220c4b60 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f71357fa640 1 -- 172.21.15.78:0/2932574024 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7148004280 con 0x55ba220c4b60 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f71357fa640 1 -- 172.21.15.78:0/2932574024 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7148060050 con 0x55ba220c4b60 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f71563b6740 10 client.17787 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f71563b6740 1 -- 172.21.15.78:0/2932574024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ba2214b320 con 0x55ba220c4b60 2024-09-16T00:26:48.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.503+0000 7f71357fa640 1 -- 172.21.15.78:0/2932574024 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713c0127c0 con 0x55ba220c4b60 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f71357fa640 1 -- 172.21.15.78:0/2932574024 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7148057640 con 0x55ba220c4b60 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f71563b6740 10 client.17787 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f71563b6740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f71563b6740 20 client.17787 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f71563b6740 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ba2214ba50 0x55ba2214de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f71563b6740 4 client.17787 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f71563b6740 1 -- 172.21.15.78:0/2932574024 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ba22031d00 con 0x55ba2214ba50 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f7151578640 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ba2214ba50 0x55ba2214de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f71357fa640 10 client.17787 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.504+0000 7f7151578640 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ba2214ba50 0x55ba2214de70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca94e124be422256 server_cookie=ce7dd4cf1448d522 in_seq=0 out_seq=0 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.505+0000 7f71357fa640 1 -- 172.21.15.78:0/2932574024 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ba22031d00 con 0x55ba2214ba50 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.505+0000 7f71357fa640 10 client.17787 handle_command_reply: tid=0 2024-09-16T00:26:48.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.505+0000 7f71563b6740 1 client.17787 shutdown 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.505+0000 7f71563b6740 1 -- 172.21.15.78:0/2932574024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c4b60 msgr2=0x55ba220d44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c4b60 0x55ba220d44c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f714802a930 tx=0x7f7148009260 comp rx=0 tx=0).stop 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 -- 172.21.15.78:0/2932574024 shutdown_connections 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ba2214ba50 0x55ba2214de70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ba220cfd40 0x55ba220d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ba220c5480 0x55ba220d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71357fa640 0 client.17787 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 --2- 172.21.15.78:0/2932574024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ba220c4b60 0x55ba220d44c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 -- 172.21.15.78:0/2932574024 >> 172.21.15.78:0/2932574024 conn(0x55ba220c10f0 msgr2=0x55ba220ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 -- 172.21.15.78:0/2932574024 shutdown_connections 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 1 -- 172.21.15.78:0/2932574024 wait complete. 2024-09-16T00:26:48.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f71563b6740 20 client.17787 trim_cache size 0 max 16384 2024-09-16T00:26:48.519 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:26:48.519 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:26:48.519 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "dcb581af-9ec5-451d-92f3-69c1fff3156a", 2024-09-16T00:26:48.519 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:26:48.519 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f7153fdd640 1 -- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f71380a3240 msgr2=0x7f71380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f7153fdd640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f71380a3240 0x7f71380a5700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f71480096c0 tx=0x7f7148031490 comp rx=0 tx=0).stop 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f7153fdd640 1 -- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c106f10 msgr2=0x7f714c19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.506+0000 7f7153fdd640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c106f10 0x7f714c19cdc0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f714400b970 tx=0x7f714400be30 comp rx=0 tx=0).stop 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.507+0000 7f7153fdd640 1 -- 172.21.15.78:0/4134142554 shutdown_connections 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.507+0000 7f7153fdd640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f71380a3240 0x7f71380a5700 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.507+0000 7f7153fdd640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714c106f10 0x7f714c19cdc0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.507+0000 7f7153fdd640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f714c102790 0x7f714c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.507+0000 7f7153fdd640 1 --2- 172.21.15.78:0/4134142554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714c101de0 0x7f714c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.507+0000 7f7153fdd640 1 -- 172.21.15.78:0/4134142554 >> 172.21.15.78:0/4134142554 conn(0x7f714c0fd8c0 msgr2=0x7f714c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.507+0000 7f7153fdd640 1 -- 172.21.15.78:0/4134142554 shutdown_connections 2024-09-16T00:26:48.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:48.507+0000 7f7153fdd640 1 -- 172.21.15.78:0/4134142554 wait complete. 2024-09-16T00:26:49.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:48 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:26:49.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:48 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:49.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:48 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:26:49.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:48 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:49.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:48 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:26:49.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:48 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:49.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:26:50.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:49 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:26:50.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:49 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:49 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:26:50.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:49 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:26:50.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:49 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:49 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.134 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:49 smithi078 ceph-mon[31157]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-09-16T00:26:50.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:26:50.176 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:49 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:26:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:49 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:49 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:26:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:49 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:26:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:49 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:49 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:49 smithi193 ceph-mon[35484]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-09-16T00:26:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:49 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:26:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:49 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:49 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:26:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:49 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:26:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:49 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:49 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:26:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:49 smithi148 ceph-mon[35599]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-09-16T00:26:50.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 -- 172.21.15.78:0/2399888564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 msgr2=0x7f4624100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:50.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 --2- 172.21.15.78:0/2399888564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 0x7f4624100840 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4614009ca0 tx=0x7f461402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 -- 172.21.15.78:0/2399888564 shutdown_connections 2024-09-16T00:26:50.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 --2- 172.21.15.78:0/2399888564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624105590 0x7f462410d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 --2- 172.21.15.78:0/2399888564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4624100e10 0x7f4624104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 --2- 172.21.15.78:0/2399888564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 0x7f4624100840 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 -- 172.21.15.78:0/2399888564 >> 172.21.15.78:0/2399888564 conn(0x7f46240fbee0 msgr2=0x7f46240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:50.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 -- 172.21.15.78:0/2399888564 shutdown_connections 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.688+0000 7f462b63f640 1 -- 172.21.15.78:0/2399888564 wait complete. 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f462b63f640 1 Processor -- start 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f462b63f640 1 -- start start 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f462b63f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 0x7f46241985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f462b63f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624100e10 0x7f4624198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f462b63f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4624105590 0x7f462419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f462b63f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4624110690 con 0x7f4624100460 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f462b63f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4624110510 con 0x7f4624100e10 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f462b63f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4624110810 con 0x7f4624105590 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 0x7f46241985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 0x7f46241985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38544/0 (socket says 172.21.15.78:38544) 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 -- 172.21.15.78:0/787223812 learned_addr learned my addr 172.21.15.78:0/787223812 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f4629bdc640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4624105590 0x7f462419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:50.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f4628bda640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624100e10 0x7f4624198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 -- 172.21.15.78:0/787223812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4624105590 msgr2=0x7f462419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4624105590 0x7f462419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 -- 172.21.15.78:0/787223812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624100e10 msgr2=0x7f4624198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624100e10 0x7f4624198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 -- 172.21.15.78:0/787223812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462419d550 con 0x7f4624100460 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f4628bda640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624100e10 0x7f4624198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f4629bdc640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4624105590 0x7f462419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.689+0000 7f46293db640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 0x7f46241985a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f46140097b0 tx=0x7f4614031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.690+0000 7f46227fc640 1 -- 172.21.15.78:0/787223812 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461402fba0 con 0x7f4624100460 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.690+0000 7f46227fc640 1 -- 172.21.15.78:0/787223812 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f461402fd40 con 0x7f4624100460 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.690+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462419d7e0 con 0x7f4624100460 2024-09-16T00:26:50.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.690+0000 7f46227fc640 1 -- 172.21.15.78:0/787223812 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461403b930 con 0x7f4624100460 2024-09-16T00:26:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.690+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46241a5120 con 0x7f4624100460 2024-09-16T00:26:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.691+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46241008c0 con 0x7f4624100460 2024-09-16T00:26:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.692+0000 7f46227fc640 1 -- 172.21.15.78:0/787223812 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4614030cc0 con 0x7f4624100460 2024-09-16T00:26:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.693+0000 7f46227fc640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f460c0a32c0 0x7f460c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.693+0000 7f4628bda640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f460c0a32c0 0x7f460c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.693+0000 7f46227fc640 1 -- 172.21.15.78:0/787223812 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f46140f15b0 con 0x7f4624100460 2024-09-16T00:26:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.693+0000 7f4628bda640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f460c0a32c0 0x7f460c0a5780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4618006fd0 tx=0x7f4618008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:50.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.696+0000 7f46227fc640 1 -- 172.21.15.78:0/787223812 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46140bdf10 con 0x7f4624100460 2024-09-16T00:26:50.747 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:50.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 770.0 s, 176.8 tps, lat 144.281 ms stddev 135.265 2024-09-16T00:26:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.927+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46241a5320 con 0x7f4624100460 2024-09-16T00:26:50.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.928+0000 7f46227fc640 1 -- 172.21.15.78:0/787223812 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f46140bd810 con 0x7f4624100460 2024-09-16T00:26:50.929 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:26:50.929 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:26:50.929 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f460c0a32c0 msgr2=0x7f460c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f460c0a32c0 0x7f460c0a5780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4618006fd0 tx=0x7f4618008040 comp rx=0 tx=0).stop 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 msgr2=0x7f46241985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 0x7f46241985a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f46140097b0 tx=0x7f4614031b10 comp rx=0 tx=0).stop 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 shutdown_connections 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f460c0a32c0 0x7f460c0a5780 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4624105590 0x7f462419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4624100e10 0x7f4624198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 --2- 172.21.15.78:0/787223812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4624100460 0x7f46241985a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 >> 172.21.15.78:0/787223812 conn(0x7f46240fbee0 msgr2=0x7f46240fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:50.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 shutdown_connections 2024-09-16T00:26:50.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:50.930+0000 7f462b63f640 1 -- 172.21.15.78:0/787223812 wait complete. 2024-09-16T00:26:51.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:50 smithi193 ceph-mon[35484]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 498 op/s 2024-09-16T00:26:51.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:50 smithi148 ceph-mon[35599]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 498 op/s 2024-09-16T00:26:51.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:50 smithi078 ceph-mon[31157]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.1 MiB/s wr, 498 op/s 2024-09-16T00:26:51.778 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id f9bfcfb --release --await' 2024-09-16T00:26:51.778 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id f9bfcfb --release --await 2024-09-16T00:26:52.004 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:51 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/787223812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:52.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.003+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/2606284458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 msgr2=0x7f6ae410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.003+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/2606284458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 0x7f6ae410eaa0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009ca0 tx=0x7f6adc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.003+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/2606284458 shutdown_connections 2024-09-16T00:26:52.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.003+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/2606284458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 0x7f6ae410eaa0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.003+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/2606284458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ae41027a0 0x7f6ae41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.003+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/2606284458 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae4101df0 0x7f6ae41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.003+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/2606284458 >> 172.21.15.78:0/2606284458 conn(0x7f6ae40fd8f0 msgr2=0x7f6ae40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.003+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/2606284458 shutdown_connections 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.004+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/2606284458 wait complete. 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.004+0000 7f6ae9cb0640 1 Processor -- start 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.004+0000 7f6ae9cb0640 1 -- start start 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae9cb0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae4101df0 0x7f6ae41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae9cb0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ae41027a0 0x7f6ae41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae9cb0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 0x7f6ae419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae9cb0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4111fc0 con 0x7f6ae4106f20 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae9cb0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4111e40 con 0x7f6ae4101df0 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae9cb0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4112140 con 0x7f6ae41027a0 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 0x7f6ae419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 0x7f6ae419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38564/0 (socket says 172.21.15.78:38564) 2024-09-16T00:26:52.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae4101df0 0x7f6ae41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae3fff640 1 -- 172.21.15.78:0/3733385665 learned_addr learned my addr 172.21.15.78:0/3733385665 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:52.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ae41027a0 0x7f6ae41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae3fff640 1 -- 172.21.15.78:0/3733385665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ae41027a0 msgr2=0x7f6ae41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae3fff640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ae41027a0 0x7f6ae41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae3fff640 1 -- 172.21.15.78:0/3733385665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae4101df0 msgr2=0x7f6ae41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae3fff640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae4101df0 0x7f6ae41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae3fff640 1 -- 172.21.15.78:0/3733385665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae419d450 con 0x7f6ae4106f20 2024-09-16T00:26:52.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.005+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae4101df0 0x7f6ae41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:52.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.006+0000 7f6ae3fff640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 0x7f6ae419cd70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6adc031a20 tx=0x7f6adc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:52.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.006+0000 7f6ae0ff9640 1 -- 172.21.15.78:0/3733385665 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc003b70 con 0x7f6ae4106f20 2024-09-16T00:26:52.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.006+0000 7f6ae0ff9640 1 -- 172.21.15.78:0/3733385665 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc003d10 con 0x7f6ae4106f20 2024-09-16T00:26:52.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.006+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/3733385665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae419d6e0 con 0x7f6ae4106f20 2024-09-16T00:26:52.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.006+0000 7f6ae0ff9640 1 -- 172.21.15.78:0/3733385665 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc02fc80 con 0x7f6ae4106f20 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.006+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/3733385665 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae41a5020 con 0x7f6ae4106f20 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.008+0000 7f6ae0ff9640 1 -- 172.21.15.78:0/3733385665 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f6adc0046a0 con 0x7f6ae4106f20 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6aec089740 1 Processor -- start 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6aec089740 1 -- start start 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6aec089740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89c380 0x55ee4a89c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6aec089740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89cca0 0x55ee4a8ab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6aec089740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a8aba90 0x55ee4a8ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6aec089740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee4a8022e0 con 0x55ee4a8aba90 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6aec089740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ee4a7a9ce0 con 0x55ee4a89cca0 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6aec089740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee4a7fe2a0 con 0x55ee4a89c380 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae0ff9640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6acc0a3240 0x7f6acc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a8aba90 0x55ee4a8ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89cca0 0x55ee4a8ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae3fff640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6acc0a3240 0x7f6acc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89cca0 0x55ee4a8ab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:57026/0 (socket says 172.21.15.78:57026) 2024-09-16T00:26:52.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae37fe640 1 -- 172.21.15.78:0/2715947093 learned_addr learned my addr 172.21.15.78:0/2715947093 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:52.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae0ff9640 1 -- 172.21.15.78:0/3733385665 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6adc0f0e40 con 0x7f6ae4106f20 2024-09-16T00:26:52.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89c380 0x55ee4a89c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae37fe640 1 -- 172.21.15.78:0/2715947093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89c380 msgr2=0x55ee4a89c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:52.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89c380 0x55ee4a89c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae37fe640 1 -- 172.21.15.78:0/2715947093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a8aba90 msgr2=0x55ee4a8ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a8aba90 0x55ee4a8ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.009+0000 7f6ae37fe640 1 -- 172.21.15.78:0/2715947093 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4a78e440 con 0x55ee4a89cca0 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.010+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a8aba90 0x55ee4a8ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.010+0000 7f6ae3fff640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6acc0a3240 0x7f6acc0a5700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0b9b60 tx=0x7f6adc03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.010+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89cca0 0x55ee4a8ab550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40104d0 tx=0x7f6ad4010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ffaf5a7ea75932 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.010+0000 7f6ac9ffb640 1 -- 172.21.15.78:0/2715947093 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4011760 con 0x55ee4a89cca0 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.010+0000 7f6ac9ffb640 1 -- 172.21.15.78:0/2715947093 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad4011de0 con 0x55ee4a89cca0 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.010+0000 7f6ac9ffb640 1 -- 172.21.15.78:0/2715947093 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400e430 con 0x55ee4a89cca0 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 -- 172.21.15.78:0/2715947093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89cca0 msgr2=0x55ee4a8ab550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89cca0 0x55ee4a8ab550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40104d0 tx=0x7f6ad4010990 comp rx=0 tx=0).stop 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 -- 172.21.15.78:0/2715947093 shutdown_connections 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a8aba90 0x55ee4a8ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89cca0 0x55ee4a8ab550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 --2- 172.21.15.78:0/2715947093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89c380 0x55ee4a89c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 -- 172.21.15.78:0/2715947093 >> 172.21.15.78:0/2715947093 conn(0x55ee4a898a30 msgr2=0x55ee4a898e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 -- 172.21.15.78:0/2715947093 shutdown_connections 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 -- 172.21.15.78:0/2715947093 wait complete. 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 -- start start 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.011+0000 7f6aec089740 1 Processor -- start 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6aec089740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8ab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6aec089740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 0x55ee4a8a8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6aec089740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a8aba90 0x55ee4a8a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6aec089740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55ee4a8ae660 con 0x55ee4a89c380 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6aec089740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55ee4a7bd050 con 0x55ee4a8aba90 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6aec089740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee4a8ae7e0 con 0x55ee4a89cca0 2024-09-16T00:26:52.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6aec089740 10 client.0 _ll_register_callbacks cb 0x55ee4a7fd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 0x55ee4a8a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38572/0 (socket says 172.21.15.78:38572) 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 -- 172.21.15.78:0/2536701568 learned_addr learned my addr 172.21.15.78:0/2536701568 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 -- 172.21.15.78:0/2536701568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 msgr2=0x55ee4a8a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 0x55ee4a8a8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 -- 172.21.15.78:0/2536701568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a8aba90 msgr2=0x55ee4a8a8bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a8aba90 0x55ee4a8a8bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 -- 172.21.15.78:0/2536701568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4a5733c0 con 0x55ee4a89c380 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8ab390 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80155e0 tx=0x7f6ad8015aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121e68694c387f3c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/2536701568 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80178c0 con 0x55ee4a89c380 2024-09-16T00:26:52.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6aec089740 1 -- 172.21.15.78:0/2536701568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad4011de0 con 0x55ee4a89c380 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.012+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/2536701568 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6ad8017a60 con 0x55ee4a89c380 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/2536701568 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ad8028050 con 0x55ee4a89c380 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6aec089740 10 client.17805 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6aec089740 1 -- 172.21.15.78:0/2536701568 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ee4a8ba120 con 0x55ee4a89c380 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/2536701568 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad4011de0 con 0x55ee4a89c380 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/2536701568 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6ad8016370 con 0x55ee4a89c380 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6aec089740 10 client.17805 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6aec089740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6aec089740 20 client.17805 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6aec089740 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ee4a8bb530 0x55ee4a8bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6aec089740 4 client.17805 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.013+0000 7f6aec089740 1 -- 172.21.15.78:0/2536701568 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee4a80f850 con 0x55ee4a8bb530 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.014+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ee4a8bb530 0x55ee4a8bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.014+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ee4a8bb530 0x55ee4a8bd920 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943fc2f27b32d535 server_cookie=d33a860b0185c956 in_seq=0 out_seq=0 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.014+0000 7f6ac2ffd640 10 client.17805 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/2536701568 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee4a80f850 con 0x55ee4a8bb530 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6ac2ffd640 10 client.17805 handle_command_reply: tid=0 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 client.17805 shutdown 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 -- 172.21.15.78:0/2536701568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 msgr2=0x55ee4a8ab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8ab390 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6ad80155e0 tx=0x7f6ad8015aa0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 -- 172.21.15.78:0/2536701568 shutdown_connections 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ee4a8bb530 0x55ee4a8bd920 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a8aba90 0x55ee4a8a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 0x55ee4a8a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6ac2ffd640 0 client.17805 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 --2- 172.21.15.78:0/2536701568 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8ab390 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 -- 172.21.15.78:0/2536701568 >> 172.21.15.78:0/2536701568 conn(0x55ee4a898a30 msgr2=0x55ee4a89e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 -- 172.21.15.78:0/2536701568 shutdown_connections 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 1 -- 172.21.15.78:0/2536701568 wait complete. 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.015+0000 7f6aec089740 20 client.17805 trim_cache size 0 max 16384 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6aec089740 1 Processor -- start 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6aec089740 1 -- start start 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6aec089740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6aec089740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 0x55ee4a8c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6aec089740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a8aba90 0x55ee4a8c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6aec089740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41aca50 con 0x55ee4a89c380 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6aec089740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ae590 con 0x55ee4a8aba90 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6aec089740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ad120 con 0x55ee4a89cca0 2024-09-16T00:26:52.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8b7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38576/0 (socket says 172.21.15.78:38576) 2024-09-16T00:26:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae2ffd640 1 -- 172.21.15.78:0/2340918987 learned_addr learned my addr 172.21.15.78:0/2340918987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/2340918987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 0x55ee4a8c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae2ffd640 1 -- 172.21.15.78:0/2340918987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 msgr2=0x55ee4a8c31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/2340918987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 0x55ee4a8c31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae2ffd640 1 -- 172.21.15.78:0/2340918987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a8aba90 msgr2=0x55ee4a8c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/2340918987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a8aba90 0x55ee4a8c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.032+0000 7f6ae2ffd640 1 -- 172.21.15.78:0/2340918987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4a8a8f60 con 0x55ee4a89c380 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/2340918987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8b7900 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009b50 tx=0x7f6ad8009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1f760770805f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6ac97fa640 1 -- 172.21.15.78:0/2340918987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8025040 con 0x55ee4a89c380 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6ac97fa640 1 -- 172.21.15.78:0/2340918987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad8017240 con 0x55ee4a89c380 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6ac97fa640 1 -- 172.21.15.78:0/2340918987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8017b60 con 0x55ee4a89c380 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6aec089740 1 -- 172.21.15.78:0/2340918987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 msgr2=0x55ee4a8b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6aec089740 1 --2- 172.21.15.78:0/2340918987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8b7900 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009b50 tx=0x7f6ad8009d90 comp rx=0 tx=0).stop 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6aec089740 1 -- 172.21.15.78:0/2340918987 shutdown_connections 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6aec089740 1 --2- 172.21.15.78:0/2340918987 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a8aba90 0x55ee4a8c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6aec089740 1 --2- 172.21.15.78:0/2340918987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a89cca0 0x55ee4a8c31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6aec089740 1 --2- 172.21.15.78:0/2340918987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89c380 0x55ee4a8b7900 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6aec089740 1 -- 172.21.15.78:0/2340918987 >> 172.21.15.78:0/2340918987 conn(0x55ee4a898a30 msgr2=0x55ee4a8a3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.033+0000 7f6aec089740 1 -- 172.21.15.78:0/2340918987 shutdown_connections 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 -- 172.21.15.78:0/2340918987 wait complete. 2024-09-16T00:26:52.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 -- start start 2024-09-16T00:26:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 Processor -- start 2024-09-16T00:26:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89c380 0x55ee4a8afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89cca0 0x55ee4a8afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a8aba90 0x55ee4a8b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41adec0 con 0x55ee4a89cca0 2024-09-16T00:26:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ad7f0 con 0x55ee4a89c380 2024-09-16T00:26:52.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41aaa30 con 0x55ee4a8aba90 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6aec089740 10 client.0 _ll_register_callbacks cb 0x7f6ad80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89cca0 0x55ee4a8afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6ae37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89cca0 0x55ee4a8afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38578/0 (socket says 172.21.15.78:38578) 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.034+0000 7f6ae37fe640 1 -- 172.21.15.78:0/820363031 learned_addr learned my addr 172.21.15.78:0/820363031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89c380 0x55ee4a8afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ae37fe640 1 -- 172.21.15.78:0/820363031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a8aba90 msgr2=0x55ee4a8b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a8aba90 0x55ee4a8b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ae37fe640 1 -- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89c380 msgr2=0x55ee4a8afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89c380 0x55ee4a8afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ae37fe640 1 -- 172.21.15.78:0/820363031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4a925440 con 0x55ee4a89cca0 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89c380 0x55ee4a8afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:52.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ae37fe640 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89cca0 0x55ee4a8afff0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6ad401cc10 tx=0x7f6ad4005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780faf33bb32046e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/820363031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad402c030 con 0x55ee4a89cca0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/820363031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f6ad401c8b0 con 0x55ee4a89cca0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6aec089740 1 -- 172.21.15.78:0/820363031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad8017240 con 0x55ee4a89cca0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/820363031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ad4039050 con 0x55ee4a89cca0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6aec089740 10 client.17811 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6aec089740 1 -- 172.21.15.78:0/820363031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ee4a926730 con 0x55ee4a89cca0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/820363031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad8017240 con 0x55ee4a89cca0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/820363031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6ad4003ab0 con 0x55ee4a89cca0 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6aec089740 10 client.17811 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6aec089740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:26:52.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6aec089740 20 client.17811 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.035+0000 7f6aec089740 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ee4a926bd0 0x55ee4a929000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.036+0000 7f6aec089740 4 client.17811 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.036+0000 7f6aec089740 1 -- 172.21.15.78:0/820363031 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee4a809d60 con 0x55ee4a926bd0 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.036+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ee4a926bd0 0x55ee4a929000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.036+0000 7f6ac2ffd640 10 client.17811 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.036+0000 7f6ae2ffd640 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ee4a926bd0 0x55ee4a929000 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980da44b6f0687d6 server_cookie=13b9630511428454 in_seq=0 out_seq=0 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.036+0000 7f6ac2ffd640 1 -- 172.21.15.78:0/820363031 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ee4a809d60 con 0x55ee4a926bd0 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.036+0000 7f6ac2ffd640 10 client.17811 handle_command_reply: tid=0 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.036+0000 7f6aec089740 1 client.17811 shutdown 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 -- 172.21.15.78:0/820363031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89cca0 msgr2=0x55ee4a8afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89cca0 0x55ee4a8afff0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6ad401cc10 tx=0x7f6ad4005c80 comp rx=0 tx=0).stop 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 -- 172.21.15.78:0/820363031 shutdown_connections 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55ee4a926bd0 0x55ee4a929000 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6ac2ffd640 0 client.17811 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee4a8aba90 0x55ee4a8b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55ee4a89cca0 0x55ee4a8afff0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 --2- 172.21.15.78:0/820363031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55ee4a89c380 0x55ee4a8afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 -- 172.21.15.78:0/820363031 >> 172.21.15.78:0/820363031 conn(0x55ee4a898a30 msgr2=0x55ee4a897350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 -- 172.21.15.78:0/820363031 shutdown_connections 2024-09-16T00:26:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 1 -- 172.21.15.78:0/820363031 wait complete. 2024-09-16T00:26:52.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6aec089740 20 client.17811 trim_cache size 0 max 16384 2024-09-16T00:26:52.059 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:26:52.060 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:26:52.060 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:26:52.060 INFO:teuthology.orchestra.run.smithi078.stdout: "dcb581af-9ec5-451d-92f3-69c1fff3156a": { 2024-09-16T00:26:52.060 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:26:52.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "dcb581af-9ec5-451d-92f3-69c1fff3156a", 2024-09-16T00:26:52.060 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:26:52.060 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:52.060 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:52.061 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:26:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.037+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6acc0a3240 msgr2=0x7f6acc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6acc0a3240 0x7f6acc0a5700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0b9b60 tx=0x7f6adc03d040 comp rx=0 tx=0).stop 2024-09-16T00:26:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 msgr2=0x7f6ae419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 0x7f6ae419cd70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6adc031a20 tx=0x7f6adc031a50 comp rx=0 tx=0).stop 2024-09-16T00:26:52.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/3733385665 shutdown_connections 2024-09-16T00:26:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f6acc0a3240 0x7f6acc0a5700 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae4106f20 0x7f6ae419cd70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ae41027a0 0x7f6ae41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 --2- 172.21.15.78:0/3733385665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6ae4101df0 0x7f6ae41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/3733385665 >> 172.21.15.78:0/3733385665 conn(0x7f6ae40fd8f0 msgr2=0x7f6ae40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/3733385665 shutdown_connections 2024-09-16T00:26:52.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:52.038+0000 7f6ae9cb0640 1 -- 172.21.15.78:0/3733385665 wait complete. 2024-09-16T00:26:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:51 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/787223812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:51 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/787223812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:26:52.786 INFO:tasks.cephfs.filesystem:scrub status for tag:dcb581af-9ec5-451d-92f3-69c1fff3156a - {'path': '/', 'tag': 'dcb581af-9ec5-451d-92f3-69c1fff3156a', 'options': 'recursive,force'} 2024-09-16T00:26:52.867 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:53.017 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:52 smithi078 ceph-mon[31157]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 364 op/s 2024-09-16T00:26:53.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:52 smithi193 ceph-mon[35484]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 364 op/s 2024-09-16T00:26:53.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:52 smithi148 ceph-mon[35599]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.6 MiB/s wr, 364 op/s 2024-09-16T00:26:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 -- 172.21.15.78:0/1408773718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c101420 msgr2=0x7f7c7c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/1408773718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c101420 0x7f7c7c10d950 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009cd0 tx=0x7f7c7402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 -- 172.21.15.78:0/1408773718 shutdown_connections 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/1408773718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c101420 0x7f7c7c10d950 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/1408773718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c7c100a80 0x7f7c7c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/1408773718 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7c106a80 0x7f7c7c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 -- 172.21.15.78:0/1408773718 >> 172.21.15.78:0/1408773718 conn(0x7f7c7c0fc7a0 msgr2=0x7f7c7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 -- 172.21.15.78:0/1408773718 shutdown_connections 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.354+0000 7f7c80ee2640 1 -- 172.21.15.78:0/1408773718 wait complete. 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c80ee2640 1 Processor -- start 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c80ee2640 1 -- start start 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c80ee2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c100a80 0x7f7c7c10e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c80ee2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7c101420 0x7f7c7c10e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c80ee2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c7c106a80 0x7f7c7c10ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c80ee2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c111fa0 con 0x7f7c7c100a80 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c80ee2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c111e20 con 0x7f7c7c101420 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c80ee2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c112120 con 0x7f7c7c106a80 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c100a80 0x7f7c7c10e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c100a80 0x7f7c7c10e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38586/0 (socket says 172.21.15.78:38586) 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7a59c640 1 -- 172.21.15.78:0/3244440573 learned_addr learned my addr 172.21.15.78:0/3244440573 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7c101420 0x7f7c7c10e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c7c106a80 0x7f7c7c10ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7a59c640 1 -- 172.21.15.78:0/3244440573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c7c106a80 msgr2=0x7f7c7c10ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c7c106a80 0x7f7c7c10ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7a59c640 1 -- 172.21.15.78:0/3244440573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7c101420 msgr2=0x7f7c7c10e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7c101420 0x7f7c7c10e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c7a59c640 1 -- 172.21.15.78:0/3244440573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7c1aad90 con 0x7f7c7c100a80 2024-09-16T00:26:53.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.355+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7c101420 0x7f7c7c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.356+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c100a80 0x7f7c7c10e3e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00c890 tx=0x7f7c6c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.356+0000 7f7c737fe640 1 -- 172.21.15.78:0/3244440573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c004220 con 0x7f7c7c100a80 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.356+0000 7f7c80ee2640 1 -- 172.21.15.78:0/3244440573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7c1aafc0 con 0x7f7c7c100a80 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.356+0000 7f7c737fe640 1 -- 172.21.15.78:0/3244440573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c6c0043c0 con 0x7f7c7c100a80 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.356+0000 7f7c737fe640 1 -- 172.21.15.78:0/3244440573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c004a00 con 0x7f7c7c100a80 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.356+0000 7f7c80ee2640 1 -- 172.21.15.78:0/3244440573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7c1ab470 con 0x7f7c7c100a80 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.358+0000 7f7c737fe640 1 -- 172.21.15.78:0/3244440573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7c6c0079b0 con 0x7f7c7c100a80 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.358+0000 7f7c832bb740 1 Processor -- start 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.358+0000 7f7c832bb740 1 -- start start 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47980 0x557ad6d51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d52240 0x557ad6d54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c832bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557ad6cac620 con 0x557ad6d47060 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c832bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557ad6c53e50 con 0x557ad6d52240 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c832bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557ad6ca85e0 con 0x557ad6d47980 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c737fe640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c680a3240 0x7f7c680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d47440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38592/0 (socket says 172.21.15.78:38592) 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 -- 172.21.15.78:0/3582051419 learned_addr learned my addr 172.21.15.78:0/3582051419 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d52240 0x557ad6d54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c737fe640 1 -- 172.21.15.78:0/3244440573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7c6c0cf9d0 con 0x7f7c7c100a80 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c680a3240 0x7f7c680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47980 0x557ad6d51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 -- 172.21.15.78:0/3582051419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47980 msgr2=0x557ad6d51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47980 0x557ad6d51d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 -- 172.21.15.78:0/3582051419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d52240 msgr2=0x557ad6d54630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d52240 0x557ad6d54630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.359+0000 7f7c7ad9d640 1 -- 172.21.15.78:0/3582051419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ad6c44d40 con 0x557ad6d47060 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.360+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d47440 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009430 tx=0x7f7c740039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d912975742ed519 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.360+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c680a3240 0x7f7c680a5700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7c74005fe0 tx=0x7f7c74049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.360+0000 7f7c70ff9640 1 -- 172.21.15.78:0/3582051419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74050070 con 0x557ad6d47060 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.360+0000 7f7c70ff9640 1 -- 172.21.15.78:0/3582051419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c74036040 con 0x557ad6d47060 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.360+0000 7f7c70ff9640 1 -- 172.21.15.78:0/3582051419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74030b00 con 0x557ad6d47060 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- 172.21.15.78:0/3582051419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 msgr2=0x557ad6d47440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d47440 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009430 tx=0x7f7c740039f0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- 172.21.15.78:0/3582051419 shutdown_connections 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d52240 0x557ad6d54630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47980 0x557ad6d51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3582051419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d47440 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- 172.21.15.78:0/3582051419 >> 172.21.15.78:0/3582051419 conn(0x557ad6d435f0 msgr2=0x557ad6d439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- 172.21.15.78:0/3582051419 shutdown_connections 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- 172.21.15.78:0/3582051419 wait complete. 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- start start 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 Processor -- start 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47060 0x557ad6d4f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d4f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d52240 0x557ad6d4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557ad6d54e10 con 0x557ad6d52240 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557ad6c34f10 con 0x557ad6d47980 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.361+0000 7f7c832bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557ad6d54f90 con 0x557ad6d47060 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c832bb740 10 client.0 _ll_register_callbacks cb 0x557ad6c58db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:53.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d52240 0x557ad6d4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d52240 0x557ad6d4fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38602/0 (socket says 172.21.15.78:38602) 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 -- 172.21.15.78:0/3817070764 learned_addr learned my addr 172.21.15.78:0/3817070764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47060 0x557ad6d4f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d4f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 -- 172.21.15.78:0/3817070764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47060 msgr2=0x557ad6d4f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47060 0x557ad6d4f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 -- 172.21.15.78:0/3817070764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 msgr2=0x557ad6d4f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d4f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 -- 172.21.15.78:0/3817070764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ad69bb580 con 0x557ad6d52240 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d52240 0x557ad6d4fbe0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009a30 tx=0x7f7c740305a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78634df99dd66ba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c61ffb640 1 -- 172.21.15.78:0/3817070764 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74050040 con 0x557ad6d52240 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c61ffb640 1 -- 172.21.15.78:0/3817070764 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7c7402e070 con 0x557ad6d52240 2024-09-16T00:26:53.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c832bb740 1 -- 172.21.15.78:0/3817070764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c74036040 con 0x557ad6d52240 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c61ffb640 1 -- 172.21.15.78:0/3817070764 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7c74003dc0 con 0x557ad6d52240 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c832bb740 10 client.17823 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.362+0000 7f7c832bb740 1 -- 172.21.15.78:0/3817070764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557ad6a85960 con 0x557ad6d52240 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c61ffb640 1 -- 172.21.15.78:0/3817070764 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c74036040 con 0x557ad6d52240 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c61ffb640 1 -- 172.21.15.78:0/3817070764 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7c74053070 con 0x557ad6d52240 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c832bb740 10 client.17823 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c832bb740 10 client.17823 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c832bb740 20 client.17823 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557ad6d61c70 0x557ad6d64060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c832bb740 4 client.17823 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c832bb740 1 -- 172.21.15.78:0/3817070764 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ad6cb9b90 con 0x557ad6d61c70 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.363+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557ad6d61c70 0x557ad6d64060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.364+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557ad6d61c70 0x557ad6d64060 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44637a16ac3c1ac0 server_cookie=ee827994b5f1e936 in_seq=0 out_seq=0 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.364+0000 7f7c61ffb640 10 client.17823 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.364+0000 7f7c61ffb640 1 -- 172.21.15.78:0/3817070764 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ad6cb9b90 con 0x557ad6d61c70 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.364+0000 7f7c61ffb640 10 client.17823 handle_command_reply: tid=0 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.364+0000 7f7c832bb740 1 client.17823 shutdown 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.364+0000 7f7c832bb740 1 -- 172.21.15.78:0/3817070764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d52240 msgr2=0x557ad6d4fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.364+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d52240 0x557ad6d4fbe0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009a30 tx=0x7f7c740305a0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 1 -- 172.21.15.78:0/3817070764 shutdown_connections 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557ad6d61c70 0x557ad6d64060 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d52240 0x557ad6d4fbe0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d4f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 1 --2- 172.21.15.78:0/3817070764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d47060 0x557ad6d4f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c61ffb640 0 client.17823 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 1 -- 172.21.15.78:0/3817070764 >> 172.21.15.78:0/3817070764 conn(0x557ad6d435f0 msgr2=0x557ad6d492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 1 -- 172.21.15.78:0/3817070764 shutdown_connections 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 1 -- 172.21.15.78:0/3817070764 wait complete. 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.365+0000 7f7c832bb740 20 client.17823 trim_cache size 0 max 16384 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.381+0000 7f7c832bb740 1 Processor -- start 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.381+0000 7f7c832bb740 1 -- start start 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.381+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.381+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d49ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.381+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 0x557ad6d69660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.381+0000 7f7c832bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c1b5270 con 0x557ad6d47060 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.381+0000 7f7c832bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c1b6db0 con 0x557ad6d47980 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.381+0000 7f7c832bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c1b5940 con 0x557ad6d52240 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d51030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38608/0 (socket says 172.21.15.78:38608) 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 -- 172.21.15.78:0/2543102511 learned_addr learned my addr 172.21.15.78:0/2543102511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 -- 172.21.15.78:0/2543102511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 msgr2=0x557ad6d69660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 0x557ad6d69660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 -- 172.21.15.78:0/2543102511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 msgr2=0x557ad6d49ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d49ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d49ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 -- 172.21.15.78:0/2543102511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ad6d446a0 con 0x557ad6d47060 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d49ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d51030 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c005840 tx=0x7f7c6c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3cac61bfc2c9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c63fff640 1 -- 172.21.15.78:0/2543102511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c01d040 con 0x557ad6d47060 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c63fff640 1 -- 172.21.15.78:0/2543102511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c6c0a48a0 con 0x557ad6d47060 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.382+0000 7f7c63fff640 1 -- 172.21.15.78:0/2543102511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c09b080 con 0x557ad6d47060 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- 172.21.15.78:0/2543102511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 msgr2=0x557ad6d51030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d51030 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c005840 tx=0x7f7c6c086040 comp rx=0 tx=0).stop 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- 172.21.15.78:0/2543102511 shutdown_connections 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 0x557ad6d69660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d49ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 --2- 172.21.15.78:0/2543102511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d51030 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- 172.21.15.78:0/2543102511 >> 172.21.15.78:0/2543102511 conn(0x557ad6d435f0 msgr2=0x557ad6d42750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- 172.21.15.78:0/2543102511 shutdown_connections 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- 172.21.15.78:0/2543102511 wait complete. 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- start start 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 Processor -- start 2024-09-16T00:26:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d5a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d5aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 0x557ad6d573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c1b66e0 con 0x557ad6d47060 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c1b6010 con 0x557ad6d47980 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c1b3250 con 0x557ad6d52240 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.383+0000 7f7c832bb740 10 client.0 _ll_register_callbacks cb 0x7f7c6c004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d5a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d5a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38622/0 (socket says 172.21.15.78:38622) 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 -- 172.21.15.78:0/235080484 learned_addr learned my addr 172.21.15.78:0/235080484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d5aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7ad9d640 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 0x557ad6d573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 -- 172.21.15.78:0/235080484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 msgr2=0x557ad6d573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 0x557ad6d573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 -- 172.21.15.78:0/235080484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 msgr2=0x557ad6d5aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d5aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 -- 172.21.15.78:0/235080484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ad6d44280 con 0x557ad6d47060 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c7a59c640 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d5a590 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0a4370 tx=0x7f7c6c09bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2a570b5ac9c0f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c61ffb640 1 -- 172.21.15.78:0/235080484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c01d070 con 0x557ad6d47060 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c832bb740 1 -- 172.21.15.78:0/235080484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c6c0a48a0 con 0x557ad6d47060 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c61ffb640 1 -- 172.21.15.78:0/235080484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f7c6c007ca0 con 0x557ad6d47060 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c61ffb640 1 -- 172.21.15.78:0/235080484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7c6c0ba350 con 0x557ad6d47060 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c832bb740 10 client.17829 fetch_fsmap learned FSMap version 34 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c832bb740 1 -- 172.21.15.78:0/235080484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557ad6dcd110 con 0x557ad6d47060 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c61ffb640 1 -- 172.21.15.78:0/235080484 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c6c0a48a0 con 0x557ad6d47060 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c61ffb640 1 -- 172.21.15.78:0/235080484 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7c6c0860b0 con 0x557ad6d47060 2024-09-16T00:26:53.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c832bb740 10 client.17829 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:26:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c832bb740 10 client.17829 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:26:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.384+0000 7f7c832bb740 20 client.17829 populate_metadata read hostname 'smithi078' 2024-09-16T00:26:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.385+0000 7f7c832bb740 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557ad6dcd720 0x557ad6dcfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.385+0000 7f7c832bb740 4 client.17829 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f9bfcfb", "release": true, "await": true}] 2024-09-16T00:26:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.385+0000 7f7c832bb740 1 -- 172.21.15.78:0/235080484 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f9bfcfb", "release": true, "await": true}) -- 0x557ad6cb40a0 con 0x557ad6dcd720 2024-09-16T00:26:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.385+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557ad6dcd720 0x557ad6dcfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.385+0000 7f7c61ffb640 10 client.17829 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:53.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.385+0000 7f7c79d9b640 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557ad6dcd720 0x557ad6dcfb40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e92f68b8aa65014 server_cookie=10bacab16bb7be9f in_seq=0 out_seq=0 2024-09-16T00:26:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c61ffb640 1 -- 172.21.15.78:0/235080484 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557ad6cb40a0 con 0x557ad6dcd720 2024-09-16T00:26:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c61ffb640 10 client.17829 handle_command_reply: tid=0 2024-09-16T00:26:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 client.17829 shutdown 2024-09-16T00:26:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 -- 172.21.15.78:0/235080484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 msgr2=0x557ad6d5a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d5a590 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0a4370 tx=0x7f7c6c09bdb0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 -- 172.21.15.78:0/235080484 shutdown_connections 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x557ad6dcd720 0x557ad6dcfb40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ad6d52240 0x557ad6d573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c61ffb640 0 client.17829 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557ad6d47980 0x557ad6d5aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 --2- 172.21.15.78:0/235080484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557ad6d47060 0x557ad6d5a590 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 -- 172.21.15.78:0/235080484 >> 172.21.15.78:0/235080484 conn(0x557ad6d435f0 msgr2=0x557ad6d5d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 -- 172.21.15.78:0/235080484 shutdown_connections 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 1 -- 172.21.15.78:0/235080484 wait complete. 2024-09-16T00:26:53.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.430+0000 7f7c832bb740 20 client.17829 trim_cache size 0 max 16384 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 48, 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: "f9bfcfb": { 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 48, 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:26:53.433 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 131.4, 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:53.434 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:53.435 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:53.435 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:26:53.435 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:26:53.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 -- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c680a3240 msgr2=0x7f7c680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c680a3240 0x7f7c680a5700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7c74005fe0 tx=0x7f7c74049000 comp rx=0 tx=0).stop 2024-09-16T00:26:53.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 -- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c100a80 msgr2=0x7f7c7c10e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c100a80 0x7f7c7c10e3e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00c890 tx=0x7f7c6c00cd50 comp rx=0 tx=0).stop 2024-09-16T00:26:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 -- 172.21.15.78:0/3244440573 shutdown_connections 2024-09-16T00:26:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f7c680a3240 0x7f7c680a5700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c7c106a80 0x7f7c7c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c7c101420 0x7f7c7c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 --2- 172.21.15.78:0/3244440573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c7c100a80 0x7f7c7c10e3e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 -- 172.21.15.78:0/3244440573 >> 172.21.15.78:0/3244440573 conn(0x7f7c7c0fc7a0 msgr2=0x7f7c7c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 -- 172.21.15.78:0/3244440573 shutdown_connections 2024-09-16T00:26:53.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:26:53.431+0000 7f7c80ee2640 1 -- 172.21.15.78:0/3244440573 wait complete. 2024-09-16T00:26:54.292 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f9bfcfb', total seconds elapsed: 15.8 2024-09-16T00:26:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 775.0 s, 250.0 tps, lat 686.328 ms stddev 1739.639 2024-09-16T00:26:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:55 smithi193 ceph-mon[35484]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-09-16T00:26:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:55 smithi148 ceph-mon[35599]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-09-16T00:26:56.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:55 smithi078 ceph-mon[31157]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-09-16T00:26:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:57 smithi193 ceph-mon[35484]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-09-16T00:26:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:57 smithi148 ceph-mon[35599]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-09-16T00:26:58.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:57 smithi078 ceph-mon[31157]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-09-16T00:27:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:59 smithi193 ceph-mon[35484]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-09-16T00:27:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:27:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:27:00.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:26:59 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:27:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:59 smithi148 ceph-mon[35599]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-09-16T00:27:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:27:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:27:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:26:59 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:27:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:59 smithi078 ceph-mon[31157]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-09-16T00:27:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:27:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:27:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:26:59 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:27:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 780.0 s, 624.2 tps, lat 160.851 ms stddev 210.425 2024-09-16T00:27:02.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:01 smithi193 ceph-mon[35484]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-09-16T00:27:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:01 smithi148 ceph-mon[35599]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-09-16T00:27:02.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:01 smithi078 ceph-mon[31157]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-09-16T00:27:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:03 smithi193 ceph-mon[35484]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.5 MiB/s wr, 368 op/s 2024-09-16T00:27:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:03 smithi148 ceph-mon[35599]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.5 MiB/s wr, 368 op/s 2024-09-16T00:27:04.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:03 smithi078 ceph-mon[31157]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.5 MiB/s wr, 368 op/s 2024-09-16T00:27:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 785.0 s, 553.0 tps, lat 184.827 ms stddev 219.055 2024-09-16T00:27:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:05 smithi193 ceph-mon[35484]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2024-09-16T00:27:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:05 smithi148 ceph-mon[35599]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2024-09-16T00:27:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:05 smithi078 ceph-mon[31157]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.5 MiB/s wr, 529 op/s 2024-09-16T00:27:08.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:07 smithi193 ceph-mon[35484]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 6.0 MiB/s wr, 572 op/s 2024-09-16T00:27:08.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:07 smithi148 ceph-mon[35599]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 6.0 MiB/s wr, 572 op/s 2024-09-16T00:27:08.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:07 smithi078 ceph-mon[31157]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 6.0 MiB/s wr, 572 op/s 2024-09-16T00:27:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:09 smithi193 ceph-mon[35484]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-09-16T00:27:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:09 smithi148 ceph-mon[35599]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-09-16T00:27:10.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:09 smithi078 ceph-mon[31157]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 6.1 MiB/s wr, 580 op/s 2024-09-16T00:27:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 790.0 s, 701.2 tps, lat 143.207 ms stddev 203.554 2024-09-16T00:27:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:11 smithi193 ceph-mon[35484]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 7.6 MiB/s wr, 707 op/s 2024-09-16T00:27:12.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:11 smithi148 ceph-mon[35599]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 7.6 MiB/s wr, 707 op/s 2024-09-16T00:27:12.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:11 smithi078 ceph-mon[31157]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 7.6 MiB/s wr, 707 op/s 2024-09-16T00:27:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:13 smithi193 ceph-mon[35484]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-09-16T00:27:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:13 smithi148 ceph-mon[35599]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-09-16T00:27:14.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:13 smithi078 ceph-mon[31157]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-09-16T00:27:15.465 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.5 2024-09-16T00:27:15.466 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:27:15.860 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 795.0 s, 716.3 tps, lat 136.805 ms stddev 150.965 2024-09-16T00:27:15.929 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:16.154 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:15 smithi078 ceph-mon[31157]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-09-16T00:27:16.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:15 smithi193 ceph-mon[35484]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-09-16T00:27:16.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:15 smithi148 ceph-mon[35599]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.2 MiB/s wr, 564 op/s 2024-09-16T00:27:16.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.693+0000 7f1b08095640 1 -- 172.21.15.78:0/3448968796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b00109450 msgr2=0x7f1b00110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:16.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.693+0000 7f1b08095640 1 --2- 172.21.15.78:0/3448968796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b00109450 0x7f1b00110250 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009be0 tx=0x7f1af802a710 comp rx=0 tx=0).stop 2024-09-16T00:27:16.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.694+0000 7f1b08095640 1 -- 172.21.15.78:0/3448968796 shutdown_connections 2024-09-16T00:27:16.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.694+0000 7f1b08095640 1 --2- 172.21.15.78:0/3448968796 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b0011b140 0x7f1b0011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.694+0000 7f1b08095640 1 --2- 172.21.15.78:0/3448968796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b00109830 0x7f1b00109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.694+0000 7f1b08095640 1 --2- 172.21.15.78:0/3448968796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b00109450 0x7f1b00110250 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.694+0000 7f1b08095640 1 -- 172.21.15.78:0/3448968796 >> 172.21.15.78:0/3448968796 conn(0x7f1b0006e140 msgr2=0x7f1b0006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:16.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.697+0000 7f1b08095640 1 -- 172.21.15.78:0/3448968796 shutdown_connections 2024-09-16T00:27:16.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.699+0000 7f1b08095640 1 -- 172.21.15.78:0/3448968796 wait complete. 2024-09-16T00:27:16.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.699+0000 7f1b08095640 1 Processor -- start 2024-09-16T00:27:16.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.699+0000 7f1b08095640 1 -- start start 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b08095640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b00109450 0x7f1b001a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b08095640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b00109830 0x7f1b001a7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b05e31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b00109450 0x7f1b001a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b05e31640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b00109450 0x7f1b001a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47106/0 (socket says 172.21.15.78:47106) 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b05e31640 1 -- 172.21.15.78:0/2261211327 learned_addr learned my addr 172.21.15.78:0/2261211327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b08095640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b0011b140 0x7f1b001abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00120e10 con 0x7f1b00109450 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00120c90 con 0x7f1b0011b140 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.700+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00120f90 con 0x7f1b00109830 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.702+0000 7f1b06632640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b0011b140 0x7f1b001abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.702+0000 7f1b05e31640 1 -- 172.21.15.78:0/2261211327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b00109830 msgr2=0x7f1b001a7830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:27:16.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.702+0000 7f1b05e31640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b00109830 0x7f1b001a7830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.702+0000 7f1b05e31640 1 -- 172.21.15.78:0/2261211327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b0011b140 msgr2=0x7f1b001abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:16.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.702+0000 7f1b05e31640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b0011b140 0x7f1b001abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:16.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.702+0000 7f1b05e31640 1 -- 172.21.15.78:0/2261211327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b001ac2a0 con 0x7f1b00109450 2024-09-16T00:27:16.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.707+0000 7f1b05e31640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b00109450 0x7f1b001a72f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1af8000f80 tx=0x7f1af8030bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:16.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.710+0000 7f1af6ffd640 1 -- 172.21.15.78:0/2261211327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8041020 con 0x7f1b00109450 2024-09-16T00:27:16.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.710+0000 7f1af6ffd640 1 -- 172.21.15.78:0/2261211327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af803c950 con 0x7f1b00109450 2024-09-16T00:27:16.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.710+0000 7f1af6ffd640 1 -- 172.21.15.78:0/2261211327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8036040 con 0x7f1b00109450 2024-09-16T00:27:16.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.710+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b001ac530 con 0x7f1b00109450 2024-09-16T00:27:16.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.710+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001ac9c0 con 0x7f1b00109450 2024-09-16T00:27:16.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.719+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae4005290 con 0x7f1b00109450 2024-09-16T00:27:16.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.719+0000 7f1af6ffd640 1 -- 172.21.15.78:0/2261211327 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1af8044020 con 0x7f1b00109450 2024-09-16T00:27:16.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.721+0000 7f1af6ffd640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1aec0a3450 0x7f1aec0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:16.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.721+0000 7f1af6ffd640 1 -- 172.21.15.78:0/2261211327 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1af80f0cb0 con 0x7f1b00109450 2024-09-16T00:27:16.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.722+0000 7f1b05630640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1aec0a3450 0x7f1aec0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:16.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.730+0000 7f1af6ffd640 1 -- 172.21.15.78:0/2261211327 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1af804b020 con 0x7f1b00109450 2024-09-16T00:27:16.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:16.737+0000 7f1b05630640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1aec0a3450 0x7f1aec0a5910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1af00058c0 tx=0x7f1af0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:17.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.118+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ae4005580 con 0x7f1b00109450 2024-09-16T00:27:17.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.119+0000 7f1af6ffd640 1 -- 172.21.15.78:0/2261211327 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f1af802f3a0 con 0x7f1b00109450 2024-09-16T00:27:17.119 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:27:17.120 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:27:17.120 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1aec0a3450 msgr2=0x7f1aec0a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1aec0a3450 0x7f1aec0a5910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1af00058c0 tx=0x7f1af0005850 comp rx=0 tx=0).stop 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b00109450 msgr2=0x7f1b001a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b00109450 0x7f1b001a72f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1af8000f80 tx=0x7f1af8030bb0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 shutdown_connections 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1aec0a3450 0x7f1aec0a5910 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1b0011b140 0x7f1b001abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b00109830 0x7f1b001a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 --2- 172.21.15.78:0/2261211327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b00109450 0x7f1b001a72f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 >> 172.21.15.78:0/2261211327 conn(0x7f1b0006e140 msgr2=0x7f1b0011bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 shutdown_connections 2024-09-16T00:27:17.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:17.121+0000 7f1b08095640 1 -- 172.21.15.78:0/2261211327 wait complete. 2024-09-16T00:27:18.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:17 smithi193 ceph-mon[35484]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 512 op/s 2024-09-16T00:27:18.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:17 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2261211327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:18.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:17 smithi148 ceph-mon[35599]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 512 op/s 2024-09-16T00:27:18.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:17 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2261211327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:17 smithi078 ceph-mon[31157]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.5 MiB/s wr, 512 op/s 2024-09-16T00:27:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:17 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2261211327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:19 smithi193 ceph-mon[35484]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-09-16T00:27:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:19 smithi148 ceph-mon[35599]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-09-16T00:27:20.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:19 smithi078 ceph-mon[31157]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-09-16T00:27:20.784 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0' 2024-09-16T00:27:20.785 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0 2024-09-16T00:27:20.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 800.0 s, 423.6 tps, lat 227.516 ms stddev 262.846 2024-09-16T00:27:21.228 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.976+0000 7fe392589640 1 -- 172.21.15.78:0/3457815477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 msgr2=0x7fe38c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.976+0000 7fe392589640 1 --2- 172.21.15.78:0/3457815477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 0x7fe38c06ee70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe384009ca0 tx=0x7fe38402a8f0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.976+0000 7fe392589640 1 -- 172.21.15.78:0/3457815477 shutdown_connections 2024-09-16T00:27:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.976+0000 7fe392589640 1 --2- 172.21.15.78:0/3457815477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe38c11b140 0x7fe38c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.976+0000 7fe392589640 1 --2- 172.21.15.78:0/3457815477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 0x7fe38c06ee70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.976+0000 7fe392589640 1 --2- 172.21.15.78:0/3457815477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c110190 0x7fe38c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.976+0000 7fe392589640 1 -- 172.21.15.78:0/3457815477 >> 172.21.15.78:0/3457815477 conn(0x7fe38c073180 msgr2=0x7fe38c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:21.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.984+0000 7fe392589640 1 -- 172.21.15.78:0/3457815477 shutdown_connections 2024-09-16T00:27:21.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.984+0000 7fe392589640 1 -- 172.21.15.78:0/3457815477 wait complete. 2024-09-16T00:27:21.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.985+0000 7fe392589640 1 Processor -- start 2024-09-16T00:27:21.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.985+0000 7fe392589640 1 -- start start 2024-09-16T00:27:21.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe392589640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 0x7fe38c1a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe392589640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c110190 0x7fe38c1a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe392589640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe38c11b140 0x7fe38c1aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe392589640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c120c00 con 0x7fe38c11b140 2024-09-16T00:27:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe392589640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c120a80 con 0x7fe38c110190 2024-09-16T00:27:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe392589640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c120d80 con 0x7fe38c06ea30 2024-09-16T00:27:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe390d86640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c110190 0x7fe38c1a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe391587640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 0x7fe38c1a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe390d86640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c110190 0x7fe38c1a76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:53404/0 (socket says 172.21.15.78:53404) 2024-09-16T00:27:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe391587640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 0x7fe38c1a7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48046/0 (socket says 172.21.15.78:48046) 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe390d86640 1 -- 172.21.15.78:0/168248608 learned_addr learned my addr 172.21.15.78:0/168248608 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.989+0000 7fe391d88640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe38c11b140 0x7fe38c1aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.990+0000 7fe391587640 1 -- 172.21.15.78:0/168248608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c110190 msgr2=0x7fe38c1a76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.990+0000 7fe391587640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c110190 0x7fe38c1a76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.990+0000 7fe391587640 1 -- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe38c11b140 msgr2=0x7fe38c1aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.990+0000 7fe391587640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe38c11b140 0x7fe38c1aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.990+0000 7fe391587640 1 -- 172.21.15.78:0/168248608 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c1ac140 con 0x7fe38c06ea30 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.991+0000 7fe391587640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 0x7fe38c1a7190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe38800d6d0 tx=0x7fe38800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.991+0000 7fe3827fc640 1 -- 172.21.15.78:0/168248608 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388017070 con 0x7fe38c06ea30 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.991+0000 7fe392589640 1 -- 172.21.15.78:0/168248608 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c1ac430 con 0x7fe38c06ea30 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.991+0000 7fe392589640 1 -- 172.21.15.78:0/168248608 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c1b3d10 con 0x7fe38c06ea30 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.991+0000 7fe3827fc640 1 -- 172.21.15.78:0/168248608 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe388009820 con 0x7fe38c06ea30 2024-09-16T00:27:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.991+0000 7fe3827fc640 1 -- 172.21.15.78:0/168248608 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388004ce0 con 0x7fe38c06ea30 2024-09-16T00:27:21.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.995+0000 7fe3827fc640 1 -- 172.21.15.78:0/168248608 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe388005220 con 0x7fe38c06ea30 2024-09-16T00:27:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.996+0000 7fe3827fc640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3780a3380 0x7fe3780a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.997+0000 7fe3827fc640 1 -- 172.21.15.78:0/168248608 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe3880cf3a0 con 0x7fe38c06ea30 2024-09-16T00:27:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.998+0000 7fe390d86640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3780a3380 0x7fe3780a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.998+0000 7fe395bc4740 1 Processor -- start 2024-09-16T00:27:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe395bc4740 1 -- start start 2024-09-16T00:27:21.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cfe4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cfef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe395bc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563a5cf4a2a0 con 0x563a5cfe5220 2024-09-16T00:27:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe395bc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563a5cef1b10 con 0x563a5cfe4900 2024-09-16T00:27:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe395bc4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a5cf46260 con 0x563a5cfefae0 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe391d88640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe391d88640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48058/0 (socket says 172.21.15.78:48058) 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:21.999+0000 7fe391d88640 1 -- 172.21.15.78:0/280608232 learned_addr learned my addr 172.21.15.78:0/280608232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.000+0000 7fe391d88640 1 -- 172.21.15.78:0/280608232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 msgr2=0x563a5cfe4ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.000+0000 7fe391d88640 1 --2- 172.21.15.78:0/280608232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cfe4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.000+0000 7fe391d88640 1 -- 172.21.15.78:0/280608232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 msgr2=0x563a5cfef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.000+0000 7fe391d88640 1 --2- 172.21.15.78:0/280608232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cfef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.000+0000 7fe391d88640 1 -- 172.21.15.78:0/280608232 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5ccbbb90 con 0x563a5cfefae0 2024-09-16T00:27:22.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.001+0000 7fe391d88640 1 --2- 172.21.15.78:0/280608232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff1ed0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00b600 tx=0x7fe37c00bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b73cb8c40a1c7dcd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.001+0000 7fe3777fe640 1 -- 172.21.15.78:0/280608232 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c003b20 con 0x563a5cfefae0 2024-09-16T00:27:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.001+0000 7fe3777fe640 1 -- 172.21.15.78:0/280608232 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe37c015070 con 0x563a5cfefae0 2024-09-16T00:27:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.002+0000 7fe395bc4740 1 -- 172.21.15.78:0/280608232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 msgr2=0x563a5cff1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.002+0000 7fe395bc4740 1 --2- 172.21.15.78:0/280608232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff1ed0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00b600 tx=0x7fe37c00bac0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.004+0000 7fe390d86640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3780a3380 0x7fe3780a5840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe38c11aca0 tx=0x7fe38402adf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:22.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.006+0000 7fe395bc4740 1 -- 172.21.15.78:0/280608232 shutdown_connections 2024-09-16T00:27:22.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.006+0000 7fe395bc4740 1 --2- 172.21.15.78:0/280608232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff1ed0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.006+0000 7fe395bc4740 1 --2- 172.21.15.78:0/280608232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cfef5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.006+0000 7fe395bc4740 1 --2- 172.21.15.78:0/280608232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cfe4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.006+0000 7fe395bc4740 1 -- 172.21.15.78:0/280608232 >> 172.21.15.78:0/280608232 conn(0x563a5cfe0e90 msgr2=0x563a5cfe1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:22.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.009+0000 7fe395bc4740 1 -- 172.21.15.78:0/280608232 shutdown_connections 2024-09-16T00:27:22.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.010+0000 7fe395bc4740 1 -- 172.21.15.78:0/280608232 wait complete. 2024-09-16T00:27:22.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.010+0000 7fe395bc4740 1 -- start start 2024-09-16T00:27:22.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.010+0000 7fe395bc4740 1 Processor -- start 2024-09-16T00:27:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.011+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cfec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.011+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe5220 0x563a5cfecee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.011+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5cfed420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.011+0000 7fe395bc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563a5cff2700 con 0x563a5cfefae0 2024-09-16T00:27:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.011+0000 7fe395bc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563a5ced26e0 con 0x563a5cfe4900 2024-09-16T00:27:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.011+0000 7fe395bc4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a5cff2880 con 0x563a5cfe5220 2024-09-16T00:27:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.011+0000 7fe395bc4740 10 client.0 _ll_register_callbacks cb 0x563a5cef6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:22.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:21 smithi078 ceph-mon[31157]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.0 MiB/s wr, 695 op/s 2024-09-16T00:27:22.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.021+0000 7fe390d86640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe5220 0x563a5cfecee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.021+0000 7fe390d86640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe5220 0x563a5cfecee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48060/0 (socket says 172.21.15.78:48060) 2024-09-16T00:27:22.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.021+0000 7fe390d86640 1 -- 172.21.15.78:0/37165869 learned_addr learned my addr 172.21.15.78:0/37165869 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:22.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.021+0000 7fe391587640 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cfec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.022+0000 7fe391d88640 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5cfed420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.023+0000 7fe391d88640 1 -- 172.21.15.78:0/37165869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe5220 msgr2=0x563a5cfecee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:22.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.023+0000 7fe391d88640 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe5220 0x563a5cfecee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.023+0000 7fe391d88640 1 -- 172.21.15.78:0/37165869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 msgr2=0x563a5cfec9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:22.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.023+0000 7fe391d88640 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cfec9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.023+0000 7fe391d88640 1 -- 172.21.15.78:0/37165869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5d00cc80 con 0x563a5cfefae0 2024-09-16T00:27:22.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.024+0000 7fe391d88640 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5cfed420 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe37c018000 tx=0x7fe37c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7999c4b2d61c39b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:22.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.024+0000 7fe374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:22.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.030+0000 7fe374ff9640 1 -- 172.21.15.78:0/37165869 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c004f10 con 0x563a5cfefae0 2024-09-16T00:27:22.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.030+0000 7fe374ff9640 1 -- 172.21.15.78:0/37165869 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe37c0050b0 con 0x563a5cfefae0 2024-09-16T00:27:22.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.030+0000 7fe395bc4740 1 -- 172.21.15.78:0/37165869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe37c015070 con 0x563a5cfefae0 2024-09-16T00:27:22.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.030+0000 7fe374ff9640 1 -- 172.21.15.78:0/37165869 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe37c02c050 con 0x563a5cfefae0 2024-09-16T00:27:22.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.031+0000 7fe395bc4740 10 client.17841 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:22.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.031+0000 7fe395bc4740 1 -- 172.21.15.78:0/37165869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a5d00f610 con 0x563a5cfefae0 2024-09-16T00:27:22.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.031+0000 7fe374ff9640 1 -- 172.21.15.78:0/37165869 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe37c015070 con 0x563a5cfefae0 2024-09-16T00:27:22.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.031+0000 7fe374ff9640 1 -- 172.21.15.78:0/37165869 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe37c019420 con 0x563a5cfefae0 2024-09-16T00:27:22.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.031+0000 7fe395bc4740 10 client.17841 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:22.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.031+0000 7fe395bc4740 10 client.17841 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:27:22.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.032+0000 7fe395bc4740 20 client.17841 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:22.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.032+0000 7fe395bc4740 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a5d010a20 0x563a5d012e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.032+0000 7fe395bc4740 4 client.17841 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:22.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.032+0000 7fe395bc4740 1 -- 172.21.15.78:0/37165869 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a5cf579d0 con 0x563a5d010a20 2024-09-16T00:27:22.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.033+0000 7fe391587640 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a5d010a20 0x563a5d012e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.033+0000 7fe391587640 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a5d010a20 0x563a5d012e10 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0664fd73870cade server_cookie=a0678287d5b9c364 in_seq=0 out_seq=0 2024-09-16T00:27:22.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.033+0000 7fe374ff9640 10 client.17841 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:22.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe374ff9640 1 -- 172.21.15.78:0/37165869 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a5cf579d0 con 0x563a5d010a20 2024-09-16T00:27:22.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe374ff9640 10 client.17841 handle_command_reply: tid=0 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 client.17841 shutdown 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 -- 172.21.15.78:0/37165869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 msgr2=0x563a5cfed420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5cfed420 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe37c018000 tx=0x7fe37c003b20 comp rx=0 tx=0).stop 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 -- 172.21.15.78:0/37165869 shutdown_connections 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a5d010a20 0x563a5d012e10 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5cfed420 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe5220 0x563a5cfecee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 --2- 172.21.15.78:0/37165869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cfec9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.034+0000 7fe395bc4740 1 -- 172.21.15.78:0/37165869 >> 172.21.15.78:0/37165869 conn(0x563a5cfe0e90 msgr2=0x563a5cfe6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:22.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.035+0000 7fe374ff9640 0 client.17841 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:22.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.035+0000 7fe395bc4740 1 -- 172.21.15.78:0/37165869 shutdown_connections 2024-09-16T00:27:22.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.037+0000 7fe395bc4740 1 -- 172.21.15.78:0/37165869 wait complete. 2024-09-16T00:27:22.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.038+0000 7fe395bc4740 20 client.17841 trim_cache size 0 max 16384 2024-09-16T00:27:22.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.067+0000 7fe395bc4740 1 Processor -- start 2024-09-16T00:27:22.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.067+0000 7fe395bc4740 1 -- start start 2024-09-16T00:27:22.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.067+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe4900 0x563a5cfed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.067+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe5220 0x563a5cfdfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.067+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5d0183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.067+0000 7fe391587640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe4900 0x563a5cfed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.067+0000 7fe391587640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe4900 0x563a5cfed270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:48066/0 (socket says 172.21.15.78:48066) 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.067+0000 7fe391d88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5d0183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe391d88640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5d0183b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47152/0 (socket says 172.21.15.78:47152) 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe391587640 1 -- 172.21.15.78:0/1808028526 learned_addr learned my addr 172.21.15.78:0/1808028526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe395bc4740 1 -- 172.21.15.78:0/1808028526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe37000fc00 con 0x563a5cfefae0 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe395bc4740 1 -- 172.21.15.78:0/1808028526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe370011740 con 0x563a5cfe5220 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe395bc4740 1 -- 172.21.15.78:0/1808028526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe3700102d0 con 0x563a5cfe4900 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe391587640 1 -- 172.21.15.78:0/1808028526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe5220 msgr2=0x563a5cfdfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe391587640 1 --2- 172.21.15.78:0/1808028526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe5220 0x563a5cfdfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe391587640 1 -- 172.21.15.78:0/1808028526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 msgr2=0x563a5d0183b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe391587640 1 --2- 172.21.15.78:0/1808028526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5d0183b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.068+0000 7fe391587640 1 -- 172.21.15.78:0/1808028526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5cfe5aa0 con 0x563a5cfe4900 2024-09-16T00:27:22.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.069+0000 7fe391587640 1 --2- 172.21.15.78:0/1808028526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe4900 0x563a5cfed270 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe388098080 tx=0x7fe3880ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633e179476057c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:22.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.074+0000 7fe376ffd640 1 -- 172.21.15.78:0/1808028526 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3880b4070 con 0x563a5cfe4900 2024-09-16T00:27:22.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.074+0000 7fe376ffd640 1 -- 172.21.15.78:0/1808028526 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3880a4270 con 0x563a5cfe4900 2024-09-16T00:27:22.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.074+0000 7fe376ffd640 1 -- 172.21.15.78:0/1808028526 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388020040 con 0x563a5cfe4900 2024-09-16T00:27:22.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.076+0000 7fe395bc4740 1 -- 172.21.15.78:0/1808028526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe4900 msgr2=0x563a5cfed270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:22.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.076+0000 7fe395bc4740 1 --2- 172.21.15.78:0/1808028526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe4900 0x563a5cfed270 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe388098080 tx=0x7fe3880ce040 comp rx=0 tx=0).stop 2024-09-16T00:27:22.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.078+0000 7fe395bc4740 1 -- 172.21.15.78:0/1808028526 shutdown_connections 2024-09-16T00:27:22.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.078+0000 7fe395bc4740 1 --2- 172.21.15.78:0/1808028526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfefae0 0x563a5d0183b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.078+0000 7fe395bc4740 1 --2- 172.21.15.78:0/1808028526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe5220 0x563a5cfdfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.078+0000 7fe395bc4740 1 --2- 172.21.15.78:0/1808028526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfe4900 0x563a5cfed270 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.078+0000 7fe395bc4740 1 -- 172.21.15.78:0/1808028526 >> 172.21.15.78:0/1808028526 conn(0x563a5cfe0e90 msgr2=0x563a5cfdf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:22.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.080+0000 7fe395bc4740 1 -- 172.21.15.78:0/1808028526 shutdown_connections 2024-09-16T00:27:22.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.080+0000 7fe395bc4740 1 -- 172.21.15.78:0/1808028526 wait complete. 2024-09-16T00:27:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.081+0000 7fe395bc4740 1 -- start start 2024-09-16T00:27:22.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.081+0000 7fe395bc4740 1 Processor -- start 2024-09-16T00:27:22.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.081+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cff7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.082+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cff80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.082+0000 7fe395bc4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.082+0000 7fe395bc4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe370011070 con 0x563a5cfe5220 2024-09-16T00:27:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.082+0000 7fe395bc4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe3700109a0 con 0x563a5cfe4900 2024-09-16T00:27:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.082+0000 7fe395bc4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe37000dbe0 con 0x563a5cfefae0 2024-09-16T00:27:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.082+0000 7fe395bc4740 10 client.0 _ll_register_callbacks cb 0x7fe37c0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.083+0000 7fe390d86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cff80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.083+0000 7fe390d86640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cff80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47160/0 (socket says 172.21.15.78:47160) 2024-09-16T00:27:22.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.083+0000 7fe390d86640 1 -- 172.21.15.78:0/2942026374 learned_addr learned my addr 172.21.15.78:0/2942026374 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.084+0000 7fe391d88640 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.084+0000 7fe391587640 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cff7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.087+0000 7fe390d86640 1 -- 172.21.15.78:0/2942026374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 msgr2=0x563a5cff4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:22.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.087+0000 7fe390d86640 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.087+0000 7fe390d86640 1 -- 172.21.15.78:0/2942026374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 msgr2=0x563a5cff7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:22.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.087+0000 7fe390d86640 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cff7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.087+0000 7fe390d86640 1 -- 172.21.15.78:0/2942026374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5cff5090 con 0x563a5cfe5220 2024-09-16T00:27:22.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.087+0000 7fe390d86640 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cff80c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe384005e30 tx=0x7fe38402f530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2f1e86383e6663 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.091+0000 7fe391d88640 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:27:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.091+0000 7fe374ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.091+0000 7fe374ff9640 1 -- 172.21.15.78:0/2942026374 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384004040 con 0x563a5cfe5220 2024-09-16T00:27:22.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.091+0000 7fe395bc4740 1 -- 172.21.15.78:0/2942026374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3880a4270 con 0x563a5cfe5220 2024-09-16T00:27:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.091+0000 7fe374ff9640 1 -- 172.21.15.78:0/2942026374 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe3840041e0 con 0x563a5cfe5220 2024-09-16T00:27:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.095+0000 7fe374ff9640 1 -- 172.21.15.78:0/2942026374 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe384004400 con 0x563a5cfe5220 2024-09-16T00:27:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.096+0000 7fe395bc4740 10 client.17847 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.096+0000 7fe395bc4740 1 -- 172.21.15.78:0/2942026374 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a5d07ba60 con 0x563a5cfe5220 2024-09-16T00:27:22.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.101+0000 7fe374ff9640 1 -- 172.21.15.78:0/2942026374 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe38402d070 con 0x563a5cfe5220 2024-09-16T00:27:22.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.101+0000 7fe374ff9640 1 -- 172.21.15.78:0/2942026374 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe384043040 con 0x563a5cfe5220 2024-09-16T00:27:22.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.101+0000 7fe395bc4740 10 client.17847 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:22.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.101+0000 7fe395bc4740 10 client.17847 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:27:22.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.101+0000 7fe395bc4740 20 client.17847 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:22.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.101+0000 7fe395bc4740 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a5d07c220 0x563a5d07e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.101+0000 7fe395bc4740 4 client.17847 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}] 2024-09-16T00:27:22.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.101+0000 7fe395bc4740 1 -- 172.21.15.78:0/2942026374 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}) -- 0x7fe3880a4270 con 0x563a5d07c220 2024-09-16T00:27:22.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.102+0000 7fe391587640 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a5d07c220 0x563a5d07e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.108+0000 7fe391587640 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a5d07c220 0x563a5d07e640 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde203798b2045d3 server_cookie=9e5fb0bbbf1bb857 in_seq=0 out_seq=0 2024-09-16T00:27:22.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.109+0000 7fe374ff9640 10 client.17847 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:21 smithi193 ceph-mon[35484]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.0 MiB/s wr, 695 op/s 2024-09-16T00:27:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:21 smithi148 ceph-mon[35599]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.0 MiB/s wr, 695 op/s 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe374ff9640 1 -- 172.21.15.78:0/2942026374 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fe3880a4270 con 0x563a5d07c220 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe374ff9640 10 client.17847 handle_command_reply: tid=0 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 client.17847 shutdown 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 -- 172.21.15.78:0/2942026374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 msgr2=0x563a5cff80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cff80c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe384005e30 tx=0x7fe38402f530 comp rx=0 tx=0).stop 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 -- 172.21.15.78:0/2942026374 shutdown_connections 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x563a5d07c220 0x563a5d07e640 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a5cfefae0 0x563a5cff4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563a5cfe5220 0x563a5cff80c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 --2- 172.21.15.78:0/2942026374 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563a5cfe4900 0x563a5cff7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe395bc4740 1 -- 172.21.15.78:0/2942026374 >> 172.21.15.78:0/2942026374 conn(0x563a5cfe0e90 msgr2=0x563a5cfe0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.367+0000 7fe374ff9640 0 client.17847 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe395bc4740 1 -- 172.21.15.78:0/2942026374 shutdown_connections 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe395bc4740 1 -- 172.21.15.78:0/2942026374 wait complete. 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe395bc4740 20 client.17847 trim_cache size 0 max 16384 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:27:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 50, 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "15f44a70": { 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 50, 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 146.5, 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:27:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:27:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:27:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:27:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:22.372 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:27:22.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 -- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3780a3380 msgr2=0x7fe3780a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:22.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3780a3380 0x7fe3780a5840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe38c11aca0 tx=0x7fe38402adf0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 -- 172.21.15.78:0/168248608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 msgr2=0x7fe38c1a7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:22.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 0x7fe38c1a7190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe38800d6d0 tx=0x7fe38800db90 comp rx=0 tx=0).stop 2024-09-16T00:27:22.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 -- 172.21.15.78:0/168248608 shutdown_connections 2024-09-16T00:27:22.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe3780a3380 0x7fe3780a5840 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe38c11b140 0x7fe38c1aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c110190 0x7fe38c1a76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 --2- 172.21.15.78:0/168248608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe38c06ea30 0x7fe38c1a7190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:22.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 -- 172.21.15.78:0/168248608 >> 172.21.15.78:0/168248608 conn(0x7fe38c073180 msgr2=0x7fe38c1193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:22.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 -- 172.21.15.78:0/168248608 shutdown_connections 2024-09-16T00:27:22.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:22.368+0000 7fe392589640 1 -- 172.21.15.78:0/168248608 wait complete. 2024-09-16T00:27:22.787 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:27:24.169 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:23 smithi078 ceph-mon[31157]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 MiB/s wr, 584 op/s 2024-09-16T00:27:24.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:23 smithi193 ceph-mon[35484]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 MiB/s wr, 584 op/s 2024-09-16T00:27:24.202 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '15f44a70', quiesce took 8.7 seconds. Will release after: 17.8 2024-09-16T00:27:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:23 smithi148 ceph-mon[35599]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.6 MiB/s wr, 584 op/s 2024-09-16T00:27:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 -- 172.21.15.78:0/3193663400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc103650 msgr2=0x7f20dc10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 --2- 172.21.15.78:0/3193663400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc103650 0x7f20dc10bb40 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009900 tx=0x7f20c402a480 comp rx=0 tx=0).stop 2024-09-16T00:27:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 -- 172.21.15.78:0/3193663400 shutdown_connections 2024-09-16T00:27:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 --2- 172.21.15.78:0/3193663400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20dc10c160 0x7f20dc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 --2- 172.21.15.78:0/3193663400 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc103650 0x7f20dc10bb40 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 --2- 172.21.15.78:0/3193663400 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc102d30 0x7f20dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 -- 172.21.15.78:0/3193663400 >> 172.21.15.78:0/3193663400 conn(0x7f20dc0fc790 msgr2=0x7f20dc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 -- 172.21.15.78:0/3193663400 shutdown_connections 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 -- 172.21.15.78:0/3193663400 wait complete. 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 Processor -- start 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.772+0000 7f20e1769640 1 -- start start 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20e1769640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc102d30 0x7f20dc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20e1769640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20dc103650 0x7f20dc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20e1769640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc10c160 0x7f20dc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20e1769640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc111f60 con 0x7f20dc10c160 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20e1769640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc111de0 con 0x7f20dc102d30 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20e1769640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1120e0 con 0x7f20dc103650 2024-09-16T00:27:24.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc10c160 0x7f20dc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc10c160 0x7f20dc101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47182/0 (socket says 172.21.15.78:47182) 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 -- 172.21.15.78:0/1827180011 learned_addr learned my addr 172.21.15.78:0/1827180011 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20daffd640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc102d30 0x7f20dc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20dc103650 0x7f20dc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 -- 172.21.15.78:0/1827180011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20dc103650 msgr2=0x7f20dc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20dc103650 0x7f20dc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 -- 172.21.15.78:0/1827180011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc102d30 msgr2=0x7f20dc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc102d30 0x7f20dc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 -- 172.21.15.78:0/1827180011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc1a68c0 con 0x7f20dc10c160 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20db7fe640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc10c160 0x7f20dc101410 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f20d400cad0 tx=0x7f20d400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20d8ff9640 1 -- 172.21.15.78:0/1827180011 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d400ad60 con 0x7f20dc10c160 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.773+0000 7f20d8ff9640 1 -- 172.21.15.78:0/1827180011 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d4005020 con 0x7f20dc10c160 2024-09-16T00:27:24.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.774+0000 7f20e1769640 1 -- 172.21.15.78:0/1827180011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc1a6bb0 con 0x7f20dc10c160 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.774+0000 7f20d8ff9640 1 -- 172.21.15.78:0/1827180011 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d400f6b0 con 0x7f20dc10c160 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.774+0000 7f20e1769640 1 -- 172.21.15.78:0/1827180011 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1a70f0 con 0x7f20dc10c160 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20d8ff9640 1 -- 172.21.15.78:0/1827180011 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f20d4005560 con 0x7f20dc10c160 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20e3b42740 1 Processor -- start 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20e3b42740 1 -- start start 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 0x560c383a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a1dc0 0x560c383ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20e3b42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560c383072e0 con 0x560c383ac680 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20e3b42740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560c382aece0 con 0x560c383a1dc0 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20e3b42740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560c383032a0 con 0x560c383a14a0 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20d8ff9640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20c80a32c0 0x7f20c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.776+0000 7f20d27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47194/0 (socket says 172.21.15.78:47194) 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 -- 172.21.15.78:0/1222489274 learned_addr learned my addr 172.21.15.78:0/1222489274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 0x560c383a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20daffd640 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a1dc0 0x560c383ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20db7fe640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20c80a32c0 0x7f20c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 -- 172.21.15.78:0/1222489274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 msgr2=0x560c383a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 0x560c383a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d8ff9640 1 -- 172.21.15.78:0/1827180011 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f20d40cfc00 con 0x7f20dc10c160 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 -- 172.21.15.78:0/1222489274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a1dc0 msgr2=0x560c383ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a1dc0 0x560c383ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 -- 172.21.15.78:0/1222489274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c38293440 con 0x560c383ac680 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383aea70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f20c40057c0 tx=0x7f20c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaecafd2866646dd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20db7fe640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20c80a32c0 0x7f20c80a5780 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f20d4098890 tx=0x7f20d40054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d17fa640 1 -- 172.21.15.78:0/1222489274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4035040 con 0x560c383ac680 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d17fa640 1 -- 172.21.15.78:0/1222489274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c40035f0 con 0x560c383ac680 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.777+0000 7f20d17fa640 1 -- 172.21.15.78:0/1222489274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c403c650 con 0x560c383ac680 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 -- 172.21.15.78:0/1222489274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 msgr2=0x560c383aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383aea70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f20c40057c0 tx=0x7f20c4000bc0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 -- 172.21.15.78:0/1222489274 shutdown_connections 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383aea70 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a1dc0 0x560c383ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1222489274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 0x560c383a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 -- 172.21.15.78:0/1222489274 >> 172.21.15.78:0/1222489274 conn(0x560c3839da30 msgr2=0x560c3839de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 -- 172.21.15.78:0/1222489274 shutdown_connections 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 -- 172.21.15.78:0/1222489274 wait complete. 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.778+0000 7f20e3b42740 1 -- start start 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20e3b42740 1 Processor -- start 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a14a0 0x560c383abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 0x560c383a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383a9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20e3b42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560c383af250 con 0x560c383ac680 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20e3b42740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560c382c2050 con 0x560c383a14a0 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20e3b42740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560c383af3d0 con 0x560c383a1dc0 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20e3b42740 10 client.0 _ll_register_callbacks cb 0x560c38302160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:24.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383a9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47202/0 (socket says 172.21.15.78:47202) 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 -- 172.21.15.78:0/1060698643 learned_addr learned my addr 172.21.15.78:0/1060698643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20daffd640 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 0x560c383a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a14a0 0x560c383abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 -- 172.21.15.78:0/1060698643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 msgr2=0x560c383a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 0x560c383a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 -- 172.21.15.78:0/1060698643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a14a0 msgr2=0x560c383abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a14a0 0x560c383abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 -- 172.21.15.78:0/1060698643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c380783c0 con 0x560c383ac680 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20d27fc640 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383a9720 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009900 tx=0x7f20c402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc208a7e484d8bd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.779+0000 7f20c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:24.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20c27fc640 1 -- 172.21.15.78:0/1060698643 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4035070 con 0x560c383ac680 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20c27fc640 1 -- 172.21.15.78:0/1060698643 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f20c4003f70 con 0x560c383ac680 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 1 -- 172.21.15.78:0/1060698643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c40035f0 con 0x560c383ac680 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20c27fc640 1 -- 172.21.15.78:0/1060698643 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f20c4004190 con 0x560c383ac680 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 10 client.17859 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 1 -- 172.21.15.78:0/1060698643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560c383babd0 con 0x560c383ac680 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20c27fc640 1 -- 172.21.15.78:0/1060698643 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c40035f0 con 0x560c383ac680 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20c27fc640 1 -- 172.21.15.78:0/1060698643 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f20c404a040 con 0x560c383ac680 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 10 client.17859 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 20 client.17859 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560c383bc080 0x560c383be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 4 client.17859 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.780+0000 7f20e3b42740 1 -- 172.21.15.78:0/1060698643 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c38314850 con 0x560c383bc080 2024-09-16T00:27:24.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.781+0000 7f20daffd640 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560c383bc080 0x560c383be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.781+0000 7f20daffd640 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560c383bc080 0x560c383be4b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220a8c2d3cc83239 server_cookie=6a6d038c8c55a186 in_seq=0 out_seq=0 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.781+0000 7f20c27fc640 10 client.17859 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20c27fc640 1 -- 172.21.15.78:0/1060698643 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c38314850 con 0x560c383bc080 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20c27fc640 10 client.17859 handle_command_reply: tid=0 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 client.17859 shutdown 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 -- 172.21.15.78:0/1060698643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 msgr2=0x560c383a9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383a9720 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009900 tx=0x7f20c402a480 comp rx=0 tx=0).stop 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 -- 172.21.15.78:0/1060698643 shutdown_connections 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560c383bc080 0x560c383be4b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383ac680 0x560c383a9720 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 0x560c383a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20c27fc640 0 client.17859 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:24.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 --2- 172.21.15.78:0/1060698643 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c383a14a0 0x560c383abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 -- 172.21.15.78:0/1060698643 >> 172.21.15.78:0/1060698643 conn(0x560c3839da30 msgr2=0x560c383a3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 -- 172.21.15.78:0/1060698643 shutdown_connections 2024-09-16T00:27:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 1 -- 172.21.15.78:0/1060698643 wait complete. 2024-09-16T00:27:24.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.782+0000 7f20e3b42740 20 client.17859 trim_cache size 0 max 16384 2024-09-16T00:27:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.799+0000 7f20e3b42740 1 Processor -- start 2024-09-16T00:27:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.799+0000 7f20e3b42740 1 -- start start 2024-09-16T00:27:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 0x560c383b8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a14a0 0x560c383ab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 0x560c383c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20e3b42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1b0e20 con 0x560c383a14a0 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20e3b42740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1b2960 con 0x560c3839da30 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20e3b42740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1b14f0 con 0x560c383a1dc0 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a14a0 0x560c383ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a14a0 0x560c383ab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47210/0 (socket says 172.21.15.78:47210) 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 -- 172.21.15.78:0/156724090 learned_addr learned my addr 172.21.15.78:0/156724090 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20d27fc640 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 0x560c383c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20d27fc640 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 0x560c383b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 -- 172.21.15.78:0/156724090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 msgr2=0x560c383c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 0x560c383c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 -- 172.21.15.78:0/156724090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 msgr2=0x560c383b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 0x560c383b8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 -- 172.21.15.78:0/156724090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c383a1bf0 con 0x560c383a14a0 2024-09-16T00:27:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.800+0000 7f20daffd640 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a14a0 0x560c383ab150 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x560c383a8a70 tx=0x7f20cc00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec24d14099bc4281 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20d0ff9640 1 -- 172.21.15.78:0/156724090 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc005220 con 0x560c383a14a0 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20d0ff9640 1 -- 172.21.15.78:0/156724090 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20cc004c80 con 0x560c383a14a0 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20d0ff9640 1 -- 172.21.15.78:0/156724090 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc01ea30 con 0x560c383a14a0 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20e3b42740 1 -- 172.21.15.78:0/156724090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a14a0 msgr2=0x560c383ab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20e3b42740 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a14a0 0x560c383ab150 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x560c383a8a70 tx=0x7f20cc00a2d0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20e3b42740 1 -- 172.21.15.78:0/156724090 shutdown_connections 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20e3b42740 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a1dc0 0x560c383c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20e3b42740 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a14a0 0x560c383ab150 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20e3b42740 1 --2- 172.21.15.78:0/156724090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 0x560c383b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.801+0000 7f20e3b42740 1 -- 172.21.15.78:0/156724090 >> 172.21.15.78:0/156724090 conn(0x560c383a9880 msgr2=0x560c383aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 -- 172.21.15.78:0/156724090 shutdown_connections 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 -- 172.21.15.78:0/156724090 wait complete. 2024-09-16T00:27:24.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 -- start start 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 Processor -- start 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 0x560c3841f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 0x560c3841f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a1dc0 0x560c383afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1b2290 con 0x560c383a1dc0 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1b1bc0 con 0x560c3839da30 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1aee00 con 0x560c383a14a0 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20e3b42740 10 client.0 _ll_register_callbacks cb 0x7f20cc00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20d27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a1dc0 0x560c383afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20d27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a1dc0 0x560c383afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47220/0 (socket says 172.21.15.78:47220) 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20d27fc640 1 -- 172.21.15.78:0/4217915938 learned_addr learned my addr 172.21.15.78:0/4217915938 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20daffd640 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 0x560c3841f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.802+0000 7f20d27fc640 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 0x560c3841f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20d27fc640 1 -- 172.21.15.78:0/4217915938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 msgr2=0x560c3841f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20d27fc640 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 0x560c3841f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20d27fc640 1 -- 172.21.15.78:0/4217915938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 msgr2=0x560c3841f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20d27fc640 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 0x560c3841f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20d27fc640 1 -- 172.21.15.78:0/4217915938 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c383b0390 con 0x560c383a1dc0 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20d27fc640 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a1dc0 0x560c383afca0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f20c4033000 tx=0x7f20c404a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10879ac3ab65199e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20c27fc640 1 -- 172.21.15.78:0/4217915938 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4035070 con 0x560c383a1dc0 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20c27fc640 1 -- 172.21.15.78:0/4217915938 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f20c40039b0 con 0x560c383a1dc0 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20e3b42740 1 -- 172.21.15.78:0/4217915938 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20cc004c80 con 0x560c383a1dc0 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20c27fc640 1 -- 172.21.15.78:0/4217915938 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f20c40035f0 con 0x560c383a1dc0 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20e3b42740 10 client.17865 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20e3b42740 1 -- 172.21.15.78:0/4217915938 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560c384268b0 con 0x560c383a1dc0 2024-09-16T00:27:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20c27fc640 1 -- 172.21.15.78:0/4217915938 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c4067070 con 0x560c383a1dc0 2024-09-16T00:27:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.803+0000 7f20c27fc640 1 -- 172.21.15.78:0/4217915938 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f20c403a040 con 0x560c383a1dc0 2024-09-16T00:27:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20e3b42740 10 client.17865 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20e3b42740 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:27:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20e3b42740 20 client.17865 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20e3b42740 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560c383ac680 0x560c38428df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20e3b42740 4 client.17865 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:27:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20e3b42740 1 -- 172.21.15.78:0/4217915938 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c3830ed60 con 0x560c383ac680 2024-09-16T00:27:24.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20daffd640 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560c383ac680 0x560c38428df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20daffd640 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560c383ac680 0x560c38428df0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a51cdcc77f78c73 server_cookie=b06f926f983224e6 in_seq=0 out_seq=0 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20c27fc640 10 client.17865 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20c27fc640 1 -- 172.21.15.78:0/4217915938 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560c3830ed60 con 0x560c383ac680 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20c27fc640 10 client.17865 handle_command_reply: tid=0 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.804+0000 7f20e3b42740 1 client.17865 shutdown 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 -- 172.21.15.78:0/4217915938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a1dc0 msgr2=0x560c383afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a1dc0 0x560c383afca0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f20c4033000 tx=0x7f20c404a040 comp rx=0 tx=0).stop 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 -- 172.21.15.78:0/4217915938 shutdown_connections 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560c383ac680 0x560c38428df0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560c383a1dc0 0x560c383afca0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c383a14a0 0x560c3841f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 --2- 172.21.15.78:0/4217915938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560c3839da30 0x560c3841f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 -- 172.21.15.78:0/4217915938 >> 172.21.15.78:0/4217915938 conn(0x560c383a9880 msgr2=0x560c383ab260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20c27fc640 0 client.17865 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 -- 172.21.15.78:0/4217915938 shutdown_connections 2024-09-16T00:27:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 1 -- 172.21.15.78:0/4217915938 wait complete. 2024-09-16T00:27:24.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e3b42740 20 client.17865 trim_cache size 0 max 16384 2024-09-16T00:27:24.812 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:27:24.812 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:27:24.812 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:27:24.812 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:27:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 -- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20c80a32c0 msgr2=0x7f20c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20c80a32c0 0x7f20c80a5780 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f20d4098890 tx=0x7f20d40054d0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 -- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc10c160 msgr2=0x7f20dc101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc10c160 0x7f20dc101410 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f20d400cad0 tx=0x7f20d400ced0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 -- 172.21.15.78:0/1827180011 shutdown_connections 2024-09-16T00:27:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20c80a32c0 0x7f20c80a5780 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20dc10c160 0x7f20dc101410 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20dc103650 0x7f20dc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 --2- 172.21.15.78:0/1827180011 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20dc102d30 0x7f20dc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.805+0000 7f20e1769640 1 -- 172.21.15.78:0/1827180011 >> 172.21.15.78:0/1827180011 conn(0x7f20dc0fc790 msgr2=0x7f20dc0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.806+0000 7f20e1769640 1 -- 172.21.15.78:0/1827180011 shutdown_connections 2024-09-16T00:27:24.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:24.806+0000 7f20e1769640 1 -- 172.21.15.78:0/1827180011 wait complete. 2024-09-16T00:27:25.524 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:27:25.524 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:27:25.771 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:25.861 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 805.0 s, 142.5 tps, lat 224.258 ms stddev 301.179 2024-09-16T00:27:26.048 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:25 smithi078 ceph-mon[31157]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 MiB/s wr, 565 op/s 2024-09-16T00:27:26.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:25 smithi193 ceph-mon[35484]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 MiB/s wr, 565 op/s 2024-09-16T00:27:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:25 smithi148 ceph-mon[35599]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.2 MiB/s wr, 565 op/s 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 -- 172.21.15.78:0/2224793656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0108300 msgr2=0x7ff2f010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/2224793656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0108300 0x7ff2f010ec70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009cd0 tx=0x7ff2e802a8d0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 -- 172.21.15.78:0/2224793656 shutdown_connections 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/2224793656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0108300 0x7ff2f010ec70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/2224793656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0103780 0x7ff2f0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/2224793656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f0102dd0 0x7ff2f01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 -- 172.21.15.78:0/2224793656 >> 172.21.15.78:0/2224793656 conn(0x7ff2f00fc7d0 msgr2=0x7ff2f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 -- 172.21.15.78:0/2224793656 shutdown_connections 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 -- 172.21.15.78:0/2224793656 wait complete. 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 Processor -- start 2024-09-16T00:27:26.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.289+0000 7ff2f69b9640 1 -- start start 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2f69b9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0102dd0 0x7ff2f0100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2f69b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0103780 0x7ff2f0100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2f69b9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f0108300 0x7ff2f01014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2f69b9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f01120d0 con 0x7ff2f0103780 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2f69b9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0111f50 con 0x7ff2f0102dd0 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2f69b9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0112250 con 0x7ff2f0108300 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0103780 0x7ff2f0100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0103780 0x7ff2f0100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36264/0 (socket says 172.21.15.78:36264) 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 -- 172.21.15.78:0/3417865335 learned_addr learned my addr 172.21.15.78:0/3417865335 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2effff640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0102dd0 0x7ff2f0100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2f4f56640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f0108300 0x7ff2f01014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 -- 172.21.15.78:0/3417865335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f0108300 msgr2=0x7ff2f01014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:26.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f0108300 0x7ff2f01014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 -- 172.21.15.78:0/3417865335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0102dd0 msgr2=0x7ff2f0100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0102dd0 0x7ff2f0100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 -- 172.21.15.78:0/3417865335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f01a6a50 con 0x7ff2f0103780 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.290+0000 7ff2ef7fe640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0103780 0x7ff2f0100f70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400d950 tx=0x7ff2e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.291+0000 7ff2ed7fa640 1 -- 172.21.15.78:0/3417865335 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4017070 con 0x7ff2f0103780 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.291+0000 7ff2ed7fa640 1 -- 172.21.15.78:0/3417865335 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e4009c30 con 0x7ff2f0103780 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.291+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f01a6d40 con 0x7ff2f0103780 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.291+0000 7ff2ed7fa640 1 -- 172.21.15.78:0/3417865335 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40047a0 con 0x7ff2f0103780 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.291+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f01a7220 con 0x7ff2f0103780 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.292+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f01031b0 con 0x7ff2f0103780 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.293+0000 7ff2ed7fa640 1 -- 172.21.15.78:0/3417865335 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff2e4004020 con 0x7ff2f0103780 2024-09-16T00:27:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.294+0000 7ff2ed7fa640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2d80a32c0 0x7ff2d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:26.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.294+0000 7ff2effff640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2d80a32c0 0x7ff2d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:26.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.294+0000 7ff2ed7fa640 1 -- 172.21.15.78:0/3417865335 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff2e40cfbe0 con 0x7ff2f0103780 2024-09-16T00:27:26.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.294+0000 7ff2effff640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2d80a32c0 0x7ff2d80a5780 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009bd0 tx=0x7ff2e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:26.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.297+0000 7ff2ed7fa640 1 -- 172.21.15.78:0/3417865335 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2e4020020 con 0x7ff2f0103780 2024-09-16T00:27:26.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.500+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2f0101f60 con 0x7ff2f0103780 2024-09-16T00:27:26.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.501+0000 7ff2ed7fa640 1 -- 172.21.15.78:0/3417865335 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7ff2e409c620 con 0x7ff2f0103780 2024-09-16T00:27:26.501 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:27:26.502 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:27:26.502 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:27:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2d80a32c0 msgr2=0x7ff2d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2d80a32c0 0x7ff2d80a5780 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009bd0 tx=0x7ff2e0005f90 comp rx=0 tx=0).stop 2024-09-16T00:27:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0103780 msgr2=0x7ff2f0100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0103780 0x7ff2f0100f70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400d950 tx=0x7ff2e400de10 comp rx=0 tx=0).stop 2024-09-16T00:27:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 shutdown_connections 2024-09-16T00:27:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff2d80a32c0 0x7ff2d80a5780 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2f0108300 0x7ff2f01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff2f0103780 0x7ff2f0100f70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 --2- 172.21.15.78:0/3417865335 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff2f0102dd0 0x7ff2f0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 >> 172.21.15.78:0/3417865335 conn(0x7ff2f00fc7d0 msgr2=0x7ff2f0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 shutdown_connections 2024-09-16T00:27:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:26.503+0000 7ff2f69b9640 1 -- 172.21.15.78:0/3417865335 wait complete. 2024-09-16T00:27:27.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:26 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:27.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:26 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:26 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:26 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:27:27.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:26 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3417865335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:26 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:26 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:26 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:26 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:27:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:26 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3417865335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:27.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:26 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:27.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:26 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:27.225 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:26 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:27.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:26 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:27:27.226 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:26 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3417865335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:27.259 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:27:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:27.935 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:27 smithi078 ceph-mon[31157]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 527 op/s 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.016+0000 7fb70b3b0640 1 -- 172.21.15.78:0/3575378558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704101450 msgr2=0x7fb704105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.016+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/3575378558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704101450 0x7fb704105920 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009900 tx=0x7fb6f402a470 comp rx=0 tx=0).stop 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.016+0000 7fb70b3b0640 1 -- 172.21.15.78:0/3575378558 shutdown_connections 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.016+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/3575378558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb704105f40 0x7fb7041063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.016+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/3575378558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704101450 0x7fb704105920 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.016+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/3575378558 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704100b30 0x7fb704100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.016+0000 7fb70b3b0640 1 -- 172.21.15.78:0/3575378558 >> 172.21.15.78:0/3575378558 conn(0x7fb7040fc7a0 msgr2=0x7fb7040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.017+0000 7fb70b3b0640 1 -- 172.21.15.78:0/3575378558 shutdown_connections 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.017+0000 7fb70b3b0640 1 -- 172.21.15.78:0/3575378558 wait complete. 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.017+0000 7fb70b3b0640 1 Processor -- start 2024-09-16T00:27:28.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.017+0000 7fb70b3b0640 1 -- start start 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70b3b0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704100b30 0x7fb7041a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70b3b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb704101450 0x7fb7041a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70b3b0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704105f40 0x7fb70419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70b3b0640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041185b0 con 0x7fb704100b30 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70b3b0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb704118430 con 0x7fb704105f40 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70b3b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb704118730 con 0x7fb704101450 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704100b30 0x7fb7041a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704100b30 0x7fb7041a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36280/0 (socket says 172.21.15.78:36280) 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 -- 172.21.15.78:0/2142256766 learned_addr learned my addr 172.21.15.78:0/2142256766 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70894b640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb704101450 0x7fb7041a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70994d640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704105f40 0x7fb70419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 -- 172.21.15.78:0/2142256766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb704101450 msgr2=0x7fb7041a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb704101450 0x7fb7041a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 -- 172.21.15.78:0/2142256766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704105f40 msgr2=0x7fb70419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704105f40 0x7fb70419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 -- 172.21.15.78:0/2142256766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70419eff0 con 0x7fb704100b30 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70914c640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704100b30 0x7fb7041a4700 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb70000d950 tx=0x7fb70000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb6fa7fc640 1 -- 172.21.15.78:0/2142256766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700017070 con 0x7fb704100b30 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb6fa7fc640 1 -- 172.21.15.78:0/2142256766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb700009c30 con 0x7fb704100b30 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb6fa7fc640 1 -- 172.21.15.78:0/2142256766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70000f6b0 con 0x7fb704100b30 2024-09-16T00:27:28.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70b3b0640 1 -- 172.21.15.78:0/2142256766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70419f2e0 con 0x7fb704100b30 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.018+0000 7fb70b3b0640 1 -- 172.21.15.78:0/2142256766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7041ab5d0 con 0x7fb704100b30 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.020+0000 7fb6fa7fc640 1 -- 172.21.15.78:0/2142256766 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb7000109d0 con 0x7fb704100b30 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70d789740 1 Processor -- start 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70d789740 1 -- start start 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70d789740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563893585b40 0x563893584090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70d789740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638935845d0 0x563893584a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70d789740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56389358f370 0x563893591760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70d789740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5638934ea2f0 con 0x56389358f370 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70d789740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563893491cf0 con 0x563893585b40 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70d789740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5638934e62b0 con 0x5638935845d0 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb6fa7fc640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6f00a32c0 0x7fb6f00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70894b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56389358f370 0x563893591760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70894b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56389358f370 0x563893591760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36284/0 (socket says 172.21.15.78:36284) 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70894b640 1 -- 172.21.15.78:0/1715655139 learned_addr learned my addr 172.21.15.78:0/1715655139 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70994d640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6f00a32c0 0x7fb6f00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70994d640 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563893585b40 0x563893584090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.021+0000 7fb70914c640 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638935845d0 0x563893584a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb6fa7fc640 1 -- 172.21.15.78:0/2142256766 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb7000cf970 con 0x7fb704100b30 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb70894b640 1 -- 172.21.15.78:0/1715655139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638935845d0 msgr2=0x563893584a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb70894b640 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638935845d0 0x563893584a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb70894b640 1 -- 172.21.15.78:0/1715655139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563893585b40 msgr2=0x563893584090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb70894b640 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563893585b40 0x563893584090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb70894b640 1 -- 172.21.15.78:0/1715655139 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638934763a0 con 0x56389358f370 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb70894b640 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56389358f370 0x563893591760 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009900 tx=0x7fb6f4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6886b777270f5a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb70994d640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6f00a32c0 0x7fb6f00a5780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc005c40 tx=0x7fb6fc009c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb6ef7fe640 1 -- 172.21.15.78:0/1715655139 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4035030 con 0x56389358f370 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb6ef7fe640 1 -- 172.21.15.78:0/1715655139 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f40043c0 con 0x56389358f370 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.022+0000 7fb6ef7fe640 1 -- 172.21.15.78:0/1715655139 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4004dc0 con 0x56389358f370 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 -- 172.21.15.78:0/1715655139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56389358f370 msgr2=0x563893591760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56389358f370 0x563893591760 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009900 tx=0x7fb6f4005e60 comp rx=0 tx=0).stop 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 -- 172.21.15.78:0/1715655139 shutdown_connections 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56389358f370 0x563893591760 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638935845d0 0x563893584a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 --2- 172.21.15.78:0/1715655139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563893585b40 0x563893584090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 -- 172.21.15.78:0/1715655139 >> 172.21.15.78:0/1715655139 conn(0x56389354d310 msgr2=0x563893580c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 -- 172.21.15.78:0/1715655139 shutdown_connections 2024-09-16T00:27:28.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 -- 172.21.15.78:0/1715655139 wait complete. 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 -- start start 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.023+0000 7fb70d789740 1 Processor -- start 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70d789740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638935845d0 0x56389358ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70d789740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563893585b40 0x56389358bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70d789740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56389358f370 0x56389358c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70d789740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563893592010 con 0x5638935845d0 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70d789740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5638934a5060 con 0x56389358f370 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70d789740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563893592190 con 0x563893585b40 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70d789740 10 client.0 _ll_register_callbacks cb 0x5638931f88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638935845d0 0x56389358ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638935845d0 0x56389358ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36292/0 (socket says 172.21.15.78:36292) 2024-09-16T00:27:28.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 -- 172.21.15.78:0/1023440494 learned_addr learned my addr 172.21.15.78:0/1023440494 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70994d640 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56389358f370 0x56389358c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70894b640 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563893585b40 0x56389358bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 -- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563893585b40 msgr2=0x56389358bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563893585b40 0x56389358bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 -- 172.21.15.78:0/1023440494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56389358f370 msgr2=0x56389358c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56389358f370 0x56389358c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 -- 172.21.15.78:0/1023440494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389325b3c0 con 0x5638935845d0 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70894b640 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563893585b40 0x56389358bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70914c640 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638935845d0 0x56389358ed30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb700009610 tx=0x7fb700000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bf1d932d628269 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb6ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb6ecff9640 1 -- 172.21.15.78:0/1023440494 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700020040 con 0x5638935845d0 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb6ecff9640 1 -- 172.21.15.78:0/1023440494 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb70009b020 con 0x5638935845d0 2024-09-16T00:27:28.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.024+0000 7fb70d789740 1 -- 172.21.15.78:0/1023440494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f40043c0 con 0x5638935845d0 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb6ecff9640 1 -- 172.21.15.78:0/1023440494 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb70009bbd0 con 0x5638935845d0 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70d789740 10 client.17883 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70d789740 1 -- 172.21.15.78:0/1023440494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56389359dac0 con 0x5638935845d0 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb6ecff9640 1 -- 172.21.15.78:0/1023440494 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f40043c0 con 0x5638935845d0 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb6ecff9640 1 -- 172.21.15.78:0/1023440494 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb700097920 con 0x5638935845d0 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70d789740 10 client.17883 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70d789740 10 client.17883 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70d789740 20 client.17883 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70d789740 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56389359ef80 0x5638935a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70d789740 4 client.17883 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70d789740 1 -- 172.21.15.78:0/1023440494 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638934f7860 con 0x56389359ef80 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.025+0000 7fb70894b640 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56389359ef80 0x5638935a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.026+0000 7fb70894b640 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56389359ef80 0x5638935a13f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7dbf448f3996602 server_cookie=1de2adaea26721e5 in_seq=0 out_seq=0 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.026+0000 7fb6ecff9640 10 client.17883 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb6ecff9640 1 -- 172.21.15.78:0/1023440494 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638934f7860 con 0x56389359ef80 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb6ecff9640 10 client.17883 handle_command_reply: tid=0 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 client.17883 shutdown 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 -- 172.21.15.78:0/1023440494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638935845d0 msgr2=0x56389358ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638935845d0 0x56389358ed30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb700009610 tx=0x7fb700000bc0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 -- 172.21.15.78:0/1023440494 shutdown_connections 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56389358f370 0x56389358c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56389359ef80 0x5638935a13f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563893585b40 0x56389358bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 --2- 172.21.15.78:0/1023440494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638935845d0 0x56389358ed30 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb6ecff9640 0 client.17883 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:27:28.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 -- 172.21.15.78:0/1023440494 >> 172.21.15.78:0/1023440494 conn(0x56389354d310 msgr2=0x563893586960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 -- 172.21.15.78:0/1023440494 shutdown_connections 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 1 -- 172.21.15.78:0/1023440494 wait complete. 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.027+0000 7fb70d789740 20 client.17883 trim_cache size 0 max 16384 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70d789740 1 Processor -- start 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70d789740 1 -- start start 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70d789740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893590760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70d789740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893590ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70d789740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x56389358df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70d789740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041b5150 con 0x563893585b40 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70d789740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041b6c90 con 0x5638935845d0 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70d789740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041b5820 con 0x56389358f370 2024-09-16T00:27:28.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893590ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893590ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36294/0 (socket says 172.21.15.78:36294) 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 -- 172.21.15.78:0/3088698600 learned_addr learned my addr 172.21.15.78:0/3088698600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70994d640 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x56389358df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70914c640 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893590760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 -- 172.21.15.78:0/3088698600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 msgr2=0x56389358df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x56389358df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 -- 172.21.15.78:0/3088698600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 msgr2=0x563893590760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893590760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 -- 172.21.15.78:0/3088698600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563893581960 con 0x563893585b40 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70914c640 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893590760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.040+0000 7fb70894b640 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893590ca0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb6f402a990 tx=0x7fb6f402a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f8f5a381aa52ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb6eeffd640 1 -- 172.21.15.78:0/3088698600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4035070 con 0x563893585b40 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb6eeffd640 1 -- 172.21.15.78:0/3088698600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f4031a90 con 0x563893585b40 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb6eeffd640 1 -- 172.21.15.78:0/3088698600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f404fa60 con 0x563893585b40 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 -- 172.21.15.78:0/3088698600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 msgr2=0x563893590ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893590ca0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb6f402a990 tx=0x7fb6f402a470 comp rx=0 tx=0).stop 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 -- 172.21.15.78:0/3088698600 shutdown_connections 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x56389358df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893590ca0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 --2- 172.21.15.78:0/3088698600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893590760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 -- 172.21.15.78:0/3088698600 >> 172.21.15.78:0/3088698600 conn(0x56389354d310 msgr2=0x56389358d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 -- 172.21.15.78:0/3088698600 shutdown_connections 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.041+0000 7fb70d789740 1 -- 172.21.15.78:0/3088698600 wait complete. 2024-09-16T00:27:28.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 1 -- start start 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 1 Processor -- start 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893597430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893597970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x5638935923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041b65c0 con 0x563893585b40 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041b5ef0 con 0x5638935845d0 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7041b3130 con 0x56389358f370 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70d789740 10 client.0 _ll_register_callbacks cb 0x7fb700097e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:28.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70894b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893597970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70894b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893597970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36304/0 (socket says 172.21.15.78:36304) 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70894b640 1 -- 172.21.15.78:0/3707980351 learned_addr learned my addr 172.21.15.78:0/3707980351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70914c640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893597430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70994d640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x5638935923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70894b640 1 -- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 msgr2=0x5638935923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70894b640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x5638935923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70894b640 1 -- 172.21.15.78:0/3707980351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 msgr2=0x563893597430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70894b640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893597430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.042+0000 7fb70894b640 1 -- 172.21.15.78:0/3707980351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563893597fb0 con 0x563893585b40 2024-09-16T00:27:28.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.043+0000 7fb70994d640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x5638935923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.043+0000 7fb70914c640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893597430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.043+0000 7fb70894b640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893597970 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005ec0 tx=0x7fb6f404f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8adae39a25fcba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.043+0000 7fb6ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.043+0000 7fb6ecff9640 1 -- 172.21.15.78:0/3707980351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4035040 con 0x563893585b40 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.043+0000 7fb6ecff9640 1 -- 172.21.15.78:0/3707980351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fb6f4032cd0 con 0x563893585b40 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 1 -- 172.21.15.78:0/3707980351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f4031a90 con 0x563893585b40 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb6ecff9640 1 -- 172.21.15.78:0/3707980351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb6f40315b0 con 0x563893585b40 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 10 client.17889 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 1 -- 172.21.15.78:0/3707980351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56389360a170 con 0x563893585b40 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb6ecff9640 1 -- 172.21.15.78:0/3707980351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f4031a90 con 0x563893585b40 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb6ecff9640 1 -- 172.21.15.78:0/3707980351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb6f4061030 con 0x563893585b40 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 10 client.17889 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:28.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 10 client.17889 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 20 client.17889 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56389360a4b0 0x56389360c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 4 client.17889 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.044+0000 7fb70d789740 1 -- 172.21.15.78:0/3707980351 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638934f1d70 con 0x56389360a4b0 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70914c640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56389360a4b0 0x56389360c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb6ecff9640 10 client.17889 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70914c640 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56389360a4b0 0x56389360c8a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=509d0c7faacc34a4 server_cookie=f0e0886f86bc47f2 in_seq=0 out_seq=0 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb6ecff9640 1 -- 172.21.15.78:0/3707980351 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638934f1d70 con 0x56389360a4b0 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb6ecff9640 10 client.17889 handle_command_reply: tid=0 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 client.17889 shutdown 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 -- 172.21.15.78:0/3707980351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 msgr2=0x563893597970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893597970 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4005ec0 tx=0x7fb6f404f470 comp rx=0 tx=0).stop 2024-09-16T00:27:28.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 -- 172.21.15.78:0/3707980351 shutdown_connections 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x56389360a4b0 0x56389360c8a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56389358f370 0x5638935923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563893585b40 0x563893597970 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 --2- 172.21.15.78:0/3707980351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638935845d0 0x563893597430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 -- 172.21.15.78:0/3707980351 >> 172.21.15.78:0/3707980351 conn(0x56389354d310 msgr2=0x56389358e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb6ecff9640 0 client.17889 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.045+0000 7fb70d789740 1 -- 172.21.15.78:0/3707980351 shutdown_connections 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70d789740 1 -- 172.21.15.78:0/3707980351 wait complete. 2024-09-16T00:27:28.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70d789740 20 client.17889 trim_cache size 0 max 16384 2024-09-16T00:27:28.065 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:27:28.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 -- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6f00a32c0 msgr2=0x7fb6f00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6f00a32c0 0x7fb6f00a5780 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc005c40 tx=0x7fb6fc009c90 comp rx=0 tx=0).stop 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 -- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704100b30 msgr2=0x7fb7041a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704100b30 0x7fb7041a4700 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb70000d950 tx=0x7fb70000de10 comp rx=0 tx=0).stop 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 -- 172.21.15.78:0/2142256766 shutdown_connections 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fb6f00a32c0 0x7fb6f00a5780 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb704105f40 0x7fb70419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb704101450 0x7fb7041a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 --2- 172.21.15.78:0/2142256766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb704100b30 0x7fb7041a4700 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 -- 172.21.15.78:0/2142256766 >> 172.21.15.78:0/2142256766 conn(0x7fb7040fc7a0 msgr2=0x7fb7041038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 -- 172.21.15.78:0/2142256766 shutdown_connections 2024-09-16T00:27:28.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:28.046+0000 7fb70b3b0640 1 -- 172.21.15.78:0/2142256766 wait complete. 2024-09-16T00:27:28.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:27 smithi193 ceph-mon[35484]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 527 op/s 2024-09-16T00:27:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:27 smithi148 ceph-mon[35599]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 527 op/s 2024-09-16T00:27:28.902 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:27:29.148 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:29.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 -- 172.21.15.78:0/3286823931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 msgr2=0x7f0e60106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/3286823931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 0x7f0e60106e60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009930 tx=0x7f0e5c02a420 comp rx=0 tx=0).stop 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 -- 172.21.15.78:0/3286823931 shutdown_connections 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/3286823931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60101420 0x7f0e6010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/3286823931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60100a80 0x7f0e60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/3286823931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 0x7f0e60106e60 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 -- 172.21.15.78:0/3286823931 >> 172.21.15.78:0/3286823931 conn(0x7f0e600fc7a0 msgr2=0x7f0e600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 -- 172.21.15.78:0/3286823931 shutdown_connections 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.677+0000 7f0e67b4c640 1 -- 172.21.15.78:0/3286823931 wait complete. 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e67b4c640 1 Processor -- start 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e67b4c640 1 -- start start 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e67b4c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60100a80 0x7f0e60199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e67b4c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60101420 0x7f0e60199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e67b4c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 0x7f0e601a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e67b4c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60110ea0 con 0x7f0e60106a80 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e67b4c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60110d20 con 0x7f0e60100a80 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e67b4c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60111020 con 0x7f0e60101420 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e660e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 0x7f0e601a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e660e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 0x7f0e601a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36310/0 (socket says 172.21.15.78:36310) 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e660e9640 1 -- 172.21.15.78:0/117683200 learned_addr learned my addr 172.21.15.78:0/117683200 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e658e8640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60100a80 0x7f0e60199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e660e9640 1 -- 172.21.15.78:0/117683200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60101420 msgr2=0x7f0e60199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e660e9640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60101420 0x7f0e60199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e660e9640 1 -- 172.21.15.78:0/117683200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60100a80 msgr2=0x7f0e60199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e660e9640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60100a80 0x7f0e60199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e660e9640 1 -- 172.21.15.78:0/117683200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e601a48f0 con 0x7f0e60106a80 2024-09-16T00:27:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.678+0000 7f0e658e8640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60100a80 0x7f0e60199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.679+0000 7f0e660e9640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 0x7f0e601a43b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800cce0 tx=0x7f0e5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.679+0000 7f0e56ffd640 1 -- 172.21.15.78:0/117683200 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800f450 con 0x7f0e60106a80 2024-09-16T00:27:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.679+0000 7f0e67b4c640 1 -- 172.21.15.78:0/117683200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e601a4ac0 con 0x7f0e60106a80 2024-09-16T00:27:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.679+0000 7f0e56ffd640 1 -- 172.21.15.78:0/117683200 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e5800fad0 con 0x7f0e60106a80 2024-09-16T00:27:29.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.679+0000 7f0e56ffd640 1 -- 172.21.15.78:0/117683200 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58018c10 con 0x7f0e60106a80 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.679+0000 7f0e67b4c640 1 -- 172.21.15.78:0/117683200 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e601a4f80 con 0x7f0e60106a80 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.681+0000 7f0e56ffd640 1 -- 172.21.15.78:0/117683200 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0e5800f5f0 con 0x7f0e60106a80 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.681+0000 7f0e69f25740 1 Processor -- start 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e69f25740 1 -- start start 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfdab40 0x55f9adfd9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfd95d0 0x55f9adfd9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 0x55f9adfe6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e69f25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9adf3f2f0 con 0x55f9adfdab40 2024-09-16T00:27:29.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e69f25740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f9adee6cf0 con 0x55f9adfe4370 2024-09-16T00:27:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e69f25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9adf3b2b0 con 0x55f9adfd95d0 2024-09-16T00:27:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfdab40 0x55f9adfd9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfdab40 0x55f9adfd9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36316/0 (socket says 172.21.15.78:36316) 2024-09-16T00:27:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 -- 172.21.15.78:0/2541532749 learned_addr learned my addr 172.21.15.78:0/2541532749 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e56ffd640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a3240 0x7f0e4c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e658e8640 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfd95d0 0x55f9adfd9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 0x55f9adfe6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e660e9640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a3240 0x7f0e4c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 -- 172.21.15.78:0/2541532749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfd95d0 msgr2=0x55f9adfd9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:29.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfd95d0 0x55f9adfd9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 -- 172.21.15.78:0/2541532749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 msgr2=0x55f9adfe6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:29.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 0x55f9adfe6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e650e7640 1 -- 172.21.15.78:0/2541532749 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9adecb3a0 con 0x55f9adfdab40 2024-09-16T00:27:29.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.682+0000 7f0e56ffd640 1 -- 172.21.15.78:0/117683200 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0e580d03e0 con 0x7f0e60106a80 2024-09-16T00:27:29.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfdab40 0x55f9adfd9090 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009dc0 tx=0x7f0e500092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259096cb58f62182 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:29.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e4bfff640 1 -- 172.21.15.78:0/2541532749 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000c050 con 0x55f9adfdab40 2024-09-16T00:27:29.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e4bfff640 1 -- 172.21.15.78:0/2541532749 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e50010040 con 0x55f9adfdab40 2024-09-16T00:27:29.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e660e9640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a3240 0x7f0e4c0a5700 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0e58098b00 tx=0x7f0e58067030 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:29.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e4bfff640 1 -- 172.21.15.78:0/2541532749 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000f040 con 0x55f9adfdab40 2024-09-16T00:27:29.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e69f25740 1 -- 172.21.15.78:0/2541532749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfdab40 msgr2=0x55f9adfd9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfdab40 0x55f9adfd9090 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009dc0 tx=0x7f0e500092d0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e69f25740 1 -- 172.21.15.78:0/2541532749 shutdown_connections 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 0x55f9adfe6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfd95d0 0x55f9adfd9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2541532749 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfdab40 0x55f9adfd9090 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.683+0000 7f0e69f25740 1 -- 172.21.15.78:0/2541532749 >> 172.21.15.78:0/2541532749 conn(0x55f9adfa2310 msgr2=0x55f9adfd5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 -- 172.21.15.78:0/2541532749 shutdown_connections 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 -- 172.21.15.78:0/2541532749 wait complete. 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 -- start start 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 Processor -- start 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 0x55f9adfe3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adfe0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adfe14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9adfe7070 con 0x55f9adfe4370 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f9adefa060 con 0x55f9adfd95d0 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f9adfe71f0 con 0x55f9adfdab40 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.684+0000 7f0e69f25740 10 client.0 _ll_register_callbacks cb 0x55f9adc4d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adfe14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adfe14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36320/0 (socket says 172.21.15.78:36320) 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 -- 172.21.15.78:0/2932850570 learned_addr learned my addr 172.21.15.78:0/2932850570 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e658e8640 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adfe0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 0x55f9adfe3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 -- 172.21.15.78:0/2932850570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 msgr2=0x55f9adfe0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adfe0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 -- 172.21.15.78:0/2932850570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 msgr2=0x55f9adfe3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 0x55f9adfe3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 -- 172.21.15.78:0/2932850570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9adcb03c0 con 0x55f9adfe4370 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adfe14f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009fd0 tx=0x7f0e500039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a612b55d944ec2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e497fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e497fa640 1 -- 172.21.15.78:0/2932850570 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5002a060 con 0x55f9adfe4370 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e497fa640 1 -- 172.21.15.78:0/2932850570 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0e50003d60 con 0x55f9adfe4370 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e69f25740 1 -- 172.21.15.78:0/2932850570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e50010040 con 0x55f9adfe4370 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e497fa640 1 -- 172.21.15.78:0/2932850570 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0e50031050 con 0x55f9adfe4370 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e69f25740 10 client.17901 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e69f25740 1 -- 172.21.15.78:0/2932850570 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f9adff2ae0 con 0x55f9adfe4370 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.685+0000 7f0e497fa640 1 -- 172.21.15.78:0/2932850570 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e50010040 con 0x55f9adfe4370 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e497fa640 1 -- 172.21.15.78:0/2932850570 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0e50016920 con 0x55f9adfe4370 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e69f25740 10 client.17901 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e69f25740 10 client.17901 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e69f25740 20 client.17901 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55f9adff3f40 0x55f9adff6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e69f25740 4 client.17901 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e69f25740 1 -- 172.21.15.78:0/2932850570 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9adf4c860 con 0x55f9adff3f40 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e658e8640 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55f9adff3f40 0x55f9adff6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e658e8640 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55f9adff3f40 0x55f9adff6330 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=edab5b39e33ad2b1 server_cookie=47001898129e335d in_seq=0 out_seq=0 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.686+0000 7f0e497fa640 10 client.17901 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e497fa640 1 -- 172.21.15.78:0/2932850570 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9adf4c860 con 0x55f9adff3f40 2024-09-16T00:27:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e497fa640 10 client.17901 handle_command_reply: tid=0 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 client.17901 shutdown 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 -- 172.21.15.78:0/2932850570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 msgr2=0x55f9adfe14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adfe14f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009fd0 tx=0x7f0e500039b0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 -- 172.21.15.78:0/2932850570 shutdown_connections 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adfe14f0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adfe0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55f9adff3f40 0x55f9adff6330 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2932850570 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 0x55f9adfe3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 -- 172.21.15.78:0/2932850570 >> 172.21.15.78:0/2932850570 conn(0x55f9adfa2310 msgr2=0x55f9adfdb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:29.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e497fa640 0 client.17901 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:27:29.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 -- 172.21.15.78:0/2932850570 shutdown_connections 2024-09-16T00:27:29.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.687+0000 7f0e69f25740 1 -- 172.21.15.78:0/2932850570 wait complete. 2024-09-16T00:27:29.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.688+0000 7f0e69f25740 20 client.17901 trim_cache size 0 max 16384 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e69f25740 1 Processor -- start 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e69f25740 1 -- start start 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 0x55f9adfdc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adff94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e69f25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601aec10 con 0x55f9adfe4370 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e69f25740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601b0750 con 0x55f9adfd95d0 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e69f25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601af2e0 con 0x55f9adfdab40 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.700+0000 7f0e650e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adff94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adff94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36336/0 (socket says 172.21.15.78:36336) 2024-09-16T00:27:29.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 -- 172.21.15.78:0/727317566 learned_addr learned my addr 172.21.15.78:0/727317566 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e658e8640 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 0x55f9adfdc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 -- 172.21.15.78:0/727317566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 msgr2=0x55f9adff8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 -- 172.21.15.78:0/727317566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 msgr2=0x55f9adfdc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 0x55f9adfdc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 -- 172.21.15.78:0/727317566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9adfd6960 con 0x55f9adfe4370 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e658e8640 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e650e7640 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adff94b0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0e50016fb0 tx=0x7f0e50009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a021b52389890af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e4b7fe640 1 -- 172.21.15.78:0/727317566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5002a040 con 0x55f9adfe4370 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e4b7fe640 1 -- 172.21.15.78:0/727317566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e50009810 con 0x55f9adfe4370 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.701+0000 7f0e4b7fe640 1 -- 172.21.15.78:0/727317566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000c070 con 0x55f9adfe4370 2024-09-16T00:27:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 -- 172.21.15.78:0/727317566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 msgr2=0x55f9adff94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adff94b0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0e50016fb0 tx=0x7f0e50009b90 comp rx=0 tx=0).stop 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 -- 172.21.15.78:0/727317566 shutdown_connections 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfe4370 0x55f9adff94b0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 --2- 172.21.15.78:0/727317566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfd95d0 0x55f9adfdc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 -- 172.21.15.78:0/727317566 >> 172.21.15.78:0/727317566 conn(0x55f9adfa2310 msgr2=0x55f9adfdf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 -- 172.21.15.78:0/727317566 shutdown_connections 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 -- 172.21.15.78:0/727317566 wait complete. 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 -- start start 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.702+0000 7f0e69f25740 1 Processor -- start 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfd95d0 0x55f9adff9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e69f25740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 0x55f9adff6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e69f25740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601b0080 con 0x55f9adfd95d0 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e69f25740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601af9b0 con 0x55f9adfe4370 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e69f25740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601acbf0 con 0x55f9adfdab40 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e69f25740 10 client.0 _ll_register_callbacks cb 0x7f0e50009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfd95d0 0x55f9adff9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfd95d0 0x55f9adff9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36352/0 (socket says 172.21.15.78:36352) 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 -- 172.21.15.78:0/2661771894 learned_addr learned my addr 172.21.15.78:0/2661771894 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e658e8640 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 -- 172.21.15.78:0/2661771894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 msgr2=0x55f9adff68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 -- 172.21.15.78:0/2661771894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 msgr2=0x55f9adff6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 0x55f9adff6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 -- 172.21.15.78:0/2661771894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9adfdc120 con 0x55f9adfd95d0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e650e7640 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfd95d0 0x55f9adff9570 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006ec0 tx=0x7f0e500092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9ab0bb01293549 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e497fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e497fa640 1 -- 172.21.15.78:0/2661771894 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50039020 con 0x55f9adfd95d0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e497fa640 1 -- 172.21.15.78:0/2661771894 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0e50005ab0 con 0x55f9adfd95d0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.703+0000 7f0e69f25740 1 -- 172.21.15.78:0/2661771894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e50009810 con 0x55f9adfd95d0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e497fa640 1 -- 172.21.15.78:0/2661771894 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0e50041050 con 0x55f9adfd95d0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e69f25740 10 client.17907 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e69f25740 1 -- 172.21.15.78:0/2661771894 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f9ae05e540 con 0x55f9adfd95d0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e497fa640 1 -- 172.21.15.78:0/2661771894 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e50009810 con 0x55f9adfd95d0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e497fa640 1 -- 172.21.15.78:0/2661771894 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0e50003e30 con 0x55f9adfd95d0 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e69f25740 10 client.17907 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:29.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e69f25740 10 client.17907 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e69f25740 20 client.17907 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55f9ae05ebf0 0x55f9ae060fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e69f25740 4 client.17907 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e69f25740 1 -- 172.21.15.78:0/2661771894 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9adf46d70 con 0x55f9ae05ebf0 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.704+0000 7f0e658e8640 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55f9ae05ebf0 0x55f9ae060fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e658e8640 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55f9ae05ebf0 0x55f9ae060fe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=231a92d4bed1d63b server_cookie=4b5846fef3da5698 in_seq=0 out_seq=0 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e497fa640 10 client.17907 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e497fa640 1 -- 172.21.15.78:0/2661771894 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9adf46d70 con 0x55f9ae05ebf0 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e497fa640 10 client.17907 handle_command_reply: tid=0 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 client.17907 shutdown 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 -- 172.21.15.78:0/2661771894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfd95d0 msgr2=0x55f9adff9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfd95d0 0x55f9adff9570 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006ec0 tx=0x7f0e500092b0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 -- 172.21.15.78:0/2661771894 shutdown_connections 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x55f9ae05ebf0 0x55f9ae060fe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9adfe4370 0x55f9adff6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9adfdab40 0x55f9adff68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e497fa640 0 client.17907 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 --2- 172.21.15.78:0/2661771894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9adfd95d0 0x55f9adff9570 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 -- 172.21.15.78:0/2661771894 >> 172.21.15.78:0/2661771894 conn(0x55f9adfa2310 msgr2=0x55f9adfdb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 -- 172.21.15.78:0/2661771894 shutdown_connections 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 1 -- 172.21.15.78:0/2661771894 wait complete. 2024-09-16T00:27:29.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.705+0000 7f0e69f25740 20 client.17907 trim_cache size 0 max 16384 2024-09-16T00:27:29.714 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:27:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 -- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a3240 msgr2=0x7f0e4c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a3240 0x7f0e4c0a5700 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0e58098b00 tx=0x7f0e58067030 comp rx=0 tx=0).stop 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 -- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 msgr2=0x7f0e601a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 0x7f0e601a43b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800cce0 tx=0x7f0e5800a620 comp rx=0 tx=0).stop 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 -- 172.21.15.78:0/117683200 shutdown_connections 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0e4c0a3240 0x7f0e4c0a5700 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e60106a80 0x7f0e601a43b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e60101420 0x7f0e60199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 --2- 172.21.15.78:0/117683200 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0e60100a80 0x7f0e60199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 -- 172.21.15.78:0/117683200 >> 172.21.15.78:0/117683200 conn(0x7f0e600fc7a0 msgr2=0x7f0e600fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 -- 172.21.15.78:0/117683200 shutdown_connections 2024-09-16T00:27:29.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:29.706+0000 7f0e67b4c640 1 -- 172.21.15.78:0/117683200 wait complete. 2024-09-16T00:27:30.015 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:29 smithi078 ceph-mon[31157]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-16T00:27:30.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:29 smithi193 ceph-mon[35484]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-16T00:27:30.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:29 smithi148 ceph-mon[35599]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-16T00:27:30.445 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:27:30.692 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 810.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:27:31.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 -- 172.21.15.78:0/839153897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc103660 msgr2=0x7f83dc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 --2- 172.21.15.78:0/839153897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc103660 0x7f83dc10bb50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009900 tx=0x7f83cc02a480 comp rx=0 tx=0).stop 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 -- 172.21.15.78:0/839153897 shutdown_connections 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 --2- 172.21.15.78:0/839153897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc10c170 0x7f83dc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 --2- 172.21.15.78:0/839153897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc103660 0x7f83dc10bb50 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 --2- 172.21.15.78:0/839153897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83dc102d40 0x7f83dc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 -- 172.21.15.78:0/839153897 >> 172.21.15.78:0/839153897 conn(0x7f83dc0fc7a0 msgr2=0x7f83dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 -- 172.21.15.78:0/839153897 shutdown_connections 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.195+0000 7f83e341d640 1 -- 172.21.15.78:0/839153897 wait complete. 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e341d640 1 Processor -- start 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e341d640 1 -- start start 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e341d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83dc102d40 0x7f83dc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e341d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc103660 0x7f83dc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e341d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc10c170 0x7f83dc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e341d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc111f70 con 0x7f83dc10c170 2024-09-16T00:27:31.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e341d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc111df0 con 0x7f83dc103660 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e341d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1120f0 con 0x7f83dc102d40 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e19ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc10c170 0x7f83dc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e19ba640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc10c170 0x7f83dc101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36378/0 (socket says 172.21.15.78:36378) 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e19ba640 1 -- 172.21.15.78:0/4205411668 learned_addr learned my addr 172.21.15.78:0/4205411668 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e09b8640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc103660 0x7f83dc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.196+0000 7f83e11b9640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83dc102d40 0x7f83dc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e19ba640 1 -- 172.21.15.78:0/4205411668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83dc102d40 msgr2=0x7f83dc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e19ba640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83dc102d40 0x7f83dc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e19ba640 1 -- 172.21.15.78:0/4205411668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc103660 msgr2=0x7f83dc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e19ba640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc103660 0x7f83dc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e19ba640 1 -- 172.21.15.78:0/4205411668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc1a6930 con 0x7f83dc10c170 2024-09-16T00:27:31.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e11b9640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83dc102d40 0x7f83dc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e19ba640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc10c170 0x7f83dc101420 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009fd0 tx=0x7f83d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83d27fc640 1 -- 172.21.15.78:0/4205411668 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4017070 con 0x7f83dc10c170 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83d27fc640 1 -- 172.21.15.78:0/4205411668 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d4003e50 con 0x7f83dc10c170 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e341d640 1 -- 172.21.15.78:0/4205411668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc1a6c20 con 0x7f83dc10c170 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83d27fc640 1 -- 172.21.15.78:0/4205411668 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40106c0 con 0x7f83dc10c170 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.197+0000 7f83e341d640 1 -- 172.21.15.78:0/4205411668 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc1a70e0 con 0x7f83dc10c170 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.199+0000 7f83d27fc640 1 -- 172.21.15.78:0/4205411668 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f83d4003970 con 0x7f83dc10c170 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e57f6740 1 Processor -- start 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e57f6740 1 -- start start 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 0x55a109ab7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 0x55a109ab7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ac4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e57f6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a109a1d2f0 con 0x55a109ac2370 2024-09-16T00:27:31.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e57f6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a1099c4cf0 con 0x55a109ab8b40 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e57f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a109a192b0 con 0x55a109ab75d0 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83d27fc640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f83c80a3240 0x7f83c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e09b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ac4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e09b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ac4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36384/0 (socket says 172.21.15.78:36384) 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e09b8640 1 -- 172.21.15.78:0/3128950341 learned_addr learned my addr 172.21.15.78:0/3128950341 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e11b9640 1 --2- 172.21.15.78:0/3128950341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 0x55a109ab7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.200+0000 7f83e19ba640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f83c80a3240 0x7f83c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83d27fc640 1 -- 172.21.15.78:0/4205411668 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f83d40cf9a0 con 0x7f83dc10c170 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83e09b8640 1 -- 172.21.15.78:0/3128950341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 msgr2=0x55a109ab7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83e09b8640 1 --2- 172.21.15.78:0/3128950341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 0x55a109ab7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83e09b8640 1 -- 172.21.15.78:0/3128950341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 msgr2=0x55a109ab7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83e09b8640 1 --2- 172.21.15.78:0/3128950341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 0x55a109ab7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83e09b8640 1 -- 172.21.15.78:0/3128950341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1099a93a0 con 0x55a109ac2370 2024-09-16T00:27:31.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83e09b8640 1 --2- 172.21.15.78:0/3128950341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ac4760 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f83cc031550 tx=0x7f83cc005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3408bf4a846b23b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83e19ba640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f83c80a3240 0x7f83c80a5700 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f83d40c10c0 tx=0x7f83d40c6a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83c77fe640 1 -- 172.21.15.78:0/3128950341 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc035070 con 0x55a109ac2370 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83c77fe640 1 -- 172.21.15.78:0/3128950341 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc041030 con 0x55a109ac2370 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.201+0000 7f83c77fe640 1 -- 172.21.15.78:0/3128950341 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc003a10 con 0x55a109ac2370 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 -- 172.21.15.78:0/3128950341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 msgr2=0x55a109ac4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 --2- 172.21.15.78:0/3128950341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ac4760 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f83cc031550 tx=0x7f83cc005e00 comp rx=0 tx=0).stop 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 -- 172.21.15.78:0/3128950341 shutdown_connections 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 --2- 172.21.15.78:0/3128950341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ac4760 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 --2- 172.21.15.78:0/3128950341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 0x55a109ab7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 --2- 172.21.15.78:0/3128950341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 0x55a109ab7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 -- 172.21.15.78:0/3128950341 >> 172.21.15.78:0/3128950341 conn(0x55a109a80310 msgr2=0x55a109ab3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:31.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 -- 172.21.15.78:0/3128950341 shutdown_connections 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 -- 172.21.15.78:0/3128950341 wait complete. 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 -- start start 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.202+0000 7f83e57f6740 1 Processor -- start 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 0x55a109ac1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 0x55a109abf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109abf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e57f6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a109ac5010 con 0x55a109ac2370 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e57f6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a1099d8060 con 0x55a109ab8b40 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e57f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a109ac5190 con 0x55a109ab75d0 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e57f6740 10 client.0 _ll_register_callbacks cb 0x55a10972b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109abf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109abf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36390/0 (socket says 172.21.15.78:36390) 2024-09-16T00:27:31.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 -- 172.21.15.78:0/2216962137 learned_addr learned my addr 172.21.15.78:0/2216962137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e11b9640 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 0x55a109abf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 0x55a109ac1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 -- 172.21.15.78:0/2216962137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 msgr2=0x55a109ac1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 0x55a109ac1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 -- 172.21.15.78:0/2216962137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 msgr2=0x55a109abf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 0x55a109abf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 -- 172.21.15.78:0/2216962137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a10978e3c0 con 0x55a109ac2370 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e09b8640 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109abf640 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009430 tx=0x7f83cc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43f0a563c890a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83c4ff9640 1 -- 172.21.15.78:0/2216962137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc035040 con 0x55a109ac2370 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83c4ff9640 1 -- 172.21.15.78:0/2216962137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f83cc051470 con 0x55a109ac2370 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.203+0000 7f83e57f6740 1 -- 172.21.15.78:0/2216962137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83cc041030 con 0x55a109ac2370 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83c4ff9640 1 -- 172.21.15.78:0/2216962137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f83cc0319c0 con 0x55a109ac2370 2024-09-16T00:27:31.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e57f6740 10 client.17919 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e57f6740 1 -- 172.21.15.78:0/2216962137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a109ad0b10 con 0x55a109ac2370 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83c4ff9640 1 -- 172.21.15.78:0/2216962137 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83cc041030 con 0x55a109ac2370 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83c4ff9640 1 -- 172.21.15.78:0/2216962137 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f83cc03c900 con 0x55a109ac2370 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e57f6740 10 client.17919 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e57f6740 10 client.17919 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e57f6740 20 client.17919 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a109ad1e60 0x55a109ad4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e57f6740 4 client.17919 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e57f6740 1 -- 172.21.15.78:0/2216962137 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a109a2a860 con 0x55a109ad1e60 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.204+0000 7f83e11b9640 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a109ad1e60 0x55a109ad4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.205+0000 7f83c4ff9640 10 client.17919 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.205+0000 7f83e11b9640 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a109ad1e60 0x55a109ad4250 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b18f73923b9e232c server_cookie=fbcd293ac56aaee in_seq=0 out_seq=0 2024-09-16T00:27:31.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.205+0000 7f83c4ff9640 1 -- 172.21.15.78:0/2216962137 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a109a2a860 con 0x55a109ad1e60 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.205+0000 7f83c4ff9640 10 client.17919 handle_command_reply: tid=0 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 client.17919 shutdown 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 -- 172.21.15.78:0/2216962137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 msgr2=0x55a109abf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109abf640 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009430 tx=0x7f83cc005780 comp rx=0 tx=0).stop 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 -- 172.21.15.78:0/2216962137 shutdown_connections 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109abf640 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a109ad1e60 0x55a109ad4250 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab8b40 0x55a109abf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2216962137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab75d0 0x55a109ac1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83c4ff9640 0 client.17919 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 -- 172.21.15.78:0/2216962137 >> 172.21.15.78:0/2216962137 conn(0x55a109a80310 msgr2=0x55a109ab9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 -- 172.21.15.78:0/2216962137 shutdown_connections 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 1 -- 172.21.15.78:0/2216962137 wait complete. 2024-09-16T00:27:31.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.206+0000 7f83e57f6740 20 client.17919 trim_cache size 0 max 16384 2024-09-16T00:27:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e57f6740 1 Processor -- start 2024-09-16T00:27:31.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e57f6740 1 -- start start 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 0x55a109ace460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ab8b40 0x55a109ab3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ac2370 0x55a109ad6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e57f6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1b0c90 con 0x55a109ab8b40 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e57f6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1b27d0 con 0x55a109ab75d0 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e57f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1b1360 con 0x55a109ac2370 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e11b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ab8b40 0x55a109ab3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e11b9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ab8b40 0x55a109ab3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36406/0 (socket says 172.21.15.78:36406) 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.219+0000 7f83e11b9640 1 -- 172.21.15.78:0/2766909130 learned_addr learned my addr 172.21.15.78:0/2766909130 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83e09b8640 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ac2370 0x55a109ad6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83e09b8640 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 0x55a109ace460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83e11b9640 1 -- 172.21.15.78:0/2766909130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ac2370 msgr2=0x55a109ad6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83e11b9640 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ac2370 0x55a109ad6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83e11b9640 1 -- 172.21.15.78:0/2766909130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 msgr2=0x55a109ace460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83e11b9640 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 0x55a109ace460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83e11b9640 1 -- 172.21.15.78:0/2766909130 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a109abd5c0 con 0x55a109ab8b40 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83e11b9640 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ab8b40 0x55a109ab3e10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f83d8005fd0 tx=0x7f83d800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e85a0d1d427a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83c6ffd640 1 -- 172.21.15.78:0/2766909130 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8005220 con 0x55a109ab8b40 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83c6ffd640 1 -- 172.21.15.78:0/2766909130 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8004c80 con 0x55a109ab8b40 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.220+0000 7f83c6ffd640 1 -- 172.21.15.78:0/2766909130 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d801da30 con 0x55a109ab8b40 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 -- 172.21.15.78:0/2766909130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ab8b40 msgr2=0x55a109ab3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ab8b40 0x55a109ab3e10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f83d8005fd0 tx=0x7f83d800be70 comp rx=0 tx=0).stop 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 -- 172.21.15.78:0/2766909130 shutdown_connections 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ac2370 0x55a109ad6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ab8b40 0x55a109ab3e10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 --2- 172.21.15.78:0/2766909130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 0x55a109ace460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 -- 172.21.15.78:0/2766909130 >> 172.21.15.78:0/2766909130 conn(0x55a109a80310 msgr2=0x55a109ab3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 -- 172.21.15.78:0/2766909130 shutdown_connections 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 -- 172.21.15.78:0/2766909130 wait complete. 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 -- start start 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.221+0000 7f83e57f6740 1 Processor -- start 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 0x55a109ad7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab8b40 0x55a109ad4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e57f6740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ad52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e57f6740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1b2100 con 0x55a109ac2370 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e57f6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1b1a30 con 0x55a109ab75d0 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e57f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1aec70 con 0x55a109ab8b40 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e57f6740 10 client.0 _ll_register_callbacks cb 0x7f83d800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ad52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ad52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36408/0 (socket says 172.21.15.78:36408) 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 -- 172.21.15.78:0/695675714 learned_addr learned my addr 172.21.15.78:0/695675714 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 0x55a109ad7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e11b9640 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab8b40 0x55a109ad4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 -- 172.21.15.78:0/695675714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab8b40 msgr2=0x55a109ad4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab8b40 0x55a109ad4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 -- 172.21.15.78:0/695675714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 msgr2=0x55a109ad7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 0x55a109ad7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 -- 172.21.15.78:0/695675714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a109b3b7a0 con 0x55a109ac2370 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e11b9640 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab8b40 0x55a109ad4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e09b8640 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ad52d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f83cc02a9a0 tx=0x7f83cc03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83be97f09cdc5cf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83c4ff9640 1 -- 172.21.15.78:0/695675714 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc035040 con 0x55a109ac2370 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83c4ff9640 1 -- 172.21.15.78:0/695675714 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f83cc03ca00 con 0x55a109ac2370 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.222+0000 7f83e57f6740 1 -- 172.21.15.78:0/695675714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d8004c80 con 0x55a109ac2370 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83c4ff9640 1 -- 172.21.15.78:0/695675714 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f83cc005d50 con 0x55a109ac2370 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e57f6740 10 client.17925 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e57f6740 1 -- 172.21.15.78:0/695675714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a109b3c900 con 0x55a109ac2370 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83c4ff9640 1 -- 172.21.15.78:0/695675714 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8004c80 con 0x55a109ac2370 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83c4ff9640 1 -- 172.21.15.78:0/695675714 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f83cc03d7a0 con 0x55a109ac2370 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e57f6740 10 client.17925 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e57f6740 10 client.17925 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e57f6740 20 client.17925 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e57f6740 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a109b3cd80 0x55a109b3f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e57f6740 4 client.17925 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e57f6740 1 -- 172.21.15.78:0/695675714 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a109a24d70 con 0x55a109b3cd80 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.223+0000 7f83e11b9640 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a109b3cd80 0x55a109b3f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83c4ff9640 10 client.17925 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e11b9640 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a109b3cd80 0x55a109b3f170 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e3d1b6de2b5953e server_cookie=883a22942082120f in_seq=0 out_seq=0 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83c4ff9640 1 -- 172.21.15.78:0/695675714 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a109a24d70 con 0x55a109b3cd80 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83c4ff9640 10 client.17925 handle_command_reply: tid=0 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 client.17925 shutdown 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 -- 172.21.15.78:0/695675714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 msgr2=0x55a109ad52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ad52d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f83cc02a9a0 tx=0x7f83cc03f040 comp rx=0 tx=0).stop 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 -- 172.21.15.78:0/695675714 shutdown_connections 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a109ac2370 0x55a109ad52d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a109ab8b40 0x55a109ad4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x55a109b3cd80 0x55a109b3f170 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 --2- 172.21.15.78:0/695675714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a109ab75d0 0x55a109ad7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83c4ff9640 0 client.17925 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:27:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 -- 172.21.15.78:0/695675714 >> 172.21.15.78:0/695675714 conn(0x55a109a80310 msgr2=0x55a109ac0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 -- 172.21.15.78:0/695675714 shutdown_connections 2024-09-16T00:27:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.224+0000 7f83e57f6740 1 -- 172.21.15.78:0/695675714 wait complete. 2024-09-16T00:27:31.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e57f6740 20 client.17925 trim_cache size 0 max 16384 2024-09-16T00:27:31.235 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:27:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 -- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f83c80a3240 msgr2=0x7f83c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f83c80a3240 0x7f83c80a5700 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f83d40c10c0 tx=0x7f83d40c6a30 comp rx=0 tx=0).stop 2024-09-16T00:27:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 -- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc10c170 msgr2=0x7f83dc101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc10c170 0x7f83dc101420 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009fd0 tx=0x7f83d400b260 comp rx=0 tx=0).stop 2024-09-16T00:27:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 -- 172.21.15.78:0/4205411668 shutdown_connections 2024-09-16T00:27:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f83c80a3240 0x7f83c80a5700 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83dc10c170 0x7f83dc101420 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f83dc103660 0x7f83dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 --2- 172.21.15.78:0/4205411668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83dc102d40 0x7f83dc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 -- 172.21.15.78:0/4205411668 >> 172.21.15.78:0/4205411668 conn(0x7f83dc0fc7a0 msgr2=0x7f83dc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 -- 172.21.15.78:0/4205411668 shutdown_connections 2024-09-16T00:27:31.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:31.225+0000 7f83e341d640 1 -- 172.21.15.78:0/4205411668 wait complete. 2024-09-16T00:27:31.928 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:31 smithi078 ceph-mon[31157]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-09-16T00:27:31.956 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:27:32.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:31 smithi193 ceph-mon[35484]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-09-16T00:27:32.204 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:31 smithi148 ceph-mon[35599]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-09-16T00:27:32.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 -- 172.21.15.78:0/1310055026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90100a80 msgr2=0x7f5b90100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 --2- 172.21.15.78:0/1310055026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90100a80 0x7f5b90100ee0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009930 tx=0x7f5b7c02a420 comp rx=0 tx=0).stop 2024-09-16T00:27:32.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 -- 172.21.15.78:0/1310055026 shutdown_connections 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 --2- 172.21.15.78:0/1310055026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b90101420 0x7f5b9010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 --2- 172.21.15.78:0/1310055026 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90100a80 0x7f5b90100ee0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 --2- 172.21.15.78:0/1310055026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b90106a80 0x7f5b90106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 -- 172.21.15.78:0/1310055026 >> 172.21.15.78:0/1310055026 conn(0x7f5b900fc7a0 msgr2=0x7f5b900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 -- 172.21.15.78:0/1310055026 shutdown_connections 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.755+0000 7f5b970a3640 1 -- 172.21.15.78:0/1310055026 wait complete. 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b970a3640 1 Processor -- start 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b970a3640 1 -- start start 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b970a3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b90100a80 0x7f5b901984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b970a3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b90101420 0x7f5b901989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b970a3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90106a80 0x7f5b9019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b970a3640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b90110ea0 con 0x7f5b90106a80 2024-09-16T00:27:32.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b970a3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b90110d20 con 0x7f5b90100a80 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b970a3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5b90111020 con 0x7f5b90101420 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b95640640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90106a80 0x7f5b9019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b95640640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90106a80 0x7f5b9019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36412/0 (socket says 172.21.15.78:36412) 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b95640640 1 -- 172.21.15.78:0/4001500988 learned_addr learned my addr 172.21.15.78:0/4001500988 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b90100a80 0x7f5b901984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b90101420 0x7f5b901989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b95640640 1 -- 172.21.15.78:0/4001500988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b90101420 msgr2=0x7f5b901989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b95640640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b90101420 0x7f5b901989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b95640640 1 -- 172.21.15.78:0/4001500988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b90100a80 msgr2=0x7f5b901984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b95640640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b90100a80 0x7f5b901984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b95640640 1 -- 172.21.15.78:0/4001500988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9019d460 con 0x7f5b90106a80 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.756+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b90101420 0x7f5b901989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:32.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.757+0000 7f5b95640640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90106a80 0x7f5b9019cd80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b970 tx=0x7f5b8800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.757+0000 7f5b8dffb640 1 -- 172.21.15.78:0/4001500988 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88019070 con 0x7f5b90106a80 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.757+0000 7f5b970a3640 1 -- 172.21.15.78:0/4001500988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9019d750 con 0x7f5b90106a80 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.757+0000 7f5b8dffb640 1 -- 172.21.15.78:0/4001500988 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8800cb40 con 0x7f5b90106a80 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.757+0000 7f5b8dffb640 1 -- 172.21.15.78:0/4001500988 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88007650 con 0x7f5b90106a80 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.757+0000 7f5b970a3640 1 -- 172.21.15.78:0/4001500988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b901a5030 con 0x7f5b90106a80 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.759+0000 7f5b8dffb640 1 -- 172.21.15.78:0/4001500988 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5b8801c070 con 0x7f5b90106a80 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.759+0000 7f5b9947c740 1 Processor -- start 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b9947c740 1 -- start start 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d435d0 0x55b8b3d43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 0x55b8b3d50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b9947c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8b3ca92f0 con 0x55b8b3d44b40 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b9947c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b8b3c50cf0 con 0x55b8b3d4e370 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b9947c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b8b3ca52b0 con 0x55b8b3d435d0 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d43090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36416/0 (socket says 172.21.15.78:36416) 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 -- 172.21.15.78:0/605043954 learned_addr learned my addr 172.21.15.78:0/605043954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8dffb640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5b800a3240 0x7f5b800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 0x55b8b3d50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d435d0 0x55b8b3d43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b95640640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5b800a3240 0x7f5b800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 -- 172.21.15.78:0/605043954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d435d0 msgr2=0x55b8b3d43a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d435d0 0x55b8b3d43a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 -- 172.21.15.78:0/605043954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 msgr2=0x55b8b3d50760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:32.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 0x55b8b3d50760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 -- 172.21.15.78:0/605043954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b3c353a0 con 0x55b8b3d44b40 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8dffb640 1 -- 172.21.15.78:0/4001500988 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5b880cf2c0 con 0x7f5b90106a80 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.760+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d43090 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x55b8b3d44070 tx=0x7f5b7c031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1ec24a1450a682 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b95640640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5b800a3240 0x7f5b800a5700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5b88097a30 tx=0x7f5b8800e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b7affd640 1 -- 172.21.15.78:0/605043954 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c035050 con 0x55b8b3d44b40 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b7affd640 1 -- 172.21.15.78:0/605043954 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c031a20 con 0x55b8b3d44b40 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b7affd640 1 -- 172.21.15.78:0/605043954 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c004920 con 0x55b8b3d44b40 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b9947c740 1 -- 172.21.15.78:0/605043954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 msgr2=0x55b8b3d43090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b9947c740 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d43090 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x55b8b3d44070 tx=0x7f5b7c031480 comp rx=0 tx=0).stop 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b9947c740 1 -- 172.21.15.78:0/605043954 shutdown_connections 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b9947c740 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 0x55b8b3d50760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b9947c740 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d435d0 0x55b8b3d43a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.761+0000 7f5b9947c740 1 --2- 172.21.15.78:0/605043954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d43090 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 -- 172.21.15.78:0/605043954 >> 172.21.15.78:0/605043954 conn(0x55b8b3d0c310 msgr2=0x55b8b3d3fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 -- 172.21.15.78:0/605043954 shutdown_connections 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 -- 172.21.15.78:0/605043954 wait complete. 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 -- start start 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 Processor -- start 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d4dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d4b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55b8b3d50fb0 con 0x55b8b3d435d0 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b8b3c64060 con 0x55b8b3d44b40 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b8b3d51130 con 0x55b8b3d4e370 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b9947c740 10 client.0 _ll_register_callbacks cb 0x55b8b39b78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:32.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d4dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d4dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36422/0 (socket says 172.21.15.78:36422) 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.762+0000 7f5b8ffff640 1 -- 172.21.15.78:0/2203963014 learned_addr learned my addr 172.21.15.78:0/2203963014 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d4b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b8ffff640 1 -- 172.21.15.78:0/2203963014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 msgr2=0x55b8b3d4b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d4b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b8ffff640 1 -- 172.21.15.78:0/2203963014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d44b40 msgr2=0x55b8b3d4b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b8ffff640 1 -- 172.21.15.78:0/2203963014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b3a1a3c0 con 0x55b8b3d435d0 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d4dd50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c02a940 tx=0x7f5b7c005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92dd6dcdc36f434 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b6ffff640 1 -- 172.21.15.78:0/2203963014 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c056060 con 0x55b8b3d435d0 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b6ffff640 1 -- 172.21.15.78:0/2203963014 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5b7c004470 con 0x55b8b3d435d0 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b9947c740 1 -- 172.21.15.78:0/2203963014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b7c031a20 con 0x55b8b3d435d0 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b6ffff640 1 -- 172.21.15.78:0/2203963014 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5b7c035040 con 0x55b8b3d435d0 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b9947c740 10 client.17937 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b9947c740 1 -- 172.21.15.78:0/2203963014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b8b3d5ca30 con 0x55b8b3d435d0 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.763+0000 7f5b6ffff640 1 -- 172.21.15.78:0/2203963014 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b7c031a20 con 0x55b8b3d435d0 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b6ffff640 1 -- 172.21.15.78:0/2203963014 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5b7c045740 con 0x55b8b3d435d0 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b9947c740 10 client.17937 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b9947c740 10 client.17937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b9947c740 20 client.17937 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55b8b3d5dd80 0x55b8b3d60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b9947c740 4 client.17937 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:32.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b9947c740 1 -- 172.21.15.78:0/2203963014 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8b3cb6860 con 0x55b8b3d5dd80 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55b8b3d5dd80 0x55b8b3d60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b6ffff640 10 client.17937 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.764+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55b8b3d5dd80 0x55b8b3d60170 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6ef44c3d15d373 server_cookie=3a199a00e535c62f in_seq=0 out_seq=0 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b6ffff640 1 -- 172.21.15.78:0/2203963014 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8b3cb6860 con 0x55b8b3d5dd80 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b6ffff640 10 client.17937 handle_command_reply: tid=0 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 client.17937 shutdown 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 -- 172.21.15.78:0/2203963014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 msgr2=0x55b8b3d4dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d4dd50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c02a940 tx=0x7f5b7c005610 comp rx=0 tx=0).stop 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 -- 172.21.15.78:0/2203963014 shutdown_connections 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55b8b3d5dd80 0x55b8b3d60170 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d4b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b6ffff640 0 client.17937 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:27:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2203963014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d4dd50 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 -- 172.21.15.78:0/2203963014 >> 172.21.15.78:0/2203963014 conn(0x55b8b3d0c310 msgr2=0x55b8b3d45960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 -- 172.21.15.78:0/2203963014 shutdown_connections 2024-09-16T00:27:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.765+0000 7f5b9947c740 1 -- 172.21.15.78:0/2203963014 wait complete. 2024-09-16T00:27:32.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.766+0000 7f5b9947c740 20 client.17937 trim_cache size 0 max 16384 2024-09-16T00:27:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.778+0000 7f5b9947c740 1 Processor -- start 2024-09-16T00:27:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.778+0000 7f5b9947c740 1 -- start start 2024-09-16T00:27:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.778+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d435d0 0x55b8b3d3fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.778+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.778+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.778+0000 7f5b9947c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901ac8f0 con 0x55b8b3d44b40 2024-09-16T00:27:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b9947c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901ae430 con 0x55b8b3d435d0 2024-09-16T00:27:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b9947c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901acfc0 con 0x55b8b3d4e370 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36438/0 (socket says 172.21.15.78:36438) 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 -- 172.21.15.78:0/2428320001 learned_addr learned my addr 172.21.15.78:0/2428320001 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d435d0 0x55b8b3d3fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 -- 172.21.15.78:0/2428320001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 msgr2=0x55b8b3d46120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d46120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 -- 172.21.15.78:0/2428320001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d435d0 msgr2=0x55b8b3d3fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d435d0 0x55b8b3d3fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 -- 172.21.15.78:0/2428320001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8b3d3e310 con 0x55b8b3d44b40 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4cf70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x55b8b3d4c640 tx=0x7f5b84012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e419c82d2175fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b7a7fc640 1 -- 172.21.15.78:0/2428320001 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84004050 con 0x55b8b3d44b40 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b7a7fc640 1 -- 172.21.15.78:0/2428320001 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b840041f0 con 0x55b8b3d44b40 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.779+0000 7f5b7a7fc640 1 -- 172.21.15.78:0/2428320001 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8401d6c0 con 0x55b8b3d44b40 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 -- 172.21.15.78:0/2428320001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 msgr2=0x55b8b3d4cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4cf70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x55b8b3d4c640 tx=0x7f5b84012820 comp rx=0 tx=0).stop 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 -- 172.21.15.78:0/2428320001 shutdown_connections 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d4e370 0x55b8b3d46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d44b40 0x55b8b3d4cf70 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 --2- 172.21.15.78:0/2428320001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d435d0 0x55b8b3d3fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 -- 172.21.15.78:0/2428320001 >> 172.21.15.78:0/2428320001 conn(0x55b8b3d0c310 msgr2=0x55b8b3d4c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 -- 172.21.15.78:0/2428320001 shutdown_connections 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 -- 172.21.15.78:0/2428320001 wait complete. 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 -- start start 2024-09-16T00:27:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.780+0000 7f5b9947c740 1 Processor -- start 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d52110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d44b40 0x55b8b3d52650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b9947c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 0x55b8b3d52b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b9947c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901add60 con 0x55b8b3d435d0 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b9947c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901ad690 con 0x55b8b3d4e370 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b9947c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901aa8d0 con 0x55b8b3d44b40 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b9947c740 10 client.0 _ll_register_callbacks cb 0x7f5b7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d52110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d52110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36454/0 (socket says 172.21.15.78:36454) 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 -- 172.21.15.78:0/3036903892 learned_addr learned my addr 172.21.15.78:0/3036903892 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d44b40 0x55b8b3d52650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 -- 172.21.15.78:0/3036903892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d44b40 msgr2=0x55b8b3d52650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d44b40 0x55b8b3d52650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 -- 172.21.15.78:0/3036903892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 msgr2=0x55b8b3d52b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 0x55b8b3d52b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 -- 172.21.15.78:0/3036903892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8b3dc76e0 con 0x55b8b3d435d0 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b8ffff640 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d52110 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0055e0 tx=0x7f5b7c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b2947491993f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.781+0000 7f5b6ffff640 1 -- 172.21.15.78:0/3036903892 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c031820 con 0x55b8b3d435d0 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b6ffff640 1 -- 172.21.15.78:0/3036903892 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5b7c0319c0 con 0x55b8b3d435d0 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 1 -- 172.21.15.78:0/3036903892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b840041f0 con 0x55b8b3d435d0 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b6ffff640 1 -- 172.21.15.78:0/3036903892 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5b7c031bc0 con 0x55b8b3d435d0 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 10 client.17943 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 1 -- 172.21.15.78:0/3036903892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b8b3dc8930 con 0x55b8b3d435d0 2024-09-16T00:27:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b6ffff640 1 -- 172.21.15.78:0/3036903892 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b840041f0 con 0x55b8b3d435d0 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b6ffff640 1 -- 172.21.15.78:0/3036903892 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5b7c060070 con 0x55b8b3d435d0 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 10 client.17943 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 10 client.17943 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 20 client.17943 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55b8b3dc8e90 0x55b8b3dcb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 4 client.17943 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.782+0000 7f5b9947c740 1 -- 172.21.15.78:0/3036903892 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8b3cb0d70 con 0x55b8b3dc8e90 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55b8b3dc8e90 0x55b8b3dcb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b6ffff640 10 client.17943 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b94e3f640 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55b8b3dc8e90 0x55b8b3dcb2b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64f6e4f18687721e server_cookie=b681763d80f06eac in_seq=0 out_seq=0 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b6ffff640 1 -- 172.21.15.78:0/3036903892 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8b3cb0d70 con 0x55b8b3dc8e90 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b6ffff640 10 client.17943 handle_command_reply: tid=0 2024-09-16T00:27:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 client.17943 shutdown 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 -- 172.21.15.78:0/3036903892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 msgr2=0x55b8b3d52110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d52110 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c0055e0 tx=0x7f5b7c0315b0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 -- 172.21.15.78:0/3036903892 shutdown_connections 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55b8b3dc8e90 0x55b8b3dcb2b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b8b3d4e370 0x55b8b3d52b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b6ffff640 0 client.17943 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b8b3d44b40 0x55b8b3d52650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 --2- 172.21.15.78:0/3036903892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55b8b3d435d0 0x55b8b3d52110 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.783+0000 7f5b9947c740 1 -- 172.21.15.78:0/3036903892 >> 172.21.15.78:0/3036903892 conn(0x55b8b3d0c310 msgr2=0x55b8b3d4d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b9947c740 1 -- 172.21.15.78:0/3036903892 shutdown_connections 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b9947c740 1 -- 172.21.15.78:0/3036903892 wait complete. 2024-09-16T00:27:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b9947c740 20 client.17943 trim_cache size 0 max 16384 2024-09-16T00:27:32.797 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:27:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 -- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5b800a3240 msgr2=0x7f5b800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5b800a3240 0x7f5b800a5700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5b88097a30 tx=0x7f5b8800e040 comp rx=0 tx=0).stop 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 -- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90106a80 msgr2=0x7f5b9019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90106a80 0x7f5b9019cd80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b970 tx=0x7f5b8800be30 comp rx=0 tx=0).stop 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 -- 172.21.15.78:0/4001500988 shutdown_connections 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5b800a3240 0x7f5b800a5700 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b90106a80 0x7f5b9019cd80 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b90101420 0x7f5b901989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 --2- 172.21.15.78:0/4001500988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5b90100a80 0x7f5b901984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 -- 172.21.15.78:0/4001500988 >> 172.21.15.78:0/4001500988 conn(0x7f5b900fc7a0 msgr2=0x7f5b90104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 -- 172.21.15.78:0/4001500988 shutdown_connections 2024-09-16T00:27:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:32.784+0000 7f5b970a3640 1 -- 172.21.15.78:0/4001500988 wait complete. 2024-09-16T00:27:33.598 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:27:33.852 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:33.961 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:33 smithi078 ceph-mon[31157]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-09-16T00:27:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:33 smithi193 ceph-mon[35484]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-09-16T00:27:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:33 smithi148 ceph-mon[35599]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 -- 172.21.15.78:0/2533147095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 msgr2=0x7fbee4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 --2- 172.21.15.78:0/2533147095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 0x7fbee4101200 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009900 tx=0x7fbed402a480 comp rx=0 tx=0).stop 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 -- 172.21.15.78:0/2533147095 shutdown_connections 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 --2- 172.21.15.78:0/2533147095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee4102ef0 0x7fbee410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 --2- 172.21.15.78:0/2533147095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbee4102570 0x7fbee41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 --2- 172.21.15.78:0/2533147095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 0x7fbee4101200 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 -- 172.21.15.78:0/2533147095 >> 172.21.15.78:0/2533147095 conn(0x7fbee40fc7d0 msgr2=0x7fbee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 -- 172.21.15.78:0/2533147095 shutdown_connections 2024-09-16T00:27:34.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.456+0000 7fbeea393640 1 -- 172.21.15.78:0/2533147095 wait complete. 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbeea393640 1 Processor -- start 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbeea393640 1 -- start start 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbeea393640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 0x7fbee41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbeea393640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee4102570 0x7fbee4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbeea393640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbee4102ef0 0x7fbee41a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbeea393640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4111020 con 0x7fbee4100e20 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbeea393640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4110ea0 con 0x7fbee4102570 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbeea393640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41111a0 con 0x7fbee4102ef0 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 0x7fbee41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 0x7fbee41994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36478/0 (socket says 172.21.15.78:36478) 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee3fff640 1 -- 172.21.15.78:0/2805500415 learned_addr learned my addr 172.21.15.78:0/2805500415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:34.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee37fe640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee4102570 0x7fbee4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee3fff640 1 -- 172.21.15.78:0/2805500415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbee4102ef0 msgr2=0x7fbee41a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee3fff640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbee4102ef0 0x7fbee41a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee3fff640 1 -- 172.21.15.78:0/2805500415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee4102570 msgr2=0x7fbee4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee3fff640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee4102570 0x7fbee4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee3fff640 1 -- 172.21.15.78:0/2805500415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee41a4ab0 con 0x7fbee4100e20 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.457+0000 7fbee37fe640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee4102570 0x7fbee4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.458+0000 7fbee3fff640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 0x7fbee41994f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbed4006010 tx=0x7fbed4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.458+0000 7fbee17fa640 1 -- 172.21.15.78:0/2805500415 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed403f070 con 0x7fbee4100e20 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.458+0000 7fbeea393640 1 -- 172.21.15.78:0/2805500415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee41a4c80 con 0x7fbee4100e20 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.458+0000 7fbee17fa640 1 -- 172.21.15.78:0/2805500415 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed4031e30 con 0x7fbee4100e20 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.458+0000 7fbee17fa640 1 -- 172.21.15.78:0/2805500415 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4035030 con 0x7fbee4100e20 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.458+0000 7fbeea393640 1 -- 172.21.15.78:0/2805500415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee41a5130 con 0x7fbee4100e20 2024-09-16T00:27:34.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.460+0000 7fbee17fa640 1 -- 172.21.15.78:0/2805500415 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fbed4042070 con 0x7fbee4100e20 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbeec76c740 1 Processor -- start 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbeec76c740 1 -- start start 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfbaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbeec76c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559a0df202f0 con 0x559a0dfbbb40 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbeec76c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559a0dec7cf0 con 0x559a0dfc5370 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbeec76c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a0df1c2b0 con 0x559a0dfba5d0 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbee8930640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbee8930640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfba090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36488/0 (socket says 172.21.15.78:36488) 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbee8930640 1 -- 172.21.15.78:0/2880226515 learned_addr learned my addr 172.21.15.78:0/2880226515 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:34.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbee3fff640 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfbaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbee37fe640 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.461+0000 7fbee17fa640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbecc0a3240 0x7fbecc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee8930640 1 -- 172.21.15.78:0/2880226515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 msgr2=0x559a0dfbaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee8930640 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfbaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee8930640 1 -- 172.21.15.78:0/2880226515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 msgr2=0x559a0dfc7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee8930640 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee8930640 1 -- 172.21.15.78:0/2880226515 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a0deac3a0 con 0x559a0dfbbb40 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee3fff640 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfbaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee37fe640 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee17fa640 1 -- 172.21.15.78:0/2805500415 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbed40f06d0 con 0x7fbee4100e20 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee8930640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbecc0a3240 0x7fbecc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbee8930640 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfba090 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009ec0 tx=0x7fbedc00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efea51b241ae906 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbeca7fc640 1 -- 172.21.15.78:0/2880226515 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc019040 con 0x559a0dfbbb40 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbeca7fc640 1 -- 172.21.15.78:0/2880226515 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc01a020 con 0x559a0dfbbb40 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.462+0000 7fbeca7fc640 1 -- 172.21.15.78:0/2880226515 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc01f8c0 con 0x559a0dfbbb40 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 -- 172.21.15.78:0/2880226515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 msgr2=0x559a0dfba090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbee8930640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbecc0a3240 0x7fbecc0a5700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbedc01b000 tx=0x7fbedc01c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfba090 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009ec0 tx=0x7fbedc00b420 comp rx=0 tx=0).stop 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 -- 172.21.15.78:0/2880226515 shutdown_connections 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfbaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 --2- 172.21.15.78:0/2880226515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfba090 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 -- 172.21.15.78:0/2880226515 >> 172.21.15.78:0/2880226515 conn(0x559a0df83310 msgr2=0x559a0dfb6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 -- 172.21.15.78:0/2880226515 shutdown_connections 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 -- 172.21.15.78:0/2880226515 wait complete. 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.463+0000 7fbeec76c740 1 -- start start 2024-09-16T00:27:34.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbeec76c740 1 Processor -- start 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfba5d0 0x559a0dfc4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfc5370 0x559a0dfc2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbeec76c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x559a0dfc8070 con 0x559a0dfc5370 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbeec76c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559a0dedb060 con 0x559a0dfba5d0 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbeec76c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a0dfc81f0 con 0x559a0dfbbb40 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbeec76c740 10 client.0 _ll_register_callbacks cb 0x559a0dc2e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee8930640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfc5370 0x559a0dfc2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee8930640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfc5370 0x559a0dfc2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36504/0 (socket says 172.21.15.78:36504) 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee8930640 1 -- 172.21.15.78:0/751709601 learned_addr learned my addr 172.21.15.78:0/751709601 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee3fff640 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfba5d0 0x559a0dfc4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee37fe640 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee8930640 1 -- 172.21.15.78:0/751709601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 msgr2=0x559a0dfc1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:34.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee8930640 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee8930640 1 -- 172.21.15.78:0/751709601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfba5d0 msgr2=0x559a0dfc4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee8930640 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfba5d0 0x559a0dfc4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee8930640 1 -- 172.21.15.78:0/751709601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a0dc913c0 con 0x559a0dfc5370 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.464+0000 7fbee37fe640 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbee8930640 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfc5370 0x559a0dfc2530 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbedc004c00 tx=0x7fbedc020e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3935546ad3c6edb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbec37fe640 1 -- 172.21.15.78:0/751709601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc02c030 con 0x559a0dfc5370 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 1 -- 172.21.15.78:0/751709601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbedc01a020 con 0x559a0dfc5370 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbec37fe640 1 -- 172.21.15.78:0/751709601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbedc014830 con 0x559a0dfc5370 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbec37fe640 1 -- 172.21.15.78:0/751709601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbedc036050 con 0x559a0dfc5370 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 10 client.17955 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 1 -- 172.21.15.78:0/751709601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a0dfd3ad0 con 0x559a0dfc5370 2024-09-16T00:27:34.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbec37fe640 1 -- 172.21.15.78:0/751709601 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc01a020 con 0x559a0dfc5370 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbec37fe640 1 -- 172.21.15.78:0/751709601 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbedc027030 con 0x559a0dfc5370 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 10 client.17955 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 10 client.17955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 20 client.17955 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559a0dfd4ed0 0x559a0dfd72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 4 client.17955 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.465+0000 7fbeec76c740 1 -- 172.21.15.78:0/751709601 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a0df2d860 con 0x559a0dfd4ed0 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.466+0000 7fbee3fff640 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559a0dfd4ed0 0x559a0dfd72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.466+0000 7fbec37fe640 10 client.17955 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.466+0000 7fbee3fff640 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559a0dfd4ed0 0x559a0dfd72c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4786a1ca91df7a2 server_cookie=8ada197f3e19b5ae in_seq=0 out_seq=0 2024-09-16T00:27:34.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbec37fe640 1 -- 172.21.15.78:0/751709601 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a0df2d860 con 0x559a0dfd4ed0 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbec37fe640 10 client.17955 handle_command_reply: tid=0 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 client.17955 shutdown 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 -- 172.21.15.78:0/751709601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfc5370 msgr2=0x559a0dfc2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfc5370 0x559a0dfc2530 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbedc004c00 tx=0x7fbedc020e00 comp rx=0 tx=0).stop 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 -- 172.21.15.78:0/751709601 shutdown_connections 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559a0dfd4ed0 0x559a0dfd72c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbec37fe640 0 client.17955 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfc5370 0x559a0dfc2530 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 --2- 172.21.15.78:0/751709601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfba5d0 0x559a0dfc4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 -- 172.21.15.78:0/751709601 >> 172.21.15.78:0/751709601 conn(0x559a0df83310 msgr2=0x559a0dfbc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 -- 172.21.15.78:0/751709601 shutdown_connections 2024-09-16T00:27:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 1 -- 172.21.15.78:0/751709601 wait complete. 2024-09-16T00:27:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.467+0000 7fbeec76c740 20 client.17955 trim_cache size 0 max 16384 2024-09-16T00:27:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbeec76c740 1 Processor -- start 2024-09-16T00:27:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbeec76c740 1 -- start start 2024-09-16T00:27:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfba5d0 0x559a0dfd1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.548 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:27:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfdcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbeec76c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41aec30 con 0x559a0dfba5d0 2024-09-16T00:27:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbeec76c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41b0770 con 0x559a0dfc5370 2024-09-16T00:27:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbeec76c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41af300 con 0x559a0dfbbb40 2024-09-16T00:27:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbee3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfba5d0 0x559a0dfd1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbee3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfba5d0 0x559a0dfd1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36516/0 (socket says 172.21.15.78:36516) 2024-09-16T00:27:34.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.493+0000 7fbee3fff640 1 -- 172.21.15.78:0/1561377250 learned_addr learned my addr 172.21.15.78:0/1561377250 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbee8930640 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfdcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbee37fe640 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbee3fff640 1 -- 172.21.15.78:0/1561377250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 msgr2=0x559a0dfc0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbee3fff640 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbee3fff640 1 -- 172.21.15.78:0/1561377250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 msgr2=0x559a0dfdcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbee3fff640 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfdcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbee3fff640 1 -- 172.21.15.78:0/1561377250 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a0dfc5070 con 0x559a0dfba5d0 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbee3fff640 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfba5d0 0x559a0dfd1340 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbed40b8d90 tx=0x7fbed4034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b738bc58fecac5db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbec9ffb640 1 -- 172.21.15.78:0/1561377250 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4049040 con 0x559a0dfba5d0 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbec9ffb640 1 -- 172.21.15.78:0/1561377250 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed40bc930 con 0x559a0dfba5d0 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.494+0000 7fbec9ffb640 1 -- 172.21.15.78:0/1561377250 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed40f03d0 con 0x559a0dfba5d0 2024-09-16T00:27:34.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 -- 172.21.15.78:0/1561377250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfba5d0 msgr2=0x559a0dfd1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfba5d0 0x559a0dfd1340 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbed40b8d90 tx=0x7fbed4034000 comp rx=0 tx=0).stop 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 -- 172.21.15.78:0/1561377250 shutdown_connections 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfdcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfbbb40 0x559a0dfc0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 --2- 172.21.15.78:0/1561377250 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfba5d0 0x559a0dfd1340 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 -- 172.21.15.78:0/1561377250 >> 172.21.15.78:0/1561377250 conn(0x559a0df83310 msgr2=0x559a0dfc0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 -- 172.21.15.78:0/1561377250 shutdown_connections 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 -- 172.21.15.78:0/1561377250 wait complete. 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 -- start start 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.495+0000 7fbeec76c740 1 Processor -- start 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfcd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfc8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbeec76c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbeec76c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41b00a0 con 0x559a0dfbbb40 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbeec76c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41af9d0 con 0x559a0dfc5370 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbeec76c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41acc10 con 0x559a0dfba5d0 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbeec76c740 10 client.0 _ll_register_callbacks cb 0x7fbedc009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfc8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfc8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36520/0 (socket says 172.21.15.78:36520) 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 -- 172.21.15.78:0/593174358 learned_addr learned my addr 172.21.15.78:0/593174358 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee8930640 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee3fff640 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfcd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 -- 172.21.15.78:0/593174358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 msgr2=0x559a0dfcd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfcd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 -- 172.21.15.78:0/593174358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 msgr2=0x559a0dfc9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 -- 172.21.15.78:0/593174358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a0dfc9920 con 0x559a0dfbbb40 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbee37fe640 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfc8e20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fbee406aac0 tx=0x7fbed8012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c66e3c3dc69825 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.496+0000 7fbec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbec37fe640 1 -- 172.21.15.78:0/593174358 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed801a070 con 0x559a0dfbbb40 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbec37fe640 1 -- 172.21.15.78:0/593174358 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbed8015600 con 0x559a0dfbbb40 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 1 -- 172.21.15.78:0/593174358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed40bc930 con 0x559a0dfbbb40 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbec37fe640 1 -- 172.21.15.78:0/593174358 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbed8026050 con 0x559a0dfbbb40 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 10 client.17961 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 1 -- 172.21.15.78:0/593174358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a0e03f920 con 0x559a0dfbbb40 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbec37fe640 1 -- 172.21.15.78:0/593174358 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbed40bc930 con 0x559a0dfbbb40 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbec37fe640 1 -- 172.21.15.78:0/593174358 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbed801e930 con 0x559a0dfbbb40 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 10 client.17961 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:34.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 10 client.17961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 20 client.17961 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559a0dfd4ed0 0x559a0e042140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 4 client.17961 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbeec76c740 1 -- 172.21.15.78:0/593174358 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a0df27d70 con 0x559a0dfd4ed0 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.497+0000 7fbee3fff640 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559a0dfd4ed0 0x559a0e042140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbec37fe640 10 client.17961 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbee3fff640 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559a0dfd4ed0 0x559a0e042140 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce070aff28b649e server_cookie=d137a00b70fd8ceb in_seq=0 out_seq=0 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbec37fe640 1 -- 172.21.15.78:0/593174358 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a0df27d70 con 0x559a0dfd4ed0 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbec37fe640 10 client.17961 handle_command_reply: tid=0 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 client.17961 shutdown 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 -- 172.21.15.78:0/593174358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 msgr2=0x559a0dfc8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfc8e20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fbee406aac0 tx=0x7fbed8012700 comp rx=0 tx=0).stop 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 -- 172.21.15.78:0/593174358 shutdown_connections 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559a0dfd4ed0 0x559a0e042140 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559a0dfc5370 0x559a0dfc9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x559a0dfbbb40 0x559a0dfc8e20 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 --2- 172.21.15.78:0/593174358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a0dfba5d0 0x559a0dfcd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 -- 172.21.15.78:0/593174358 >> 172.21.15.78:0/593174358 conn(0x559a0df83310 msgr2=0x559a0dfc10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbec37fe640 0 client.17961 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 -- 172.21.15.78:0/593174358 shutdown_connections 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.498+0000 7fbeec76c740 1 -- 172.21.15.78:0/593174358 wait complete. 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeec76c740 20 client.17961 trim_cache size 0 max 16384 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 -- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbecc0a3240 msgr2=0x7fbecc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbecc0a3240 0x7fbecc0a5700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbedc01b000 tx=0x7fbedc01c000 comp rx=0 tx=0).stop 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 -- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 msgr2=0x7fbee41994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 0x7fbee41994f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fbed4006010 tx=0x7fbed4003c80 comp rx=0 tx=0).stop 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 -- 172.21.15.78:0/2805500415 shutdown_connections 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbecc0a3240 0x7fbecc0a5700 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbee4102ef0 0x7fbee41a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbee4102570 0x7fbee4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 --2- 172.21.15.78:0/2805500415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbee4100e20 0x7fbee41994f0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 -- 172.21.15.78:0/2805500415 >> 172.21.15.78:0/2805500415 conn(0x7fbee40fc7d0 msgr2=0x7fbee40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 -- 172.21.15.78:0/2805500415 shutdown_connections 2024-09-16T00:27:34.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:34.499+0000 7fbeea393640 1 -- 172.21.15.78:0/2805500415 wait complete. 2024-09-16T00:27:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 815.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:27:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:35 smithi193 ceph-mon[35484]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 543 KiB/s wr, 80 op/s 2024-09-16T00:27:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:35 smithi148 ceph-mon[35599]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 543 KiB/s wr, 80 op/s 2024-09-16T00:27:36.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:35 smithi078 ceph-mon[31157]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 543 KiB/s wr, 80 op/s 2024-09-16T00:27:36.303 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:27:36.303 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:27:36.571 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:37.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 -- 172.21.15.78:0/2241310716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c102790 msgr2=0x7fdb5c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 --2- 172.21.15.78:0/2241310716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c102790 0x7fdb5c1067e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009900 tx=0x7fdb5002a410 comp rx=0 tx=0).stop 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 -- 172.21.15.78:0/2241310716 shutdown_connections 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 --2- 172.21.15.78:0/2241310716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb5c106f10 0x7fdb5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 --2- 172.21.15.78:0/2241310716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c102790 0x7fdb5c1067e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 --2- 172.21.15.78:0/2241310716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb5c101de0 0x7fdb5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 -- 172.21.15.78:0/2241310716 >> 172.21.15.78:0/2241310716 conn(0x7fdb5c0fd8c0 msgr2=0x7fdb5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 -- 172.21.15.78:0/2241310716 shutdown_connections 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 -- 172.21.15.78:0/2241310716 wait complete. 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 Processor -- start 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 -- start start 2024-09-16T00:27:37.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c101de0 0x7fdb5c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.095+0000 7fdb646db640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb5c102790 0x7fdb5c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb646db640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb5c106f10 0x7fdb5c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb646db640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c111f30 con 0x7fdb5c101de0 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb646db640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c111db0 con 0x7fdb5c102790 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb646db640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1120b0 con 0x7fdb5c106f10 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c101de0 0x7fdb5c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c101de0 0x7fdb5c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38168/0 (socket says 172.21.15.78:38168) 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 -- 172.21.15.78:0/2986509956 learned_addr learned my addr 172.21.15.78:0/2986509956 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62c78640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb5c106f10 0x7fdb5c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 -- 172.21.15.78:0/2986509956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb5c106f10 msgr2=0x7fdb5c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb5c106f10 0x7fdb5c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 -- 172.21.15.78:0/2986509956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb5c102790 msgr2=0x7fdb5c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb5c102790 0x7fdb5c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 -- 172.21.15.78:0/2986509956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c19d3a0 con 0x7fdb5c101de0 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62c78640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb5c106f10 0x7fdb5c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb62477640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c101de0 0x7fdb5c1983f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00cce0 tx=0x7fdb4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb5b7fe640 1 -- 172.21.15.78:0/2986509956 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00f650 con 0x7fdb5c101de0 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb5b7fe640 1 -- 172.21.15.78:0/2986509956 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c00fcd0 con 0x7fdb5c101de0 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb646db640 1 -- 172.21.15.78:0/2986509956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c19d690 con 0x7fdb5c101de0 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb5b7fe640 1 -- 172.21.15.78:0/2986509956 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c018c70 con 0x7fdb5c101de0 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.096+0000 7fdb646db640 1 -- 172.21.15.78:0/2986509956 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1a4f70 con 0x7fdb5c101de0 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb5b7fe640 1 -- 172.21.15.78:0/2986509956 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fdb4c00f7f0 con 0x7fdb5c101de0 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb66ab4740 1 Processor -- start 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb66ab4740 1 -- start start 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad44e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad45360 0x55af6ad53c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad56540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af6acaa280 con 0x55af6ad54150 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55af6ac51b10 con 0x55af6ad45360 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af6aca6240 con 0x55af6ad44a40 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb61c76640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad56540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb5b7fe640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb440a3240 0x7fdb440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb61c76640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad56540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38174/0 (socket says 172.21.15.78:38174) 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb61c76640 1 -- 172.21.15.78:0/493832483 learned_addr learned my addr 172.21.15.78:0/493832483 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.099+0000 7fdb62477640 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad45360 0x55af6ad53c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb62c78640 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad44e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb5b7fe640 1 -- 172.21.15.78:0/2986509956 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdb4c0d01d0 con 0x7fdb5c101de0 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb61c76640 1 -- 172.21.15.78:0/493832483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 msgr2=0x55af6ad44e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb61c76640 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad44e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb61c76640 1 -- 172.21.15.78:0/493832483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad45360 msgr2=0x55af6ad53c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb61c76640 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad45360 0x55af6ad53c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb61c76640 1 -- 172.21.15.78:0/493832483 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6ac5e1f0 con 0x55af6ad54150 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb62477640 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad45360 0x55af6ad53c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb62c78640 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad44e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb62c78640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb440a3240 0x7fdb440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb61c76640 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad56540 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009a20 tx=0x7fdb50031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88a067d193b3553 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb58ff9640 1 -- 172.21.15.78:0/493832483 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50040070 con 0x55af6ad54150 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb58ff9640 1 -- 172.21.15.78:0/493832483 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb500317b0 con 0x55af6ad54150 2024-09-16T00:27:37.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.100+0000 7fdb58ff9640 1 -- 172.21.15.78:0/493832483 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50035040 con 0x55af6ad54150 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 -- 172.21.15.78:0/493832483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 msgr2=0x55af6ad56540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb62c78640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb440a3240 0x7fdb440a5700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400aa60 tx=0x7fdb540046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad56540 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009a20 tx=0x7fdb50031510 comp rx=0 tx=0).stop 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 -- 172.21.15.78:0/493832483 shutdown_connections 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad56540 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad45360 0x55af6ad53c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/493832483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad44e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 -- 172.21.15.78:0/493832483 >> 172.21.15.78:0/493832483 conn(0x55af6ad410f0 msgr2=0x55af6ad414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 -- 172.21.15.78:0/493832483 shutdown_connections 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 -- 172.21.15.78:0/493832483 wait complete. 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 -- start start 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.101+0000 7fdb66ab4740 1 Processor -- start 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad45360 0x55af6ad51570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad51ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55af6ad56d20 con 0x55af6ad54150 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55af6ac64f10 con 0x55af6ad44a40 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af6ad56ea0 con 0x55af6ad45360 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb66ab4740 10 client.0 _ll_register_callbacks cb 0x55af6ac2da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad51ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad51ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38180/0 (socket says 172.21.15.78:38180) 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 -- 172.21.15.78:0/3970941569 learned_addr learned my addr 172.21.15.78:0/3970941569 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb61c76640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad45360 0x55af6ad51570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62477640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 -- 172.21.15.78:0/3970941569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad45360 msgr2=0x55af6ad51570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad45360 0x55af6ad51570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 -- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 msgr2=0x55af6ad51030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:37.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad51030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 -- 172.21.15.78:0/3970941569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6aa1b6a0 con 0x55af6ad54150 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb61c76640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad45360 0x55af6ad51570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62477640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad51030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb62c78640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad51ab0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005f70 tx=0x7fdb54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d85dded299b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.102+0000 7fdb41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb41ffb640 1 -- 172.21.15.78:0/3970941569 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54020070 con 0x55af6ad54150 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb41ffb640 1 -- 172.21.15.78:0/3970941569 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdb5400fd50 con 0x55af6ad54150 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 1 -- 172.21.15.78:0/3970941569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb500317b0 con 0x55af6ad54150 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb41ffb640 1 -- 172.21.15.78:0/3970941569 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb5402c050 con 0x55af6ad54150 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 10 client.17973 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 1 -- 172.21.15.78:0/3970941569 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55af6ad62920 con 0x55af6ad54150 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb41ffb640 1 -- 172.21.15.78:0/3970941569 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb500317b0 con 0x55af6ad54150 2024-09-16T00:27:37.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb41ffb640 1 -- 172.21.15.78:0/3970941569 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdb54018070 con 0x55af6ad54150 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 10 client.17973 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 20 client.17973 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55af6ad63d30 0x55af6ad661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 4 client.17973 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.103+0000 7fdb66ab4740 1 -- 172.21.15.78:0/3970941569 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af6acb77f0 con 0x55af6ad63d30 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.104+0000 7fdb62477640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55af6ad63d30 0x55af6ad661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.104+0000 7fdb41ffb640 10 client.17973 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.104+0000 7fdb62477640 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55af6ad63d30 0x55af6ad661a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737c125cdf44a496 server_cookie=3193341528348338 in_seq=0 out_seq=0 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb41ffb640 1 -- 172.21.15.78:0/3970941569 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af6acb77f0 con 0x55af6ad63d30 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb41ffb640 10 client.17973 handle_command_reply: tid=0 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 client.17973 shutdown 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 -- 172.21.15.78:0/3970941569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 msgr2=0x55af6ad51ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad51ab0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005f70 tx=0x7fdb54004730 comp rx=0 tx=0).stop 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 -- 172.21.15.78:0/3970941569 shutdown_connections 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55af6ad63d30 0x55af6ad661a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad54150 0x55af6ad51ab0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad45360 0x55af6ad51570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/3970941569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad51030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb41ffb640 0 client.17973 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 -- 172.21.15.78:0/3970941569 >> 172.21.15.78:0/3970941569 conn(0x55af6ad410f0 msgr2=0x55af6ad46e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 -- 172.21.15.78:0/3970941569 shutdown_connections 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 1 -- 172.21.15.78:0/3970941569 wait complete. 2024-09-16T00:27:37.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.105+0000 7fdb66ab4740 20 client.17973 trim_cache size 0 max 16384 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.122+0000 7fdb66ab4740 1 Processor -- start 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.122+0000 7fdb66ab4740 1 -- start start 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.122+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad4fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.122+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad6ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad54150 0x55af6ad6bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ac950 con 0x55af6ad45360 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ae490 con 0x55af6ad54150 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ad020 con 0x55af6ad44a40 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad6ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad6ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38184/0 (socket says 172.21.15.78:38184) 2024-09-16T00:27:37.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 -- 172.21.15.78:0/1276962285 learned_addr learned my addr 172.21.15.78:0/1276962285 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb62c78640 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad54150 0x55af6ad6bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb62477640 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad4fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 -- 172.21.15.78:0/1276962285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 msgr2=0x55af6ad4fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad4fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 -- 172.21.15.78:0/1276962285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad54150 msgr2=0x55af6ad6bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad54150 0x55af6ad6bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 -- 172.21.15.78:0/1276962285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6ad45190 con 0x55af6ad45360 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb61c76640 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad6ba70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009230 tx=0x7fdb50005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c70c1e5abe0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb43fff640 1 -- 172.21.15.78:0/1276962285 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50035040 con 0x55af6ad45360 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb43fff640 1 -- 172.21.15.78:0/1276962285 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb500317b0 con 0x55af6ad45360 2024-09-16T00:27:37.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.123+0000 7fdb43fff640 1 -- 172.21.15.78:0/1276962285 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50040040 con 0x55af6ad45360 2024-09-16T00:27:37.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 -- 172.21.15.78:0/1276962285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 msgr2=0x55af6ad6ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad6ba70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009230 tx=0x7fdb50005e40 comp rx=0 tx=0).stop 2024-09-16T00:27:37.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 -- 172.21.15.78:0/1276962285 shutdown_connections 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad54150 0x55af6ad6bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad6ba70 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/1276962285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad44a40 0x55af6ad4fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 -- 172.21.15.78:0/1276962285 >> 172.21.15.78:0/1276962285 conn(0x55af6ad410f0 msgr2=0x55af6ad4f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 -- 172.21.15.78:0/1276962285 shutdown_connections 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 -- 172.21.15.78:0/1276962285 wait complete. 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.124+0000 7fdb66ab4740 1 -- start start 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb66ab4740 1 Processor -- start 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad58070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb66ab4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad54150 0x55af6ad58af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1addc0 con 0x55af6ad45360 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1ad6f0 con 0x55af6ad44a40 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb66ab4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1aa930 con 0x55af6ad54150 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb66ab4740 10 client.0 _ll_register_callbacks cb 0x7fdb5400a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38192/0 (socket says 172.21.15.78:38192) 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 -- 172.21.15.78:0/595111110 learned_addr learned my addr 172.21.15.78:0/595111110 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb62477640 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad58070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb62c78640 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad54150 0x55af6ad58af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 -- 172.21.15.78:0/595111110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad54150 msgr2=0x55af6ad58af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad54150 0x55af6ad58af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 -- 172.21.15.78:0/595111110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 msgr2=0x55af6ad58070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad58070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 -- 172.21.15.78:0/595111110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6adcdbe0 con 0x55af6ad45360 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb61c76640 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad585b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fdb500092a0 tx=0x7fdb500095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4654a64d2592c7ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.125+0000 7fdb41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb41ffb640 1 -- 172.21.15.78:0/595111110 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5005c060 con 0x55af6ad45360 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb41ffb640 1 -- 172.21.15.78:0/595111110 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fdb50045860 con 0x55af6ad45360 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 1 -- 172.21.15.78:0/595111110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb500317b0 con 0x55af6ad45360 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb41ffb640 1 -- 172.21.15.78:0/595111110 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb50063050 con 0x55af6ad45360 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 10 client.17979 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 1 -- 172.21.15.78:0/595111110 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55af6adcef00 con 0x55af6ad45360 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb41ffb640 1 -- 172.21.15.78:0/595111110 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb500317b0 con 0x55af6ad45360 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb41ffb640 1 -- 172.21.15.78:0/595111110 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdb5003a3f0 con 0x55af6ad45360 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 10 client.17979 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:27:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 20 client.17979 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55af6adcf170 0x55af6add1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 4 client.17979 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.126+0000 7fdb66ab4740 1 -- 172.21.15.78:0/595111110 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55af6acb1d00 con 0x55af6adcf170 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.127+0000 7fdb62477640 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55af6adcf170 0x55af6add1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.127+0000 7fdb62477640 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55af6adcf170 0x55af6add1560 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4927ebc49331caf server_cookie=d19ba23dbb06076b in_seq=0 out_seq=0 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.127+0000 7fdb41ffb640 10 client.17979 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb41ffb640 1 -- 172.21.15.78:0/595111110 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55af6acb1d00 con 0x55af6adcf170 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb41ffb640 10 client.17979 handle_command_reply: tid=0 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 client.17979 shutdown 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 -- 172.21.15.78:0/595111110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 msgr2=0x55af6ad585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad585b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fdb500092a0 tx=0x7fdb500095a0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 -- 172.21.15.78:0/595111110 shutdown_connections 2024-09-16T00:27:37.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55af6adcf170 0x55af6add1560 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af6ad54150 0x55af6ad58af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55af6ad45360 0x55af6ad585b0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 --2- 172.21.15.78:0/595111110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55af6ad44a40 0x55af6ad58070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.128+0000 7fdb66ab4740 1 -- 172.21.15.78:0/595111110 >> 172.21.15.78:0/595111110 conn(0x55af6ad410f0 msgr2=0x55af6ad4fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb41ffb640 0 client.17979 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb66ab4740 1 -- 172.21.15.78:0/595111110 shutdown_connections 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb66ab4740 1 -- 172.21.15.78:0/595111110 wait complete. 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb66ab4740 20 client.17979 trim_cache size 0 max 16384 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:27:37.135 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:27:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "f75592b2-2187-42f8-87fe-f7db668db440", 2024-09-16T00:27:37.136 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:27:37.136 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 -- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb440a3240 msgr2=0x7fdb440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb440a3240 0x7fdb440a5700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400aa60 tx=0x7fdb540046c0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 -- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c101de0 msgr2=0x7fdb5c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c101de0 0x7fdb5c1983f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00cce0 tx=0x7fdb4c00a620 comp rx=0 tx=0).stop 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 -- 172.21.15.78:0/2986509956 shutdown_connections 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fdb440a3240 0x7fdb440a5700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb5c106f10 0x7fdb5c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb5c102790 0x7fdb5c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 --2- 172.21.15.78:0/2986509956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb5c101de0 0x7fdb5c1983f0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 -- 172.21.15.78:0/2986509956 >> 172.21.15.78:0/2986509956 conn(0x7fdb5c0fd8c0 msgr2=0x7fdb5c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 -- 172.21.15.78:0/2986509956 shutdown_connections 2024-09-16T00:27:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:37.129+0000 7fdb646db640 1 -- 172.21.15.78:0/2986509956 wait complete. 2024-09-16T00:27:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:37 smithi078 ceph-mon[31157]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-09-16T00:27:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:37 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:27:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:37 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:27:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:37 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:38.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:27:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:37 smithi193 ceph-mon[35484]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-09-16T00:27:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:37 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:27:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:37 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:27:38.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:37 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:37 smithi148 ceph-mon[35599]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-09-16T00:27:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:37 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:27:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:37 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:27:38.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:37 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:38.309 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:38.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 -- 172.21.15.78:0/715380485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56741036c0 msgr2=0x7f567410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 --2- 172.21.15.78:0/715380485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56741036c0 0x7f567410bbb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5668009900 tx=0x7f566802a480 comp rx=0 tx=0).stop 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 -- 172.21.15.78:0/715380485 shutdown_connections 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 --2- 172.21.15.78:0/715380485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f567410c180 0x7f567410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 --2- 172.21.15.78:0/715380485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56741036c0 0x7f567410bbb0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 --2- 172.21.15.78:0/715380485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5674102da0 0x7f5674103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 -- 172.21.15.78:0/715380485 >> 172.21.15.78:0/715380485 conn(0x7f56740fc7a0 msgr2=0x7f56740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 -- 172.21.15.78:0/715380485 shutdown_connections 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.870+0000 7f567ae37640 1 -- 172.21.15.78:0/715380485 wait complete. 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f567ae37640 1 Processor -- start 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f567ae37640 1 -- start start 2024-09-16T00:27:38.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f567ae37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674102da0 0x7f5674100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f567ae37640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56741036c0 0x7f5674100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f567ae37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f567410c180 0x7f5674101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f567ae37640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5674111f60 con 0x7f567410c180 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f567ae37640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5674111de0 con 0x7f5674102da0 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f567ae37640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56741120e0 con 0x7f56741036c0 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f56793d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f567410c180 0x7f5674101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f56793d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f567410c180 0x7f5674101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38206/0 (socket says 172.21.15.78:38206) 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f56793d4640 1 -- 172.21.15.78:0/2897499809 learned_addr learned my addr 172.21.15.78:0/2897499809 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.871+0000 7f5678bd3640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674102da0 0x7f5674100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f5673fff640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56741036c0 0x7f5674100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f56793d4640 1 -- 172.21.15.78:0/2897499809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56741036c0 msgr2=0x7f5674100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f56793d4640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56741036c0 0x7f5674100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f56793d4640 1 -- 172.21.15.78:0/2897499809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674102da0 msgr2=0x7f5674100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f56793d4640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674102da0 0x7f5674100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f56793d4640 1 -- 172.21.15.78:0/2897499809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741a68a0 con 0x7f567410c180 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f5678bd3640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674102da0 0x7f5674100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f5673fff640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56741036c0 0x7f5674100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f56793d4640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f567410c180 0x7f5674101480 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f566400c970 tx=0x7f566400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f5671ffb640 1 -- 172.21.15.78:0/2897499809 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664007bb0 con 0x7f567410c180 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f567ae37640 1 -- 172.21.15.78:0/2897499809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741a6b90 con 0x7f567410c180 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f5671ffb640 1 -- 172.21.15.78:0/2897499809 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5664007d50 con 0x7f567410c180 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f5671ffb640 1 -- 172.21.15.78:0/2897499809 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664018800 con 0x7f567410c180 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.872+0000 7f567ae37640 1 -- 172.21.15.78:0/2897499809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741a7080 con 0x7f567410c180 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f5671ffb640 1 -- 172.21.15.78:0/2897499809 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f566400fa50 con 0x7f567410c180 2024-09-16T00:27:38.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f567d210740 1 Processor -- start 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f567d210740 1 -- start start 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f5671ffb640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f565c0a3240 0x7f565c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f567d210740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f567d210740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 0x562ea50df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f567d210740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 0x562ea50e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f567d210740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562ea503a2e0 con 0x562ea50d44a0 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f567d210740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ea4fe1ce0 con 0x562ea50df680 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f567d210740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ea50362a0 con 0x562ea50d4dc0 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f5671ffb640 1 -- 172.21.15.78:0/2897499809 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f56640cfd00 con 0x7f567410c180 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f5678bd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f5678bd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50d4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38220/0 (socket says 172.21.15.78:38220) 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.875+0000 7f5678bd3640 1 -- 172.21.15.78:0/2321795214 learned_addr learned my addr 172.21.15.78:0/2321795214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:38.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f5673fff640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f565c0a3240 0x7f565c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f5678bd3640 1 -- 172.21.15.78:0/2321795214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 msgr2=0x562ea50df140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f5678bd3640 1 --2- 172.21.15.78:0/2321795214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 0x562ea50df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f5678bd3640 1 -- 172.21.15.78:0/2321795214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 msgr2=0x562ea50e1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f5678bd3640 1 --2- 172.21.15.78:0/2321795214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 0x562ea50e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f5678bd3640 1 -- 172.21.15.78:0/2321795214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea4fc6440 con 0x562ea50d44a0 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f5678bd3640 1 --2- 172.21.15.78:0/2321795214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50d4880 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f566c008e70 tx=0x7f566c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e994353e5c1a214c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f565affd640 1 -- 172.21.15.78:0/2321795214 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00f830 con 0x562ea50d44a0 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f5673fff640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f565c0a3240 0x7f565c0a5700 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x562ea50d1890 tx=0x7f5668032e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f565affd640 1 -- 172.21.15.78:0/2321795214 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f566c00e3b0 con 0x562ea50d44a0 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.876+0000 7f565affd640 1 -- 172.21.15.78:0/2321795214 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00b600 con 0x562ea50d44a0 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 -- 172.21.15.78:0/2321795214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 msgr2=0x562ea50d4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 --2- 172.21.15.78:0/2321795214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50d4880 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f566c008e70 tx=0x7f566c00d710 comp rx=0 tx=0).stop 2024-09-16T00:27:38.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 -- 172.21.15.78:0/2321795214 shutdown_connections 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 --2- 172.21.15.78:0/2321795214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 0x562ea50e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 --2- 172.21.15.78:0/2321795214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 0x562ea50df140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 --2- 172.21.15.78:0/2321795214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50d4880 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 -- 172.21.15.78:0/2321795214 >> 172.21.15.78:0/2321795214 conn(0x562ea50d0a30 msgr2=0x562ea50d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 -- 172.21.15.78:0/2321795214 shutdown_connections 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 -- 172.21.15.78:0/2321795214 wait complete. 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 -- start start 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.877+0000 7f567d210740 1 Processor -- start 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f567d210740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50df010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f567d210740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 0x562ea50dc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f567d210740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 0x562ea50dc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f567d210740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562ea50e2250 con 0x562ea50d44a0 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f567d210740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ea4ff5050 con 0x562ea50df680 2024-09-16T00:27:38.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f567d210740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ea50e23d0 con 0x562ea50d4dc0 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f567d210740 10 client.0 _ll_register_callbacks cb 0x562ea5035160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50df010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38234/0 (socket says 172.21.15.78:38234) 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 -- 172.21.15.78:0/454674241 learned_addr learned my addr 172.21.15.78:0/454674241 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f56793d4640 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 0x562ea50dc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5673fff640 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 0x562ea50dc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 -- 172.21.15.78:0/454674241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 msgr2=0x562ea50dc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 0x562ea50dc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 -- 172.21.15.78:0/454674241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 msgr2=0x562ea50dc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 0x562ea50dc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 -- 172.21.15.78:0/454674241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea4dab3c0 con 0x562ea50d44a0 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5678bd3640 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50df010 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f566c0083b0 tx=0x7f566c008bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e272f4a049b4d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5653fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.878+0000 7f5653fff640 1 -- 172.21.15.78:0/454674241 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c027060 con 0x562ea50d44a0 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f5653fff640 1 -- 172.21.15.78:0/454674241 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f566c00f840 con 0x562ea50d44a0 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 1 -- 172.21.15.78:0/454674241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566c00e3b0 con 0x562ea50d44a0 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f5653fff640 1 -- 172.21.15.78:0/454674241 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f566c02e050 con 0x562ea50d44a0 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 10 client.17991 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 1 -- 172.21.15.78:0/454674241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562ea50edc70 con 0x562ea50d44a0 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f5653fff640 1 -- 172.21.15.78:0/454674241 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f566c00e3b0 con 0x562ea50d44a0 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f5653fff640 1 -- 172.21.15.78:0/454674241 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f566c018af0 con 0x562ea50d44a0 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 10 client.17991 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 10 client.17991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 20 client.17991 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ea50ef0d0 0x562ea50f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 4 client.17991 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f567d210740 1 -- 172.21.15.78:0/454674241 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ea5047850 con 0x562ea50ef0d0 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.879+0000 7f5673fff640 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ea50ef0d0 0x562ea50f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.880+0000 7f5673fff640 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ea50ef0d0 0x562ea50f14c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1f952568741163 server_cookie=3a38100b846b44d7 in_seq=0 out_seq=0 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.880+0000 7f5653fff640 10 client.17991 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f5653fff640 1 -- 172.21.15.78:0/454674241 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ea5047850 con 0x562ea50ef0d0 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f5653fff640 10 client.17991 handle_command_reply: tid=0 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 client.17991 shutdown 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 -- 172.21.15.78:0/454674241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 msgr2=0x562ea50df010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50df010 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f566c0083b0 tx=0x7f566c008bb0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 -- 172.21.15.78:0/454674241 shutdown_connections 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ea50ef0d0 0x562ea50f14c0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50df680 0x562ea50dc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f5653fff640 0 client.17991 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:38.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d4dc0 0x562ea50dc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 --2- 172.21.15.78:0/454674241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50df010 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 -- 172.21.15.78:0/454674241 >> 172.21.15.78:0/454674241 conn(0x562ea50d0a30 msgr2=0x562ea50d6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:38.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 -- 172.21.15.78:0/454674241 shutdown_connections 2024-09-16T00:27:38.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 1 -- 172.21.15.78:0/454674241 wait complete. 2024-09-16T00:27:38.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.881+0000 7f567d210740 20 client.17991 trim_cache size 0 max 16384 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.898+0000 7f567d210740 1 Processor -- start 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.898+0000 7f567d210740 1 -- start start 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.898+0000 7f567d210740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 0x562ea50dbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f567d210740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f567d210740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 0x562ea50f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f567d210740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56741b0df0 con 0x562ea50d44a0 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f567d210740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56741b2930 con 0x562ea50d4dc0 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f567d210740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56741b14c0 con 0x562ea50d0a30 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38248/0 (socket says 172.21.15.78:38248) 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 -- 172.21.15.78:0/3358963547 learned_addr learned my addr 172.21.15.78:0/3358963547 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5678bd3640 1 --2- 172.21.15.78:0/3358963547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 0x562ea50dbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 -- 172.21.15.78:0/3358963547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 msgr2=0x562ea50dbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:38.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 --2- 172.21.15.78:0/3358963547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 0x562ea50dbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 -- 172.21.15.78:0/3358963547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 msgr2=0x562ea50f4a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:27:38.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 --2- 172.21.15.78:0/3358963547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 0x562ea50f4a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 -- 172.21.15.78:0/3358963547 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea50da810 con 0x562ea50d44a0 2024-09-16T00:27:38.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f5673fff640 1 --2- 172.21.15.78:0/3358963547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f44e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5668033000 tx=0x7f566803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3868991254917402 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f565a7fc640 1 -- 172.21.15.78:0/3358963547 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668035050 con 0x562ea50d44a0 2024-09-16T00:27:38.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f565a7fc640 1 -- 172.21.15.78:0/3358963547 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56680319e0 con 0x562ea50d44a0 2024-09-16T00:27:38.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.899+0000 7f565a7fc640 1 -- 172.21.15.78:0/3358963547 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566804ac80 con 0x562ea50d44a0 2024-09-16T00:27:38.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 -- 172.21.15.78:0/3358963547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 msgr2=0x562ea50f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 --2- 172.21.15.78:0/3358963547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f44e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5668033000 tx=0x7f566803c000 comp rx=0 tx=0).stop 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 -- 172.21.15.78:0/3358963547 shutdown_connections 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 --2- 172.21.15.78:0/3358963547 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 0x562ea50f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 --2- 172.21.15.78:0/3358963547 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f44e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 --2- 172.21.15.78:0/3358963547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 0x562ea50dbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 -- 172.21.15.78:0/3358963547 >> 172.21.15.78:0/3358963547 conn(0x562ea50d7cb0 msgr2=0x562ea50d6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 -- 172.21.15.78:0/3358963547 shutdown_connections 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 -- 172.21.15.78:0/3358963547 wait complete. 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.900+0000 7f567d210740 1 -- start start 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f567d210740 1 Processor -- start 2024-09-16T00:27:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f567d210740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 0x562ea50f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f567d210740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f567d210740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 0x562ea50f2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f567d210740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f56741b2260 con 0x562ea50d44a0 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f567d210740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56741b1b90 con 0x562ea50d4dc0 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f567d210740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56741aedd0 con 0x562ea50d0a30 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f567d210740 10 client.0 _ll_register_callbacks cb 0x7f566c008cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5673fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5673fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38264/0 (socket says 172.21.15.78:38264) 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5673fff640 1 -- 172.21.15.78:0/1896377066 learned_addr learned my addr 172.21.15.78:0/1896377066 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5678bd3640 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 0x562ea50f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f56793d4640 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 0x562ea50f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5673fff640 1 -- 172.21.15.78:0/1896377066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 msgr2=0x562ea50f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5673fff640 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 0x562ea50f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5673fff640 1 -- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 msgr2=0x562ea50f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5673fff640 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 0x562ea50f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f5673fff640 1 -- 172.21.15.78:0/1896377066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea50f2d50 con 0x562ea50d44a0 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.901+0000 7f56793d4640 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 0x562ea50f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f5673fff640 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f2010 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5668000c00 tx=0x7f566804a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e1190880e14330 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f5653fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f5653fff640 1 -- 172.21.15.78:0/1896377066 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668035070 con 0x562ea50d44a0 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f5653fff640 1 -- 172.21.15.78:0/1896377066 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f566804ac00 con 0x562ea50d44a0 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 1 -- 172.21.15.78:0/1896377066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56680319e0 con 0x562ea50d44a0 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f5653fff640 1 -- 172.21.15.78:0/1896377066 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f566804ae00 con 0x562ea50d44a0 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 10 client.17997 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 1 -- 172.21.15.78:0/1896377066 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562ea5159a50 con 0x562ea50d44a0 2024-09-16T00:27:38.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f5653fff640 1 -- 172.21.15.78:0/1896377066 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56680319e0 con 0x562ea50d44a0 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f5653fff640 1 -- 172.21.15.78:0/1896377066 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5668054770 con 0x562ea50d44a0 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 10 client.17997 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 10 client.17997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 20 client.17997 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ea50df680 0x562ea515c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 4 client.17997 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.902+0000 7f567d210740 1 -- 172.21.15.78:0/1896377066 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ea5041d60 con 0x562ea50df680 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.903+0000 7f5678bd3640 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ea50df680 0x562ea515c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.903+0000 7f5653fff640 10 client.17997 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.903+0000 7f5678bd3640 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ea50df680 0x562ea515c1b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc8c62fd97540ac server_cookie=163021750b08e33b in_seq=0 out_seq=0 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.903+0000 7f5653fff640 1 -- 172.21.15.78:0/1896377066 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562ea5041d60 con 0x562ea50df680 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.903+0000 7f5653fff640 10 client.17997 handle_command_reply: tid=0 2024-09-16T00:27:38.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.903+0000 7f567d210740 1 client.17997 shutdown 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.903+0000 7f567d210740 1 -- 172.21.15.78:0/1896377066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 msgr2=0x562ea50f2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.903+0000 7f567d210740 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f2010 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5668000c00 tx=0x7f566804a900 comp rx=0 tx=0).stop 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 1 -- 172.21.15.78:0/1896377066 shutdown_connections 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562ea50df680 0x562ea515c1b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ea50d4dc0 0x562ea50f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562ea50d44a0 0x562ea50f2010 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 1 --2- 172.21.15.78:0/1896377066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea50d0a30 0x562ea50f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 1 -- 172.21.15.78:0/1896377066 >> 172.21.15.78:0/1896377066 conn(0x562ea50d7cb0 msgr2=0x562ea50d5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f5653fff640 0 client.17997 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 1 -- 172.21.15.78:0/1896377066 shutdown_connections 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 1 -- 172.21.15.78:0/1896377066 wait complete. 2024-09-16T00:27:38.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567d210740 20 client.17997 trim_cache size 0 max 16384 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "f75592b2-2187-42f8-87fe-f7db668db440": { 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "f75592b2-2187-42f8-87fe-f7db668db440", 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:38.911 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567ae37640 1 -- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f565c0a3240 msgr2=0x7f565c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567ae37640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f565c0a3240 0x7f565c0a5700 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x562ea50d1890 tx=0x7f5668032e30 comp rx=0 tx=0).stop 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.904+0000 7f567ae37640 1 -- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f567410c180 msgr2=0x7f5674101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f567410c180 0x7f5674101480 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f566400c970 tx=0x7f566400ce30 comp rx=0 tx=0).stop 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 -- 172.21.15.78:0/2897499809 shutdown_connections 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f565c0a3240 0x7f565c0a5700 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f567410c180 0x7f5674101480 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56741036c0 0x7f5674100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 --2- 172.21.15.78:0/2897499809 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5674102da0 0x7f5674100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 -- 172.21.15.78:0/2897499809 >> 172.21.15.78:0/2897499809 conn(0x7f56740fc7a0 msgr2=0x7f56740fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 -- 172.21.15.78:0/2897499809 shutdown_connections 2024-09-16T00:27:38.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:38.905+0000 7f567ae37640 1 -- 172.21.15.78:0/2897499809 wait complete. 2024-09-16T00:27:39.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:38 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:27:39.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:38 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:38 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:27:39.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:38 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:38 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:38 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:27:39.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:38 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:38 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:27:39.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:38 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:38 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:38 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:27:39.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:38 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:38 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:27:39.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:38 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:38 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:27:39.800 INFO:tasks.cephfs.filesystem:scrub status for tag:f75592b2-2187-42f8-87fe-f7db668db440 - {'path': '/', 'tag': 'f75592b2-2187-42f8-87fe-f7db668db440', 'options': 'recursive,force'} 2024-09-16T00:27:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:39 smithi193 ceph-mon[35484]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-16T00:27:40.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:39 smithi148 ceph-mon[35599]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-16T00:27:40.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:39 smithi078 ceph-mon[31157]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-16T00:27:40.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 820.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:27:42.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:27:42.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:41 smithi193 ceph-mon[35484]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-16T00:27:42.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:41 smithi148 ceph-mon[35599]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-16T00:27:42.262 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:42.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:41 smithi078 ceph-mon[31157]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-16T00:27:42.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 -- 172.21.15.78:0/581909738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838106a80 msgr2=0x7fe838106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:42.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 --2- 172.21.15.78:0/581909738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838106a80 0x7fe838106e60 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe830009930 tx=0x7fe83002a420 comp rx=0 tx=0).stop 2024-09-16T00:27:42.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 -- 172.21.15.78:0/581909738 shutdown_connections 2024-09-16T00:27:42.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 --2- 172.21.15.78:0/581909738 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe838101420 0x7fe83810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 --2- 172.21.15.78:0/581909738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe838100a80 0x7fe838100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 --2- 172.21.15.78:0/581909738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838106a80 0x7fe838106e60 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 -- 172.21.15.78:0/581909738 >> 172.21.15.78:0/581909738 conn(0x7fe8380fc7a0 msgr2=0x7fe8380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 -- 172.21.15.78:0/581909738 shutdown_connections 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.745+0000 7fe83f855640 1 -- 172.21.15.78:0/581909738 wait complete. 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83f855640 1 Processor -- start 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83f855640 1 -- start start 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83f855640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838100a80 0x7fe838102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83f855640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe838101420 0x7fe838102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83f855640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe838106a80 0x7fe8381030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83f855640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe838110e80 con 0x7fe838100a80 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83f855640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe838110d00 con 0x7fe838101420 2024-09-16T00:27:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83f855640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe838111000 con 0x7fe838106a80 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83d5f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838100a80 0x7fe838102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83d5f1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838100a80 0x7fe838102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38280/0 (socket says 172.21.15.78:38280) 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83d5f1640 1 -- 172.21.15.78:0/2157645439 learned_addr learned my addr 172.21.15.78:0/2157645439 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.746+0000 7fe83cdf0640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe838101420 0x7fe838102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83ddf2640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe838106a80 0x7fe8381030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83d5f1640 1 -- 172.21.15.78:0/2157645439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe838106a80 msgr2=0x7fe8381030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83d5f1640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe838106a80 0x7fe8381030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83d5f1640 1 -- 172.21.15.78:0/2157645439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe838101420 msgr2=0x7fe838102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83d5f1640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe838101420 0x7fe838102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83d5f1640 1 -- 172.21.15.78:0/2157645439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8381a6840 con 0x7fe838100a80 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83d5f1640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838100a80 0x7fe838102650 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe830009230 tx=0x7fe830004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe8367fc640 1 -- 172.21.15.78:0/2157645439 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83003f030 con 0x7fe838100a80 2024-09-16T00:27:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8381a6ad0 con 0x7fe838100a80 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe8367fc640 1 -- 172.21.15.78:0/2157645439 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe830003ad0 con 0x7fe838100a80 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe8367fc640 1 -- 172.21.15.78:0/2157645439 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830035070 con 0x7fe838100a80 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.747+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8381a6fb0 con 0x7fe838100a80 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.748+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe838100ee0 con 0x7fe838100a80 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.749+0000 7fe8367fc640 1 -- 172.21.15.78:0/2157645439 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe830042030 con 0x7fe838100a80 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.750+0000 7fe8367fc640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe8200a3240 0x7fe8200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.750+0000 7fe83cdf0640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe8200a3240 0x7fe8200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.750+0000 7fe8367fc640 1 -- 172.21.15.78:0/2157645439 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe8300f0160 con 0x7fe838100a80 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.751+0000 7fe83cdf0640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe8200a3240 0x7fe8200a5700 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe8381040e0 tx=0x7fe82c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.755+0000 7fe8367fc640 1 -- 172.21.15.78:0/2157645439 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8300bcb20 con 0x7fe838100a80 2024-09-16T00:27:42.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.961+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8381a7290 con 0x7fe838100a80 2024-09-16T00:27:42.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.961+0000 7fe8367fc640 1 -- 172.21.15.78:0/2157645439 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fe8300bc420 con 0x7fe838100a80 2024-09-16T00:27:42.963 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:27:42.963 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:27:42.963 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:27:42.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe8200a3240 msgr2=0x7fe8200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:42.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe8200a3240 0x7fe8200a5700 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe8381040e0 tx=0x7fe82c009680 comp rx=0 tx=0).stop 2024-09-16T00:27:42.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838100a80 msgr2=0x7fe838102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:42.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838100a80 0x7fe838102650 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fe830009230 tx=0x7fe830004270 comp rx=0 tx=0).stop 2024-09-16T00:27:42.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 shutdown_connections 2024-09-16T00:27:42.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe8200a3240 0x7fe8200a5700 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe838106a80 0x7fe8381030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe838101420 0x7fe838102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 --2- 172.21.15.78:0/2157645439 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe838100a80 0x7fe838102650 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 >> 172.21.15.78:0/2157645439 conn(0x7fe8380fc7a0 msgr2=0x7fe83806a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 shutdown_connections 2024-09-16T00:27:42.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:42.964+0000 7fe83f855640 1 -- 172.21.15.78:0/2157645439 wait complete. 2024-09-16T00:27:43.789 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 15f44a70 --release --await' 2024-09-16T00:27:43.789 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 15f44a70 --release --await 2024-09-16T00:27:44.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:43 smithi078 ceph-mon[31157]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-16T00:27:44.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:43 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2157645439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:44.037 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:27:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:43 smithi193 ceph-mon[35484]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-16T00:27:44.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:43 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2157645439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:43 smithi148 ceph-mon[35599]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-16T00:27:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:43 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2157645439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:27:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- 172.21.15.78:0/909238370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 msgr2=0x7fbc3410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/909238370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 0x7fbc3410da40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009ca0 tx=0x7fbc2c02a950 comp rx=0 tx=0).stop 2024-09-16T00:27:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- 172.21.15.78:0/909238370 shutdown_connections 2024-09-16T00:27:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/909238370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 0x7fbc3410da40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/909238370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc34102570 0x7fbc341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/909238370 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc34100e20 0x7fbc34101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- 172.21.15.78:0/909238370 >> 172.21.15.78:0/909238370 conn(0x7fbc340fc7d0 msgr2=0x7fbc340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:44.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- 172.21.15.78:0/909238370 shutdown_connections 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- 172.21.15.78:0/909238370 wait complete. 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 Processor -- start 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- start start 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc34100e20 0x7fbc34198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc34102570 0x7fbc34198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 0x7fbc3419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34111020 con 0x7fbc34102ef0 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34110ea0 con 0x7fbc34100e20 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.556+0000 7fbc3ade6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341111a0 con 0x7fbc34102570 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 0x7fbc3419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 0x7fbc3419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38294/0 (socket says 172.21.15.78:38294) 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 -- 172.21.15.78:0/4208818010 learned_addr learned my addr 172.21.15.78:0/4208818010 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:44.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc33fff640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc34102570 0x7fbc34198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc38b82640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc34100e20 0x7fbc34198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 -- 172.21.15.78:0/4208818010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc34102570 msgr2=0x7fbc34198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc34102570 0x7fbc34198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 -- 172.21.15.78:0/4208818010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc34100e20 msgr2=0x7fbc34198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc34100e20 0x7fbc34198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 -- 172.21.15.78:0/4208818010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3419d5c0 con 0x7fbc34102ef0 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc33fff640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc34102570 0x7fbc34198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc39383640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 0x7fbc3419cee0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c006010 tx=0x7fbc2c02f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc31ffb640 1 -- 172.21.15.78:0/4208818010 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c034040 con 0x7fbc34102ef0 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc3ade6640 1 -- 172.21.15.78:0/4208818010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3419d850 con 0x7fbc34102ef0 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc31ffb640 1 -- 172.21.15.78:0/4208818010 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c003e00 con 0x7fbc34102ef0 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.557+0000 7fbc31ffb640 1 -- 172.21.15.78:0/4208818010 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c045480 con 0x7fbc34102ef0 2024-09-16T00:27:44.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.558+0000 7fbc3ade6640 1 -- 172.21.15.78:0/4208818010 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc341a5190 con 0x7fbc34102ef0 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc31ffb640 1 -- 172.21.15.78:0/4208818010 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fbc2c037020 con 0x7fbc34102ef0 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc3d1bf740 1 Processor -- start 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc3d1bf740 1 -- start start 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc31ffb640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbc240a3240 0x7fbc240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72829320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72838110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da7283aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da7278e620 con 0x55da72829860 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55da72735e50 con 0x55da72828f40 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.560+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da7278a5e0 con 0x55da72838650 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72838110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72838110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38302/0 (socket says 172.21.15.78:38302) 2024-09-16T00:27:44.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc38b82640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbc240a3240 0x7fbc240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc31ffb640 1 -- 172.21.15.78:0/4208818010 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbc2c0f0ea0 con 0x7fbc34102ef0 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 -- 172.21.15.78:0/2837397279 learned_addr learned my addr 172.21.15.78:0/2837397279 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc39383640 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da7283aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72829320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 -- 172.21.15.78:0/2837397279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 msgr2=0x55da7283aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da7283aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 -- 172.21.15.78:0/2837397279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 msgr2=0x55da72829320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72829320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 -- 172.21.15.78:0/2837397279 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da72726d40 con 0x55da72829860 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc39383640 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da7283aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72838110 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ae70 tx=0x7fbc2000a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b3ba617255774d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc38b82640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbc240a3240 0x7fbc240a5700 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009890 tx=0x7fbc28008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:44.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc1effd640 1 -- 172.21.15.78:0/2837397279 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2001c070 con 0x55da72829860 2024-09-16T00:27:44.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc1effd640 1 -- 172.21.15.78:0/2837397279 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2000fe30 con 0x55da72829860 2024-09-16T00:27:44.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.561+0000 7fbc1effd640 1 -- 172.21.15.78:0/2837397279 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20014790 con 0x55da72829860 2024-09-16T00:27:44.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2837397279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 msgr2=0x55da72838110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72838110 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000ae70 tx=0x7fbc2000a520 comp rx=0 tx=0).stop 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2837397279 shutdown_connections 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da7283aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72838110 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2837397279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72829320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2837397279 >> 172.21.15.78:0/2837397279 conn(0x55da728255f0 msgr2=0x55da728259d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2837397279 shutdown_connections 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2837397279 wait complete. 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.562+0000 7fbc3d1bf740 1 -- start start 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc3d1bf740 1 Processor -- start 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72835610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72835b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da72836090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55da7283b220 con 0x55da72829860 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55da72716f10 con 0x55da72828f40 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da7283b3a0 con 0x55da72838650 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc3d1bf740 10 client.0 _ll_register_callbacks cb 0x55da7273adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72835b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72835b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38306/0 (socket says 172.21.15.78:38306) 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc33fff640 1 -- 172.21.15.78:0/2371724829 learned_addr learned my addr 172.21.15.78:0/2371724829 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc38b82640 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72835610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc39383640 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da72836090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc33fff640 1 -- 172.21.15.78:0/2371724829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 msgr2=0x55da72836090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da72836090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc33fff640 1 -- 172.21.15.78:0/2371724829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 msgr2=0x55da72835610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:44.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72835610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.563+0000 7fbc33fff640 1 -- 172.21.15.78:0/2371724829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da7249d580 con 0x55da72829860 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72835b50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbc2001a040 tx=0x7fbc20017900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbac4111e54c57b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc13fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc13fff640 1 -- 172.21.15.78:0/2371724829 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20025020 con 0x55da72829860 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc13fff640 1 -- 172.21.15.78:0/2371724829 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbc20014570 con 0x55da72829860 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2371724829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2000fe30 con 0x55da72829860 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc13fff640 1 -- 172.21.15.78:0/2371724829 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc2001c070 con 0x55da72829860 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 10 client.18015 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2371724829 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da72567960 con 0x55da72829860 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc13fff640 1 -- 172.21.15.78:0/2371724829 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2000fe30 con 0x55da72829860 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc13fff640 1 -- 172.21.15.78:0/2371724829 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbc20015040 con 0x55da72829860 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 10 client.18015 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 10 client.18015 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:27:44.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 20 client.18015 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da728481e0 0x55da7284a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 4 client.18015 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.564+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2371724829 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da7279bb90 con 0x55da728481e0 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.565+0000 7fbc38b82640 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da728481e0 0x55da7284a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.567+0000 7fbc13fff640 10 client.18015 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.567+0000 7fbc38b82640 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da728481e0 0x55da7284a5d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a545aff6ef32f91e server_cookie=b109be72a9486762 in_seq=0 out_seq=0 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc13fff640 1 -- 172.21.15.78:0/2371724829 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da7279bb90 con 0x55da728481e0 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc13fff640 10 client.18015 handle_command_reply: tid=0 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 client.18015 shutdown 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2371724829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 msgr2=0x55da72835b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72835b50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbc2001a040 tx=0x7fbc20017900 comp rx=0 tx=0).stop 2024-09-16T00:27:44.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2371724829 shutdown_connections 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da728481e0 0x55da7284a5d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da72836090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72835b50 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc13fff640 0 client.18015 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2371724829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72835610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2371724829 >> 172.21.15.78:0/2371724829 conn(0x55da728255f0 msgr2=0x55da7282b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2371724829 shutdown_connections 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.568+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2371724829 wait complete. 2024-09-16T00:27:44.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.569+0000 7fbc3d1bf740 20 client.18015 trim_cache size 0 max 16384 2024-09-16T00:27:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc3d1bf740 1 Processor -- start 2024-09-16T00:27:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc3d1bf740 1 -- start start 2024-09-16T00:27:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72824640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72832890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da72832dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341aca00 con 0x55da72829860 2024-09-16T00:27:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341ae540 con 0x55da72828f40 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341ad0d0 con 0x55da72838650 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72832890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72832890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38318/0 (socket says 172.21.15.78:38318) 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.585+0000 7fbc33fff640 1 -- 172.21.15.78:0/2138959377 learned_addr learned my addr 172.21.15.78:0/2138959377 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc33fff640 1 -- 172.21.15.78:0/2138959377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 msgr2=0x55da72832dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2138959377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da72832dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc33fff640 1 -- 172.21.15.78:0/2138959377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 msgr2=0x55da72824640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2138959377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72824640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc33fff640 1 -- 172.21.15.78:0/2138959377 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da72823bb0 con 0x55da72829860 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc33fff640 1 --2- 172.21.15.78:0/2138959377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72832890 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbc200178d0 tx=0x7fbc2000a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e3e7e2758b3a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc1e7fc640 1 -- 172.21.15.78:0/2138959377 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20025040 con 0x55da72829860 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc1e7fc640 1 -- 172.21.15.78:0/2138959377 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2001d040 con 0x55da72829860 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.586+0000 7fbc1e7fc640 1 -- 172.21.15.78:0/2138959377 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200149e0 con 0x55da72829860 2024-09-16T00:27:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2138959377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 msgr2=0x55da72832890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2138959377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72832890 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbc200178d0 tx=0x7fbc2000a580 comp rx=0 tx=0).stop 2024-09-16T00:27:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2138959377 shutdown_connections 2024-09-16T00:27:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2138959377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72838650 0x55da72832dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2138959377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da72832890 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/2138959377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72828f40 0x55da72824640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2138959377 >> 172.21.15.78:0/2138959377 conn(0x55da728255f0 msgr2=0x55da72823f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2138959377 shutdown_connections 2024-09-16T00:27:44.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/2138959377 wait complete. 2024-09-16T00:27:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 -- start start 2024-09-16T00:27:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 Processor -- start 2024-09-16T00:27:44.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72828f40 0x55da7284ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da7284bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.587+0000 7fbc3d1bf740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72838650 0x55da728b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341ade70 con 0x55da72829860 2024-09-16T00:27:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341ad7a0 con 0x55da72838650 2024-09-16T00:27:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc3d1bf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc341aa9e0 con 0x55da72828f40 2024-09-16T00:27:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc3d1bf740 10 client.0 _ll_register_callbacks cb 0x7fbc20017790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:27:44.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da7284bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da7284bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38320/0 (socket says 172.21.15.78:38320) 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 -- 172.21.15.78:0/1474334745 learned_addr learned my addr 172.21.15.78:0/1474334745 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc38b82640 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72828f40 0x55da7284ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 -- 172.21.15.78:0/1474334745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72828f40 msgr2=0x55da7284ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72828f40 0x55da7284ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc39383640 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72838650 0x55da728b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 -- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72838650 msgr2=0x55da728b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72838650 0x55da728b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 -- 172.21.15.78:0/1474334745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da728b2060 con 0x55da72829860 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc39383640 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72838650 0x55da728b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc33fff640 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da7284bf70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000fb40 tx=0x7fbc20022f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91862f3cbaf83f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc13fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:27:44.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc13fff640 1 -- 172.21.15.78:0/1474334745 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2003b070 con 0x55da72829860 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc13fff640 1 -- 172.21.15.78:0/1474334745 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbc20014500 con 0x55da72829860 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/1474334745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2001d040 con 0x55da72829860 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.588+0000 7fbc13fff640 1 -- 172.21.15.78:0/1474334745 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc20025040 con 0x55da72829860 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc3d1bf740 10 client.18021 fetch_fsmap learned FSMap version 34 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/1474334745 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da728b32b0 con 0x55da72829860 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc13fff640 1 -- 172.21.15.78:0/1474334745 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2001d040 con 0x55da72829860 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc13fff640 1 -- 172.21.15.78:0/1474334745 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbc2003f400 con 0x55da72829860 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc3d1bf740 10 client.18021 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc3d1bf740 10 client.18021 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:27:44.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc3d1bf740 20 client.18021 populate_metadata read hostname 'smithi078' 2024-09-16T00:27:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da728b3800 0x55da728b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc3d1bf740 4 client.18021 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "15f44a70", "release": true, "await": true}] 2024-09-16T00:27:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/1474334745 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "15f44a70", "release": true, "await": true}) -- 0x55da727960a0 con 0x55da728b3800 2024-09-16T00:27:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.589+0000 7fbc38b82640 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da728b3800 0x55da728b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.590+0000 7fbc38b82640 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da728b3800 0x55da728b5c20 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=139d3c8f2171041b server_cookie=79b4ea484230978b in_seq=0 out_seq=0 2024-09-16T00:27:44.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.590+0000 7fbc13fff640 10 client.18021 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:44.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc13fff640 1 -- 172.21.15.78:0/1474334745 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55da727960a0 con 0x55da728b3800 2024-09-16T00:27:44.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc13fff640 10 client.18021 handle_command_reply: tid=0 2024-09-16T00:27:44.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 client.18021 shutdown 2024-09-16T00:27:44.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/1474334745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 msgr2=0x55da7284bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da7284bf70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000fb40 tx=0x7fbc20022f30 comp rx=0 tx=0).stop 2024-09-16T00:27:44.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/1474334745 shutdown_connections 2024-09-16T00:27:44.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55da728b3800 0x55da728b5c20 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55da72838650 0x55da728b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55da72829860 0x55da7284bf70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 --2- 172.21.15.78:0/1474334745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da72828f40 0x55da7284ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/1474334745 >> 172.21.15.78:0/1474334745 conn(0x55da728255f0 msgr2=0x55da7282b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc13fff640 0 client.18021 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/1474334745 shutdown_connections 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 1 -- 172.21.15.78:0/1474334745 wait complete. 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.654+0000 7fbc3d1bf740 20 client.18021 trim_cache size 0 max 16384 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:27:44.658 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 52, 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "15f44a70": { 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 52, 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 146.5, 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:27:44.659 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:27:44.660 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:27:44.660 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:27:44.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:27:44.660 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:27:44.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:44.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:44.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:44.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:44.661 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:27:44.661 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:27:44.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 -- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbc240a3240 msgr2=0x7fbc240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbc240a3240 0x7fbc240a5700 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009890 tx=0x7fbc28008040 comp rx=0 tx=0).stop 2024-09-16T00:27:44.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 -- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 msgr2=0x7fbc3419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:44.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 0x7fbc3419cee0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c006010 tx=0x7fbc2c02f6f0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 -- 172.21.15.78:0/4208818010 shutdown_connections 2024-09-16T00:27:44.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbc240a3240 0x7fbc240a5700 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbc34102ef0 0x7fbc3419cee0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc34102570 0x7fbc34198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 --2- 172.21.15.78:0/4208818010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbc34100e20 0x7fbc34198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:44.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 -- 172.21.15.78:0/4208818010 >> 172.21.15.78:0/4208818010 conn(0x7fbc340fc7d0 msgr2=0x7fbc34069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:44.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.655+0000 7fbc3ade6640 1 -- 172.21.15.78:0/4208818010 shutdown_connections 2024-09-16T00:27:44.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:27:44.656+0000 7fbc3ade6640 1 -- 172.21.15.78:0/4208818010 wait complete. 2024-09-16T00:27:45.564 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '15f44a70', total seconds elapsed: 30.1 2024-09-16T00:27:45.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 825.0 s, 132.7 tps, lat 3559.217 ms stddev 8115.049 2024-09-16T00:27:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:45 smithi193 ceph-mon[35484]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-16T00:27:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:45 smithi148 ceph-mon[35599]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-16T00:27:46.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:45 smithi078 ceph-mon[31157]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-16T00:27:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:47 smithi193 ceph-mon[35484]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 167 KiB/s wr, 30 op/s 2024-09-16T00:27:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:47 smithi148 ceph-mon[35599]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 167 KiB/s wr, 30 op/s 2024-09-16T00:27:48.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:47 smithi078 ceph-mon[31157]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 167 KiB/s wr, 30 op/s 2024-09-16T00:27:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:49 smithi193 ceph-mon[35484]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 643 KiB/s wr, 67 op/s 2024-09-16T00:27:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:27:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:27:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:27:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:49 smithi148 ceph-mon[35599]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 643 KiB/s wr, 67 op/s 2024-09-16T00:27:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:27:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:27:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:27:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:49 smithi078 ceph-mon[31157]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 643 KiB/s wr, 67 op/s 2024-09-16T00:27:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:27:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:27:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:27:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 830.0 s, 568.0 tps, lat 177.144 ms stddev 216.172 2024-09-16T00:27:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:51 smithi193 ceph-mon[35484]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-16T00:27:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:51 smithi148 ceph-mon[35599]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-16T00:27:52.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:51 smithi078 ceph-mon[31157]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-09-16T00:27:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:53 smithi193 ceph-mon[35484]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 MiB/s wr, 261 op/s 2024-09-16T00:27:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:53 smithi148 ceph-mon[35599]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 MiB/s wr, 261 op/s 2024-09-16T00:27:54.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:53 smithi078 ceph-mon[31157]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.5 MiB/s wr, 261 op/s 2024-09-16T00:27:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 835.0 s, 549.2 tps, lat 180.206 ms stddev 198.587 2024-09-16T00:27:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:55 smithi193 ceph-mon[35484]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-09-16T00:27:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:55 smithi148 ceph-mon[35599]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-09-16T00:27:56.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:55 smithi078 ceph-mon[31157]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-09-16T00:27:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:57 smithi193 ceph-mon[35484]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-09-16T00:27:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:57 smithi148 ceph-mon[35599]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-09-16T00:27:58.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:57 smithi078 ceph-mon[31157]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.3 MiB/s wr, 452 op/s 2024-09-16T00:28:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:27:59 smithi193 ceph-mon[35484]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2024-09-16T00:28:00.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:27:59 smithi148 ceph-mon[35599]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2024-09-16T00:28:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:27:59 smithi078 ceph-mon[31157]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2024-09-16T00:28:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 840.0 s, 790.8 tps, lat 128.056 ms stddev 152.369 2024-09-16T00:28:02.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:01 smithi193 ceph-mon[35484]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-09-16T00:28:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:01 smithi148 ceph-mon[35599]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-09-16T00:28:02.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:01 smithi078 ceph-mon[31157]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 570 op/s 2024-09-16T00:28:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:03 smithi193 ceph-mon[35484]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-09-16T00:28:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:03 smithi148 ceph-mon[35599]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-09-16T00:28:04.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:03 smithi078 ceph-mon[31157]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 521 op/s 2024-09-16T00:28:05.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 845.0 s, 576.4 tps, lat 172.425 ms stddev 200.124 2024-09-16T00:28:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:05 smithi193 ceph-mon[35484]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.7 MiB/s wr, 578 op/s 2024-09-16T00:28:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:05 smithi148 ceph-mon[35599]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.7 MiB/s wr, 578 op/s 2024-09-16T00:28:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:05 smithi078 ceph-mon[31157]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.7 MiB/s wr, 578 op/s 2024-09-16T00:28:08.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:07 smithi193 ceph-mon[35484]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.7 MiB/s wr, 632 op/s 2024-09-16T00:28:08.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:07 smithi148 ceph-mon[35599]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.7 MiB/s wr, 632 op/s 2024-09-16T00:28:08.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:07 smithi078 ceph-mon[31157]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.7 MiB/s wr, 632 op/s 2024-09-16T00:28:09.802 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:28:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:09 smithi193 ceph-mon[35484]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-09-16T00:28:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:09 smithi148 ceph-mon[35599]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-09-16T00:28:10.273 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:09 smithi078 ceph-mon[31157]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.5 MiB/s wr, 610 op/s 2024-09-16T00:28:10.348 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:10.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 850.0 s, 536.4 tps, lat 185.301 ms stddev 240.026 2024-09-16T00:28:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.023+0000 7faf88f47640 1 -- 172.21.15.78:0/1660901326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf84109830 msgr2=0x7faf84109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.023+0000 7faf88f47640 1 --2- 172.21.15.78:0/1660901326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf84109830 0x7faf84109c90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf7c01acb0 tx=0x7faf7c03b920 comp rx=0 tx=0).stop 2024-09-16T00:28:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.024+0000 7faf88f47640 1 -- 172.21.15.78:0/1660901326 shutdown_connections 2024-09-16T00:28:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.024+0000 7faf88f47640 1 --2- 172.21.15.78:0/1660901326 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf8411b140 0x7faf8411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.024+0000 7faf88f47640 1 --2- 172.21.15.78:0/1660901326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf84109830 0x7faf84109c90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.024+0000 7faf88f47640 1 --2- 172.21.15.78:0/1660901326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf84109450 0x7faf84110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.024+0000 7faf88f47640 1 -- 172.21.15.78:0/1660901326 >> 172.21.15.78:0/1660901326 conn(0x7faf8406e140 msgr2=0x7faf8406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.025+0000 7faf88f47640 1 -- 172.21.15.78:0/1660901326 shutdown_connections 2024-09-16T00:28:12.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.025+0000 7faf88f47640 1 -- 172.21.15.78:0/1660901326 wait complete. 2024-09-16T00:28:12.030 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:11 smithi078 ceph-mon[31157]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.3 MiB/s wr, 682 op/s 2024-09-16T00:28:12.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.026+0000 7faf88f47640 1 Processor -- start 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.026+0000 7faf88f47640 1 -- start start 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.026+0000 7faf88f47640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf84109450 0x7faf8411a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.026+0000 7faf88f47640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf84109830 0x7faf84116830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.027+0000 7faf88f47640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf8411b140 0x7faf84116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.027+0000 7faf88f47640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf84113390 con 0x7faf8411b140 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.027+0000 7faf88f47640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faf84113210 con 0x7faf84109830 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.027+0000 7faf88f47640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf84113510 con 0x7faf84109450 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.027+0000 7faf81d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf84109830 0x7faf84116830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.027+0000 7faf81d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf84109830 0x7faf84116830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34942/0 (socket says 172.21.15.78:34942) 2024-09-16T00:28:12.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.027+0000 7faf81d9b640 1 -- 172.21.15.78:0/2790809523 learned_addr learned my addr 172.21.15.78:0/2790809523 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.028+0000 7faf8259c640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf84109450 0x7faf8411a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.031+0000 7faf82d9d640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf8411b140 0x7faf84116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.031+0000 7faf81d9b640 1 -- 172.21.15.78:0/2790809523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf84109450 msgr2=0x7faf8411a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.031+0000 7faf81d9b640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf84109450 0x7faf8411a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.031+0000 7faf81d9b640 1 -- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf8411b140 msgr2=0x7faf84116d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.031+0000 7faf81d9b640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf8411b140 0x7faf84116d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.031+0000 7faf81d9b640 1 -- 172.21.15.78:0/2790809523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf84117360 con 0x7faf84109830 2024-09-16T00:28:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.031+0000 7faf82d9d640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf8411b140 0x7faf84116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:12.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.032+0000 7faf81d9b640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf84109830 0x7faf84116830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf7c003c50 tx=0x7faf7c003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.032+0000 7faf737fe640 1 -- 172.21.15.78:0/2790809523 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c045070 con 0x7faf84109830 2024-09-16T00:28:12.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.032+0000 7faf737fe640 1 -- 172.21.15.78:0/2790809523 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c040e20 con 0x7faf84109830 2024-09-16T00:28:12.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.032+0000 7faf737fe640 1 -- 172.21.15.78:0/2790809523 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c04b430 con 0x7faf84109830 2024-09-16T00:28:12.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.035+0000 7faf88f47640 1 -- 172.21.15.78:0/2790809523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf841175f0 con 0x7faf84109830 2024-09-16T00:28:12.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.035+0000 7faf88f47640 1 -- 172.21.15.78:0/2790809523 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf841ab340 con 0x7faf84109830 2024-09-16T00:28:12.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.040+0000 7faf8b320740 1 Processor -- start 2024-09-16T00:28:12.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.040+0000 7faf8b320740 1 -- start start 2024-09-16T00:28:12.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.040+0000 7faf8b320740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d5c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.040+0000 7faf8b320740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5cdc0 0x555bb3d67140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.040+0000 7faf8b320740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d67680 0x555bb3d69a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.040+0000 7faf8b320740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555bb3cc22e0 con 0x555bb3d67680 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.040+0000 7faf8b320740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555bb3c69ce0 con 0x555bb3d5c4a0 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.040+0000 7faf8b320740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555bb3cbe2a0 con 0x555bb3d5cdc0 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf82d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d5c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf8259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5cdc0 0x555bb3d67140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf82d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d5c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34950/0 (socket says 172.21.15.78:34950) 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf82d9d640 1 -- 172.21.15.78:0/2951768079 learned_addr learned my addr 172.21.15.78:0/2951768079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf82d9d640 1 -- 172.21.15.78:0/2951768079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5cdc0 msgr2=0x555bb3d67140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf82d9d640 1 --2- 172.21.15.78:0/2951768079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5cdc0 0x555bb3d67140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf82d9d640 1 -- 172.21.15.78:0/2951768079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d67680 msgr2=0x555bb3d69a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf82d9d640 1 --2- 172.21.15.78:0/2951768079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d67680 0x555bb3d69a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.041+0000 7faf82d9d640 1 -- 172.21.15.78:0/2951768079 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb3c4e440 con 0x555bb3d5c4a0 2024-09-16T00:28:12.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.042+0000 7faf82d9d640 1 --2- 172.21.15.78:0/2951768079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d5c880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf7800a920 tx=0x7faf7800ade0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5656cbebdf721561 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.043+0000 7faf737fe640 1 -- 172.21.15.78:0/2790809523 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7faf7c048020 con 0x7faf84109830 2024-09-16T00:28:12.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.043+0000 7faf70ff9640 1 -- 172.21.15.78:0/2951768079 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78015070 con 0x555bb3d5c4a0 2024-09-16T00:28:12.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.043+0000 7faf70ff9640 1 -- 172.21.15.78:0/2951768079 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7800cc00 con 0x555bb3d5c4a0 2024-09-16T00:28:12.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.043+0000 7faf70ff9640 1 -- 172.21.15.78:0/2951768079 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78018690 con 0x555bb3d5c4a0 2024-09-16T00:28:12.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.044+0000 7faf737fe640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf6c0a3410 0x7faf6c0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.044+0000 7faf737fe640 1 -- 172.21.15.78:0/2790809523 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faf7c101c70 con 0x7faf84109830 2024-09-16T00:28:12.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.045+0000 7faf8259c640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf6c0a3410 0x7faf6c0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.046+0000 7faf8b320740 1 -- 172.21.15.78:0/2951768079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 msgr2=0x555bb3d5c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.046+0000 7faf8b320740 1 --2- 172.21.15.78:0/2951768079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d5c880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf7800a920 tx=0x7faf7800ade0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.046+0000 7faf8259c640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf6c0a3410 0x7faf6c0a58d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7faf74006fd0 tx=0x7faf74008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.047+0000 7faf8b320740 1 -- 172.21.15.78:0/2951768079 shutdown_connections 2024-09-16T00:28:12.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.047+0000 7faf8b320740 1 --2- 172.21.15.78:0/2951768079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d67680 0x555bb3d69a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.047+0000 7faf8b320740 1 --2- 172.21.15.78:0/2951768079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5cdc0 0x555bb3d67140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.047+0000 7faf8b320740 1 --2- 172.21.15.78:0/2951768079 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d5c880 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.047+0000 7faf8b320740 1 -- 172.21.15.78:0/2951768079 >> 172.21.15.78:0/2951768079 conn(0x555bb3d58a30 msgr2=0x555bb3d58e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.048+0000 7faf8b320740 1 -- 172.21.15.78:0/2951768079 shutdown_connections 2024-09-16T00:28:12.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.048+0000 7faf8b320740 1 -- 172.21.15.78:0/2951768079 wait complete. 2024-09-16T00:28:12.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.048+0000 7faf8b320740 1 -- start start 2024-09-16T00:28:12.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.049+0000 7faf8b320740 1 Processor -- start 2024-09-16T00:28:12.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.049+0000 7faf8b320740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5c4a0 0x555bb3d67070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.049+0000 7faf8b320740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5cdc0 0x555bb3d64320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.049+0000 7faf8b320740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 0x555bb3d64910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.049+0000 7faf8b320740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x555bb3d6a250 con 0x555bb3d5c4a0 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.049+0000 7faf8b320740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555bb3c7d050 con 0x555bb3d5cdc0 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.049+0000 7faf8b320740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555bb3d6a3d0 con 0x555bb3d67680 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.050+0000 7faf81d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5cdc0 0x555bb3d64320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.050+0000 7faf81d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5cdc0 0x555bb3d64320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:34952/0 (socket says 172.21.15.78:34952) 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.050+0000 7faf81d9b640 1 -- 172.21.15.78:0/763189928 learned_addr learned my addr 172.21.15.78:0/763189928 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.050+0000 7faf82d9d640 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 0x555bb3d64910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.050+0000 7faf8b320740 10 client.0 _ll_register_callbacks cb 0x555bb3cbd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:12.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.052+0000 7faf82d9d640 1 -- 172.21.15.78:0/763189928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5cdc0 msgr2=0x555bb3d64320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.052+0000 7faf82d9d640 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5cdc0 0x555bb3d64320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.052+0000 7faf82d9d640 1 -- 172.21.15.78:0/763189928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5c4a0 msgr2=0x555bb3d67070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.052+0000 7faf82d9d640 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5c4a0 0x555bb3d67070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.052+0000 7faf82d9d640 1 -- 172.21.15.78:0/763189928 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb3a333c0 con 0x555bb3d67680 2024-09-16T00:28:12.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.052+0000 7faf82d9d640 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 0x555bb3d64910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c520 tx=0x7faf7800cfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff39a2ca4516380 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.052+0000 7faf69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:28:12.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.053+0000 7faf69ffb640 1 -- 172.21.15.78:0/763189928 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78020070 con 0x555bb3d67680 2024-09-16T00:28:12.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.053+0000 7faf69ffb640 1 -- 172.21.15.78:0/763189928 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faf7801a660 con 0x555bb3d67680 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.053+0000 7faf8b320740 1 -- 172.21.15.78:0/763189928 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7800cc00 con 0x555bb3d67680 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.053+0000 7faf69ffb640 1 -- 172.21.15.78:0/763189928 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faf78015040 con 0x555bb3d67680 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.053+0000 7faf8b320740 10 client.27001 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.053+0000 7faf8b320740 1 -- 172.21.15.78:0/763189928 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bb3d75cc0 con 0x555bb3d67680 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.055+0000 7faf69ffb640 1 -- 172.21.15.78:0/763189928 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7800cc00 con 0x555bb3d67680 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.055+0000 7faf69ffb640 1 -- 172.21.15.78:0/763189928 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7faf7802e790 con 0x555bb3d67680 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.056+0000 7faf8b320740 10 client.27001 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.056+0000 7faf8b320740 10 client.27001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.056+0000 7faf8b320740 20 client.27001 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.056+0000 7faf8b320740 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555bb3d77070 0x555bb3d79460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.056+0000 7faf8b320740 4 client.27001 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:12.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.056+0000 7faf8259c640 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555bb3d77070 0x555bb3d79460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.057+0000 7faf8b320740 1 -- 172.21.15.78:0/763189928 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bb3ccf850 con 0x555bb3d77070 2024-09-16T00:28:12.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.057+0000 7faf8259c640 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555bb3d77070 0x555bb3d79460 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=873375b06a1bacbf server_cookie=ef585244e1a4de97 in_seq=0 out_seq=0 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.057+0000 7faf69ffb640 10 client.27001 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf69ffb640 1 -- 172.21.15.78:0/763189928 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bb3ccf850 con 0x555bb3d77070 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf69ffb640 10 client.27001 handle_command_reply: tid=0 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 client.27001 shutdown 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 -- 172.21.15.78:0/763189928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 msgr2=0x555bb3d64910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 0x555bb3d64910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c520 tx=0x7faf7800cfc0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 -- 172.21.15.78:0/763189928 shutdown_connections 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555bb3d77070 0x555bb3d79460 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 0x555bb3d64910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5cdc0 0x555bb3d64320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 --2- 172.21.15.78:0/763189928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5c4a0 0x555bb3d67070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf8b320740 1 -- 172.21.15.78:0/763189928 >> 172.21.15.78:0/763189928 conn(0x555bb3d58a30 msgr2=0x555bb3d5e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.058+0000 7faf69ffb640 0 client.27001 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:12.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.059+0000 7faf8b320740 1 -- 172.21.15.78:0/763189928 shutdown_connections 2024-09-16T00:28:12.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.059+0000 7faf8b320740 1 -- 172.21.15.78:0/763189928 wait complete. 2024-09-16T00:28:12.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.059+0000 7faf8b320740 20 client.27001 trim_cache size 0 max 16384 2024-09-16T00:28:12.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.089+0000 7faf8b320740 1 Processor -- start 2024-09-16T00:28:12.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8b320740 1 -- start start 2024-09-16T00:28:12.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8b320740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5c4a0 0x555bb3d66650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8b320740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d7c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8b320740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d67680 0x555bb3d7c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8b320740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf841b50e0 con 0x555bb3d5cdc0 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8b320740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faf841b6c20 con 0x555bb3d67680 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8b320740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf841b57b0 con 0x555bb3d5c4a0 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5c4a0 0x555bb3d66650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5c4a0 0x555bb3d66650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:50186/0 (socket says 172.21.15.78:50186) 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf8259c640 1 -- 172.21.15.78:0/1999808869 learned_addr learned my addr 172.21.15.78:0/1999808869 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:12.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf81d9b640 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d7c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.090+0000 7faf82d9d640 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d67680 0x555bb3d7c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.093+0000 7faf82d9d640 1 -- 172.21.15.78:0/1999808869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5c4a0 msgr2=0x555bb3d66650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.093+0000 7faf82d9d640 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5c4a0 0x555bb3d66650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.093+0000 7faf82d9d640 1 -- 172.21.15.78:0/1999808869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 msgr2=0x555bb3d7c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.093+0000 7faf82d9d640 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d7c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.093+0000 7faf82d9d640 1 -- 172.21.15.78:0/1999808869 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb3d596c0 con 0x555bb3d67680 2024-09-16T00:28:12.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.093+0000 7faf81d9b640 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d7c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:12.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.095+0000 7faf82d9d640 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d67680 0x555bb3d7c9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf78002010 tx=0x7faf7800c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12be0bf42b0da568 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.099+0000 7faf6bfff640 1 -- 172.21.15.78:0/1999808869 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78020070 con 0x555bb3d67680 2024-09-16T00:28:12.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.099+0000 7faf6bfff640 1 -- 172.21.15.78:0/1999808869 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7800cc00 con 0x555bb3d67680 2024-09-16T00:28:12.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.099+0000 7faf6bfff640 1 -- 172.21.15.78:0/1999808869 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78004020 con 0x555bb3d67680 2024-09-16T00:28:12.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.104+0000 7faf8b320740 1 -- 172.21.15.78:0/1999808869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d67680 msgr2=0x555bb3d7c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.104+0000 7faf8b320740 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d67680 0x555bb3d7c9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf78002010 tx=0x7faf7800c6c0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.107+0000 7faf8b320740 1 -- 172.21.15.78:0/1999808869 shutdown_connections 2024-09-16T00:28:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.107+0000 7faf8b320740 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d67680 0x555bb3d7c9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.107+0000 7faf8b320740 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d7c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.107+0000 7faf8b320740 1 --2- 172.21.15.78:0/1999808869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d5c4a0 0x555bb3d66650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.107+0000 7faf8b320740 1 -- 172.21.15.78:0/1999808869 >> 172.21.15.78:0/1999808869 conn(0x555bb3d58a30 msgr2=0x555bb3d5f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.114+0000 7faf8b320740 1 -- 172.21.15.78:0/1999808869 shutdown_connections 2024-09-16T00:28:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.117+0000 7faf8b320740 1 -- 172.21.15.78:0/1999808869 wait complete. 2024-09-16T00:28:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 1 -- start start 2024-09-16T00:28:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 1 Processor -- start 2024-09-16T00:28:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d7ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d79d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 0x555bb3d7a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf841b6550 con 0x555bb3d5cdc0 2024-09-16T00:28:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faf841b5e80 con 0x555bb3d5c4a0 2024-09-16T00:28:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf841b30c0 con 0x555bb3d67680 2024-09-16T00:28:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.118+0000 7faf8b320740 10 client.0 _ll_register_callbacks cb 0x7faf7400b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.119+0000 7faf8259c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d7ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.119+0000 7faf81d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d79d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.120+0000 7faf81d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d79d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57094/0 (socket says 172.21.15.78:57094) 2024-09-16T00:28:12.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.120+0000 7faf81d9b640 1 -- 172.21.15.78:0/452593972 learned_addr learned my addr 172.21.15.78:0/452593972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.120+0000 7faf81d9b640 1 -- 172.21.15.78:0/452593972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 msgr2=0x555bb3d7a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.120+0000 7faf81d9b640 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 0x555bb3d7a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.120+0000 7faf81d9b640 1 -- 172.21.15.78:0/452593972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 msgr2=0x555bb3d7ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.120+0000 7faf81d9b640 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d7ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.121+0000 7faf81d9b640 1 -- 172.21.15.78:0/452593972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb3d7a8e0 con 0x555bb3d5cdc0 2024-09-16T00:28:12.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.121+0000 7faf81d9b640 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d79d60 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7faf7c03bee0 tx=0x7faf7c0043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25af684fa7157f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.121+0000 7faf69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.121+0000 7faf69ffb640 1 -- 172.21.15.78:0/452593972 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0e9020 con 0x555bb3d5cdc0 2024-09-16T00:28:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.121+0000 7faf8b320740 1 -- 172.21.15.78:0/452593972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7800cc00 con 0x555bb3d5cdc0 2024-09-16T00:28:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.121+0000 7faf69ffb640 1 -- 172.21.15.78:0/452593972 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faf7c0ca540 con 0x555bb3d5cdc0 2024-09-16T00:28:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.122+0000 7faf69ffb640 1 -- 172.21.15.78:0/452593972 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faf7c0db050 con 0x555bb3d5cdc0 2024-09-16T00:28:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.122+0000 7faf8b320740 10 client.18027 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:12.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.122+0000 7faf8b320740 1 -- 172.21.15.78:0/452593972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bb3e03890 con 0x555bb3d5cdc0 2024-09-16T00:28:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.123+0000 7faf69ffb640 1 -- 172.21.15.78:0/452593972 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c0ce5d0 con 0x555bb3d5cdc0 2024-09-16T00:28:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.123+0000 7faf69ffb640 1 -- 172.21.15.78:0/452593972 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7faf7c004b40 con 0x555bb3d5cdc0 2024-09-16T00:28:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.123+0000 7faf8b320740 10 client.18027 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.123+0000 7faf8b320740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.123+0000 7faf8b320740 20 client.18027 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.123+0000 7faf8b320740 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555bb3e04160 0x555bb3e06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.123+0000 7faf8b320740 4 client.18027 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:28:12.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.123+0000 7faf8b320740 1 -- 172.21.15.78:0/452593972 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7faf7800cc00 con 0x555bb3e04160 2024-09-16T00:28:12.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.127+0000 7faf8259c640 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555bb3e04160 0x555bb3e06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:12.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.131+0000 7faf69ffb640 10 client.18027 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:12.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.131+0000 7faf8259c640 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555bb3e04160 0x555bb3e06580 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4241bbddca23f116 server_cookie=f1430052868c18f5 in_seq=0 out_seq=0 2024-09-16T00:28:12.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.135+0000 7faf69ffb640 1 -- 172.21.15.78:0/452593972 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7faf7800cc00 con 0x555bb3e04160 2024-09-16T00:28:12.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.135+0000 7faf69ffb640 10 client.18027 handle_command_reply: tid=0 2024-09-16T00:28:12.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.135+0000 7faf8b320740 1 client.18027 shutdown 2024-09-16T00:28:12.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.136+0000 7faf8b320740 1 -- 172.21.15.78:0/452593972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 msgr2=0x555bb3d79d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.136+0000 7faf8b320740 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d79d60 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7faf7c03bee0 tx=0x7faf7c0043a0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.141+0000 7faf8b320740 1 -- 172.21.15.78:0/452593972 shutdown_connections 2024-09-16T00:28:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.141+0000 7faf8b320740 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x555bb3e04160 0x555bb3e06580 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.141+0000 7faf8b320740 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555bb3d67680 0x555bb3d7a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.141+0000 7faf8b320740 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x555bb3d5cdc0 0x555bb3d79d60 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.141+0000 7faf8b320740 1 --2- 172.21.15.78:0/452593972 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555bb3d5c4a0 0x555bb3d7ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.141+0000 7faf8b320740 1 -- 172.21.15.78:0/452593972 >> 172.21.15.78:0/452593972 conn(0x555bb3d58a30 msgr2=0x555bb3d57370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.141+0000 7faf69ffb640 0 client.18027 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:12.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.144+0000 7faf8b320740 1 -- 172.21.15.78:0/452593972 shutdown_connections 2024-09-16T00:28:12.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.144+0000 7faf8b320740 1 -- 172.21.15.78:0/452593972 wait complete. 2024-09-16T00:28:12.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.144+0000 7faf8b320740 20 client.18027 trim_cache size 0 max 16384 2024-09-16T00:28:12.145 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:28:12.145 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:28:12.145 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:28:12.145 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:28:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 -- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf6c0a3410 msgr2=0x7faf6c0a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf6c0a3410 0x7faf6c0a58d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7faf74006fd0 tx=0x7faf74008040 comp rx=0 tx=0).stop 2024-09-16T00:28:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 -- 172.21.15.78:0/2790809523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf84109830 msgr2=0x7faf84116830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:12.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf84109830 0x7faf84116830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf7c003c50 tx=0x7faf7c003c80 comp rx=0 tx=0).stop 2024-09-16T00:28:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 -- 172.21.15.78:0/2790809523 shutdown_connections 2024-09-16T00:28:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faf6c0a3410 0x7faf6c0a58d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf8411b140 0x7faf84116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faf84109830 0x7faf84116830 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 --2- 172.21.15.78:0/2790809523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf84109450 0x7faf8411a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 -- 172.21.15.78:0/2790809523 >> 172.21.15.78:0/2790809523 conn(0x7faf8406e140 msgr2=0x7faf84070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 -- 172.21.15.78:0/2790809523 shutdown_connections 2024-09-16T00:28:12.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:12.147+0000 7faf88f47640 1 -- 172.21.15.78:0/2790809523 wait complete. 2024-09-16T00:28:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:11 smithi193 ceph-mon[35484]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.3 MiB/s wr, 682 op/s 2024-09-16T00:28:12.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:11 smithi148 ceph-mon[35599]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.3 MiB/s wr, 682 op/s 2024-09-16T00:28:13.132 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:28:13.133 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:13.622 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:14.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:13 smithi078 ceph-mon[31157]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.0 MiB/s wr, 644 op/s 2024-09-16T00:28:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:13 smithi193 ceph-mon[35484]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.0 MiB/s wr, 644 op/s 2024-09-16T00:28:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:13 smithi148 ceph-mon[35599]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.0 MiB/s wr, 644 op/s 2024-09-16T00:28:14.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.236+0000 7f12b5348640 1 -- 172.21.15.78:0/127642720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109830 msgr2=0x7f12b0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.236+0000 7f12b5348640 1 --2- 172.21.15.78:0/127642720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109830 0x7f12b0109c90 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009900 tx=0x7f12a402a440 comp rx=0 tx=0).stop 2024-09-16T00:28:14.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.238+0000 7f12b5348640 1 -- 172.21.15.78:0/127642720 shutdown_connections 2024-09-16T00:28:14.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.238+0000 7f12b5348640 1 --2- 172.21.15.78:0/127642720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b011b140 0x7f12b011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.238+0000 7f12b5348640 1 --2- 172.21.15.78:0/127642720 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109830 0x7f12b0109c90 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.238+0000 7f12b5348640 1 --2- 172.21.15.78:0/127642720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b0109450 0x7f12b0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.238+0000 7f12b5348640 1 -- 172.21.15.78:0/127642720 >> 172.21.15.78:0/127642720 conn(0x7f12b006e140 msgr2=0x7f12b006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:14.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.242+0000 7f12b5348640 1 -- 172.21.15.78:0/127642720 shutdown_connections 2024-09-16T00:28:14.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.242+0000 7f12b5348640 1 -- 172.21.15.78:0/127642720 wait complete. 2024-09-16T00:28:14.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.242+0000 7f12b5348640 1 Processor -- start 2024-09-16T00:28:14.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.243+0000 7f12b5348640 1 -- start start 2024-09-16T00:28:14.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12b5348640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109450 0x7f12b0118a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12b5348640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b0109830 0x7f12b01109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109450 0x7f12b0118a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12aeffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109450 0x7f12b0118a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57114/0 (socket says 172.21.15.78:57114) 2024-09-16T00:28:14.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12aeffd640 1 -- 172.21.15.78:0/3755049593 learned_addr learned my addr 172.21.15.78:0/3755049593 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:14.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12b5348640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b011b140 0x7f12b0110f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0120e30 con 0x7f12b0109450 2024-09-16T00:28:14.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0120cb0 con 0x7f12b0109830 2024-09-16T00:28:14.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.247+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0120fb0 con 0x7f12b011b140 2024-09-16T00:28:14.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.249+0000 7f12aeffd640 1 -- 172.21.15.78:0/3755049593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b011b140 msgr2=0x7f12b0110f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:28:14.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.249+0000 7f12aeffd640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b011b140 0x7f12b0110f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.249+0000 7f12aeffd640 1 -- 172.21.15.78:0/3755049593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b0109830 msgr2=0x7f12b01109f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:28:14.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.249+0000 7f12aeffd640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b0109830 0x7f12b01109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.249+0000 7f12aeffd640 1 -- 172.21.15.78:0/3755049593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b0111750 con 0x7f12b0109450 2024-09-16T00:28:14.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.250+0000 7f12aeffd640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109450 0x7f12b0118a20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f12a000d950 tx=0x7f12a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:14.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.251+0000 7f1297fff640 1 -- 172.21.15.78:0/3755049593 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0017070 con 0x7f12b0109450 2024-09-16T00:28:14.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.251+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b01bc140 con 0x7f12b0109450 2024-09-16T00:28:14.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.251+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01bc6d0 con 0x7f12b0109450 2024-09-16T00:28:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.251+0000 7f1297fff640 1 -- 172.21.15.78:0/3755049593 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12a0009c30 con 0x7f12b0109450 2024-09-16T00:28:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.251+0000 7f1297fff640 1 -- 172.21.15.78:0/3755049593 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0017070 con 0x7f12b0109450 2024-09-16T00:28:14.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.253+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c005290 con 0x7f12b0109450 2024-09-16T00:28:14.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.258+0000 7f1297fff640 1 -- 172.21.15.78:0/3755049593 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f12a00109d0 con 0x7f12b0109450 2024-09-16T00:28:14.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.259+0000 7f1297fff640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12980a3680 0x7f12980a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:14.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.260+0000 7f1297fff640 1 -- 172.21.15.78:0/3755049593 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f12a0009a30 con 0x7f12b0109450 2024-09-16T00:28:14.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.263+0000 7f1297fff640 1 -- 172.21.15.78:0/3755049593 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12a009bcc0 con 0x7f12b0109450 2024-09-16T00:28:14.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.264+0000 7f12ae7fc640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12980a3680 0x7f12980a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:14.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.266+0000 7f12ae7fc640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12980a3680 0x7f12980a5b40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005c20 tx=0x7f12a4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:14.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.626+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f129c005580 con 0x7f12b0109450 2024-09-16T00:28:14.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.634+0000 7f1297fff640 1 -- 172.21.15.78:0/3755049593 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f12a009bcc0 con 0x7f12b0109450 2024-09-16T00:28:14.635 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:28:14.635 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:28:14.635 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:28:14.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.640+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12980a3680 msgr2=0x7f12980a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.640+0000 7f12b5348640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12980a3680 0x7f12980a5b40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005c20 tx=0x7f12a4000bc0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.641+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109450 msgr2=0x7f12b0118a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:14.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.641+0000 7f12b5348640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109450 0x7f12b0118a20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f12a000d950 tx=0x7f12a000de10 comp rx=0 tx=0).stop 2024-09-16T00:28:14.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.641+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 shutdown_connections 2024-09-16T00:28:14.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.641+0000 7f12b5348640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f12980a3680 0x7f12980a5b40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.641+0000 7f12b5348640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b011b140 0x7f12b0110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.641+0000 7f12b5348640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f12b0109830 0x7f12b01109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.641+0000 7f12b5348640 1 --2- 172.21.15.78:0/3755049593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12b0109450 0x7f12b0118a20 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:14.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.641+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 >> 172.21.15.78:0/3755049593 conn(0x7f12b006e140 msgr2=0x7f12b011bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:14.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.643+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 shutdown_connections 2024-09-16T00:28:14.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:14.643+0000 7f12b5348640 1 -- 172.21.15.78:0/3755049593 wait complete. 2024-09-16T00:28:15.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:14 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3755049593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:15.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:14 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3755049593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:15.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:14 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3755049593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:15.402 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:28:15.860 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 855.0 s, 535.7 tps, lat 185.415 ms stddev 217.116 2024-09-16T00:28:15.873 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:16.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:15 smithi193 ceph-mon[35484]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.6 MiB/s wr, 623 op/s 2024-09-16T00:28:16.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:15 smithi148 ceph-mon[35599]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.6 MiB/s wr, 623 op/s 2024-09-16T00:28:16.211 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:15 smithi078 ceph-mon[31157]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.6 MiB/s wr, 623 op/s 2024-09-16T00:28:18.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:17 smithi193 ceph-mon[35484]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 603 op/s 2024-09-16T00:28:18.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:17 smithi148 ceph-mon[35599]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 603 op/s 2024-09-16T00:28:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:17 smithi078 ceph-mon[31157]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.2 MiB/s wr, 603 op/s 2024-09-16T00:28:18.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 -- 172.21.15.78:0/2847551941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd0101290 msgr2=0x7fddd0107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/2847551941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd0101290 0x7fddd0107a50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009900 tx=0x7fddc402a410 comp rx=0 tx=0).stop 2024-09-16T00:28:18.440 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 -- 172.21.15.78:0/2847551941 shutdown_connections 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/2847551941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddd0108180 0x7fddd010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/2847551941 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd0101290 0x7fddd0107a50 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/2847551941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddd01008e0 0x7fddd0100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 -- 172.21.15.78:0/2847551941 >> 172.21.15.78:0/2847551941 conn(0x7fddd0077210 msgr2=0x7fddd0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 -- 172.21.15.78:0/2847551941 shutdown_connections 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.225+0000 7fddd5c2d640 1 -- 172.21.15.78:0/2847551941 wait complete. 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddd5c2d640 1 Processor -- start 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddd5c2d640 1 -- start start 2024-09-16T00:28:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddd5c2d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd01008e0 0x7fddd0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddd5c2d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddd0101290 0x7fddd0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddd5c2d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddd0108180 0x7fddd019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddd5c2d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0112070 con 0x7fddd01008e0 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddd5c2d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0111ef0 con 0x7fddd0101290 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddd5c2d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01121f0 con 0x7fddd0108180 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd01008e0 0x7fddd0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd01008e0 0x7fddd0198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59510/0 (socket says 172.21.15.78:59510) 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcf7fe640 1 -- 172.21.15.78:0/623509058 learned_addr learned my addr 172.21.15.78:0/623509058 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddceffd640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddd0101290 0x7fddd0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcffff640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddd0108180 0x7fddd019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcf7fe640 1 -- 172.21.15.78:0/623509058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddd0108180 msgr2=0x7fddd019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddd0108180 0x7fddd019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcf7fe640 1 -- 172.21.15.78:0/623509058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddd0101290 msgr2=0x7fddd0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddd0101290 0x7fddd0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.226+0000 7fddcf7fe640 1 -- 172.21.15.78:0/623509058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd019d4b0 con 0x7fddd01008e0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.227+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd01008e0 0x7fddd0198500 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fddc000d950 tx=0x7fddc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.227+0000 7fddccff9640 1 -- 172.21.15.78:0/623509058 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0017070 con 0x7fddd01008e0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.227+0000 7fddd5c2d640 1 -- 172.21.15.78:0/623509058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd019d7a0 con 0x7fddd01008e0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.227+0000 7fddccff9640 1 -- 172.21.15.78:0/623509058 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc0009c30 con 0x7fddd01008e0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.227+0000 7fddd5c2d640 1 -- 172.21.15.78:0/623509058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd01a5080 con 0x7fddd01008e0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.227+0000 7fddccff9640 1 -- 172.21.15.78:0/623509058 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc000f6b0 con 0x7fddd01008e0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.229+0000 7fddccff9640 1 -- 172.21.15.78:0/623509058 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fddc00109d0 con 0x7fddd01008e0 2024-09-16T00:28:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.229+0000 7fddd8006740 1 Processor -- start 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.229+0000 7fddd8006740 1 -- start start 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddd8006740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddd8006740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddd8006740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddd8006740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607969152f0 con 0x5607969af5d0 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddd8006740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5607968bccf0 con 0x5607969b0b40 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddd8006740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607969112b0 con 0x5607969be780 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59516/0 (socket says 172.21.15.78:59516) 2024-09-16T00:28:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcf7fe640 1 -- 172.21.15.78:0/1126551849 learned_addr learned my addr 172.21.15.78:0/1126551849 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddceffd640 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddccff9640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fddb80a32c0 0x7fddb80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcf7fe640 1 -- 172.21.15.78:0/1126551849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 msgr2=0x5607969c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969c0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcf7fe640 1 -- 172.21.15.78:0/1126551849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 msgr2=0x5607969af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcf7fe640 1 -- 172.21.15.78:0/1126551849 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607968a13a0 con 0x5607969af5d0 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcffff640 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddcffff640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fddb80a32c0 0x7fddb80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.230+0000 7fddccff9640 1 -- 172.21.15.78:0/623509058 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fddc00cfd50 con 0x7fddd01008e0 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.231+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969afa10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fddc001df40 tx=0x7fddc0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e83572067b3157 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.231+0000 7fddcffff640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fddb80a32c0 0x7fddb80a5780 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005fd0 tx=0x7fddc8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.231+0000 7fddb5ffb640 1 -- 172.21.15.78:0/1126551849 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc009b910 con 0x5607969af5d0 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.231+0000 7fddb5ffb640 1 -- 172.21.15.78:0/1126551849 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc00bc080 con 0x5607969af5d0 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.231+0000 7fddb5ffb640 1 -- 172.21.15.78:0/1126551849 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc001d9a0 con 0x5607969af5d0 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.231+0000 7fddd8006740 1 -- 172.21.15.78:0/1126551849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 msgr2=0x5607969afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.231+0000 7fddd8006740 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969afa10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fddc001df40 tx=0x7fddc0023000 comp rx=0 tx=0).stop 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 -- 172.21.15.78:0/1126551849 shutdown_connections 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969afa10 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 --2- 172.21.15.78:0/1126551849 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 -- 172.21.15.78:0/1126551849 >> 172.21.15.78:0/1126551849 conn(0x560796978310 msgr2=0x5607969abc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 -- 172.21.15.78:0/1126551849 shutdown_connections 2024-09-16T00:28:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 -- 172.21.15.78:0/1126551849 wait complete. 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 -- start start 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 Processor -- start 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969be180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.232+0000 7fddd8006740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddd8006740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969bba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddd8006740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5607969c1420 con 0x5607969af5d0 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddd8006740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5607968d0060 con 0x5607969b0b40 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddd8006740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607969c15a0 con 0x5607969be780 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969be180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969be180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59528/0 (socket says 172.21.15.78:59528) 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 -- 172.21.15.78:0/2790490704 learned_addr learned my addr 172.21.15.78:0/2790490704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:18.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddd8006740 10 client.0 _ll_register_callbacks cb 0x5607966238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddceffd640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcffff640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 -- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 msgr2=0x5607969bba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969bba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 -- 172.21.15.78:0/2790490704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 msgr2=0x5607969bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969bb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 -- 172.21.15.78:0/2790490704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607966863c0 con 0x5607969af5d0 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddceffd640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcffff640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969bba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969be180 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fddc00a46f0 tx=0x7fddc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2bceb38d7361da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:18.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddaeffd640 1 -- 172.21.15.78:0/2790490704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc001dba0 con 0x5607969af5d0 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddaeffd640 1 -- 172.21.15.78:0/2790490704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fddc009a2b0 con 0x5607969af5d0 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddd8006740 1 -- 172.21.15.78:0/2790490704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc00bc080 con 0x5607969af5d0 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddaeffd640 1 -- 172.21.15.78:0/2790490704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fddc001dda0 con 0x5607969af5d0 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddd8006740 10 client.18045 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.233+0000 7fddd8006740 1 -- 172.21.15.78:0/2790490704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607969ccea0 con 0x5607969af5d0 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddaeffd640 1 -- 172.21.15.78:0/2790490704 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc0009de0 con 0x5607969af5d0 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddaeffd640 1 -- 172.21.15.78:0/2790490704 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fddc0098620 con 0x5607969af5d0 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddd8006740 10 client.18045 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddd8006740 10 client.18045 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddd8006740 20 client.18045 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddd8006740 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5607969ce020 0x5607969d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddd8006740 4 client.18045 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddd8006740 1 -- 172.21.15.78:0/2790490704 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fddc00bc080 con 0x5607969ce020 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.234+0000 7fddceffd640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5607969ce020 0x5607969d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.235+0000 7fddceffd640 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5607969ce020 0x5607969d0410 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f25a379db88db867 server_cookie=aba753ca3e4153a7 in_seq=0 out_seq=0 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.235+0000 7fddaeffd640 10 client.18045 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.235+0000 7fddaeffd640 1 -- 172.21.15.78:0/2790490704 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fddc00bc080 con 0x5607969ce020 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.235+0000 7fddaeffd640 10 client.18045 handle_command_reply: tid=0 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 client.18045 shutdown 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 -- 172.21.15.78:0/2790490704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 msgr2=0x5607969be180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969be180 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fddc00a46f0 tx=0x7fddc0005880 comp rx=0 tx=0).stop 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 -- 172.21.15.78:0/2790490704 shutdown_connections 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5607969ce020 0x5607969d0410 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddaeffd640 0 client.18045 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:28:18.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969bba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 --2- 172.21.15.78:0/2790490704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969af5d0 0x5607969be180 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 -- 172.21.15.78:0/2790490704 >> 172.21.15.78:0/2790490704 conn(0x560796978310 msgr2=0x5607969b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 -- 172.21.15.78:0/2790490704 shutdown_connections 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 1 -- 172.21.15.78:0/2790490704 wait complete. 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.236+0000 7fddd8006740 20 client.18045 trim_cache size 0 max 16384 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddd8006740 1 Processor -- start 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddd8006740 1 -- start start 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddd8006740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969af5d0 0x5607969ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddd8006740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969ba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddd8006740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969be780 0x5607969d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddd8006740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01aca40 con 0x5607969be780 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddd8006740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01ae580 con 0x5607969b0b40 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddd8006740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01ad110 con 0x5607969af5d0 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969be780 0x5607969d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969be780 0x5607969d59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59534/0 (socket says 172.21.15.78:59534) 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 -- 172.21.15.78:0/4079554475 learned_addr learned my addr 172.21.15.78:0/4079554475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddceffd640 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969ba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969af5d0 0x5607969ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 -- 172.21.15.78:0/4079554475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969af5d0 msgr2=0x5607969ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969af5d0 0x5607969ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 -- 172.21.15.78:0/4079554475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 msgr2=0x5607969ba410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:18.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969ba410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 -- 172.21.15.78:0/4079554475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607969b9980 con 0x5607969be780 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddcffff640 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969be780 0x5607969d59b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fddc800f700 tx=0x7fddc8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dcc40c1e238c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.249+0000 7fddb57fa640 1 -- 172.21.15.78:0/4079554475 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc80128c0 con 0x5607969be780 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddb57fa640 1 -- 172.21.15.78:0/4079554475 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc8023030 con 0x5607969be780 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddb57fa640 1 -- 172.21.15.78:0/4079554475 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc801e420 con 0x5607969be780 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 -- 172.21.15.78:0/4079554475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969be780 msgr2=0x5607969d59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969be780 0x5607969d59b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fddc800f700 tx=0x7fddc8015000 comp rx=0 tx=0).stop 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 -- 172.21.15.78:0/4079554475 shutdown_connections 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969be780 0x5607969d59b0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969b0b40 0x5607969ba410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 --2- 172.21.15.78:0/4079554475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969af5d0 0x5607969ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 -- 172.21.15.78:0/4079554475 >> 172.21.15.78:0/4079554475 conn(0x560796978310 msgr2=0x5607969b9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:18.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 -- 172.21.15.78:0/4079554475 shutdown_connections 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 -- 172.21.15.78:0/4079554475 wait complete. 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.250+0000 7fddd8006740 1 -- start start 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddd8006740 1 Processor -- start 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddd8006740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969af5d0 0x5607969c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddd8006740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969b0b40 0x5607969c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddd8006740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddd8006740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01adeb0 con 0x5607969b0b40 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddd8006740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01ad7e0 con 0x5607969af5d0 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddd8006740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01aaa20 con 0x5607969be780 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddd8006740 10 client.0 _ll_register_callbacks cb 0x7fddc80095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969b0b40 0x5607969c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddceffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969b0b40 0x5607969c2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59538/0 (socket says 172.21.15.78:59538) 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddceffd640 1 -- 172.21.15.78:0/2015437892 learned_addr learned my addr 172.21.15.78:0/2015437892 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969af5d0 0x5607969c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddcffff640 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddceffd640 1 -- 172.21.15.78:0/2015437892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 msgr2=0x5607969c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddceffd640 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddceffd640 1 -- 172.21.15.78:0/2015437892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969af5d0 msgr2=0x5607969c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddceffd640 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969af5d0 0x5607969c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.251+0000 7fddceffd640 1 -- 172.21.15.78:0/2015437892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560796a37980 con 0x5607969b0b40 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddceffd640 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969b0b40 0x5607969c2410 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fddd006a760 tx=0x7fddc4004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec2695ac01e21a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddaeffd640 1 -- 172.21.15.78:0/2015437892 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc4035040 con 0x5607969b0b40 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 1 -- 172.21.15.78:0/2015437892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc8023030 con 0x5607969b0b40 2024-09-16T00:28:18.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddaeffd640 1 -- 172.21.15.78:0/2015437892 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fddc4005240 con 0x5607969b0b40 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddaeffd640 1 -- 172.21.15.78:0/2015437892 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fddc4052050 con 0x5607969b0b40 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 10 client.18051 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 1 -- 172.21.15.78:0/2015437892 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560796a38b90 con 0x5607969b0b40 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddaeffd640 1 -- 172.21.15.78:0/2015437892 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc8023030 con 0x5607969b0b40 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddaeffd640 1 -- 172.21.15.78:0/2015437892 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fddc40035f0 con 0x5607969b0b40 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 10 client.18051 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 10 client.18051 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 20 client.18051 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5607969ce020 0x560796a3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 4 client.18051 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:28:18.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.252+0000 7fddd8006740 1 -- 172.21.15.78:0/2015437892 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560796922860 con 0x5607969ce020 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5607969ce020 0x560796a3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddaeffd640 10 client.18051 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddcf7fe640 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5607969ce020 0x560796a3b320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b96027e9ca3c19a server_cookie=171a417e9d1b036a in_seq=0 out_seq=0 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddaeffd640 1 -- 172.21.15.78:0/2015437892 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560796922860 con 0x5607969ce020 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddaeffd640 10 client.18051 handle_command_reply: tid=0 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 client.18051 shutdown 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 -- 172.21.15.78:0/2015437892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969b0b40 msgr2=0x5607969c2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969b0b40 0x5607969c2410 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fddd006a760 tx=0x7fddc4004280 comp rx=0 tx=0).stop 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 -- 172.21.15.78:0/2015437892 shutdown_connections 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x5607969ce020 0x560796a3b320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5607969be780 0x5607969c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5607969b0b40 0x5607969c2410 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddaeffd640 0 client.18051 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 --2- 172.21.15.78:0/2015437892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5607969af5d0 0x5607969c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 -- 172.21.15.78:0/2015437892 >> 172.21.15.78:0/2015437892 conn(0x560796978310 msgr2=0x5607969ba520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 -- 172.21.15.78:0/2015437892 shutdown_connections 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.253+0000 7fddd8006740 1 -- 172.21.15.78:0/2015437892 wait complete. 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd8006740 20 client.18051 trim_cache size 0 max 16384 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 -- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fddb80a32c0 msgr2=0x7fddb80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fddb80a32c0 0x7fddb80a5780 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005fd0 tx=0x7fddc8004a40 comp rx=0 tx=0).stop 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 -- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd01008e0 msgr2=0x7fddd0198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:18.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd01008e0 0x7fddd0198500 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fddc000d950 tx=0x7fddc000de10 comp rx=0 tx=0).stop 2024-09-16T00:28:18.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 -- 172.21.15.78:0/623509058 shutdown_connections 2024-09-16T00:28:18.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fddb80a32c0 0x7fddb80a5780 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddd0108180 0x7fddd019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fddd0101290 0x7fddd0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 --2- 172.21.15.78:0/623509058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fddd01008e0 0x7fddd0198500 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:18.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 -- 172.21.15.78:0/623509058 >> 172.21.15.78:0/623509058 conn(0x7fddd0077210 msgr2=0x7fddd0105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:18.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 -- 172.21.15.78:0/623509058 shutdown_connections 2024-09-16T00:28:18.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:17.254+0000 7fddd5c2d640 1 -- 172.21.15.78:0/623509058 wait complete. 2024-09-16T00:28:19.275 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:28:19.716 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:20.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:19 smithi193 ceph-mon[35484]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-16T00:28:20.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:19 smithi148 ceph-mon[35599]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-16T00:28:20.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:19 smithi078 ceph-mon[31157]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 523 op/s 2024-09-16T00:28:20.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 860.0 s, 444.9 tps, lat 220.518 ms stddev 261.045 2024-09-16T00:28:21.727 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:28:21.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 -- 172.21.15.78:0/179500984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 msgr2=0x7fabac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 --2- 172.21.15.78:0/179500984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 0x7fabac1029b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009900 tx=0x7fab9c02a480 comp rx=0 tx=0).stop 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 -- 172.21.15.78:0/179500984 shutdown_connections 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 --2- 172.21.15.78:0/179500984 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabac102ef0 0x7fabac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 --2- 172.21.15.78:0/179500984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 0x7fabac1029b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 --2- 172.21.15.78:0/179500984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabac100e20 0x7fabac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 -- 172.21.15.78:0/179500984 >> 172.21.15.78:0/179500984 conn(0x7fabac0fc7d0 msgr2=0x7fabac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 -- 172.21.15.78:0/179500984 shutdown_connections 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.699+0000 7fabb1625640 1 -- 172.21.15.78:0/179500984 wait complete. 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabb1625640 1 Processor -- start 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabb1625640 1 -- start start 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabb1625640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabac100e20 0x7fabac198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabb1625640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 0x7fabac198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabb1625640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabac102ef0 0x7fabac19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabb1625640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabac110fa0 con 0x7fabac102570 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabb1625640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fabac110e20 con 0x7fabac102ef0 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabb1625640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fabac111120 con 0x7fabac100e20 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabaa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 0x7fabac198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabaa7fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 0x7fabac198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59554/0 (socket says 172.21.15.78:59554) 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabaa7fc640 1 -- 172.21.15.78:0/1773131619 learned_addr learned my addr 172.21.15.78:0/1773131619 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.700+0000 7fabab7fe640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabac102ef0 0x7fabac19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabaaffd640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabac100e20 0x7fabac198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabaa7fc640 1 -- 172.21.15.78:0/1773131619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabac100e20 msgr2=0x7fabac198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabaa7fc640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabac100e20 0x7fabac198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabaa7fc640 1 -- 172.21.15.78:0/1773131619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabac102ef0 msgr2=0x7fabac19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabaa7fc640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabac102ef0 0x7fabac19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabaa7fc640 1 -- 172.21.15.78:0/1773131619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac19d4f0 con 0x7fabac102570 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabaaffd640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabac100e20 0x7fabac198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabaa7fc640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 0x7fabac198a80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fab9c006010 tx=0x7fab9c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fab93fff640 1 -- 172.21.15.78:0/1773131619 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c03f070 con 0x7fabac102570 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fab93fff640 1 -- 172.21.15.78:0/1773131619 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab9c031c30 con 0x7fabac102570 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabb1625640 1 -- 172.21.15.78:0/1773131619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabac19d780 con 0x7fabac102570 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fab93fff640 1 -- 172.21.15.78:0/1773131619 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c035030 con 0x7fabac102570 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.701+0000 7fabb1625640 1 -- 172.21.15.78:0/1773131619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabac1a50c0 con 0x7fabac102570 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.703+0000 7fab93fff640 1 -- 172.21.15.78:0/1773131619 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fab9c042070 con 0x7fabac102570 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabb39fe740 1 Processor -- start 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabb39fe740 1 -- start start 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7aeb40 0x56366f7ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fab93fff640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab940a3240 0x7fab940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 0x56366f7ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabb39fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56366f7132f0 con 0x56366f7ad5d0 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabb39fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56366f6bacf0 con 0x56366f7b8370 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabb39fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56366f70f2b0 con 0x56366f7aeb40 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59556/0 (socket says 172.21.15.78:59556) 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabaaffd640 1 -- 172.21.15.78:0/71259340 learned_addr learned my addr 172.21.15.78:0/71259340 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fab93fff640 1 -- 172.21.15.78:0/1773131619 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fab9c0f04d0 con 0x7fabac102570 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabaa7fc640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab940a3240 0x7fab940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabab7fe640 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7aeb40 0x56366f7ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.704+0000 7fabab7fe640 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 0x56366f7ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fabaaffd640 1 -- 172.21.15.78:0/71259340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7aeb40 msgr2=0x56366f7ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fabaaffd640 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7aeb40 0x56366f7ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fabaaffd640 1 -- 172.21.15.78:0/71259340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 msgr2=0x56366f7ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fabaaffd640 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 0x56366f7ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fabaaffd640 1 -- 172.21.15.78:0/71259340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366f69f3a0 con 0x56366f7ad5d0 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fabab7fe640 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7aeb40 0x56366f7ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fabaaffd640 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7ada10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7faba000bd40 tx=0x7faba000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d05f1d4ee7a532e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fabaa7fc640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab940a3240 0x7fab940a5700 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0bc720 tx=0x7fab9c02a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fab917fa640 1 -- 172.21.15.78:0/71259340 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0015070 con 0x56366f7ad5d0 2024-09-16T00:28:21.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fab917fa640 1 -- 172.21.15.78:0/71259340 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba000be30 con 0x56366f7ad5d0 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.705+0000 7fab917fa640 1 -- 172.21.15.78:0/71259340 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0011a30 con 0x56366f7ad5d0 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- 172.21.15.78:0/71259340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 msgr2=0x56366f7ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7ada10 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7faba000bd40 tx=0x7faba000f730 comp rx=0 tx=0).stop 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- 172.21.15.78:0/71259340 shutdown_connections 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 0x56366f7ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7ada10 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 --2- 172.21.15.78:0/71259340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7aeb40 0x56366f7ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- 172.21.15.78:0/71259340 >> 172.21.15.78:0/71259340 conn(0x56366f776310 msgr2=0x56366f7a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- 172.21.15.78:0/71259340 shutdown_connections 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- 172.21.15.78:0/71259340 wait complete. 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- start start 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 Processor -- start 2024-09-16T00:28:21.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7ad5d0 0x56366f7b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7aeb40 0x56366f7b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7b8370 0x56366f7b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56366f7bb070 con 0x56366f7b8370 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56366f6ce060 con 0x56366f7aeb40 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.706+0000 7fabb39fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56366f7bb1f0 con 0x56366f7ad5d0 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7aeb40 0x56366f7b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7aeb40 0x56366f7b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46070/0 (socket says 172.21.15.78:46070) 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabaaffd640 1 -- 172.21.15.78:0/3834633574 learned_addr learned my addr 172.21.15.78:0/3834633574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabb39fe740 10 client.0 _ll_register_callbacks cb 0x56366f4218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7b8370 0x56366f7b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabab7fe640 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7ad5d0 0x56366f7b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabab7fe640 1 -- 172.21.15.78:0/3834633574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7ad5d0 msgr2=0x56366f7b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:21.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabab7fe640 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7ad5d0 0x56366f7b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabab7fe640 1 -- 172.21.15.78:0/3834633574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7aeb40 msgr2=0x56366f7b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabab7fe640 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7aeb40 0x56366f7b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabab7fe640 1 -- 172.21.15.78:0/3834633574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366f4843c0 con 0x56366f7b8370 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabaaffd640 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7aeb40 0x56366f7b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabab7fe640 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7b8370 0x56366f7b5640 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7faba400bf00 tx=0x7faba4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40eee342835a01ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fab8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fab8eff5640 1 -- 172.21.15.78:0/3834633574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4014040 con 0x56366f7b8370 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fab8eff5640 1 -- 172.21.15.78:0/3834633574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faba4018a90 con 0x56366f7b8370 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.707+0000 7fabb39fe740 1 -- 172.21.15.78:0/3834633574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba000be30 con 0x56366f7b8370 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fab8eff5640 1 -- 172.21.15.78:0/3834633574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faba402a050 con 0x56366f7b8370 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabb39fe740 10 client.18063 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabb39fe740 1 -- 172.21.15.78:0/3834633574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56366f7c6b70 con 0x56366f7b8370 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fab8eff5640 1 -- 172.21.15.78:0/3834633574 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba000be30 con 0x56366f7b8370 2024-09-16T00:28:21.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fab8eff5640 1 -- 172.21.15.78:0/3834633574 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7faba401cb50 con 0x56366f7b8370 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabb39fe740 10 client.18063 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabb39fe740 10 client.18063 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabb39fe740 20 client.18063 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x56366f7c84f0 0x56366f7ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabb39fe740 4 client.18063 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabb39fe740 1 -- 172.21.15.78:0/3834633574 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56366f720860 con 0x56366f7c84f0 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.708+0000 7fabaaffd640 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x56366f7c84f0 0x56366f7ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.709+0000 7fabaaffd640 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x56366f7c84f0 0x56366f7ca990 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b80c03b0d95f6e24 server_cookie=882e2f097f6dcc7a in_seq=0 out_seq=0 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.709+0000 7fab8eff5640 10 client.18063 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:28:21.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.709+0000 7fab8eff5640 1 -- 172.21.15.78:0/3834633574 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56366f720860 con 0x56366f7c84f0 2024-09-16T00:28:21.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.709+0000 7fab8eff5640 10 client.18063 handle_command_reply: tid=0 2024-09-16T00:28:21.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.709+0000 7fabb39fe740 1 client.18063 shutdown 2024-09-16T00:28:21.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.709+0000 7fabb39fe740 1 -- 172.21.15.78:0/3834633574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7b8370 msgr2=0x56366f7b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7b8370 0x56366f7b5640 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7faba400bf00 tx=0x7faba4004b50 comp rx=0 tx=0).stop 2024-09-16T00:28:21.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 -- 172.21.15.78:0/3834633574 shutdown_connections 2024-09-16T00:28:21.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7b8370 0x56366f7b5640 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x56366f7c84f0 0x56366f7ca990 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7aeb40 0x56366f7b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fab8eff5640 0 client.18063 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3834633574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7ad5d0 0x56366f7b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 -- 172.21.15.78:0/3834633574 >> 172.21.15.78:0/3834633574 conn(0x56366f776310 msgr2=0x56366f7af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 -- 172.21.15.78:0/3834633574 shutdown_connections 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 1 -- 172.21.15.78:0/3834633574 wait complete. 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.710+0000 7fabb39fe740 20 client.18063 trim_cache size 0 max 16384 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabb39fe740 1 Processor -- start 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabb39fe740 1 -- start start 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7ad5d0 0x56366f7c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7aeb40 0x56366f7b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7b8370 0x56366f7a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabb39fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabac1aebb0 con 0x56366f7aeb40 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabb39fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fabac1b06f0 con 0x56366f7ad5d0 2024-09-16T00:28:21.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabb39fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fabac1af280 con 0x56366f7b8370 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7aeb40 0x56366f7b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabaaffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7aeb40 0x56366f7b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59574/0 (socket says 172.21.15.78:59574) 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabaaffd640 1 -- 172.21.15.78:0/3968725794 learned_addr learned my addr 172.21.15.78:0/3968725794 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabab7fe640 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7ad5d0 0x56366f7c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabab7fe640 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7b8370 0x56366f7a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabaaffd640 1 -- 172.21.15.78:0/3968725794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7b8370 msgr2=0x56366f7a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabaaffd640 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7b8370 0x56366f7a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabaaffd640 1 -- 172.21.15.78:0/3968725794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7ad5d0 msgr2=0x56366f7c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabaaffd640 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7ad5d0 0x56366f7c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.723+0000 7fabaaffd640 1 -- 172.21.15.78:0/3968725794 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366f7aa960 con 0x56366f7aeb40 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fabaaffd640 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7aeb40 0x56366f7b48f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7faba0006010 tx=0x7faba001af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa6f270e255bae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fab90ff9640 1 -- 172.21.15.78:0/3968725794 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0015040 con 0x56366f7aeb40 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fab90ff9640 1 -- 172.21.15.78:0/3968725794 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0005520 con 0x56366f7aeb40 2024-09-16T00:28:21.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fab90ff9640 1 -- 172.21.15.78:0/3968725794 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba00119a0 con 0x56366f7aeb40 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fabb39fe740 1 -- 172.21.15.78:0/3968725794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7aeb40 msgr2=0x56366f7b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7aeb40 0x56366f7b48f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7faba0006010 tx=0x7faba001af90 comp rx=0 tx=0).stop 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fabb39fe740 1 -- 172.21.15.78:0/3968725794 shutdown_connections 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7b8370 0x56366f7a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7aeb40 0x56366f7b48f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fabb39fe740 1 --2- 172.21.15.78:0/3968725794 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7ad5d0 0x56366f7c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.724+0000 7fabb39fe740 1 -- 172.21.15.78:0/3968725794 >> 172.21.15.78:0/3968725794 conn(0x56366f776310 msgr2=0x56366f7b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 -- 172.21.15.78:0/3968725794 shutdown_connections 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 -- 172.21.15.78:0/3968725794 wait complete. 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 -- start start 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 Processor -- start 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 0x56366f7c09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7c84f0 0x56366f7b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabac1b0020 con 0x56366f7ad5d0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fabac1af950 con 0x56366f7b8370 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fabac1acb90 con 0x56366f7c84f0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabb39fe740 10 client.0 _ll_register_callbacks cb 0x7faba4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7c0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59580/0 (socket says 172.21.15.78:59580) 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabab7fe640 1 -- 172.21.15.78:0/2029434830 learned_addr learned my addr 172.21.15.78:0/2029434830 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.725+0000 7fabab7fe640 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7c84f0 0x56366f7b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabab7fe640 1 -- 172.21.15.78:0/2029434830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7c84f0 msgr2=0x56366f7b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabab7fe640 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7c84f0 0x56366f7b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabab7fe640 1 -- 172.21.15.78:0/2029434830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 msgr2=0x56366f7c09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:21.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabab7fe640 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 0x56366f7c09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabab7fe640 1 -- 172.21.15.78:0/2029434830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366f7b0120 con 0x56366f7ad5d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabab7fe640 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7c0480 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7faba4002230 tx=0x7faba400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fc9e67941610db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fab8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fab8eff5640 1 -- 172.21.15.78:0/2029434830 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4033060 con 0x56366f7ad5d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fab8eff5640 1 -- 172.21.15.78:0/2029434830 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faba400b030 con 0x56366f7ad5d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabb39fe740 1 -- 172.21.15.78:0/2029434830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba0005520 con 0x56366f7ad5d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fab8eff5640 1 -- 172.21.15.78:0/2029434830 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faba403a050 con 0x56366f7ad5d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabb39fe740 10 client.18069 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fabb39fe740 1 -- 172.21.15.78:0/2029434830 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56366f8325f0 con 0x56366f7ad5d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.726+0000 7fab8eff5640 1 -- 172.21.15.78:0/2029434830 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba0005520 con 0x56366f7ad5d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fab8eff5640 1 -- 172.21.15.78:0/2029434830 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7faba4018ac0 con 0x56366f7ad5d0 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabb39fe740 10 client.18069 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabb39fe740 10 client.18069 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:28:21.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabb39fe740 20 client.18069 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabb39fe740 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x56366f832c80 0x56366f8350e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabb39fe740 4 client.18069 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabb39fe740 1 -- 172.21.15.78:0/2029434830 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56366f71ad70 con 0x56366f832c80 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabaaffd640 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x56366f832c80 0x56366f8350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fab8eff5640 10 client.18069 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabaaffd640 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x56366f832c80 0x56366f8350e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22858773cee9e898 server_cookie=9ff3ab36ed0f1283 in_seq=0 out_seq=0 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fab8eff5640 1 -- 172.21.15.78:0/2029434830 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56366f71ad70 con 0x56366f832c80 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fab8eff5640 10 client.18069 handle_command_reply: tid=0 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.727+0000 7fabb39fe740 1 client.18069 shutdown 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 -- 172.21.15.78:0/2029434830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 msgr2=0x56366f7c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7c0480 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7faba4002230 tx=0x7faba400b490 comp rx=0 tx=0).stop 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 -- 172.21.15.78:0/2029434830 shutdown_connections 2024-09-16T00:28:21.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56366f7c84f0 0x56366f7b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x56366f832c80 0x56366f8350e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56366f7b8370 0x56366f7c09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fab8eff5640 0 client.18069 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 --2- 172.21.15.78:0/2029434830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56366f7ad5d0 0x56366f7c0480 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 -- 172.21.15.78:0/2029434830 >> 172.21.15.78:0/2029434830 conn(0x56366f776310 msgr2=0x56366f7b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 -- 172.21.15.78:0/2029434830 shutdown_connections 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 1 -- 172.21.15.78:0/2029434830 wait complete. 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb39fe740 20 client.18069 trim_cache size 0 max 16384 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb1625640 1 -- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab940a3240 msgr2=0x7fab940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb1625640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab940a3240 0x7fab940a5700 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0bc720 tx=0x7fab9c02a480 comp rx=0 tx=0).stop 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb1625640 1 -- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 msgr2=0x7fabac198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb1625640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 0x7fabac198a80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fab9c006010 tx=0x7fab9c003c10 comp rx=0 tx=0).stop 2024-09-16T00:28:21.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.728+0000 7fabb1625640 1 -- 172.21.15.78:0/1773131619 shutdown_connections 2024-09-16T00:28:21.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.729+0000 7fabb1625640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fab940a3240 0x7fab940a5700 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.729+0000 7fabb1625640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fabac102ef0 0x7fabac19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.729+0000 7fabb1625640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabac102570 0x7fabac198a80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.729+0000 7fabb1625640 1 --2- 172.21.15.78:0/1773131619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabac100e20 0x7fabac198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:21.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.729+0000 7fabb1625640 1 -- 172.21.15.78:0/1773131619 >> 172.21.15.78:0/1773131619 conn(0x7fabac0fc7d0 msgr2=0x7fabac069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:21.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.729+0000 7fabb1625640 1 -- 172.21.15.78:0/1773131619 shutdown_connections 2024-09-16T00:28:21.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:20.729+0000 7fabb1625640 1 -- 172.21.15.78:0/1773131619 wait complete. 2024-09-16T00:28:22.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:21 smithi078 ceph-mon[31157]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-09-16T00:28:22.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:21 smithi193 ceph-mon[35484]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-09-16T00:28:22.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:21 smithi148 ceph-mon[35599]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 MiB/s wr, 543 op/s 2024-09-16T00:28:22.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:28:23.022 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.612+0000 7f0c6e36e640 1 -- 172.21.15.78:0/2303442767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c68107440 msgr2=0x7f0c68100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.612+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/2303442767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c68107440 0x7f0c68100100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009860 tx=0x7f0c5c02a360 comp rx=0 tx=0).stop 2024-09-16T00:28:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.612+0000 7f0c6e36e640 1 -- 172.21.15.78:0/2303442767 shutdown_connections 2024-09-16T00:28:23.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.612+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/2303442767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c68100860 0x7f0c681045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.612+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/2303442767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c68107440 0x7f0c68100100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.612+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/2303442767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68106a90 0x7f0c68106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.612+0000 7f0c6e36e640 1 -- 172.21.15.78:0/2303442767 >> 172.21.15.78:0/2303442767 conn(0x7f0c68077230 msgr2=0x7f0c680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.612+0000 7f0c6e36e640 1 -- 172.21.15.78:0/2303442767 shutdown_connections 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.613+0000 7f0c6e36e640 1 -- 172.21.15.78:0/2303442767 wait complete. 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.613+0000 7f0c6e36e640 1 Processor -- start 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.613+0000 7f0c6e36e640 1 -- start start 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c6e36e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c68100860 0x7f0c68073a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c6e36e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68106a90 0x7f0c6806cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c6e36e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c6806cff0 0x7f0c6806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c6e36e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681185e0 con 0x7f0c68100860 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c6e36e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68118460 con 0x7f0c68106a90 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c6e36e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68118760 con 0x7f0c6806cff0 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68106a90 0x7f0c6806cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68106a90 0x7f0c6806cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:46092/0 (socket says 172.21.15.78:46092) 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c677fe640 1 -- 172.21.15.78:0/3972926113 learned_addr learned my addr 172.21.15.78:0/3972926113 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c67fff640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c68100860 0x7f0c68073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c6806cff0 0x7f0c6806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c677fe640 1 -- 172.21.15.78:0/3972926113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c6806cff0 msgr2=0x7f0c6806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c677fe640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c6806cff0 0x7f0c6806d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c677fe640 1 -- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c68100860 msgr2=0x7f0c68073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c677fe640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c68100860 0x7f0c68073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c677fe640 1 -- 172.21.15.78:0/3972926113 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c68071550 con 0x7f0c68106a90 2024-09-16T00:28:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c67fff640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c68100860 0x7f0c68073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.614+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c6806cff0 0x7f0c6806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.615+0000 7f0c677fe640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68106a90 0x7f0c6806cab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c02a8d0 tx=0x7f0c5c0326c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.615+0000 7f0c657fa640 1 -- 172.21.15.78:0/3972926113 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c037040 con 0x7f0c68106a90 2024-09-16T00:28:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.615+0000 7f0c6e36e640 1 -- 172.21.15.78:0/3972926113 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c680717e0 con 0x7f0c68106a90 2024-09-16T00:28:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.615+0000 7f0c657fa640 1 -- 172.21.15.78:0/3972926113 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5c032cd0 con 0x7f0c68106a90 2024-09-16T00:28:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.615+0000 7f0c657fa640 1 -- 172.21.15.78:0/3972926113 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c0052b0 con 0x7f0c68106a90 2024-09-16T00:28:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.615+0000 7f0c6e36e640 1 -- 172.21.15.78:0/3972926113 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c68071cc0 con 0x7f0c68106a90 2024-09-16T00:28:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.617+0000 7f0c657fa640 1 -- 172.21.15.78:0/3972926113 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0c5c005450 con 0x7f0c68106a90 2024-09-16T00:28:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c70747740 1 Processor -- start 2024-09-16T00:28:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c70747740 1 -- start start 2024-09-16T00:28:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c70747740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904347b40 0x561904346090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c70747740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904346a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c70747740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x561904353760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c70747740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5619042ac2f0 con 0x5619043465d0 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c70747740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561904253cf0 con 0x561904351370 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c70747740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5619042a82b0 con 0x561904347b40 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904346a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904346a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59604/0 (socket says 172.21.15.78:59604) 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c67fff640 1 -- 172.21.15.78:0/1880217686 learned_addr learned my addr 172.21.15.78:0/1880217686 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904347b40 0x561904346090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.618+0000 7f0c677fe640 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x561904353760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c67fff640 1 -- 172.21.15.78:0/1880217686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904347b40 msgr2=0x561904346090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c67fff640 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904347b40 0x561904346090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c67fff640 1 -- 172.21.15.78:0/1880217686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 msgr2=0x561904353760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c67fff640 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x561904353760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c67fff640 1 -- 172.21.15.78:0/1880217686 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619042383a0 con 0x5619043465d0 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c677fe640 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x561904353760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c657fa640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0c500a3280 0x7f0c500a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c67fff640 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904346a10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0c580114f0 tx=0x7f0c580119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faf8f0bbef34218 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0c500a3280 0x7f0c500a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.619+0000 7f0c657fa640 1 -- 172.21.15.78:0/3972926113 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0c5c0ef6a0 con 0x7f0c68106a90 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.620+0000 7f0c4e7fc640 1 -- 172.21.15.78:0/1880217686 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58004020 con 0x5619043465d0 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.620+0000 7f0c4e7fc640 1 -- 172.21.15.78:0/1880217686 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c580041c0 con 0x5619043465d0 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.620+0000 7f0c4e7fc640 1 -- 172.21.15.78:0/1880217686 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800f9c0 con 0x5619043465d0 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 -- 172.21.15.78:0/1880217686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 msgr2=0x561904346a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904346a10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0c580114f0 tx=0x7f0c580119b0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0c500a3280 0x7f0c500a5740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009fd0 tx=0x7f0c60009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 -- 172.21.15.78:0/1880217686 shutdown_connections 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x561904353760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904346a10 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 --2- 172.21.15.78:0/1880217686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904347b40 0x561904346090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 -- 172.21.15.78:0/1880217686 >> 172.21.15.78:0/1880217686 conn(0x56190430f310 msgr2=0x561904342c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 -- 172.21.15.78:0/1880217686 shutdown_connections 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 -- 172.21.15.78:0/1880217686 wait complete. 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.621+0000 7f0c70747740 1 -- start start 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c70747740 1 Processor -- start 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c70747740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619043465d0 0x561904350d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c70747740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190434e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c70747740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x56190434e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619043465d0 0x561904350d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619043465d0 0x561904350d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36110/0 (socket says 172.21.15.78:36110) 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c67fff640 1 -- 172.21.15.78:0/295477782 learned_addr learned my addr 172.21.15.78:0/295477782 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x56190434e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x561904354070 con 0x561904347b40 2024-09-16T00:28:23.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.622+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561904267060 con 0x561904351370 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5619043541f0 con 0x5619043465d0 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c67fff640 1 -- 172.21.15.78:0/295477782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 msgr2=0x56190434e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c677fe640 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190434e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c67fff640 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x56190434e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c67fff640 1 -- 172.21.15.78:0/295477782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 msgr2=0x56190434e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c67fff640 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190434e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c67fff640 1 -- 172.21.15.78:0/295477782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190401d3c0 con 0x5619043465d0 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c677fe640 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190434e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c70747740 10 client.0 _ll_register_callbacks cb 0x561903fba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c580041c0 con 0x5619043465d0 2024-09-16T00:28:23.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c67fff640 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619043465d0 0x561904350d60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009230 tx=0x7f0c58003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5538a4859e913d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c477fe640 10 client.27052 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c477fe640 1 -- 172.21.15.78:0/295477782 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58028020 con 0x5619043465d0 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.623+0000 7f0c477fe640 1 -- 172.21.15.78:0/295477782 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0c58004370 con 0x5619043465d0 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c477fe640 1 -- 172.21.15.78:0/295477782 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c58005040 con 0x5619043465d0 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c70747740 10 client.27052 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=34}) -- 0x56190435f840 con 0x5619043465d0 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c477fe640 1 -- 172.21.15.78:0/295477782 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0c580051e0 con 0x5619043465d0 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c70747740 10 client.27052 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:23.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c70747740 10 client.27052 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c70747740 20 client.27052 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c70747740 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x561904360bf0 0x561904362fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c70747740 4 client.27052 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619042b9860 con 0x561904360bf0 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c677fe640 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x561904360bf0 0x561904362fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c477fe640 10 client.27052 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.624+0000 7f0c677fe640 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x561904360bf0 0x561904362fe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7b7115868ceaa6e server_cookie=b9c0b16acc7fd5a8 in_seq=0 out_seq=0 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.625+0000 7f0c477fe640 1 -- 172.21.15.78:0/295477782 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619042b9860 con 0x561904360bf0 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.625+0000 7f0c477fe640 10 client.27052 handle_command_reply: tid=0 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.625+0000 7f0c70747740 1 client.27052 shutdown 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.625+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619043465d0 msgr2=0x561904350d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.625+0000 7f0c70747740 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619043465d0 0x561904350d60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009230 tx=0x7f0c58003c90 comp rx=0 tx=0).stop 2024-09-16T00:28:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 shutdown_connections 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x561904360bf0 0x561904362fe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904351370 0x56190434e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190434e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 1 --2- 172.21.15.78:0/295477782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619043465d0 0x561904350d60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c477fe640 0 client.27052 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 >> 172.21.15.78:0/295477782 conn(0x56190430f310 msgr2=0x561904348960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 shutdown_connections 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 1 -- 172.21.15.78:0/295477782 wait complete. 2024-09-16T00:28:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.626+0000 7f0c70747740 20 client.27052 trim_cache size 0 max 16384 2024-09-16T00:28:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c70747740 1 Processor -- start 2024-09-16T00:28:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c70747740 1 -- start start 2024-09-16T00:28:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c70747740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619043465d0 0x561904342e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c70747740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190435d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c70747740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x561904368830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c70747740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681b5230 con 0x561904347b40 2024-09-16T00:28:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c70747740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681b6d70 con 0x5619043465d0 2024-09-16T00:28:23.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c70747740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681b5900 con 0x561904351370 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190435d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190435d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59618/0 (socket says 172.21.15.78:59618) 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c677fe640 1 -- 172.21.15.78:0/655289189 learned_addr learned my addr 172.21.15.78:0/655289189 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.642+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/655289189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x561904368830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.643+0000 7f0c677fe640 1 -- 172.21.15.78:0/655289189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 msgr2=0x561904368830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.643+0000 7f0c677fe640 1 --2- 172.21.15.78:0/655289189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x561904368830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.643+0000 7f0c677fe640 1 -- 172.21.15.78:0/655289189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619043465d0 msgr2=0x561904342e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.643+0000 7f0c677fe640 1 --2- 172.21.15.78:0/655289189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619043465d0 0x561904342e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.643+0000 7f0c677fe640 1 -- 172.21.15.78:0/655289189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561904343960 con 0x561904347b40 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.643+0000 7f0c677fe640 1 --2- 172.21.15.78:0/655289189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190435d310 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0b7d60 tx=0x7f0c5c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90776108592bc5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.643+0000 7f0c4dffb640 1 -- 172.21.15.78:0/655289189 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c0bfcb0 con 0x561904347b40 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.643+0000 7f0c4dffb640 1 -- 172.21.15.78:0/655289189 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5c0c4570 con 0x561904347b40 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c4dffb640 1 -- 172.21.15.78:0/655289189 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c0ef380 con 0x561904347b40 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 -- 172.21.15.78:0/655289189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 msgr2=0x56190435d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 --2- 172.21.15.78:0/655289189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190435d310 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0b7d60 tx=0x7f0c5c005f90 comp rx=0 tx=0).stop 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 -- 172.21.15.78:0/655289189 shutdown_connections 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 --2- 172.21.15.78:0/655289189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x561904368830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 --2- 172.21.15.78:0/655289189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x561904347b40 0x56190435d310 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 --2- 172.21.15.78:0/655289189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5619043465d0 0x561904342e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 -- 172.21.15.78:0/655289189 >> 172.21.15.78:0/655289189 conn(0x56190430f310 msgr2=0x561904342700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 -- 172.21.15.78:0/655289189 shutdown_connections 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.644+0000 7f0c70747740 1 -- 172.21.15.78:0/655289189 wait complete. 2024-09-16T00:28:23.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 1 -- start start 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 1 Processor -- start 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904359d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904347b40 0x56190434af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x56190434b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681b66a0 con 0x5619043465d0 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681b5fd0 con 0x561904347b40 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68101110 con 0x561904351370 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.645+0000 7f0c70747740 10 client.0 _ll_register_callbacks cb 0x7f0c60009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904359d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c6c90b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x56190434b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c67fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904359d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59628/0 (socket says 172.21.15.78:59628) 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904347b40 0x56190434af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c67fff640 1 -- 172.21.15.78:0/63698024 learned_addr learned my addr 172.21.15.78:0/63698024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c6c90b640 1 -- 172.21.15.78:0/63698024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904347b40 msgr2=0x56190434af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904347b40 0x56190434af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c6c90b640 1 -- 172.21.15.78:0/63698024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 msgr2=0x561904359d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904359d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c6c90b640 1 -- 172.21.15.78:0/63698024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619043ca800 con 0x561904351370 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c677fe640 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904347b40 0x56190434af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c67fff640 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904359d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c6c90b640 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x56190434b690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c600237d0 tx=0x7f0c60004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbd8232a867c8d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c477fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.646+0000 7f0c477fe640 1 -- 172.21.15.78:0/63698024 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60004b20 con 0x561904351370 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 1 -- 172.21.15.78:0/63698024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c5c0c4570 con 0x561904351370 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c477fe640 1 -- 172.21.15.78:0/63698024 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0c60029030 con 0x561904351370 2024-09-16T00:28:23.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c477fe640 1 -- 172.21.15.78:0/63698024 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c60048050 con 0x561904351370 2024-09-16T00:28:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 10 client.27058 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:23.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 1 -- 172.21.15.78:0/63698024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5619043cba10 con 0x561904351370 2024-09-16T00:28:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c477fe640 1 -- 172.21.15.78:0/63698024 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5c0c4570 con 0x561904351370 2024-09-16T00:28:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c477fe640 1 -- 172.21.15.78:0/63698024 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f0c6003b450 con 0x561904351370 2024-09-16T00:28:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 10 client.27058 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 10 client.27058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 20 client.27058 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5619043cc1f0 0x5619043ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 4 client.27058 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.647+0000 7f0c70747740 1 -- 172.21.15.78:0/63698024 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619042b3d70 con 0x5619043cc1f0 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c67fff640 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5619043cc1f0 0x5619043ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c477fe640 10 client.27058 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c67fff640 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5619043cc1f0 0x5619043ce610 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d031e124b4a42818 server_cookie=ae410c79a832878d in_seq=0 out_seq=0 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c477fe640 1 -- 172.21.15.78:0/63698024 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619042b3d70 con 0x5619043cc1f0 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c477fe640 10 client.27058 handle_command_reply: tid=0 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 client.27058 shutdown 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 -- 172.21.15.78:0/63698024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 msgr2=0x56190434b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x56190434b690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0c600237d0 tx=0x7f0c60004a40 comp rx=0 tx=0).stop 2024-09-16T00:28:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 -- 172.21.15.78:0/63698024 shutdown_connections 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561904351370 0x56190434b690 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5619043cc1f0 0x5619043ce610 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561904347b40 0x56190434af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 --2- 172.21.15.78:0/63698024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5619043465d0 0x561904359d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c70747740 1 -- 172.21.15.78:0/63698024 >> 172.21.15.78:0/63698024 conn(0x56190430f310 msgr2=0x5619043414e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.648+0000 7f0c477fe640 0 client.27058 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c70747740 1 -- 172.21.15.78:0/63698024 shutdown_connections 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c70747740 1 -- 172.21.15.78:0/63698024 wait complete. 2024-09-16T00:28:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c70747740 20 client.27058 trim_cache size 0 max 16384 2024-09-16T00:28:23.657 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:28:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 -- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0c500a3280 msgr2=0x7f0c500a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0c500a3280 0x7f0c500a5740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009fd0 tx=0x7f0c60009500 comp rx=0 tx=0).stop 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 -- 172.21.15.78:0/3972926113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68106a90 msgr2=0x7f0c6806cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68106a90 0x7f0c6806cab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c02a8d0 tx=0x7f0c5c0326c0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 -- 172.21.15.78:0/3972926113 shutdown_connections 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0c500a3280 0x7f0c500a5740 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c6806cff0 0x7f0c6806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c68106a90 0x7f0c6806cab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 --2- 172.21.15.78:0/3972926113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c68100860 0x7f0c68073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.649+0000 7f0c6e36e640 1 -- 172.21.15.78:0/3972926113 >> 172.21.15.78:0/3972926113 conn(0x7f0c68077230 msgr2=0x7f0c68102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.650+0000 7f0c6e36e640 1 -- 172.21.15.78:0/3972926113 shutdown_connections 2024-09-16T00:28:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:23.650+0000 7f0c6e36e640 1 -- 172.21.15.78:0/3972926113 wait complete. 2024-09-16T00:28:24.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:23 smithi193 ceph-mon[35484]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-09-16T00:28:24.199 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:23 smithi078 ceph-mon[31157]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-09-16T00:28:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:23 smithi148 ceph-mon[35599]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-09-16T00:28:24.503 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:28:24.894 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.526+0000 7fefc956b640 1 -- 172.21.15.78:0/2382340983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 msgr2=0x7fefc4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.526+0000 7fefc956b640 1 --2- 172.21.15.78:0/2382340983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 0x7fefc4108830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009900 tx=0x7fefb802a440 comp rx=0 tx=0).stop 2024-09-16T00:28:25.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.532+0000 7fefc956b640 1 -- 172.21.15.78:0/2382340983 shutdown_connections 2024-09-16T00:28:25.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.533+0000 7fefc956b640 1 --2- 172.21.15.78:0/2382340983 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc410d530 0x7fefc41150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.533+0000 7fefc956b640 1 --2- 172.21.15.78:0/2382340983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc4108e00 0x7fefc410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.533+0000 7fefc956b640 1 --2- 172.21.15.78:0/2382340983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 0x7fefc4108830 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.533+0000 7fefc956b640 1 -- 172.21.15.78:0/2382340983 >> 172.21.15.78:0/2382340983 conn(0x7fefc4077ba0 msgr2=0x7fefc4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.534+0000 7fefc956b640 1 -- 172.21.15.78:0/2382340983 shutdown_connections 2024-09-16T00:28:25.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.534+0000 7fefc956b640 1 -- 172.21.15.78:0/2382340983 wait complete. 2024-09-16T00:28:25.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.535+0000 7fefc956b640 1 Processor -- start 2024-09-16T00:28:25.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.536+0000 7fefc956b640 1 -- start start 2024-09-16T00:28:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc956b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 0x7fefc419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc956b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4108e00 0x7fefc419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc956b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc410d530 0x7fefc41a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 0x7fefc419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 0x7fefc419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36162/0 (socket says 172.21.15.78:36162) 2024-09-16T00:28:25.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc2ffd640 1 -- 172.21.15.78:0/3659456756 learned_addr learned my addr 172.21.15.78:0/3659456756 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc27fc640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4108e00 0x7fefc419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fefc4118620 con 0x7fefc4108e00 2024-09-16T00:28:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41184a0 con 0x7fefc410d530 2024-09-16T00:28:25.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.543+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41187a0 con 0x7fefc4108450 2024-09-16T00:28:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.547+0000 7fefc37fe640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc410d530 0x7fefc41a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.547+0000 7fefc2ffd640 1 -- 172.21.15.78:0/3659456756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc410d530 msgr2=0x7fefc41a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.547+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc410d530 0x7fefc41a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.547+0000 7fefc2ffd640 1 -- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4108e00 msgr2=0x7fefc419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.547+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4108e00 0x7fefc419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.547+0000 7fefc2ffd640 1 -- 172.21.15.78:0/3659456756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc41a3b70 con 0x7fefc4108450 2024-09-16T00:28:25.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.547+0000 7fefc27fc640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4108e00 0x7fefc419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.554+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 0x7fefc419eb40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefb8004770 tx=0x7fefb8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.556+0000 7fefabfff640 1 -- 172.21.15.78:0/3659456756 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb803f070 con 0x7fefc4108450 2024-09-16T00:28:25.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.556+0000 7fefabfff640 1 -- 172.21.15.78:0/3659456756 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb8031c30 con 0x7fefc4108450 2024-09-16T00:28:25.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.556+0000 7fefabfff640 1 -- 172.21.15.78:0/3659456756 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8035030 con 0x7fefc4108450 2024-09-16T00:28:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.559+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc41a3e00 con 0x7fefc4108450 2024-09-16T00:28:25.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.559+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41a4260 con 0x7fefc4108450 2024-09-16T00:28:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.562+0000 7fefcb944740 1 Processor -- start 2024-09-16T00:28:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.563+0000 7fefcb944740 1 -- start start 2024-09-16T00:28:25.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.567+0000 7fefcb944740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a62874090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.567+0000 7fefcb944740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 0x560a62874a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.567+0000 7fefcb944740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a6287f370 0x560a62881760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.567+0000 7fefcb944740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560a627da2f0 con 0x560a62875b40 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.567+0000 7fefcb944740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560a62781cf0 con 0x560a628745d0 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.567+0000 7fefcb944740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560a627d62b0 con 0x560a6287f370 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.567+0000 7fefc27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a6287f370 0x560a62881760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.568+0000 7fefc27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a6287f370 0x560a62881760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36166/0 (socket says 172.21.15.78:36166) 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.568+0000 7fefc27fc640 1 -- 172.21.15.78:0/318371835 learned_addr learned my addr 172.21.15.78:0/318371835 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.568+0000 7fefc27fc640 1 -- 172.21.15.78:0/318371835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 msgr2=0x560a62874a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.568+0000 7fefc27fc640 1 --2- 172.21.15.78:0/318371835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 0x560a62874a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.568+0000 7fefc27fc640 1 -- 172.21.15.78:0/318371835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 msgr2=0x560a62874090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.568+0000 7fefc27fc640 1 --2- 172.21.15.78:0/318371835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a62874090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.568+0000 7fefc27fc640 1 -- 172.21.15.78:0/318371835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a627663a0 con 0x560a6287f370 2024-09-16T00:28:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.569+0000 7fefc27fc640 1 --2- 172.21.15.78:0/318371835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a6287f370 0x560a62881760 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009f90 tx=0x7fefbc00c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565cfc728c12bb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:25.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.569+0000 7fefa97fa640 1 -- 172.21.15.78:0/318371835 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc01a070 con 0x560a6287f370 2024-09-16T00:28:25.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.569+0000 7fefa97fa640 1 -- 172.21.15.78:0/318371835 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefbc00aad0 con 0x560a6287f370 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.569+0000 7fefa97fa640 1 -- 172.21.15.78:0/318371835 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc015c70 con 0x560a6287f370 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.569+0000 7fefabfff640 1 -- 172.21.15.78:0/3659456756 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fefb8042070 con 0x7fefc4108450 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.570+0000 7fefcb944740 1 -- 172.21.15.78:0/318371835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a6287f370 msgr2=0x560a62881760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.570+0000 7fefcb944740 1 --2- 172.21.15.78:0/318371835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a6287f370 0x560a62881760 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009f90 tx=0x7fefbc00c890 comp rx=0 tx=0).stop 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.570+0000 7fefabfff640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefac0a3410 0x7fefac0a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.571+0000 7fefcb944740 1 -- 172.21.15.78:0/318371835 shutdown_connections 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.571+0000 7fefcb944740 1 --2- 172.21.15.78:0/318371835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a6287f370 0x560a62881760 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.571+0000 7fefcb944740 1 --2- 172.21.15.78:0/318371835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 0x560a62874a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.571+0000 7fefcb944740 1 --2- 172.21.15.78:0/318371835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a62874090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.571+0000 7fefcb944740 1 -- 172.21.15.78:0/318371835 >> 172.21.15.78:0/318371835 conn(0x560a6283d310 msgr2=0x560a62870c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:25.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.571+0000 7fefabfff640 1 -- 172.21.15.78:0/3659456756 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fefb80f0710 con 0x7fefc4108450 2024-09-16T00:28:25.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.571+0000 7fefc37fe640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefac0a3410 0x7fefac0a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.572+0000 7fefc37fe640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefac0a3410 0x7fefac0a58d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009a80 tx=0x7fefb4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.574+0000 7fefcb944740 1 -- 172.21.15.78:0/318371835 shutdown_connections 2024-09-16T00:28:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.574+0000 7fefcb944740 1 -- 172.21.15.78:0/318371835 wait complete. 2024-09-16T00:28:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.574+0000 7fefcb944740 1 -- start start 2024-09-16T00:28:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.574+0000 7fefcb944740 1 Processor -- start 2024-09-16T00:28:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.575+0000 7fefcb944740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a628745d0 0x560a6287ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.575+0000 7fefcb944740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a6287c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.575+0000 7fefcb944740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 0x560a6287c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.575+0000 7fefcb944740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560a62882070 con 0x560a62875b40 2024-09-16T00:28:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.575+0000 7fefcb944740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560a62795060 con 0x560a6287f370 2024-09-16T00:28:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.575+0000 7fefcb944740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560a628821f0 con 0x560a628745d0 2024-09-16T00:28:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.575+0000 7fefcb944740 10 client.0 _ll_register_callbacks cb 0x560a624e88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.576+0000 7fefc27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a6287c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.576+0000 7fefc27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a6287c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59658/0 (socket says 172.21.15.78:59658) 2024-09-16T00:28:25.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.576+0000 7fefc27fc640 1 -- 172.21.15.78:0/3117691661 learned_addr learned my addr 172.21.15.78:0/3117691661 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.576+0000 7fefc27fc640 1 -- 172.21.15.78:0/3117691661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a628745d0 msgr2=0x560a6287ee10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:28:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.576+0000 7fefc27fc640 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a628745d0 0x560a6287ee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.576+0000 7fefc27fc640 1 -- 172.21.15.78:0/3117691661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 msgr2=0x560a6287c850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:28:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.576+0000 7fefc27fc640 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 0x560a6287c850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.576+0000 7fefc27fc640 1 -- 172.21.15.78:0/3117691661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6254b3c0 con 0x560a62875b40 2024-09-16T00:28:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.577+0000 7fefc27fc640 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a6287c0c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fefbc015890 tx=0x7fefbc009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c55c64e1545dce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.577+0000 7fefa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:25.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.577+0000 7fefa6ff5640 1 -- 172.21.15.78:0/3117691661 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc023020 con 0x560a62875b40 2024-09-16T00:28:25.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.578+0000 7fefcb944740 1 -- 172.21.15.78:0/3117691661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefbc00aad0 con 0x560a62875b40 2024-09-16T00:28:25.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.578+0000 7fefa6ff5640 1 -- 172.21.15.78:0/3117691661 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fefbc015c00 con 0x560a62875b40 2024-09-16T00:28:25.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.578+0000 7fefa6ff5640 1 -- 172.21.15.78:0/3117691661 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fefbc01a050 con 0x560a62875b40 2024-09-16T00:28:25.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.578+0000 7fefcb944740 10 client.18093 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:25.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.578+0000 7fefcb944740 1 -- 172.21.15.78:0/3117691661 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560a6289ebe0 con 0x560a62875b40 2024-09-16T00:28:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.579+0000 7fefa6ff5640 1 -- 172.21.15.78:0/3117691661 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefbc00aad0 con 0x560a62875b40 2024-09-16T00:28:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.579+0000 7fefa6ff5640 1 -- 172.21.15.78:0/3117691661 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fefbc0273a0 con 0x560a62875b40 2024-09-16T00:28:25.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.579+0000 7fefcb944740 10 client.18093 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:25.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.579+0000 7fefcb944740 10 client.18093 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:28:25.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.579+0000 7fefcb944740 20 client.18093 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:25.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.579+0000 7fefcb944740 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560a6289ff30 0x560a628a2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.579+0000 7fefcb944740 4 client.18093 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:25.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.579+0000 7fefcb944740 1 -- 172.21.15.78:0/3117691661 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a627e7860 con 0x560a6289ff30 2024-09-16T00:28:25.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.581+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560a6289ff30 0x560a628a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.581+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560a6289ff30 0x560a628a2320 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2cd7ad7c541f8cc9 server_cookie=d1eabbbb4bee9e83 in_seq=0 out_seq=0 2024-09-16T00:28:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.581+0000 7fefa6ff5640 10 client.18093 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:28:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefa6ff5640 1 -- 172.21.15.78:0/3117691661 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a627e7860 con 0x560a6289ff30 2024-09-16T00:28:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefa6ff5640 10 client.18093 handle_command_reply: tid=0 2024-09-16T00:28:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 client.18093 shutdown 2024-09-16T00:28:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 -- 172.21.15.78:0/3117691661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 msgr2=0x560a6287c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a6287c0c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fefbc015890 tx=0x7fefbc009d80 comp rx=0 tx=0).stop 2024-09-16T00:28:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 -- 172.21.15.78:0/3117691661 shutdown_connections 2024-09-16T00:28:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560a6289ff30 0x560a628a2320 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 0x560a6287c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a62875b40 0x560a6287c0c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 --2- 172.21.15.78:0/3117691661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a628745d0 0x560a6287ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefcb944740 1 -- 172.21.15.78:0/3117691661 >> 172.21.15.78:0/3117691661 conn(0x560a6283d310 msgr2=0x560a62876930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.582+0000 7fefa6ff5640 0 client.18093 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:28:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.583+0000 7fefcb944740 1 -- 172.21.15.78:0/3117691661 shutdown_connections 2024-09-16T00:28:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.583+0000 7fefcb944740 1 -- 172.21.15.78:0/3117691661 wait complete. 2024-09-16T00:28:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.583+0000 7fefcb944740 20 client.18093 trim_cache size 0 max 16384 2024-09-16T00:28:25.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefcb944740 1 Processor -- start 2024-09-16T00:28:25.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefcb944740 1 -- start start 2024-09-16T00:28:25.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefcb944740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a628745d0 0x560a62877120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefcb944740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62877660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefcb944740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 0x560a628a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefcb944740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ac980 con 0x560a628745d0 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefcb944740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ae4c0 con 0x560a6287f370 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefcb944740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ad050 con 0x560a62875b40 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefc2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a628745d0 0x560a62877120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefc2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a628745d0 0x560a62877120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59662/0 (socket says 172.21.15.78:59662) 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefc2ffd640 1 -- 172.21.15.78:0/3228661496 learned_addr learned my addr 172.21.15.78:0/3228661496 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefc37fe640 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 0x560a628a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefc27fc640 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62877660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.597+0000 7fefc2ffd640 1 -- 172.21.15.78:0/3228661496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 msgr2=0x560a62877660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62877660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefc2ffd640 1 -- 172.21.15.78:0/3228661496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 msgr2=0x560a628a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:25.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 0x560a628a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefc2ffd640 1 -- 172.21.15.78:0/3228661496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a62874c20 con 0x560a628745d0 2024-09-16T00:28:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefc37fe640 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 0x560a628a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a628745d0 0x560a62877120 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fefb80b8dd0 tx=0x7fefb8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcc31ef66955214 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefa8ff9640 1 -- 172.21.15.78:0/3228661496 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb80da070 con 0x560a628745d0 2024-09-16T00:28:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefa8ff9640 1 -- 172.21.15.78:0/3228661496 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb80c55e0 con 0x560a628745d0 2024-09-16T00:28:25.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.598+0000 7fefa8ff9640 1 -- 172.21.15.78:0/3228661496 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb80d5030 con 0x560a628745d0 2024-09-16T00:28:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 -- 172.21.15.78:0/3228661496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a628745d0 msgr2=0x560a62877120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a628745d0 0x560a62877120 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fefb80b8dd0 tx=0x7fefb8000bc0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 -- 172.21.15.78:0/3228661496 shutdown_connections 2024-09-16T00:28:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a6287f370 0x560a628a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62877660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 --2- 172.21.15.78:0/3228661496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a628745d0 0x560a62877120 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 -- 172.21.15.78:0/3228661496 >> 172.21.15.78:0/3228661496 conn(0x560a6283d310 msgr2=0x560a6287b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 -- 172.21.15.78:0/3228661496 shutdown_connections 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 -- 172.21.15.78:0/3228661496 wait complete. 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 -- start start 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.599+0000 7fefcb944740 1 Processor -- start 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefcb944740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 0x560a62887b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefcb944740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62882ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefcb944740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6287f370 0x560a62883400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefcb944740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41addf0 con 0x560a6287f370 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefcb944740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fefc41ad720 con 0x560a628745d0 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefcb944740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc40823a0 con 0x560a62875b40 2024-09-16T00:28:25.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefcb944740 10 client.0 _ll_register_callbacks cb 0x7fefbc018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62882ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc2ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 0x560a62887b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62882ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:36206/0 (socket says 172.21.15.78:36206) 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc27fc640 1 -- 172.21.15.78:0/4029783383 learned_addr learned my addr 172.21.15.78:0/4029783383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc37fe640 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6287f370 0x560a62883400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc2ffd640 1 -- 172.21.15.78:0/4029783383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 msgr2=0x560a62882ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62882ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc2ffd640 1 -- 172.21.15.78:0/4029783383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6287f370 msgr2=0x560a62883400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6287f370 0x560a62883400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc2ffd640 1 -- 172.21.15.78:0/4029783383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a62883c30 con 0x560a628745d0 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.600+0000 7fefc2ffd640 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 0x560a62887b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fefb80bc670 tx=0x7fefb80c0e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=405065b73b4b5477 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.601+0000 7fefa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:28:25.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.601+0000 7fefa6ff5640 1 -- 172.21.15.78:0/4029783383 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb80ca060 con 0x560a628745d0 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.601+0000 7fefa6ff5640 1 -- 172.21.15.78:0/4029783383 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fefb80db290 con 0x560a628745d0 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.601+0000 7fefcb944740 1 -- 172.21.15.78:0/4029783383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefb80c55e0 con 0x560a628745d0 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.601+0000 7fefa6ff5640 1 -- 172.21.15.78:0/4029783383 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fefb80d5070 con 0x560a628745d0 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.601+0000 7fefcb944740 10 client.27248 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.601+0000 7fefcb944740 1 -- 172.21.15.78:0/4029783383 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560a6290a8b0 con 0x560a628745d0 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.602+0000 7fefa6ff5640 1 -- 172.21.15.78:0/4029783383 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb80c55e0 con 0x560a628745d0 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.602+0000 7fefa6ff5640 1 -- 172.21.15.78:0/4029783383 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fefb804c060 con 0x560a628745d0 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.602+0000 7fefcb944740 10 client.27248 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.602+0000 7fefcb944740 10 client.27248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.602+0000 7fefcb944740 20 client.27248 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.603+0000 7fefcb944740 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560a6290ada0 0x560a6290d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.603+0000 7fefcb944740 4 client.27248 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:28:25.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.603+0000 7fefcb944740 1 -- 172.21.15.78:0/4029783383 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a627e1d70 con 0x560a6290ada0 2024-09-16T00:28:25.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.603+0000 7fefc27fc640 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560a6290ada0 0x560a6290d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:25.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.604+0000 7fefc27fc640 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560a6290ada0 0x560a6290d190 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe1b1251511e1 server_cookie=ab574208ae062ce6 in_seq=0 out_seq=0 2024-09-16T00:28:25.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.604+0000 7fefa6ff5640 10 client.27248 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:28:25.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.612+0000 7fefa6ff5640 1 -- 172.21.15.78:0/4029783383 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a627e1d70 con 0x560a6290ada0 2024-09-16T00:28:25.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.612+0000 7fefa6ff5640 10 client.27248 handle_command_reply: tid=0 2024-09-16T00:28:25.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.612+0000 7fefcb944740 1 client.27248 shutdown 2024-09-16T00:28:25.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.616+0000 7fefcb944740 1 -- 172.21.15.78:0/4029783383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 msgr2=0x560a62887b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.616+0000 7fefcb944740 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 0x560a62887b90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fefb80bc670 tx=0x7fefb80c0e50 comp rx=0 tx=0).stop 2024-09-16T00:28:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.624+0000 7fefcb944740 1 -- 172.21.15.78:0/4029783383 shutdown_connections 2024-09-16T00:28:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.624+0000 7fefcb944740 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560a6290ada0 0x560a6290d190 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.624+0000 7fefcb944740 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560a6287f370 0x560a62883400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.624+0000 7fefcb944740 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a62875b40 0x560a62882ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.624+0000 7fefcb944740 1 --2- 172.21.15.78:0/4029783383 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a628745d0 0x560a62887b90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.624+0000 7fefa6ff5640 0 client.27248 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:28:25.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.624+0000 7fefcb944740 1 -- 172.21.15.78:0/4029783383 >> 172.21.15.78:0/4029783383 conn(0x560a6283d310 msgr2=0x560a6286f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:25.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.628+0000 7fefcb944740 1 -- 172.21.15.78:0/4029783383 shutdown_connections 2024-09-16T00:28:25.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.631+0000 7fefcb944740 1 -- 172.21.15.78:0/4029783383 wait complete. 2024-09-16T00:28:25.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.631+0000 7fefcb944740 20 client.27248 trim_cache size 0 max 16384 2024-09-16T00:28:25.631 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:28:25.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefac0a3410 msgr2=0x7fefac0a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefac0a3410 0x7fefac0a58d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009a80 tx=0x7fefb4005e50 comp rx=0 tx=0).stop 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 msgr2=0x7fefc419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 0x7fefc419eb40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefb8004770 tx=0x7fefb8003c10 comp rx=0 tx=0).stop 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 shutdown_connections 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fefac0a3410 0x7fefac0a58d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fefc410d530 0x7fefc41a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fefc4108e00 0x7fefc419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 --2- 172.21.15.78:0/3659456756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc4108450 0x7fefc419eb40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 >> 172.21.15.78:0/3659456756 conn(0x7fefc4077ba0 msgr2=0x7fefc4078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 shutdown_connections 2024-09-16T00:28:25.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:25.633+0000 7fefc956b640 1 -- 172.21.15.78:0/3659456756 wait complete. 2024-09-16T00:28:25.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 865.0 s, 496.8 tps, lat 201.782 ms stddev 243.527 2024-09-16T00:28:26.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:25 smithi193 ceph-mon[35484]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.4 MiB/s wr, 482 op/s 2024-09-16T00:28:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:25 smithi148 ceph-mon[35599]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.4 MiB/s wr, 482 op/s 2024-09-16T00:28:26.205 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:25 smithi078 ceph-mon[31157]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.4 MiB/s wr, 482 op/s 2024-09-16T00:28:27.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:26 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:27.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:26 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:27.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:26 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:27 smithi148 ceph-mon[35599]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.8 MiB/s wr, 511 op/s 2024-09-16T00:28:28.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:27 smithi078 ceph-mon[31157]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.8 MiB/s wr, 511 op/s 2024-09-16T00:28:28.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:27 smithi193 ceph-mon[35484]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.8 MiB/s wr, 511 op/s 2024-09-16T00:28:28.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:28:29.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:29.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:29.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:28:29.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:29.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:29.271 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:28:29.311 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:29.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:29.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:29.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:28:30.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:29 smithi148 ceph-mon[35599]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-09-16T00:28:30.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:29 smithi078 ceph-mon[31157]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-09-16T00:28:30.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:29 smithi193 ceph-mon[35484]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2024-09-16T00:28:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.836+0000 7f3790546640 1 -- 172.21.15.78:0/3589361594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f378811b140 msgr2=0x7f378811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.836+0000 7f3790546640 1 --2- 172.21.15.78:0/3589361594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f378811b140 0x7f378811d530 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3780009ca0 tx=0x7f378002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.838+0000 7f3790546640 1 -- 172.21.15.78:0/3589361594 shutdown_connections 2024-09-16T00:28:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.838+0000 7f3790546640 1 --2- 172.21.15.78:0/3589361594 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f378811b140 0x7f378811d530 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.838+0000 7f3790546640 1 --2- 172.21.15.78:0/3589361594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3788109830 0x7f3788109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.838+0000 7f3790546640 1 --2- 172.21.15.78:0/3589361594 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3788109450 0x7f3788110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.838+0000 7f3790546640 1 -- 172.21.15.78:0/3589361594 >> 172.21.15.78:0/3589361594 conn(0x7f378806e140 msgr2=0x7f378806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.843+0000 7f3790546640 1 -- 172.21.15.78:0/3589361594 shutdown_connections 2024-09-16T00:28:30.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.844+0000 7f3790546640 1 -- 172.21.15.78:0/3589361594 wait complete. 2024-09-16T00:28:30.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.845+0000 7f3790546640 1 Processor -- start 2024-09-16T00:28:30.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.846+0000 7f3790546640 1 -- start start 2024-09-16T00:28:30.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.846+0000 7f3790546640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3788109450 0x7f378811a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.846+0000 7f3790546640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3788109830 0x7f3788114770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.846+0000 7f3790546640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f378811b140 0x7f3788114cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.846+0000 7f3790546640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3788111330 con 0x7f378811b140 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.846+0000 7f3790546640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37881111b0 con 0x7f3788109830 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.846+0000 7f3790546640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37881114b0 con 0x7f3788109450 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.847+0000 7f378eae3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f378811b140 0x7f3788114cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.847+0000 7f378eae3640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f378811b140 0x7f3788114cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38404/0 (socket says 172.21.15.78:38404) 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.847+0000 7f378eae3640 1 -- 172.21.15.78:0/3096740306 learned_addr learned my addr 172.21.15.78:0/3096740306 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.849+0000 7f378dae1640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3788109830 0x7f3788114770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.849+0000 7f378e2e2640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3788109450 0x7f378811a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.852+0000 7f378eae3640 1 -- 172.21.15.78:0/3096740306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3788109450 msgr2=0x7f378811a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.852+0000 7f378eae3640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3788109450 0x7f378811a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.852+0000 7f378eae3640 1 -- 172.21.15.78:0/3096740306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3788109830 msgr2=0x7f3788114770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.852+0000 7f378eae3640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3788109830 0x7f3788114770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.852+0000 7f378eae3640 1 -- 172.21.15.78:0/3096740306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3788115270 con 0x7f378811b140 2024-09-16T00:28:30.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.853+0000 7f378eae3640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f378811b140 0x7f3788114cb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3780030ca0 tx=0x7f3780030cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.853+0000 7f377f7fe640 1 -- 172.21.15.78:0/3096740306 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780003970 con 0x7f378811b140 2024-09-16T00:28:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.853+0000 7f3790546640 1 -- 172.21.15.78:0/3096740306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3788115500 con 0x7f378811b140 2024-09-16T00:28:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.854+0000 7f3790546640 1 -- 172.21.15.78:0/3096740306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881bc3c0 con 0x7f378811b140 2024-09-16T00:28:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.854+0000 7f377f7fe640 1 -- 172.21.15.78:0/3096740306 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3780031450 con 0x7f378811b140 2024-09-16T00:28:30.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.855+0000 7f377f7fe640 1 -- 172.21.15.78:0/3096740306 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800043e0 con 0x7f378811b140 2024-09-16T00:28:30.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 870.0 s, 470.2 tps, lat 215.076 ms stddev 286.859 2024-09-16T00:28:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.860+0000 7f379291f740 1 Processor -- start 2024-09-16T00:28:30.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.862+0000 7f379291f740 1 -- start start 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.863+0000 7f379291f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 0x5594479dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.863+0000 7f379291f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.863+0000 7f379291f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479e9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.863+0000 7f379291f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594479422f0 con 0x5594479e7370 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.863+0000 7f379291f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5594478e9cf0 con 0x5594479ddb40 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.863+0000 7f379291f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55944793e2b0 con 0x5594479dc5d0 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.863+0000 7f377f7fe640 1 -- 172.21.15.78:0/3096740306 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3780004580 con 0x7f378811b140 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.864+0000 7f377f7fe640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37740a3380 0x7f37740a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.864+0000 7f377f7fe640 1 -- 172.21.15.78:0/3096740306 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f37800f15a0 con 0x7f378811b140 2024-09-16T00:28:30.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378dae1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378eae3640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37740a3380 0x7f37740a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378dae1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479e9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38406/0 (socket says 172.21.15.78:38406) 2024-09-16T00:28:30.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378e2e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378dae1640 1 -- 172.21.15.78:0/861100212 learned_addr learned my addr 172.21.15.78:0/861100212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:30.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378dae1640 1 --2- 172.21.15.78:0/861100212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 0x5594479dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378e2e2640 1 -- 172.21.15.78:0/861100212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 msgr2=0x5594479dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:30.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378e2e2640 1 --2- 172.21.15.78:0/861100212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 0x5594479dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378e2e2640 1 -- 172.21.15.78:0/861100212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 msgr2=0x5594479e9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:30.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378e2e2640 1 --2- 172.21.15.78:0/861100212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479e9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.869+0000 7f378e2e2640 1 -- 172.21.15.78:0/861100212 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594478ce3a0 con 0x5594479dc5d0 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.870+0000 7f378e2e2640 1 --2- 172.21.15.78:0/861100212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479dca10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f378400cba0 tx=0x7f378400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b10b78e80cf760 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.871+0000 7f378eae3640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37740a3380 0x7f37740a5840 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3780009970 tx=0x7f378002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.871+0000 7f377cff9640 1 -- 172.21.15.78:0/861100212 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784004230 con 0x5594479dc5d0 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.871+0000 7f377cff9640 1 -- 172.21.15.78:0/861100212 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37840043d0 con 0x5594479dc5d0 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.871+0000 7f377cff9640 1 -- 172.21.15.78:0/861100212 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784004e00 con 0x5594479dc5d0 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 -- 172.21.15.78:0/861100212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 msgr2=0x5594479dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 --2- 172.21.15.78:0/861100212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479dca10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f378400cba0 tx=0x7f378400e720 comp rx=0 tx=0).stop 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 -- 172.21.15.78:0/861100212 shutdown_connections 2024-09-16T00:28:30.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 --2- 172.21.15.78:0/861100212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479e9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 --2- 172.21.15.78:0/861100212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479dca10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 --2- 172.21.15.78:0/861100212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 0x5594479dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 -- 172.21.15.78:0/861100212 >> 172.21.15.78:0/861100212 conn(0x5594479a5310 msgr2=0x5594479d8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 -- 172.21.15.78:0/861100212 shutdown_connections 2024-09-16T00:28:30.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.872+0000 7f379291f740 1 -- 172.21.15.78:0/861100212 wait complete. 2024-09-16T00:28:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.873+0000 7f379291f740 1 -- start start 2024-09-16T00:28:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.873+0000 7f379291f740 1 Processor -- start 2024-09-16T00:28:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.873+0000 7f379291f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479dc5d0 0x5594479e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f379291f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f379291f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479e7370 0x5594479e4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f379291f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5594479ea070 con 0x5594479dc5d0 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f379291f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5594478fd060 con 0x5594479e7370 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f379291f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594479ea1f0 con 0x5594479ddb40 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f379291f740 10 client.0 _ll_register_callbacks cb 0x5594476508c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378dae1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479e7370 0x5594479e4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378dae1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479e7370 0x5594479e4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:41348/0 (socket says 172.21.15.78:41348) 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378dae1640 1 -- 172.21.15.78:0/3275570735 learned_addr learned my addr 172.21.15.78:0/3275570735 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378e2e2640 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378dae1640 1 -- 172.21.15.78:0/3275570735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 msgr2=0x5594479e3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378dae1640 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378dae1640 1 -- 172.21.15.78:0/3275570735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479dc5d0 msgr2=0x5594479e6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:30.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378dae1640 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479dc5d0 0x5594479e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.874+0000 7f378dae1640 1 -- 172.21.15.78:0/3275570735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594476b33c0 con 0x5594479e7370 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.876+0000 7f378dae1640 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479e7370 0x5594479e4560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3778013a90 tx=0x7f3778013f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad40bd3035923a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.876+0000 7f3771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.876+0000 7f3771ffb640 1 -- 172.21.15.78:0/3275570735 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780130d0 con 0x5594479e7370 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.876+0000 7f3771ffb640 1 -- 172.21.15.78:0/3275570735 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3778005020 con 0x5594479e7370 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.877+0000 7f379291f740 1 -- 172.21.15.78:0/3275570735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37840043d0 con 0x5594479e7370 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.877+0000 7f3771ffb640 1 -- 172.21.15.78:0/3275570735 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3778013270 con 0x5594479e7370 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.877+0000 7f379291f740 10 client.27260 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.877+0000 7f379291f740 1 -- 172.21.15.78:0/3275570735 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5594479f5b90 con 0x5594479e7370 2024-09-16T00:28:30.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.877+0000 7f3771ffb640 1 -- 172.21.15.78:0/3275570735 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37840043d0 con 0x5594479e7370 2024-09-16T00:28:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.877+0000 7f3771ffb640 1 -- 172.21.15.78:0/3275570735 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f377801c460 con 0x5594479e7370 2024-09-16T00:28:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.877+0000 7f379291f740 10 client.27260 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.878+0000 7f379291f740 10 client.27260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.878+0000 7f379291f740 20 client.27260 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.878+0000 7f379291f740 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5594479f7420 0x5594479f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.878+0000 7f379291f740 4 client.27260 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.878+0000 7f379291f740 1 -- 172.21.15.78:0/3275570735 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55944794f860 con 0x5594479f7420 2024-09-16T00:28:30.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.878+0000 7f378e2e2640 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5594479f7420 0x5594479f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.880+0000 7f3771ffb640 10 client.27260 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:30.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.880+0000 7f378e2e2640 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5594479f7420 0x5594479f98c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26ec839947b60073 server_cookie=5aa66265d8740711 in_seq=0 out_seq=0 2024-09-16T00:28:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.881+0000 7f3771ffb640 1 -- 172.21.15.78:0/3275570735 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55944794f860 con 0x5594479f7420 2024-09-16T00:28:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.881+0000 7f3771ffb640 10 client.27260 handle_command_reply: tid=0 2024-09-16T00:28:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.881+0000 7f379291f740 1 client.27260 shutdown 2024-09-16T00:28:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.881+0000 7f379291f740 1 -- 172.21.15.78:0/3275570735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479e7370 msgr2=0x5594479e4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.881+0000 7f379291f740 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479e7370 0x5594479e4560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3778013a90 tx=0x7f3778013f50 comp rx=0 tx=0).stop 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 1 -- 172.21.15.78:0/3275570735 shutdown_connections 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5594479f7420 0x5594479f98c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479e7370 0x5594479e4560 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 1 --2- 172.21.15.78:0/3275570735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479dc5d0 0x5594479e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 1 -- 172.21.15.78:0/3275570735 >> 172.21.15.78:0/3275570735 conn(0x5594479a5310 msgr2=0x5594479de930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f3771ffb640 0 client.27260 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 1 -- 172.21.15.78:0/3275570735 shutdown_connections 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 1 -- 172.21.15.78:0/3275570735 wait complete. 2024-09-16T00:28:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.885+0000 7f379291f740 20 client.27260 trim_cache size 0 max 16384 2024-09-16T00:28:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f379291f740 1 Processor -- start 2024-09-16T00:28:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f379291f740 1 -- start start 2024-09-16T00:28:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f379291f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479dc5d0 0x5594479f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f379291f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f379291f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479df120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f379291f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f376c00fb20 con 0x5594479e7370 2024-09-16T00:28:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f379291f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f376c011660 con 0x5594479dc5d0 2024-09-16T00:28:30.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f379291f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f376c0101f0 con 0x5594479ddb40 2024-09-16T00:28:30.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378e2e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.907+0000 7f378dae1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479dc5d0 0x5594479f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378e2e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:51944/0 (socket says 172.21.15.78:51944) 2024-09-16T00:28:30.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378e2e2640 1 -- 172.21.15.78:0/3226833949 learned_addr learned my addr 172.21.15.78:0/3226833949 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378dae1640 1 --2- 172.21.15.78:0/3226833949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378dae1640 1 -- 172.21.15.78:0/3226833949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 msgr2=0x5594479e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378dae1640 1 --2- 172.21.15.78:0/3226833949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378dae1640 1 -- 172.21.15.78:0/3226833949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 msgr2=0x5594479df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378dae1640 1 --2- 172.21.15.78:0/3226833949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479df120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.908+0000 7f378dae1640 1 -- 172.21.15.78:0/3226833949 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594479dd3a0 con 0x5594479dc5d0 2024-09-16T00:28:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.909+0000 7f378dae1640 1 --2- 172.21.15.78:0/3226833949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479dc5d0 0x5594479f33a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3778005ee0 tx=0x7f3778013720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb954eadc06ee7f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.909+0000 7f3773fff640 1 -- 172.21.15.78:0/3226833949 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780130d0 con 0x5594479dc5d0 2024-09-16T00:28:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.909+0000 7f3773fff640 1 -- 172.21.15.78:0/3226833949 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377802d070 con 0x5594479dc5d0 2024-09-16T00:28:30.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.909+0000 7f3773fff640 1 -- 172.21.15.78:0/3226833949 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778003870 con 0x5594479dc5d0 2024-09-16T00:28:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 -- 172.21.15.78:0/3226833949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479dc5d0 msgr2=0x5594479f33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 --2- 172.21.15.78:0/3226833949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479dc5d0 0x5594479f33a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3778005ee0 tx=0x7f3778013720 comp rx=0 tx=0).stop 2024-09-16T00:28:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 -- 172.21.15.78:0/3226833949 shutdown_connections 2024-09-16T00:28:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 --2- 172.21.15.78:0/3226833949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 --2- 172.21.15.78:0/3226833949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479ddb40 0x5594479e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 --2- 172.21.15.78:0/3226833949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479dc5d0 0x5594479f33a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 -- 172.21.15.78:0/3226833949 >> 172.21.15.78:0/3226833949 conn(0x5594479a5310 msgr2=0x5594479e5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 -- 172.21.15.78:0/3226833949 shutdown_connections 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.910+0000 7f379291f740 1 -- 172.21.15.78:0/3226833949 wait complete. 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.911+0000 7f379291f740 1 -- start start 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.911+0000 7f379291f740 1 Processor -- start 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.911+0000 7f379291f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.911+0000 7f379291f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 0x5594479fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.911+0000 7f379291f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f379291f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f376c010f90 con 0x5594479e7370 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f378dae1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f379291f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f376c0108c0 con 0x5594479ddb40 2024-09-16T00:28:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f379291f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f376c00db00 con 0x5594479dc5d0 2024-09-16T00:28:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f378dae1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479fe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:51954/0 (socket says 172.21.15.78:51954) 2024-09-16T00:28:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f378dae1640 1 -- 172.21.15.78:0/1580743256 learned_addr learned my addr 172.21.15.78:0/1580743256 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f378e2e2640 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 0x5594479fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f379291f740 10 client.0 _ll_register_callbacks cb 0x7f3778005d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.912+0000 7f378dae1640 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.913+0000 7f378dae1640 1 -- 172.21.15.78:0/1580743256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 msgr2=0x5594479fe1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.913+0000 7f378dae1640 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479fe1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.913+0000 7f378dae1640 1 -- 172.21.15.78:0/1580743256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 msgr2=0x5594479fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.913+0000 7f378dae1640 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 0x5594479fe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.913+0000 7f378dae1640 1 -- 172.21.15.78:0/1580743256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594479ee780 con 0x5594479e7370 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.913+0000 7f378dae1640 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479fec40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f377800e960 tx=0x7f377800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6c0ca1c1b34e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.913+0000 7f3771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.914+0000 7f3771ffb640 1 -- 172.21.15.78:0/1580743256 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780130d0 con 0x5594479e7370 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.914+0000 7f379291f740 1 -- 172.21.15.78:0/1580743256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377802d070 con 0x5594479e7370 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.914+0000 7f3771ffb640 1 -- 172.21.15.78:0/1580743256 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3778027a60 con 0x5594479e7370 2024-09-16T00:28:30.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.914+0000 7f3771ffb640 1 -- 172.21.15.78:0/1580743256 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3778027c60 con 0x5594479e7370 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.914+0000 7f379291f740 10 client.18111 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.914+0000 7f379291f740 1 -- 172.21.15.78:0/1580743256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5594479efa80 con 0x5594479e7370 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f3771ffb640 1 -- 172.21.15.78:0/1580743256 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377802d070 con 0x5594479e7370 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f3771ffb640 1 -- 172.21.15.78:0/1580743256 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f3778003940 con 0x5594479e7370 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f379291f740 10 client.18111 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f379291f740 10 client.18111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f379291f740 20 client.18111 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f379291f740 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559447a621a0 0x559447a64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f379291f740 4 client.18111 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f379291f740 1 -- 172.21.15.78:0/1580743256 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559447949d70 con 0x559447a621a0 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.915+0000 7f378e2e2640 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559447a621a0 0x559447a64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f3771ffb640 10 client.18111 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f378e2e2640 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559447a621a0 0x559447a64590 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9c6b754e35eb4 server_cookie=7dfe283cfa10a21d in_seq=0 out_seq=0 2024-09-16T00:28:30.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f3771ffb640 1 -- 172.21.15.78:0/1580743256 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559447949d70 con 0x559447a621a0 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f3771ffb640 10 client.18111 handle_command_reply: tid=0 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 client.18111 shutdown 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 -- 172.21.15.78:0/1580743256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 msgr2=0x5594479fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479fec40 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f377800e960 tx=0x7f377800eb90 comp rx=0 tx=0).stop 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 -- 172.21.15.78:0/1580743256 shutdown_connections 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x559447a621a0 0x559447a64590 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5594479e7370 0x5594479fec40 secure :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f377800e960 tx=0x7f377800eb90 comp rx=0 tx=0).stop 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5594479ddb40 0x5594479fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 --2- 172.21.15.78:0/1580743256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594479dc5d0 0x5594479fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f379291f740 1 -- 172.21.15.78:0/1580743256 >> 172.21.15.78:0/1580743256 conn(0x5594479a5310 msgr2=0x5594479e6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.916+0000 7f3771ffb640 0 client.18111 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.917+0000 7f379291f740 1 -- 172.21.15.78:0/1580743256 shutdown_connections 2024-09-16T00:28:30.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.917+0000 7f379291f740 1 -- 172.21.15.78:0/1580743256 wait complete. 2024-09-16T00:28:30.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.917+0000 7f379291f740 20 client.18111 trim_cache size 0 max 16384 2024-09-16T00:28:30.921 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:28:30.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 -- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37740a3380 msgr2=0x7f37740a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37740a3380 0x7f37740a5840 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3780009970 tx=0x7f378002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 -- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f378811b140 msgr2=0x7f3788114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:30.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f378811b140 0x7f3788114cb0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3780030ca0 tx=0x7f3780030cd0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 -- 172.21.15.78:0/3096740306 shutdown_connections 2024-09-16T00:28:30.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37740a3380 0x7f37740a5840 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f378811b140 0x7f3788114cb0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3788109830 0x7f3788114770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 --2- 172.21.15.78:0/3096740306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3788109450 0x7f378811a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:30.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.919+0000 7f3790546640 1 -- 172.21.15.78:0/3096740306 >> 172.21.15.78:0/3096740306 conn(0x7f378806e140 msgr2=0x7f3788070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:30.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.922+0000 7f3790546640 1 -- 172.21.15.78:0/3096740306 shutdown_connections 2024-09-16T00:28:30.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:30.922+0000 7f3790546640 1 -- 172.21.15.78:0/3096740306 wait complete. 2024-09-16T00:28:31.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:30 smithi078 ceph-mon[31157]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-09-16T00:28:31.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:30 smithi193 ceph-mon[35484]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-09-16T00:28:31.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:30 smithi148 ceph-mon[35599]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.4 MiB/s wr, 494 op/s 2024-09-16T00:28:32.857 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:28:32.858 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:28:33.415 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:34.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:33 smithi193 ceph-mon[35484]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 MiB/s wr, 480 op/s 2024-09-16T00:28:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:33 smithi148 ceph-mon[35599]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 MiB/s wr, 480 op/s 2024-09-16T00:28:34.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:33 smithi078 ceph-mon[31157]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 MiB/s wr, 480 op/s 2024-09-16T00:28:34.780 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:28:34.780 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-16T00:28:34.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 875.0 s, 841.6 tps, lat 114.030 ms stddev 190.260 2024-09-16T00:28:36.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:35 smithi193 ceph-mon[35484]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-09-16T00:28:36.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:35 smithi148 ceph-mon[35599]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-09-16T00:28:36.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:35 smithi078 ceph-mon[31157]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.8 MiB/s wr, 430 op/s 2024-09-16T00:28:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/1990439172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 msgr2=0x7fe6a0104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/1990439172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 0x7fe6a0104470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe69801acb0 tx=0x7fe69803b900 comp rx=0 tx=0).stop 2024-09-16T00:28:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/1990439172 shutdown_connections 2024-09-16T00:28:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/1990439172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 0x7fe6a0104470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/1990439172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a00fff40 0x7fe6a0100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/1990439172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a0106a00 0x7fe6a0106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/1990439172 >> 172.21.15.78:0/1990439172 conn(0x7fe6a0077070 msgr2=0x7fe6a00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/1990439172 shutdown_connections 2024-09-16T00:28:36.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.663+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/1990439172 wait complete. 2024-09-16T00:28:36.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.664+0000 7fe6a4f9a640 1 Processor -- start 2024-09-16T00:28:36.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.664+0000 7fe6a4f9a640 1 -- start start 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe6a4f9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a00fff40 0x7fe6a01960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe6a4f9a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 0x7fe6a01965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe6a4f9a640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a0106a00 0x7fe6a019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe6a4f9a640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0109c50 con 0x7fe6a0106a00 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe6a4f9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0109ad0 con 0x7fe6a00fff40 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe6a4f9a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0109dd0 con 0x7fe6a01008c0 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe69effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 0x7fe6a01965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe69f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a00fff40 0x7fe6a01960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe69effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 0x7fe6a01965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34596/0 (socket says 172.21.15.78:34596) 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.665+0000 7fe69effd640 1 -- 172.21.15.78:0/2915132859 learned_addr learned my addr 172.21.15.78:0/2915132859 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:36.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69effd640 1 -- 172.21.15.78:0/2915132859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a00fff40 msgr2=0x7fe6a01960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69effd640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a00fff40 0x7fe6a01960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69ffff640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a0106a00 0x7fe6a019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69effd640 1 -- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a0106a00 msgr2=0x7fe6a019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69effd640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a0106a00 0x7fe6a019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69effd640 1 -- 172.21.15.78:0/2915132859 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a019b060 con 0x7fe6a01008c0 2024-09-16T00:28:36.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a00fff40 0x7fe6a01960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69ffff640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a0106a00 0x7fe6a019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.666+0000 7fe69effd640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 0x7fe6a01965f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe69400b450 tx=0x7fe69400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.667+0000 7fe69cff9640 1 -- 172.21.15.78:0/2915132859 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694015020 con 0x7fe6a01008c0 2024-09-16T00:28:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.667+0000 7fe69cff9640 1 -- 172.21.15.78:0/2915132859 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe694011d70 con 0x7fe6a01008c0 2024-09-16T00:28:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.667+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/2915132859 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a019b350 con 0x7fe6a01008c0 2024-09-16T00:28:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.668+0000 7fe69cff9640 1 -- 172.21.15.78:0/2915132859 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69400f340 con 0x7fe6a01008c0 2024-09-16T00:28:36.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.668+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/2915132859 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01a2c30 con 0x7fe6a01008c0 2024-09-16T00:28:36.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.670+0000 7fe69cff9640 1 -- 172.21.15.78:0/2915132859 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe694018020 con 0x7fe6a01008c0 2024-09-16T00:28:36.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.671+0000 7fe69cff9640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6880a3450 0x7fe6880a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.671+0000 7fe69cff9640 1 -- 172.21.15.78:0/2915132859 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe6940cf6d0 con 0x7fe6a01008c0 2024-09-16T00:28:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe6a85d5740 1 Processor -- start 2024-09-16T00:28:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe6a85d5740 1 -- start start 2024-09-16T00:28:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d15d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d167a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d167f50 0x56534d16a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56534d0c2280 con 0x56534d167f50 2024-09-16T00:28:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56534d069b10 con 0x56534d15d690 2024-09-16T00:28:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56534d0be240 con 0x56534d15cd70 2024-09-16T00:28:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d167f50 0x56534d16a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d167f50 0x56534d16a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59682/0 (socket says 172.21.15.78:59682) 2024-09-16T00:28:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe69ffff640 1 -- 172.21.15.78:0/870246756 learned_addr learned my addr 172.21.15.78:0/870246756 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.672+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69effd640 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d167a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69f7fe640 1 -- 172.21.15.78:0/870246756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 msgr2=0x56534d167a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d167a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69f7fe640 1 -- 172.21.15.78:0/870246756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d167f50 msgr2=0x56534d16a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d167f50 0x56534d16a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69f7fe640 1 -- 172.21.15.78:0/870246756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534d0761f0 con 0x56534d15cd70 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69ffff640 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d167f50 0x56534d16a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6880a3450 0x7fe6880a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d15d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe690011540 tx=0x7fe690011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44574b17ecaa07fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe685ffb640 1 -- 172.21.15.78:0/870246756 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69001c910 con 0x56534d15cd70 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6880a3450 0x7fe6880a5910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe690023000 tx=0x7fe690020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe685ffb640 1 -- 172.21.15.78:0/870246756 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69000e3c0 con 0x56534d15cd70 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.673+0000 7fe685ffb640 1 -- 172.21.15.78:0/870246756 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69001b5c0 con 0x56534d15cd70 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.674+0000 7fe6a85d5740 1 -- 172.21.15.78:0/870246756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 msgr2=0x56534d15d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.674+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d15d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe690011540 tx=0x7fe690011a00 comp rx=0 tx=0).stop 2024-09-16T00:28:36.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.674+0000 7fe6a85d5740 1 -- 172.21.15.78:0/870246756 shutdown_connections 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.675+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d167f50 0x56534d16a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.675+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d167a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.675+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/870246756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d15d150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.675+0000 7fe6a85d5740 1 -- 172.21.15.78:0/870246756 >> 172.21.15.78:0/870246756 conn(0x56534d159300 msgr2=0x56534d1596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.675+0000 7fe6a85d5740 1 -- 172.21.15.78:0/870246756 shutdown_connections 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.675+0000 7fe6a85d5740 1 -- 172.21.15.78:0/870246756 wait complete. 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.675+0000 7fe6a85d5740 1 -- start start 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.675+0000 7fe6a85d5740 1 Processor -- start 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d165310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 0x56534d165850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56534d16ab20 con 0x56534d15cd70 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56534d07cf10 con 0x56534d15d690 2024-09-16T00:28:36.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56534d16aca0 con 0x56534d167f50 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe6a85d5740 10 client.0 _ll_register_callbacks cb 0x56534d045a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d165310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d164dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59688/0 (socket says 172.21.15.78:59688) 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69f7fe640 1 -- 172.21.15.78:0/1885679526 learned_addr learned my addr 172.21.15.78:0/1885679526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d165310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59848/0 (socket says 172.21.15.78:59848) 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69ffff640 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 0x56534d165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69effd640 1 -- 172.21.15.78:0/1885679526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 msgr2=0x56534d165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69effd640 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 0x56534d165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69effd640 1 -- 172.21.15.78:0/1885679526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 msgr2=0x56534d164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69effd640 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69effd640 1 -- 172.21.15.78:0/1885679526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534ce336a0 con 0x56534d15d690 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe69effd640 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d165310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe69409b630 tx=0x7fe69409c160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e51a91c2d24e8e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.676+0000 7fe67effd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe67effd640 1 -- 172.21.15.78:0/1885679526 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6940b3070 con 0x56534d15d690 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe67effd640 1 -- 172.21.15.78:0/1885679526 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe694010600 con 0x56534d15d690 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1885679526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe69000e3c0 con 0x56534d15d690 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe67effd640 1 -- 172.21.15.78:0/1885679526 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe6940a45a0 con 0x56534d15d690 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 10 client.27278 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1885679526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56534d176700 con 0x56534d15d690 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe67effd640 1 -- 172.21.15.78:0/1885679526 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69000e3c0 con 0x56534d15d690 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe67effd640 1 -- 172.21.15.78:0/1885679526 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe694097680 con 0x56534d15d690 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 10 client.27278 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 10 client.27278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 20 client.27278 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56534d177b80 0x56534d179fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 4 client.27278 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.677+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1885679526 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56534d0cf7f0 con 0x56534d177b80 2024-09-16T00:28:36.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.678+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56534d177b80 0x56534d179fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.678+0000 7fe67effd640 10 client.27278 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:36.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.678+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56534d177b80 0x56534d179fa0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b6243bfb339274 server_cookie=8412b1803e3e2204 in_seq=0 out_seq=0 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.679+0000 7fe67effd640 1 -- 172.21.15.78:0/1885679526 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56534d0cf7f0 con 0x56534d177b80 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.679+0000 7fe67effd640 10 client.27278 handle_command_reply: tid=0 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.679+0000 7fe6a85d5740 1 client.27278 shutdown 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.679+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1885679526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 msgr2=0x56534d165310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.679+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d165310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe69409b630 tx=0x7fe69409c160 comp rx=0 tx=0).stop 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.679+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1885679526 shutdown_connections 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.679+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56534d177b80 0x56534d179fa0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.680+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 0x56534d165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.680+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d165310 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.680+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1885679526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.680+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1885679526 >> 172.21.15.78:0/1885679526 conn(0x56534d159300 msgr2=0x56534d15efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.680+0000 7fe67effd640 0 client.27278 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:36.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.680+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1885679526 shutdown_connections 2024-09-16T00:28:36.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.680+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1885679526 wait complete. 2024-09-16T00:28:36.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.680+0000 7fe6a85d5740 20 client.27278 trim_cache size 0 max 16384 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe6a85d5740 1 Processor -- start 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe6a85d5740 1 -- start start 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d166c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15d690 0x56534d17f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d167f50 0x56534d17fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01ac7f0 con 0x56534d15d690 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01ae330 con 0x56534d167f50 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe6a85d5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01acec0 con 0x56534d15cd70 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe69f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d166c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe69f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d166c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34630/0 (socket says 172.21.15.78:34630) 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe69f7fe640 1 -- 172.21.15.78:0/3292074336 learned_addr learned my addr 172.21.15.78:0/3292074336 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.705+0000 7fe69effd640 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15d690 0x56534d17f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe69ffff640 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d167f50 0x56534d17fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe69f7fe640 1 -- 172.21.15.78:0/3292074336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d167f50 msgr2=0x56534d17fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d167f50 0x56534d17fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe69f7fe640 1 -- 172.21.15.78:0/3292074336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15d690 msgr2=0x56534d17f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15d690 0x56534d17f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe69f7fe640 1 -- 172.21.15.78:0/3292074336 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534d1676a0 con 0x56534d15cd70 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d166c30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe690011fd0 tx=0x7fe69000ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d6f40f43067ad16 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe6857fa640 1 -- 172.21.15.78:0/3292074336 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690009b50 con 0x56534d15cd70 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe6857fa640 1 -- 172.21.15.78:0/3292074336 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe690009cf0 con 0x56534d15cd70 2024-09-16T00:28:36.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.706+0000 7fe6857fa640 1 -- 172.21.15.78:0/3292074336 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69001bb30 con 0x56534d15cd70 2024-09-16T00:28:36.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 -- 172.21.15.78:0/3292074336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 msgr2=0x56534d166c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d166c30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe690011fd0 tx=0x7fe69000ff90 comp rx=0 tx=0).stop 2024-09-16T00:28:36.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 -- 172.21.15.78:0/3292074336 shutdown_connections 2024-09-16T00:28:36.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d167f50 0x56534d17fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15d690 0x56534d17f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/3292074336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d15cd70 0x56534d166c30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 -- 172.21.15.78:0/3292074336 >> 172.21.15.78:0/3292074336 conn(0x56534d159300 msgr2=0x56534d166520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 -- 172.21.15.78:0/3292074336 shutdown_connections 2024-09-16T00:28:36.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.707+0000 7fe6a85d5740 1 -- 172.21.15.78:0/3292074336 wait complete. 2024-09-16T00:28:36.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.708+0000 7fe6a85d5740 1 -- start start 2024-09-16T00:28:36.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.708+0000 7fe6a85d5740 1 Processor -- start 2024-09-16T00:28:36.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d16c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d16c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe6a85d5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 0x56534d1e1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe69effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d16c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe69effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d16c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59860/0 (socket says 172.21.15.78:59860) 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe69effd640 1 -- 172.21.15.78:0/1043208332 learned_addr learned my addr 172.21.15.78:0/1043208332 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe69ffff640 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 0x56534d1e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01adc60 con 0x56534d15cd70 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01ad590 con 0x56534d15d690 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.709+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a01aa7d0 con 0x56534d167f50 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.710+0000 7fe6a85d5740 10 client.0 _ll_register_callbacks cb 0x7fe69409b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.710+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d16c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.710+0000 7fe69effd640 1 -- 172.21.15.78:0/1043208332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 msgr2=0x56534d1e1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.710+0000 7fe69effd640 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 0x56534d1e1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.710+0000 7fe69effd640 1 -- 172.21.15.78:0/1043208332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 msgr2=0x56534d16c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.710+0000 7fe69effd640 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d16c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.710+0000 7fe69effd640 1 -- 172.21.15.78:0/1043208332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534d1e1d60 con 0x56534d15d690 2024-09-16T00:28:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.710+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d16c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:36.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.711+0000 7fe69effd640 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d16c6e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe694097a20 tx=0x7fe69409b570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66a128a2a26297fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.711+0000 7fe67effd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.711+0000 7fe67effd640 1 -- 172.21.15.78:0/1043208332 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6940b3040 con 0x56534d15d690 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.711+0000 7fe67effd640 1 -- 172.21.15.78:0/1043208332 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe6940cf3b0 con 0x56534d15d690 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.711+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe690009cf0 con 0x56534d15d690 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe67effd640 1 -- 172.21.15.78:0/1043208332 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe69409b290 con 0x56534d15d690 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe6a85d5740 10 client.27284 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56534d1e2f60 con 0x56534d15d690 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe67effd640 1 -- 172.21.15.78:0/1043208332 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6940afe50 con 0x56534d15d690 2024-09-16T00:28:36.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe67effd640 1 -- 172.21.15.78:0/1043208332 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe6940ad020 con 0x56534d15d690 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe6a85d5740 10 client.27284 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe6a85d5740 10 client.27284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe6a85d5740 20 client.27284 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56534d1e3630 0x56534d1e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe6a85d5740 4 client.27284 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.712+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fe690009cf0 con 0x56534d1e3630 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.713+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56534d1e3630 0x56534d1e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.713+0000 7fe67effd640 10 client.27284 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:36.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.713+0000 7fe69f7fe640 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56534d1e3630 0x56534d1e5a50 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424c7487c9bc5045 server_cookie=5cebe0130fda260d in_seq=0 out_seq=0 2024-09-16T00:28:36.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.714+0000 7fe67effd640 1 -- 172.21.15.78:0/1043208332 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fe690009cf0 con 0x56534d1e3630 2024-09-16T00:28:36.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.714+0000 7fe67effd640 10 client.27284 handle_command_reply: tid=0 2024-09-16T00:28:36.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.714+0000 7fe6a85d5740 1 client.27284 shutdown 2024-09-16T00:28:36.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.714+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 msgr2=0x56534d16c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.714+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d16c6e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe694097a20 tx=0x7fe69409b570 comp rx=0 tx=0).stop 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 shutdown_connections 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x56534d1e3630 0x56534d1e5a50 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56534d167f50 0x56534d1e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56534d15d690 0x56534d16c6e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 1 --2- 172.21.15.78:0/1043208332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56534d15cd70 0x56534d16c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe67effd640 0 client.27284 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 >> 172.21.15.78:0/1043208332 conn(0x56534d159300 msgr2=0x56534d158410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 shutdown_connections 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 1 -- 172.21.15.78:0/1043208332 wait complete. 2024-09-16T00:28:36.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.715+0000 7fe6a85d5740 20 client.27284 trim_cache size 0 max 16384 2024-09-16T00:28:36.718 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:28:36.718 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:28:36.718 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "0da7e337-1063-43e2-91f5-5be5a6bfa815", 2024-09-16T00:28:36.718 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:28:36.718 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:28:36.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6880a3450 msgr2=0x7fe6880a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6880a3450 0x7fe6880a5910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe690023000 tx=0x7fe690020000 comp rx=0 tx=0).stop 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/2915132859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 msgr2=0x7fe6a01965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 0x7fe6a01965f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe69400b450 tx=0x7fe69400b910 comp rx=0 tx=0).stop 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/2915132859 shutdown_connections 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe6880a3450 0x7fe6880a5910 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6a0106a00 0x7fe6a019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a01008c0 0x7fe6a01965f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 --2- 172.21.15.78:0/2915132859 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6a00fff40 0x7fe6a01960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.716+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/2915132859 >> 172.21.15.78:0/2915132859 conn(0x7fe6a0077070 msgr2=0x7fe6a01022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.717+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/2915132859 shutdown_connections 2024-09-16T00:28:36.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:36.717+0000 7fe6a4f9a640 1 -- 172.21.15.78:0/2915132859 wait complete. 2024-09-16T00:28:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:37 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:28:38.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:37 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:37 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:28:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:37 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:28:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:37 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:37 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:28:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:37 smithi078 ceph-mon[31157]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 MiB/s wr, 636 op/s 2024-09-16T00:28:38.020 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:37 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:37 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:28:38.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:37 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:37 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:28:38.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:37 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:28:38.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:37 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:37 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:28:38.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:37 smithi193 ceph-mon[35484]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 MiB/s wr, 636 op/s 2024-09-16T00:28:38.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:37 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:37 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:28:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:37 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:37 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:28:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:37 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:28:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:37 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:37 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:28:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:37 smithi148 ceph-mon[35599]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.0 MiB/s wr, 636 op/s 2024-09-16T00:28:38.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:37 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:38.257 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:28:38.310 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:39.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.032+0000 7fe46d634640 1 -- 172.21.15.78:0/1947241363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe46810d520 msgr2=0x7fe4681150a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.032+0000 7fe46d634640 1 --2- 172.21.15.78:0/1947241363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe46810d520 0x7fe4681150a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe458009860 tx=0x7fe45802a360 comp rx=0 tx=0).stop 2024-09-16T00:28:39.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.036+0000 7fe46d634640 1 -- 172.21.15.78:0/1947241363 shutdown_connections 2024-09-16T00:28:39.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.036+0000 7fe46d634640 1 --2- 172.21.15.78:0/1947241363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe46810d520 0x7fe4681150a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.036+0000 7fe46d634640 1 --2- 172.21.15.78:0/1947241363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe468108da0 0x7fe46810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.036+0000 7fe46d634640 1 --2- 172.21.15.78:0/1947241363 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4681083f0 0x7fe4681087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.036+0000 7fe46d634640 1 -- 172.21.15.78:0/1947241363 >> 172.21.15.78:0/1947241363 conn(0x7fe468077af0 msgr2=0x7fe468079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.040+0000 7fe46d634640 1 -- 172.21.15.78:0/1947241363 shutdown_connections 2024-09-16T00:28:39.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.042+0000 7fe46d634640 1 -- 172.21.15.78:0/1947241363 wait complete. 2024-09-16T00:28:39.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.042+0000 7fe46d634640 1 Processor -- start 2024-09-16T00:28:39.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.045+0000 7fe46d634640 1 -- start start 2024-09-16T00:28:39.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.045+0000 7fe46d634640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4681083f0 0x7fe468082900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.045+0000 7fe46d634640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe468108da0 0x7fe46807b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.045+0000 7fe46d634640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe46810d520 0x7fe46807bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.045+0000 7fe46d634640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4681185e0 con 0x7fe46810d520 2024-09-16T00:28:39.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.045+0000 7fe46d634640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe468118460 con 0x7fe4681083f0 2024-09-16T00:28:39.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.045+0000 7fe46d634640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe468118760 con 0x7fe468108da0 2024-09-16T00:28:39.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.046+0000 7fe4667fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe468108da0 0x7fe46807b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.046+0000 7fe4667fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe468108da0 0x7fe46807b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34660/0 (socket says 172.21.15.78:34660) 2024-09-16T00:28:39.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.046+0000 7fe4667fc640 1 -- 172.21.15.78:0/2607677992 learned_addr learned my addr 172.21.15.78:0/2607677992 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:39.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.046+0000 7fe466ffd640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4681083f0 0x7fe468082900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.047+0000 7fe466ffd640 1 -- 172.21.15.78:0/2607677992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe468108da0 msgr2=0x7fe46807b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.047+0000 7fe466ffd640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe468108da0 0x7fe46807b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.047+0000 7fe466ffd640 1 -- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe46810d520 msgr2=0x7fe46807bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.047+0000 7fe466ffd640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe46810d520 0x7fe46807bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.047+0000 7fe466ffd640 1 -- 172.21.15.78:0/2607677992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46807c5c0 con 0x7fe4681083f0 2024-09-16T00:28:39.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.049+0000 7fe466ffd640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4681083f0 0x7fe468082900 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009fd0 tx=0x7fe45c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.053+0000 7fe44ffff640 1 -- 172.21.15.78:0/2607677992 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c015020 con 0x7fe4681083f0 2024-09-16T00:28:39.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.053+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe468080430 con 0x7fe4681083f0 2024-09-16T00:28:39.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.053+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe468080970 con 0x7fe4681083f0 2024-09-16T00:28:39.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.053+0000 7fe44ffff640 1 -- 172.21.15.78:0/2607677992 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe45c011d10 con 0x7fe4681083f0 2024-09-16T00:28:39.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.053+0000 7fe44ffff640 1 -- 172.21.15.78:0/2607677992 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00f2e0 con 0x7fe4681083f0 2024-09-16T00:28:39.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.056+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe454005290 con 0x7fe4681083f0 2024-09-16T00:28:39.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.066+0000 7fe44ffff640 1 -- 172.21.15.78:0/2607677992 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe45c018020 con 0x7fe4681083f0 2024-09-16T00:28:39.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.067+0000 7fe44ffff640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4500a3450 0x7fe4500a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.067+0000 7fe44ffff640 1 -- 172.21.15.78:0/2607677992 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe45c0cf9d0 con 0x7fe4681083f0 2024-09-16T00:28:39.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.068+0000 7fe44ffff640 1 -- 172.21.15.78:0/2607677992 <== mon.2 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe45c01f020 con 0x7fe4681083f0 2024-09-16T00:28:39.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.076+0000 7fe4667fc640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4500a3450 0x7fe4500a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.087 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:39.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.089+0000 7fe4667fc640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4500a3450 0x7fe4500a5910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe46807d150 tx=0x7fe460010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:38 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:39.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:38 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:39.265 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:38 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:28:39.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.448+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe454005580 con 0x7fe4681083f0 2024-09-16T00:28:39.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.454+0000 7fe44ffff640 1 -- 172.21.15.78:0/2607677992 <== mon.2 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fe45c09c3b0 con 0x7fe4681083f0 2024-09-16T00:28:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:28:39.458 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:28:39.459 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:28:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4500a3450 msgr2=0x7fe4500a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4500a3450 0x7fe4500a5910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe46807d150 tx=0x7fe460010040 comp rx=0 tx=0).stop 2024-09-16T00:28:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4681083f0 msgr2=0x7fe468082900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4681083f0 0x7fe468082900 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009fd0 tx=0x7fe45c00b870 comp rx=0 tx=0).stop 2024-09-16T00:28:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 shutdown_connections 2024-09-16T00:28:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4500a3450 0x7fe4500a5910 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe46807d150 tx=0x7fe460010040 comp rx=0 tx=0).stop 2024-09-16T00:28:39.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe46810d520 0x7fe46807bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe468108da0 0x7fe46807b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 --2- 172.21.15.78:0/2607677992 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4681083f0 0x7fe468082900 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.472+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 >> 172.21.15.78:0/2607677992 conn(0x7fe468077af0 msgr2=0x7fe468078f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.475+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 shutdown_connections 2024-09-16T00:28:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.475+0000 7fe46d634640 1 -- 172.21.15.78:0/2607677992 wait complete. 2024-09-16T00:28:39.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.734+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/655680908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1079c0 msgr2=0x7ff0bc107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.734+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/655680908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1079c0 0x7ff0bc107da0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009890 tx=0x7ff0ac02a390 comp rx=0 tx=0).stop 2024-09-16T00:28:39.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.734+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/655680908 shutdown_connections 2024-09-16T00:28:39.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.734+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/655680908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0bc102360 0x7ff0bc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.734+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/655680908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0bc1019c0 0x7ff0bc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.734+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/655680908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1079c0 0x7ff0bc107da0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.734+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/655680908 >> 172.21.15.78:0/655680908 conn(0x7ff0bc0fd720 msgr2=0x7ff0bc0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.737+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/655680908 shutdown_connections 2024-09-16T00:28:39.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.737+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/655680908 wait complete. 2024-09-16T00:28:39.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.738+0000 7ff0c1bf6640 1 Processor -- start 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c1bf6640 1 -- start start 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1019c0 0x7ff0bc10ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0bc102360 0x7ff0bc10bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0bc1079c0 0x7ff0bc10c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c1bf6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc0760a0 con 0x7ff0bc1019c0 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c1bf6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc075f20 con 0x7ff0bc1079c0 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c1bf6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc076220 con 0x7ff0bc102360 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0bbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0bc102360 0x7ff0bc10bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0bbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0bc102360 0x7ff0bc10bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34686/0 (socket says 172.21.15.78:34686) 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0bbfff640 1 -- 172.21.15.78:0/4179054810 learned_addr learned my addr 172.21.15.78:0/4179054810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0bc1079c0 0x7ff0bc10c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.739+0000 7ff0c0bf4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1019c0 0x7ff0bc10ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.740+0000 7ff0bbfff640 1 -- 172.21.15.78:0/4179054810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0bc1079c0 msgr2=0x7ff0bc10c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.740+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0bc1079c0 0x7ff0bc10c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.740+0000 7ff0bbfff640 1 -- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1019c0 msgr2=0x7ff0bc10ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.740+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1019c0 0x7ff0bc10ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.740+0000 7ff0bbfff640 1 -- 172.21.15.78:0/4179054810 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc1aab90 con 0x7ff0bc102360 2024-09-16T00:28:39.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.740+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0bc1079c0 0x7ff0bc10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:39.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.740+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1019c0 0x7ff0bc10ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:39.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.741+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0bc102360 0x7ff0bc10bfb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000d980 tx=0x7ff0b000de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.742+0000 7ff0b9ffb640 1 -- 172.21.15.78:0/4179054810 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0017070 con 0x7ff0bc102360 2024-09-16T00:28:39.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.742+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/4179054810 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0bc1aae80 con 0x7ff0bc102360 2024-09-16T00:28:39.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.742+0000 7ff0b9ffb640 1 -- 172.21.15.78:0/4179054810 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b0009c30 con 0x7ff0bc102360 2024-09-16T00:28:39.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.742+0000 7ff0b9ffb640 1 -- 172.21.15.78:0/4179054810 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000f620 con 0x7ff0bc102360 2024-09-16T00:28:39.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.742+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/4179054810 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0bc1ab2e0 con 0x7ff0bc102360 2024-09-16T00:28:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.748+0000 7ff0b9ffb640 1 -- 172.21.15.78:0/4179054810 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff0b000f7c0 con 0x7ff0bc102360 2024-09-16T00:28:39.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.749+0000 7ff0c5231740 1 Processor -- start 2024-09-16T00:28:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.750+0000 7ff0c5231740 1 -- start start 2024-09-16T00:28:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3670 0x55d1d00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3f90 0x55d1d00ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c5231740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1d00092e0 con 0x55d1d00ae850 2024-09-16T00:28:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c5231740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d1cffb0ce0 con 0x55d1d00a3670 2024-09-16T00:28:39.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c5231740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d1d00052a0 con 0x55d1d00a3f90 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c0bf4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3f90 0x55d1d00ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c0bf4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3f90 0x55d1d00ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34702/0 (socket says 172.21.15.78:34702) 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0c0bf4640 1 -- 172.21.15.78:0/2598622438 learned_addr learned my addr 172.21.15.78:0/2598622438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.751+0000 7ff0b9ffb640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0a40a33d0 0x7ff0a40a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.752+0000 7ff0c0bf4640 1 -- 172.21.15.78:0/2598622438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3670 msgr2=0x55d1d00a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.752+0000 7ff0b9ffb640 1 -- 172.21.15.78:0/4179054810 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff0b00cf8d0 con 0x7ff0bc102360 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.752+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3670 0x55d1d00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.752+0000 7ff0c0bf4640 1 -- 172.21.15.78:0/2598622438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 msgr2=0x55d1d00b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.752+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.752+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.752+0000 7ff0c0bf4640 1 -- 172.21.15.78:0/2598622438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cff95440 con 0x55d1d00a3f90 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.752+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.753+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3f90 0x55d1d00ae310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac034000 tx=0x7ff0ac030440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654f067939297a23 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.753+0000 7ff0a2ffd640 1 -- 172.21.15.78:0/2598622438 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac041070 con 0x55d1d00a3f90 2024-09-16T00:28:39.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.754+0000 7ff0a2ffd640 1 -- 172.21.15.78:0/2598622438 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ac030e70 con 0x55d1d00a3f90 2024-09-16T00:28:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.755+0000 7ff0c5231740 1 -- 172.21.15.78:0/2598622438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3f90 msgr2=0x55d1d00ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.755+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3f90 0x55d1d00ae310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac034000 tx=0x7ff0ac030440 comp rx=0 tx=0).stop 2024-09-16T00:28:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.756+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0a40a33d0 0x7ff0a40a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.756+0000 7ff0c5231740 1 -- 172.21.15.78:0/2598622438 shutdown_connections 2024-09-16T00:28:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.756+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.756+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3f90 0x55d1d00ae310 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.756+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2598622438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3670 0x55d1d00a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.756+0000 7ff0c5231740 1 -- 172.21.15.78:0/2598622438 >> 172.21.15.78:0/2598622438 conn(0x55d1d009fc00 msgr2=0x55d1d009ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.757+0000 7ff0c5231740 1 -- 172.21.15.78:0/2598622438 shutdown_connections 2024-09-16T00:28:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.757+0000 7ff0c5231740 1 -- 172.21.15.78:0/2598622438 wait complete. 2024-09-16T00:28:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.757+0000 7ff0c5231740 1 -- start start 2024-09-16T00:28:39.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.757+0000 7ff0c5231740 1 Processor -- start 2024-09-16T00:28:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.758+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.758+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00ab320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.758+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00ab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.758+0000 7ff0c5231740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1d00b1470 con 0x55d1d00a3670 2024-09-16T00:28:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.758+0000 7ff0c5231740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d1cffc4050 con 0x55d1d00a3f90 2024-09-16T00:28:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.758+0000 7ff0c5231740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d1d00b15f0 con 0x55d1d00ae850 2024-09-16T00:28:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.758+0000 7ff0c5231740 10 client.0 _ll_register_callbacks cb 0x55d1d0004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:39.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.759+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0a40a33d0 0x7ff0a40a5890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4005c40 tx=0x7ff0b40094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.759+0000 7ff0c13f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.759+0000 7ff0c13f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00ab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:34712/0 (socket says 172.21.15.78:34712) 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.759+0000 7ff0c13f5640 1 -- 172.21.15.78:0/2564423886 learned_addr learned my addr 172.21.15.78:0/2564423886 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.759+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.760+0000 7ff0c13f5640 1 -- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 msgr2=0x55d1d00ab320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.760+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00ab320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.760+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.760+0000 7ff0c13f5640 1 -- 172.21.15.78:0/2564423886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 msgr2=0x55d1d00ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.760+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.760+0000 7ff0c13f5640 1 -- 172.21.15.78:0/2564423886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cfd7a3c0 con 0x55d1d00ae850 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.761+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.761+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00ab860 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40127d0 tx=0x7ff0b4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=473945f9e55e77ec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.761+0000 7ff09bfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.761+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.761+0000 7ff09bfff640 1 -- 172.21.15.78:0/2564423886 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4022070 con 0x55d1d00ae850 2024-09-16T00:28:39.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.761+0000 7ff0c5231740 1 -- 172.21.15.78:0/2564423886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1d0016850 con 0x55d1d00ae850 2024-09-16T00:28:39.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.761+0000 7ff09bfff640 1 -- 172.21.15.78:0/2564423886 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff0b4011960 con 0x55d1d00ae850 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff09bfff640 1 -- 172.21.15.78:0/2564423886 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff0b402c050 con 0x55d1d00ae850 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff0c5231740 10 client.27124 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff0c5231740 1 -- 172.21.15.78:0/2564423886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d1d00bce20 con 0x55d1d00ae850 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff09bfff640 1 -- 172.21.15.78:0/2564423886 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d1d0016850 con 0x55d1d00ae850 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff09bfff640 1 -- 172.21.15.78:0/2564423886 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff0b401a070 con 0x55d1d00ae850 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff0c5231740 10 client.27124 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff0c5231740 10 client.27124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff0c5231740 20 client.27124 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1d00be6a0 0x55d1d00c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff0c5231740 4 client.27124 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.762+0000 7ff0c5231740 1 -- 172.21.15.78:0/2564423886 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1d0010d60 con 0x55d1d00be6a0 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.763+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1d00be6a0 0x55d1d00c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.763+0000 7ff09bfff640 10 client.27124 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.763+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1d00be6a0 0x55d1d00c0b40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=716068241b2a42f2 server_cookie=841422fa7bba9eef in_seq=0 out_seq=0 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff09bfff640 1 -- 172.21.15.78:0/2564423886 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1d0010d60 con 0x55d1d00be6a0 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff09bfff640 10 client.27124 handle_command_reply: tid=0 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 client.27124 shutdown 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 -- 172.21.15.78:0/2564423886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 msgr2=0x55d1d00ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00ab860 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40127d0 tx=0x7ff0b4004ab0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 -- 172.21.15.78:0/2564423886 shutdown_connections 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1d00be6a0 0x55d1d00c0b40 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00ab860 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00ab320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2564423886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff0c5231740 1 -- 172.21.15.78:0/2564423886 >> 172.21.15.78:0/2564423886 conn(0x55d1d009fc00 msgr2=0x55d1d00a58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.764+0000 7ff09bfff640 0 client.27124 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.765+0000 7ff0c5231740 1 -- 172.21.15.78:0/2564423886 shutdown_connections 2024-09-16T00:28:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.765+0000 7ff0c5231740 1 -- 172.21.15.78:0/2564423886 wait complete. 2024-09-16T00:28:39.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.765+0000 7ff0c5231740 20 client.27124 trim_cache size 0 max 16384 2024-09-16T00:28:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c5231740 1 Processor -- start 2024-09-16T00:28:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c5231740 1 -- start start 2024-09-16T00:28:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00c6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c5231740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a800fdf0 con 0x55d1d00a3670 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c5231740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8011930 con 0x55d1d00a3f90 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c5231740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a80104c0 con 0x55d1d00ae850 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0bbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0bbfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59918/0 (socket says 172.21.15.78:59918) 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0bbfff640 1 -- 172.21.15.78:0/2821816322 learned_addr learned my addr 172.21.15.78:0/2821816322 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.787+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0bbfff640 1 -- 172.21.15.78:0/2821816322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 msgr2=0x55d1d00c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0bbfff640 1 -- 172.21.15.78:0/2821816322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 msgr2=0x55d1d00ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0bbfff640 1 -- 172.21.15.78:0/2821816322 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d009fa30 con 0x55d1d00a3f90 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00c6450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0098f40 tx=0x7ff0b009ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3781b709ec29c320 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0a27fc640 1 -- 172.21.15.78:0/2821816322 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0020040 con 0x55d1d00a3f90 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0a27fc640 1 -- 172.21.15.78:0/2821816322 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b009cb70 con 0x55d1d00a3f90 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.788+0000 7ff0a27fc640 1 -- 172.21.15.78:0/2821816322 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b001bc60 con 0x55d1d00a3f90 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 -- 172.21.15.78:0/2821816322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 msgr2=0x55d1d00c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00c6450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0098f40 tx=0x7ff0b009ad70 comp rx=0 tx=0).stop 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 -- 172.21.15.78:0/2821816322 shutdown_connections 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00ae850 0x55d1d00c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00c6450 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 --2- 172.21.15.78:0/2821816322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00a3670 0x55d1d00ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 -- 172.21.15.78:0/2821816322 >> 172.21.15.78:0/2821816322 conn(0x55d1d009fc00 msgr2=0x55d1d009ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 -- 172.21.15.78:0/2821816322 shutdown_connections 2024-09-16T00:28:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 -- 172.21.15.78:0/2821816322 wait complete. 2024-09-16T00:28:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.789+0000 7ff0c5231740 1 -- start start 2024-09-16T00:28:39.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.790+0000 7ff0c5231740 1 Processor -- start 2024-09-16T00:28:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.790+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3670 0x55d1d00b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.790+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.790+0000 7ff0c5231740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c5231740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8011260 con 0x55d1d00ae850 2024-09-16T00:28:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c5231740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a8010b90 con 0x55d1d00a3f90 2024-09-16T00:28:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c5231740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a800dd70 con 0x55d1d00a3670 2024-09-16T00:28:39.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c5231740 10 client.0 _ll_register_callbacks cb 0x7ff0b40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c13f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c13f5640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59776/0 (socket says 172.21.15.78:59776) 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c13f5640 1 -- 172.21.15.78:0/3934689018 learned_addr learned my addr 172.21.15.78:0/3934689018 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0bbfff640 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3670 0x55d1d00b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c13f5640 1 -- 172.21.15.78:0/3934689018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3670 msgr2=0x55d1d00b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3670 0x55d1d00b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c13f5640 1 -- 172.21.15.78:0/3934689018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 msgr2=0x55d1d00b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c13f5640 1 -- 172.21.15.78:0/3934689018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d00b4de0 con 0x55d1d00ae850 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.791+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3670 0x55d1d00b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.792+0000 7ff0c13f5640 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b4770 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40117b0 tx=0x7ff0b4005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6c60fb60f77b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:39.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.792+0000 7ff09bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.792+0000 7ff09bfff640 1 -- 172.21.15.78:0/3934689018 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b402c050 con 0x55d1d00ae850 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.792+0000 7ff09bfff640 1 -- 172.21.15.78:0/3934689018 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff0b401de40 con 0x55d1d00ae850 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.792+0000 7ff0c5231740 1 -- 172.21.15.78:0/3934689018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b009cb70 con 0x55d1d00ae850 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.793+0000 7ff09bfff640 1 -- 172.21.15.78:0/3934689018 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff0b4044050 con 0x55d1d00ae850 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.793+0000 7ff0c5231740 10 client.18135 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.793+0000 7ff0c5231740 1 -- 172.21.15.78:0/3934689018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d1d01292b0 con 0x55d1d00ae850 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.793+0000 7ff09bfff640 1 -- 172.21.15.78:0/3934689018 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b009cb70 con 0x55d1d00ae850 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.794+0000 7ff09bfff640 1 -- 172.21.15.78:0/3934689018 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff0b401a070 con 0x55d1d00ae850 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.794+0000 7ff0c5231740 10 client.18135 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.794+0000 7ff0c5231740 10 client.18135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.794+0000 7ff0c5231740 20 client.18135 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.794+0000 7ff0c5231740 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1d0129910 0x55d1d012bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.794+0000 7ff0c5231740 4 client.18135 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.794+0000 7ff0c5231740 1 -- 172.21.15.78:0/3934689018 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1cff8d210 con 0x55d1d0129910 2024-09-16T00:28:39.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.794+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1d0129910 0x55d1d012bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff09bfff640 10 client.18135 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c0bf4640 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1d0129910 0x55d1d012bd20 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e2a5a9f64c4a88 server_cookie=63f2f2af366b625a in_seq=0 out_seq=0 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff09bfff640 1 -- 172.21.15.78:0/3934689018 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d1cff8d210 con 0x55d1d0129910 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff09bfff640 10 client.18135 handle_command_reply: tid=0 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 client.18135 shutdown 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 -- 172.21.15.78:0/3934689018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 msgr2=0x55d1d00b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b4770 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40117b0 tx=0x7ff0b4005f70 comp rx=0 tx=0).stop 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 -- 172.21.15.78:0/3934689018 shutdown_connections 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1d0129910 0x55d1d012bd20 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1d00ae850 0x55d1d00b4770 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff09bfff640 0 client.18135 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1d00a3f90 0x55d1d00b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 --2- 172.21.15.78:0/3934689018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d00a3670 0x55d1d00b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.795+0000 7ff0c5231740 1 -- 172.21.15.78:0/3934689018 >> 172.21.15.78:0/3934689018 conn(0x55d1d009fc00 msgr2=0x55d1d00aabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.796+0000 7ff0c5231740 1 -- 172.21.15.78:0/3934689018 shutdown_connections 2024-09-16T00:28:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.796+0000 7ff0c5231740 1 -- 172.21.15.78:0/3934689018 wait complete. 2024-09-16T00:28:39.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.796+0000 7ff0c5231740 20 client.18135 trim_cache size 0 max 16384 2024-09-16T00:28:39.800 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:28:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:28:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:28:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "0da7e337-1063-43e2-91f5-5be5a6bfa815": { 2024-09-16T00:28:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:28:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "0da7e337-1063-43e2-91f5-5be5a6bfa815", 2024-09-16T00:28:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:28:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:39.801 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:28:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.796+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0a40a33d0 msgr2=0x7ff0a40a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.796+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0a40a33d0 0x7ff0a40a5890 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4005c40 tx=0x7ff0b40094a0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.796+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/4179054810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0bc102360 msgr2=0x7ff0bc10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0bc102360 0x7ff0bc10bfb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000d980 tx=0x7ff0b000de40 comp rx=0 tx=0).stop 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/4179054810 shutdown_connections 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff0a40a33d0 0x7ff0a40a5890 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff0bc1079c0 0x7ff0bc10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0bc102360 0x7ff0bc10bfb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 --2- 172.21.15.78:0/4179054810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0bc1019c0 0x7ff0bc10ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/4179054810 >> 172.21.15.78:0/4179054810 conn(0x7ff0bc0fd720 msgr2=0x7ff0bc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/4179054810 shutdown_connections 2024-09-16T00:28:39.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:39.797+0000 7ff0c1bf6640 1 -- 172.21.15.78:0/4179054810 wait complete. 2024-09-16T00:28:40.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:39 smithi078 ceph-mon[31157]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 611 op/s 2024-09-16T00:28:40.153 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:39 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2607677992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:39 smithi193 ceph-mon[35484]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 611 op/s 2024-09-16T00:28:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:39 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2607677992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:40.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:39 smithi148 ceph-mon[35599]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 MiB/s wr, 611 op/s 2024-09-16T00:28:40.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:39 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2607677992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:40.620 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-16T00:28:40.621 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-16T00:28:40.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 880.0 s, 584.2 tps, lat 182.090 ms stddev 236.772 2024-09-16T00:28:41.415 INFO:tasks.cephfs.filesystem:scrub status for tag:0da7e337-1063-43e2-91f5-5be5a6bfa815 - {'path': '/', 'tag': '0da7e337-1063-43e2-91f5-5be5a6bfa815', 'options': 'recursive,force'} 2024-09-16T00:28:41.602 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:42.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:41 smithi078 ceph-mon[31157]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 658 op/s 2024-09-16T00:28:42.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:41 smithi193 ceph-mon[35484]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 658 op/s 2024-09-16T00:28:42.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.202+0000 7f5d1963f640 1 -- 172.21.15.78:0/4131818863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d14105f50 msgr2=0x7f5d141063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.202+0000 7f5d1963f640 1 --2- 172.21.15.78:0/4131818863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d14105f50 0x7f5d141063e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009900 tx=0x7f5d0402a470 comp rx=0 tx=0).stop 2024-09-16T00:28:42.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.202+0000 7f5d1963f640 1 -- 172.21.15.78:0/4131818863 shutdown_connections 2024-09-16T00:28:42.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.202+0000 7f5d1963f640 1 --2- 172.21.15.78:0/4131818863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d14105f50 0x7f5d141063e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.202+0000 7f5d1963f640 1 --2- 172.21.15.78:0/4131818863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14101460 0x7f5d14105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.202+0000 7f5d1963f640 1 --2- 172.21.15.78:0/4131818863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d14100b40 0x7f5d14100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.202+0000 7f5d1963f640 1 -- 172.21.15.78:0/4131818863 >> 172.21.15.78:0/4131818863 conn(0x7f5d140fc7d0 msgr2=0x7f5d140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.203+0000 7f5d1963f640 1 -- 172.21.15.78:0/4131818863 shutdown_connections 2024-09-16T00:28:42.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.203+0000 7f5d1963f640 1 -- 172.21.15.78:0/4131818863 wait complete. 2024-09-16T00:28:42.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:41 smithi148 ceph-mon[35599]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 658 op/s 2024-09-16T00:28:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d1963f640 1 Processor -- start 2024-09-16T00:28:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d1963f640 1 -- start start 2024-09-16T00:28:42.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d1963f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14100b40 0x7f5d141a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d1963f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d14101460 0x7f5d141a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d1963f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d14105f50 0x7f5d1419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d1963f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141185c0 con 0x7f5d14101460 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d1963f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14118440 con 0x7f5d14100b40 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d1963f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14118740 con 0x7f5d14105f50 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d12ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14100b40 0x7f5d141a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d12ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14100b40 0x7f5d141a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59936/0 (socket says 172.21.15.78:59936) 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.204+0000 7f5d12ffd640 1 -- 172.21.15.78:0/298383615 learned_addr learned my addr 172.21.15.78:0/298383615 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.205+0000 7f5d127fc640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d14101460 0x7f5d141a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.205+0000 7f5d137fe640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d14105f50 0x7f5d1419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.205+0000 7f5d12ffd640 1 -- 172.21.15.78:0/298383615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d14105f50 msgr2=0x7f5d1419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.205+0000 7f5d12ffd640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d14105f50 0x7f5d1419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.205+0000 7f5d12ffd640 1 -- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d14101460 msgr2=0x7f5d141a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.205+0000 7f5d12ffd640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d14101460 0x7f5d141a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.205+0000 7f5d12ffd640 1 -- 172.21.15.78:0/298383615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1419efb0 con 0x7f5d14100b40 2024-09-16T00:28:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.205+0000 7f5d12ffd640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14100b40 0x7f5d141a4690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e9f0 tx=0x7f5d0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.206+0000 7f5cfbfff640 1 -- 172.21.15.78:0/298383615 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800cd80 con 0x7f5d14100b40 2024-09-16T00:28:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.206+0000 7f5cfbfff640 1 -- 172.21.15.78:0/298383615 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d08010910 con 0x7f5d14100b40 2024-09-16T00:28:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.206+0000 7f5cfbfff640 1 -- 172.21.15.78:0/298383615 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08018990 con 0x7f5d14100b40 2024-09-16T00:28:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.206+0000 7f5d1963f640 1 -- 172.21.15.78:0/298383615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1419f2a0 con 0x7f5d14100b40 2024-09-16T00:28:42.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.206+0000 7f5d1963f640 1 -- 172.21.15.78:0/298383615 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d141ab4f0 con 0x7f5d14100b40 2024-09-16T00:28:42.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.209+0000 7f5d1ba18740 1 Processor -- start 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.209+0000 7f5d1ba18740 1 -- start start 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da757b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da757b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da757b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59804/0 (socket says 172.21.15.78:59804) 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d137fe640 1 -- 172.21.15.78:0/3849733979 learned_addr learned my addr 172.21.15.78:0/3849733979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da762400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da762940 0x55d1da764d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d1ba18740 1 -- 172.21.15.78:0/3849733979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1da6bd2a0 con 0x55d1da757760 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d1ba18740 1 -- 172.21.15.78:0/3849733979 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d1da664b10 con 0x55d1da758080 2024-09-16T00:28:42.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d1ba18740 1 -- 172.21.15.78:0/3849733979 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d1da6b9260 con 0x55d1da762940 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5cfbfff640 1 -- 172.21.15.78:0/298383615 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5d08010ac0 con 0x7f5d14100b40 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.210+0000 7f5d127fc640 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da762940 0x55d1da764d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.211+0000 7f5d12ffd640 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da762400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.211+0000 7f5d137fe640 1 -- 172.21.15.78:0/3849733979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da762940 msgr2=0x55d1da764d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.211+0000 7f5d137fe640 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da762940 0x55d1da764d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.211+0000 7f5d137fe640 1 -- 172.21.15.78:0/3849733979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 msgr2=0x55d1da762400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.211+0000 7f5d137fe640 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da762400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.211+0000 7f5d137fe640 1 -- 172.21.15.78:0/3849733979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1da42eb90 con 0x55d1da757760 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.211+0000 7f5d12ffd640 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da762400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:42.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.212+0000 7f5cfbfff640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cfc0a32c0 0x7f5cfc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.212+0000 7f5d12ffd640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cfc0a32c0 0x7f5cfc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.212+0000 7f5d137fe640 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da757b40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5d0402aa60 tx=0x7f5d04005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acbf56b5d8cb9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.212+0000 7f5cfbfff640 1 -- 172.21.15.78:0/298383615 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5d080cff20 con 0x7f5d14100b40 2024-09-16T00:28:42.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.212+0000 7f5cf97fa640 1 -- 172.21.15.78:0/3849733979 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04040070 con 0x55d1da757760 2024-09-16T00:28:42.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.212+0000 7f5cf97fa640 1 -- 172.21.15.78:0/3849733979 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d04004300 con 0x55d1da757760 2024-09-16T00:28:42.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.212+0000 7f5cf97fa640 1 -- 172.21.15.78:0/3849733979 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04035040 con 0x55d1da757760 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 -- 172.21.15.78:0/3849733979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 msgr2=0x55d1da757b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da757b40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5d0402aa60 tx=0x7f5d04005c00 comp rx=0 tx=0).stop 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d12ffd640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cfc0a32c0 0x7f5cfc0a5780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5d080cda30 tx=0x7f5d08018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 -- 172.21.15.78:0/3849733979 shutdown_connections 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da762940 0x55d1da764d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da762400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/3849733979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da757b40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 -- 172.21.15.78:0/3849733979 >> 172.21.15.78:0/3849733979 conn(0x55d1da753cf0 msgr2=0x55d1da7540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 -- 172.21.15.78:0/3849733979 shutdown_connections 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 -- 172.21.15.78:0/3849733979 wait complete. 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.213+0000 7f5d1ba18740 1 -- start start 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d1ba18740 1 Processor -- start 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da75fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da762940 0x55d1da7602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d1da765510 con 0x55d1da762940 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d1da6456e0 con 0x55d1da758080 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d1da765690 con 0x55d1da757760 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d1ba18740 10 client.0 _ll_register_callbacks cb 0x55d1da669a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d127fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da75fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d137fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d127fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da75fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:59950/0 (socket says 172.21.15.78:59950) 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d127fc640 1 -- 172.21.15.78:0/1358324055 learned_addr learned my addr 172.21.15.78:0/1358324055 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:42.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.214+0000 7f5d137fe640 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da762940 0x55d1da7602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.215+0000 7f5d127fc640 1 -- 172.21.15.78:0/1358324055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 msgr2=0x55d1da75f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.215+0000 7f5d127fc640 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.215+0000 7f5d127fc640 1 -- 172.21.15.78:0/1358324055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da762940 msgr2=0x55d1da7602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.215+0000 7f5d127fc640 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da762940 0x55d1da7602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.215+0000 7f5d127fc640 1 -- 172.21.15.78:0/1358324055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1da76ea50 con 0x55d1da758080 2024-09-16T00:28:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.216+0000 7f5d127fc640 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da75fdb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c011d10 tx=0x7f5d0c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ac14167e060a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.216+0000 7f5cf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:28:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.216+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/1358324055 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c00fe60 con 0x55d1da758080 2024-09-16T00:28:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.216+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/1358324055 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5d0c004b20 con 0x55d1da758080 2024-09-16T00:28:42.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.216+0000 7f5d1ba18740 1 -- 172.21.15.78:0/1358324055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d04004300 con 0x55d1da758080 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/1358324055 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5d0c024050 con 0x55d1da758080 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d1ba18740 10 client.27311 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d1ba18740 1 -- 172.21.15.78:0/1358324055 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d1da7713b0 con 0x55d1da758080 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/1358324055 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d04004300 con 0x55d1da758080 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/1358324055 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5d0c01b470 con 0x55d1da758080 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d1ba18740 10 client.27311 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d1ba18740 10 client.27311 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d1ba18740 20 client.27311 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1da772910 0x55d1da774d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d1ba18740 4 client.27311 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d1ba18740 1 -- 172.21.15.78:0/1358324055 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1da6ca9d0 con 0x55d1da772910 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.217+0000 7f5d137fe640 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1da772910 0x55d1da774d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.218+0000 7f5cf6ff5640 10 client.27311 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:42.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.218+0000 7f5d137fe640 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1da772910 0x55d1da774d00 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ce9b0ebf138e62 server_cookie=cc671cc35989bd2b in_seq=0 out_seq=0 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/1358324055 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1da6ca9d0 con 0x55d1da772910 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5cf6ff5640 10 client.27311 handle_command_reply: tid=0 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 client.27311 shutdown 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 -- 172.21.15.78:0/1358324055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 msgr2=0x55d1da75fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da75fdb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c011d10 tx=0x7f5d0c00f790 comp rx=0 tx=0).stop 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 -- 172.21.15.78:0/1358324055 shutdown_connections 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1da772910 0x55d1da774d00 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da762940 0x55d1da7602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5cf6ff5640 0 client.27311 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da758080 0x55d1da75fdb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/1358324055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 -- 172.21.15.78:0/1358324055 >> 172.21.15.78:0/1358324055 conn(0x55d1da753cf0 msgr2=0x55d1da752e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 -- 172.21.15.78:0/1358324055 shutdown_connections 2024-09-16T00:28:42.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 1 -- 172.21.15.78:0/1358324055 wait complete. 2024-09-16T00:28:42.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.219+0000 7f5d1ba18740 20 client.27311 trim_cache size 0 max 16384 2024-09-16T00:28:42.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d1ba18740 1 Processor -- start 2024-09-16T00:28:42.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d1ba18740 1 -- start start 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da7617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da758080 0x55d1da77a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 0x55d1da77aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b5260 con 0x55d1da757760 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b6da0 con 0x55d1da762940 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b5930 con 0x55d1da758080 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da7617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d127fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da758080 0x55d1da77a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d137fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da7617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59828/0 (socket says 172.21.15.78:59828) 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d137fe640 1 -- 172.21.15.78:0/2054547152 learned_addr learned my addr 172.21.15.78:0/2054547152 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.247+0000 7f5d137fe640 1 --2- 172.21.15.78:0/2054547152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 0x55d1da77aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5d127fc640 1 -- 172.21.15.78:0/2054547152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 msgr2=0x55d1da77aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5d127fc640 1 --2- 172.21.15.78:0/2054547152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 0x55d1da77aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5d127fc640 1 -- 172.21.15.78:0/2054547152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 msgr2=0x55d1da7617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5d127fc640 1 --2- 172.21.15.78:0/2054547152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da7617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5d127fc640 1 -- 172.21.15.78:0/2054547152 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1da757590 con 0x55d1da758080 2024-09-16T00:28:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5d127fc640 1 --2- 172.21.15.78:0/2054547152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da758080 0x55d1da77a5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55d1da760e20 tx=0x7f5d0c00fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd691e1c0acf1fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5cf8ff9640 1 -- 172.21.15.78:0/2054547152 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c01b470 con 0x55d1da758080 2024-09-16T00:28:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5cf8ff9640 1 -- 172.21.15.78:0/2054547152 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c003e50 con 0x55d1da758080 2024-09-16T00:28:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.248+0000 7f5cf8ff9640 1 -- 172.21.15.78:0/2054547152 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c0286f0 con 0x55d1da758080 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 -- 172.21.15.78:0/2054547152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da758080 msgr2=0x55d1da77a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/2054547152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da758080 0x55d1da77a5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55d1da760e20 tx=0x7f5d0c00fc90 comp rx=0 tx=0).stop 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 -- 172.21.15.78:0/2054547152 shutdown_connections 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/2054547152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 0x55d1da77aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/2054547152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da758080 0x55d1da77a5a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/2054547152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da757760 0x55d1da7617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 -- 172.21.15.78:0/2054547152 >> 172.21.15.78:0/2054547152 conn(0x55d1da753cf0 msgr2=0x55d1da7610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 -- 172.21.15.78:0/2054547152 shutdown_connections 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 -- 172.21.15.78:0/2054547152 wait complete. 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 -- start start 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.249+0000 7f5d1ba18740 1 Processor -- start 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da758080 0x55d1da75ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d1ba18740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 0x55d1da75d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b66d0 con 0x55d1da758080 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b6000 con 0x55d1da762940 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d1ba18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141b3240 con 0x55d1da757760 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d1ba18740 10 client.0 _ll_register_callbacks cb 0x7f5d0c011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da758080 0x55d1da75ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d137fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da758080 0x55d1da75ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59844/0 (socket says 172.21.15.78:59844) 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 -- 172.21.15.78:0/420465896 learned_addr learned my addr 172.21.15.78:0/420465896 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d137fe640 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 0x55d1da75d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 -- 172.21.15.78:0/420465896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 msgr2=0x55d1da75c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 -- 172.21.15.78:0/420465896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 msgr2=0x55d1da75d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 0x55d1da75d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 -- 172.21.15.78:0/420465896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1da74f260 con 0x55d1da758080 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d137fe640 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5d127fc640 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da758080 0x55d1da75ceb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00a010 tx=0x7f5d0c01bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2110f3113b83130 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.250+0000 7f5cf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/420465896 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c036060 con 0x55d1da758080 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/420465896 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5d0c021040 con 0x55d1da758080 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 1 -- 172.21.15.78:0/420465896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0c003e50 con 0x55d1da758080 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/420465896 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5d0c03d050 con 0x55d1da758080 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 10 client.18153 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 1 -- 172.21.15.78:0/420465896 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d1da7dd760 con 0x55d1da758080 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/420465896 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c003e50 con 0x55d1da758080 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/420465896 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5d0c0050e0 con 0x55d1da758080 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 10 client.18153 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 10 client.18153 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 20 client.18153 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:42.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1da772910 0x55d1da7dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 4 client.18153 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-16T00:28:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d1ba18740 1 -- 172.21.15.78:0/420465896 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x55d1da6c4d20 con 0x55d1da772910 2024-09-16T00:28:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.251+0000 7f5d137fe640 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1da772910 0x55d1da7dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.252+0000 7f5cf6ff5640 10 client.18153 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:42.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.252+0000 7f5d137fe640 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1da772910 0x55d1da7dfe80 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcdd041bb1005303 server_cookie=33369d1ec030ffcb in_seq=0 out_seq=0 2024-09-16T00:28:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5cf6ff5640 1 -- 172.21.15.78:0/420465896 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d1da6c4d20 con 0x55d1da772910 2024-09-16T00:28:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5cf6ff5640 10 client.18153 handle_command_reply: tid=0 2024-09-16T00:28:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 client.18153 shutdown 2024-09-16T00:28:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 -- 172.21.15.78:0/420465896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da758080 msgr2=0x55d1da75ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da758080 0x55d1da75ceb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00a010 tx=0x7f5d0c01bce0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 -- 172.21.15.78:0/420465896 shutdown_connections 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d1da772910 0x55d1da7dfe80 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5cf6ff5640 0 client.18153 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d1da762940 0x55d1da75d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d1da758080 0x55d1da75ceb0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 --2- 172.21.15.78:0/420465896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1da757760 0x55d1da75c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 -- 172.21.15.78:0/420465896 >> 172.21.15.78:0/420465896 conn(0x55d1da753cf0 msgr2=0x55d1da752e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 -- 172.21.15.78:0/420465896 shutdown_connections 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 1 -- 172.21.15.78:0/420465896 wait complete. 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.493+0000 7f5d1ba18740 20 client.18153 trim_cache size 0 max 16384 2024-09-16T00:28:42.496 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 54, 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "9c16a291": { 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 54, 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:28:42.497 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 137.7, 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:42.498 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:28:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 -- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cfc0a32c0 msgr2=0x7f5cfc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cfc0a32c0 0x7f5cfc0a5780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5d080cda30 tx=0x7f5d08018420 comp rx=0 tx=0).stop 2024-09-16T00:28:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 -- 172.21.15.78:0/298383615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14100b40 msgr2=0x7f5d141a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14100b40 0x7f5d141a4690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e9f0 tx=0x7f5d0800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 -- 172.21.15.78:0/298383615 shutdown_connections 2024-09-16T00:28:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5cfc0a32c0 0x7f5cfc0a5780 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d14105f50 0x7f5d1419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d14101460 0x7f5d141a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 --2- 172.21.15.78:0/298383615 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d14100b40 0x7f5d141a4690 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 -- 172.21.15.78:0/298383615 >> 172.21.15.78:0/298383615 conn(0x7f5d140fc7d0 msgr2=0x7f5d14103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 -- 172.21.15.78:0/298383615 shutdown_connections 2024-09-16T00:28:42.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:42.494+0000 7f5d1963f640 1 -- 172.21.15.78:0/298383615 wait complete. 2024-09-16T00:28:43.613 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9c16a291', quiesce took 8.8 seconds. Will release after: 8.899999999999999 2024-09-16T00:28:44.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:43 smithi193 ceph-mon[35484]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 MiB/s wr, 652 op/s 2024-09-16T00:28:44.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:43 smithi148 ceph-mon[35599]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 MiB/s wr, 652 op/s 2024-09-16T00:28:44.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:43 smithi078 ceph-mon[31157]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 MiB/s wr, 652 op/s 2024-09-16T00:28:45.862 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 885.0 s, 181.9 tps, lat 162.228 ms stddev 203.106 2024-09-16T00:28:46.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:45 smithi193 ceph-mon[35484]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-16T00:28:46.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:45 smithi148 ceph-mon[35599]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-16T00:28:46.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:45 smithi078 ceph-mon[31157]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-16T00:28:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:47 smithi193 ceph-mon[35484]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-09-16T00:28:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:47 smithi148 ceph-mon[35599]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-09-16T00:28:48.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:47 smithi078 ceph-mon[31157]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 613 op/s 2024-09-16T00:28:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:49 smithi193 ceph-mon[35484]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 MiB/s wr, 343 op/s 2024-09-16T00:28:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:28:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:28:50.180 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:49 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:28:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:49 smithi148 ceph-mon[35599]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 MiB/s wr, 343 op/s 2024-09-16T00:28:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:28:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:28:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:49 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:28:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:49 smithi078 ceph-mon[31157]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.0 MiB/s wr, 343 op/s 2024-09-16T00:28:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:28:50.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:28:50.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:49 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:28:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 890.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:28:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:51 smithi193 ceph-mon[35484]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.4 MiB/s wr, 274 op/s 2024-09-16T00:28:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:51 smithi148 ceph-mon[35599]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.4 MiB/s wr, 274 op/s 2024-09-16T00:28:52.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:51 smithi078 ceph-mon[31157]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.4 MiB/s wr, 274 op/s 2024-09-16T00:28:52.514 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:52.784 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 -- 172.21.15.78:0/4128110928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c100e30 msgr2=0x7f1f3c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 --2- 172.21.15.78:0/4128110928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c100e30 0x7f1f3c104e80 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009900 tx=0x7f1f3002a410 comp rx=0 tx=0).stop 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 -- 172.21.15.78:0/4128110928 shutdown_connections 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 --2- 172.21.15.78:0/4128110928 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3c1055b0 0x7f1f3c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 --2- 172.21.15.78:0/4128110928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c100e30 0x7f1f3c104e80 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 --2- 172.21.15.78:0/4128110928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f3c100480 0x7f1f3c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 -- 172.21.15.78:0/4128110928 >> 172.21.15.78:0/4128110928 conn(0x7f1f3c077210 msgr2=0x7f1f3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 -- 172.21.15.78:0/4128110928 shutdown_connections 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 -- 172.21.15.78:0/4128110928 wait complete. 2024-09-16T00:28:53.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 Processor -- start 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.287+0000 7f1f42d37640 1 -- start start 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f42d37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3c100480 0x7f1f3c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f42d37640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f3c100e30 0x7f1f3c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f42d37640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c1055b0 0x7f1f3c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f42d37640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c111fb0 con 0x7f1f3c1055b0 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f42d37640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c111e30 con 0x7f1f3c100480 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f42d37640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c112130 con 0x7f1f3c100e30 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c1055b0 0x7f1f3c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c1055b0 0x7f1f3c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54822/0 (socket says 172.21.15.78:54822) 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 -- 172.21.15.78:0/681333003 learned_addr learned my addr 172.21.15.78:0/681333003 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f40ad3640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3c100480 0x7f1f3c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f3bfff640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f3c100e30 0x7f1f3c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 -- 172.21.15.78:0/681333003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f3c100e30 msgr2=0x7f1f3c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f3c100e30 0x7f1f3c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 -- 172.21.15.78:0/681333003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3c100480 msgr2=0x7f1f3c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3c100480 0x7f1f3c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 -- 172.21.15.78:0/681333003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c19d520 con 0x7f1f3c1055b0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f40ad3640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3c100480 0x7f1f3c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f412d4640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c1055b0 0x7f1f3c19ce40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400d950 tx=0x7f1f3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f39ffb640 1 -- 172.21.15.78:0/681333003 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34017030 con 0x7f1f3c1055b0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f39ffb640 1 -- 172.21.15.78:0/681333003 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f34004020 con 0x7f1f3c1055b0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c19d810 con 0x7f1f3c1055b0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.288+0000 7f1f39ffb640 1 -- 172.21.15.78:0/681333003 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34003c00 con 0x7f1f3c1055b0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.289+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c1a50f0 con 0x7f1f3c1055b0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.290+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3c100860 con 0x7f1f3c1055b0 2024-09-16T00:28:53.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.291+0000 7f1f39ffb640 1 -- 172.21.15.78:0/681333003 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1f3401a070 con 0x7f1f3c1055b0 2024-09-16T00:28:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.291+0000 7f1f39ffb640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f240a3240 0x7f1f240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.292+0000 7f1f40ad3640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f240a3240 0x7f1f240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.292+0000 7f1f39ffb640 1 -- 172.21.15.78:0/681333003 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1f340cf610 con 0x7f1f3c1055b0 2024-09-16T00:28:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.292+0000 7f1f40ad3640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f240a3240 0x7f1f240a5700 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009fa0 tx=0x7f1f2c009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.296+0000 7f1f39ffb640 1 -- 172.21.15.78:0/681333003 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f34021070 con 0x7f1f3c1055b0 2024-09-16T00:28:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.498+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f3c108790 con 0x7f1f3c1055b0 2024-09-16T00:28:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.499+0000 7f1f39ffb640 1 -- 172.21.15.78:0/681333003 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f1f3409bfd0 con 0x7f1f3c1055b0 2024-09-16T00:28:53.501 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:28:53.501 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:28:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:28:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f240a3240 msgr2=0x7f1f240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f240a3240 0x7f1f240a5700 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009fa0 tx=0x7f1f2c009570 comp rx=0 tx=0).stop 2024-09-16T00:28:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c1055b0 msgr2=0x7f1f3c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c1055b0 0x7f1f3c19ce40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400d950 tx=0x7f1f3400de10 comp rx=0 tx=0).stop 2024-09-16T00:28:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 shutdown_connections 2024-09-16T00:28:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f240a3240 0x7f1f240a5700 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f3c1055b0 0x7f1f3c19ce40 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f3c100e30 0x7f1f3c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 --2- 172.21.15.78:0/681333003 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f3c100480 0x7f1f3c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:53.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 >> 172.21.15.78:0/681333003 conn(0x7f1f3c077210 msgr2=0x7f1f3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:53.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 shutdown_connections 2024-09-16T00:28:53.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:53.501+0000 7f1f42d37640 1 -- 172.21.15.78:0/681333003 wait complete. 2024-09-16T00:28:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:53 smithi193 ceph-mon[35484]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-16T00:28:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:53 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/681333003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:53 smithi148 ceph-mon[35599]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-16T00:28:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:53 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/681333003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:54.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:53 smithi078 ceph-mon[31157]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-16T00:28:54.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:53 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/681333003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:54.522 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 9c16a291 --release --await' 2024-09-16T00:28:54.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 9c16a291 --release --await 2024-09-16T00:28:54.773 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:55.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 -- 172.21.15.78:0/474118822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 msgr2=0x7f704810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 --2- 172.21.15.78:0/474118822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 0x7f704810ea90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7040009900 tx=0x7f704002a410 comp rx=0 tx=0).stop 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 -- 172.21.15.78:0/474118822 shutdown_connections 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 --2- 172.21.15.78:0/474118822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 0x7f704810ea90 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 --2- 172.21.15.78:0/474118822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7048102790 0x7f70481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 --2- 172.21.15.78:0/474118822 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7048101de0 0x7f70481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 -- 172.21.15.78:0/474118822 >> 172.21.15.78:0/474118822 conn(0x7f70480fd8c0 msgr2=0x7f70480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 -- 172.21.15.78:0/474118822 shutdown_connections 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 -- 172.21.15.78:0/474118822 wait complete. 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.263+0000 7f704e0a6640 1 Processor -- start 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f704e0a6640 1 -- start start 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f704e0a6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7048101de0 0x7f70480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f704e0a6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7048102790 0x7f7048074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f704e0a6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 0x7f7048074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f704e0a6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7048111f70 con 0x7f7048106f10 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f704e0a6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7048111df0 con 0x7f7048102790 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f704e0a6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70481120f0 con 0x7f7048101de0 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 0x7f7048074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 0x7f7048074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54852/0 (socket says 172.21.15.78:54852) 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 -- 172.21.15.78:0/930226394 learned_addr learned my addr 172.21.15.78:0/930226394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7046ffd640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7048102790 0x7f7048074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 -- 172.21.15.78:0/930226394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7048101de0 msgr2=0x7f70480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f70477fe640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7048101de0 0x7f70480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7048101de0 0x7f70480744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 -- 172.21.15.78:0/930226394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7048102790 msgr2=0x7f7048074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7048102790 0x7f7048074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 -- 172.21.15.78:0/930226394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70481a6820 con 0x7f7048106f10 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.264+0000 7f7047fff640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 0x7f7048074f70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7040005ab0 tx=0x7f70400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.265+0000 7f7044ff9640 1 -- 172.21.15.78:0/930226394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704003f030 con 0x7f7048106f10 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.265+0000 7f7044ff9640 1 -- 172.21.15.78:0/930226394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7040003ce0 con 0x7f7048106f10 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.265+0000 7f704e0a6640 1 -- 172.21.15.78:0/930226394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70481a6ab0 con 0x7f7048106f10 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.265+0000 7f7044ff9640 1 -- 172.21.15.78:0/930226394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040035070 con 0x7f7048106f10 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.265+0000 7f704e0a6640 1 -- 172.21.15.78:0/930226394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70481a6f90 con 0x7f7048106f10 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.267+0000 7f7044ff9640 1 -- 172.21.15.78:0/930226394 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7040042030 con 0x7f7048106f10 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.267+0000 7f705047f740 1 Processor -- start 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.267+0000 7f705047f740 1 -- start start 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f705047f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 0x5567796f6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f705047f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779700d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f705047f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779703630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f705047f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55677965b620 con 0x556779701240 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f705047f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556779602e50 con 0x5567796f6980 2024-09-16T00:28:55.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f705047f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567796575e0 con 0x5567796f6060 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779703630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779703630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54864/0 (socket says 172.21.15.78:54864) 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 -- 172.21.15.78:0/3279641958 learned_addr learned my addr 172.21.15.78:0/3279641958 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7044ff9640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70300a3240 0x7f70300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f70477fe640 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779700d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 0x5567796f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7047fff640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70300a3240 0x7f70300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 -- 172.21.15.78:0/3279641958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 msgr2=0x5567796f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 0x5567796f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 -- 172.21.15.78:0/3279641958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 msgr2=0x556779700d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779700d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 -- 172.21.15.78:0/3279641958 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567795f3d40 con 0x556779701240 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7044ff9640 1 -- 172.21.15.78:0/930226394 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f70400f0590 con 0x7f7048106f10 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f7046ffd640 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779703630 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f703c009f60 tx=0x7f703c0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1a2e434f415d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f702dffb640 1 -- 172.21.15.78:0/3279641958 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c010af0 con 0x556779701240 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f702dffb640 1 -- 172.21.15.78:0/3279641958 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c010c90 con 0x556779701240 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.268+0000 7f702dffb640 1 -- 172.21.15.78:0/3279641958 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c01d5e0 con 0x556779701240 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 -- 172.21.15.78:0/3279641958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 msgr2=0x556779703630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779703630 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f703c009f60 tx=0x7f703c0098d0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f7047fff640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70300a3240 0x7f70300a5700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7040009a20 tx=0x7f7040087180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 -- 172.21.15.78:0/3279641958 shutdown_connections 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779703630 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779700d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 --2- 172.21.15.78:0/3279641958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 0x5567796f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 -- 172.21.15.78:0/3279641958 >> 172.21.15.78:0/3279641958 conn(0x5567796f25f0 msgr2=0x5567796f29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:55.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 -- 172.21.15.78:0/3279641958 shutdown_connections 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.269+0000 7f705047f740 1 -- 172.21.15.78:0/3279641958 wait complete. 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 1 -- start start 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 1 Processor -- start 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567796f6060 0x5567796fe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x5567796fe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556779701240 0x5567796febb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x556779703e10 con 0x5567796f6060 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5567795e3f10 con 0x5567796f6980 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556779703f90 con 0x556779701240 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f705047f740 10 client.0 _ll_register_callbacks cb 0x556779607db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567796f6060 0x5567796fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567796f6060 0x5567796fe130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54872/0 (socket says 172.21.15.78:54872) 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f7046ffd640 1 -- 172.21.15.78:0/1005224412 learned_addr learned my addr 172.21.15.78:0/1005224412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:55.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f70477fe640 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x5567796fe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f7046ffd640 1 -- 172.21.15.78:0/1005224412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556779701240 msgr2=0x5567796febb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f7046ffd640 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556779701240 0x5567796febb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f7046ffd640 1 -- 172.21.15.78:0/1005224412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 msgr2=0x5567796fe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f7046ffd640 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x5567796fe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.270+0000 7f7046ffd640 1 -- 172.21.15.78:0/1005224412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55677936a580 con 0x5567796f6060 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f7046ffd640 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567796f6060 0x5567796fe130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f703c010b20 tx=0x7f703c010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfa76e5013f549c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f7026ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f7026ffd640 1 -- 172.21.15.78:0/1005224412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c010e40 con 0x5567796f6060 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f7026ffd640 1 -- 172.21.15.78:0/1005224412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f703c002030 con 0x5567796f6060 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 1 -- 172.21.15.78:0/1005224412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f703c010c90 con 0x5567796f6060 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f7026ffd640 1 -- 172.21.15.78:0/1005224412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f703c004c50 con 0x5567796f6060 2024-09-16T00:28:55.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 10 client.18171 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 1 -- 172.21.15.78:0/1005224412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556779434960 con 0x5567796f6060 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f7026ffd640 1 -- 172.21.15.78:0/1005224412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c010c90 con 0x5567796f6060 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f7026ffd640 1 -- 172.21.15.78:0/1005224412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f703c00c070 con 0x5567796f6060 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 10 client.18171 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 10 client.18171 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 20 client.18171 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556779710d00 0x5567797130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 4 client.18171 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.271+0000 7f705047f740 1 -- 172.21.15.78:0/1005224412 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556779668b90 con 0x556779710d00 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.272+0000 7f70477fe640 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556779710d00 0x5567797130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.272+0000 7f7026ffd640 10 client.18171 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:55.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.272+0000 7f70477fe640 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556779710d00 0x5567797130f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b3f6fef36d3cf8 server_cookie=ba76920813031ac4 in_seq=0 out_seq=0 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f7026ffd640 1 -- 172.21.15.78:0/1005224412 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556779668b90 con 0x556779710d00 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f7026ffd640 10 client.18171 handle_command_reply: tid=0 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 client.18171 shutdown 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 -- 172.21.15.78:0/1005224412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567796f6060 msgr2=0x5567796fe130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567796f6060 0x5567796fe130 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f703c010b20 tx=0x7f703c010b50 comp rx=0 tx=0).stop 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 -- 172.21.15.78:0/1005224412 shutdown_connections 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556779710d00 0x5567797130f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556779701240 0x5567796febb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x5567796fe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f7026ffd640 0 client.18171 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 --2- 172.21.15.78:0/1005224412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5567796f6060 0x5567796fe130 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 -- 172.21.15.78:0/1005224412 >> 172.21.15.78:0/1005224412 conn(0x5567796f25f0 msgr2=0x5567796f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 -- 172.21.15.78:0/1005224412 shutdown_connections 2024-09-16T00:28:55.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 1 -- 172.21.15.78:0/1005224412 wait complete. 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.273+0000 7f705047f740 20 client.18171 trim_cache size 0 max 16384 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f705047f740 1 Processor -- start 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f705047f740 1 -- start start 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f705047f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6060 0x5567796f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f705047f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6980 0x5567797188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f705047f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779718df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f705047f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b0d10 con 0x556779701240 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f705047f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b2850 con 0x5567796f6060 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f705047f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b13e0 con 0x5567796f6980 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779718df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779718df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54888/0 (socket says 172.21.15.78:54888) 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 -- 172.21.15.78:0/373535222 learned_addr learned my addr 172.21.15.78:0/373535222 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:55.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6060 0x5567796f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f70477fe640 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6980 0x5567797188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 -- 172.21.15.78:0/373535222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6980 msgr2=0x5567797188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6980 0x5567797188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 -- 172.21.15.78:0/373535222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6060 msgr2=0x5567796f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6060 0x5567796f81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f7046ffd640 1 -- 172.21.15.78:0/373535222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567796f0bb0 con 0x556779701240 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.290+0000 7f70477fe640 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6980 0x5567797188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f7046ffd640 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779718df0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f703c009930 tx=0x7f703c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b807583af44e205 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f702d7fa640 1 -- 172.21.15.78:0/373535222 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c010740 con 0x556779701240 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f702d7fa640 1 -- 172.21.15.78:0/373535222 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c00cba0 con 0x556779701240 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f702d7fa640 1 -- 172.21.15.78:0/373535222 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c010e20 con 0x556779701240 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 -- 172.21.15.78:0/373535222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 msgr2=0x556779718df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779718df0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f703c009930 tx=0x7f703c015040 comp rx=0 tx=0).stop 2024-09-16T00:28:55.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 -- 172.21.15.78:0/373535222 shutdown_connections 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x556779718df0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6980 0x5567797188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 --2- 172.21.15.78:0/373535222 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6060 0x5567796f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 -- 172.21.15.78:0/373535222 >> 172.21.15.78:0/373535222 conn(0x5567796f25f0 msgr2=0x5567796f7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 -- 172.21.15.78:0/373535222 shutdown_connections 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.291+0000 7f705047f740 1 -- 172.21.15.78:0/373535222 wait complete. 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 1 -- start start 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 1 Processor -- start 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 0x5567797093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779709900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x5567797061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b2180 con 0x556779701240 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f70481b1ab0 con 0x5567796f6980 2024-09-16T00:28:55.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70481aecf0 con 0x5567796f6060 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f705047f740 10 client.0 _ll_register_callbacks cb 0x7f703c00caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x5567797061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x5567797061b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54896/0 (socket says 172.21.15.78:54896) 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f7046ffd640 1 -- 172.21.15.78:0/245969043 learned_addr learned my addr 172.21.15.78:0/245969043 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f7046ffd640 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 0x5567797093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.292+0000 7f70477fe640 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779709900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7046ffd640 1 -- 172.21.15.78:0/245969043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 msgr2=0x5567797093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7046ffd640 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 0x5567797093c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7046ffd640 1 -- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 msgr2=0x556779709900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7046ffd640 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779709900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7046ffd640 1 -- 172.21.15.78:0/245969043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567797067f0 con 0x556779701240 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f70477fe640 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779709900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:55.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7046ffd640 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x5567797061b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f703c0094c0 tx=0x7f703c005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa62427d8879a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7026ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7026ffd640 1 -- 172.21.15.78:0/245969043 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c005590 con 0x556779701240 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7026ffd640 1 -- 172.21.15.78:0/245969043 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f703c010d30 con 0x556779701240 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f705047f740 1 -- 172.21.15.78:0/245969043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f703c00cba0 con 0x556779701240 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7026ffd640 1 -- 172.21.15.78:0/245969043 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f703c00c360 con 0x556779701240 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f705047f740 10 client.18177 fetch_fsmap learned FSMap version 34 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f705047f740 1 -- 172.21.15.78:0/245969043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55677977c180 con 0x556779701240 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7026ffd640 1 -- 172.21.15.78:0/245969043 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c00cba0 con 0x556779701240 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.293+0000 7f7026ffd640 1 -- 172.21.15.78:0/245969043 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f703c00c040 con 0x556779701240 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f705047f740 10 client.18177 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f705047f740 10 client.18177 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f705047f740 20 client.18177 populate_metadata read hostname 'smithi078' 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f705047f740 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556779710d00 0x55677977e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:55.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f705047f740 4 client.18177 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9c16a291", "release": true, "await": true}] 2024-09-16T00:28:55.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f705047f740 1 -- 172.21.15.78:0/245969043 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9c16a291", "release": true, "await": true}) -- 0x5567796630a0 con 0x556779710d00 2024-09-16T00:28:55.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f70477fe640 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556779710d00 0x55677977e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:55.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f70477fe640 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556779710d00 0x55677977e610 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f004e91fe4bbd3e server_cookie=6c1a5b033c328eff in_seq=0 out_seq=0 2024-09-16T00:28:55.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.294+0000 7f7026ffd640 10 client.18177 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:55.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f7026ffd640 1 -- 172.21.15.78:0/245969043 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5567796630a0 con 0x556779710d00 2024-09-16T00:28:55.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f7026ffd640 10 client.18177 handle_command_reply: tid=0 2024-09-16T00:28:55.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 client.18177 shutdown 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 -- 172.21.15.78:0/245969043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 msgr2=0x5567797061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x5567797061b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f703c0094c0 tx=0x7f703c005870 comp rx=0 tx=0).stop 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 -- 172.21.15.78:0/245969043 shutdown_connections 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x556779710d00 0x55677977e610 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x556779701240 0x5567797061b0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f7026ffd640 0 client.18177 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5567796f6980 0x556779709900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 --2- 172.21.15.78:0/245969043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567796f6060 0x5567797093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 -- 172.21.15.78:0/245969043 >> 172.21.15.78:0/245969043 conn(0x5567796f25f0 msgr2=0x5567796f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.345+0000 7f705047f740 1 -- 172.21.15.78:0/245969043 shutdown_connections 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.346+0000 7f705047f740 1 -- 172.21.15.78:0/245969043 wait complete. 2024-09-16T00:28:55.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.346+0000 7f705047f740 20 client.18177 trim_cache size 0 max 16384 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 56, 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "9c16a291": { 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 56, 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:28:55.349 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 137.7, 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:55.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:28:55.351 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:28:55.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.346+0000 7f704e0a6640 1 -- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70300a3240 msgr2=0x7f70300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.346+0000 7f704e0a6640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70300a3240 0x7f70300a5700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7040009a20 tx=0x7f7040087180 comp rx=0 tx=0).stop 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.346+0000 7f704e0a6640 1 -- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 msgr2=0x7f7048074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.346+0000 7f704e0a6640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 0x7f7048074f70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7040005ab0 tx=0x7f70400042f0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.347+0000 7f704e0a6640 1 -- 172.21.15.78:0/930226394 shutdown_connections 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.347+0000 7f704e0a6640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f70300a3240 0x7f70300a5700 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.347+0000 7f704e0a6640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7048106f10 0x7f7048074f70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.347+0000 7f704e0a6640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7048102790 0x7f7048074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.347+0000 7f704e0a6640 1 --2- 172.21.15.78:0/930226394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7048101de0 0x7f70480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.347+0000 7f704e0a6640 1 -- 172.21.15.78:0/930226394 >> 172.21.15.78:0/930226394 conn(0x7f70480fd8c0 msgr2=0x7f70480fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.347+0000 7f704e0a6640 1 -- 172.21.15.78:0/930226394 shutdown_connections 2024-09-16T00:28:55.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:55.347+0000 7f704e0a6640 1 -- 172.21.15.78:0/930226394 wait complete. 2024-09-16T00:28:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 895.0 s, 31.8 tps, lat 6611.371 ms stddev 6658.487 2024-09-16T00:28:56.132 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9c16a291', total seconds elapsed: 21.4 2024-09-16T00:28:56.132 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-16T00:28:56.133 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:28:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:55 smithi193 ceph-mon[35484]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 727 KiB/s wr, 87 op/s 2024-09-16T00:28:56.204 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:55 smithi078 ceph-mon[31157]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 727 KiB/s wr, 87 op/s 2024-09-16T00:28:56.205 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:55 smithi148 ceph-mon[35599]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 727 KiB/s wr, 87 op/s 2024-09-16T00:28:56.607 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.252+0000 7f9636cd4640 1 -- 172.21.15.78:0/2029946564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 msgr2=0x7f9630110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:57.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.252+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2029946564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 0x7f9630110250 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9620009900 tx=0x7f962002a440 comp rx=0 tx=0).stop 2024-09-16T00:28:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.262+0000 7f9636cd4640 1 -- 172.21.15.78:0/2029946564 shutdown_connections 2024-09-16T00:28:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.262+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2029946564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963011b140 0x7f963011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.262+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2029946564 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9630109830 0x7f9630109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.262+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2029946564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 0x7f9630110250 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.262+0000 7f9636cd4640 1 -- 172.21.15.78:0/2029946564 >> 172.21.15.78:0/2029946564 conn(0x7f963006e140 msgr2=0x7f963006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:57.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.268+0000 7f9636cd4640 1 -- 172.21.15.78:0/2029946564 shutdown_connections 2024-09-16T00:28:57.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.269+0000 7f9636cd4640 1 -- 172.21.15.78:0/2029946564 wait complete. 2024-09-16T00:28:57.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.270+0000 7f9636cd4640 1 Processor -- start 2024-09-16T00:28:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.270+0000 7f9636cd4640 1 -- start start 2024-09-16T00:28:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.271+0000 7f9636cd4640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 0x7f963011a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.271+0000 7f9636cd4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9630109830 0x7f9630113770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.271+0000 7f9634a70640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 0x7f963011a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.271+0000 7f9634a70640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 0x7f963011a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57822/0 (socket says 172.21.15.78:57822) 2024-09-16T00:28:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.272+0000 7f9636cd4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963011b140 0x7f9630113cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.272+0000 7f9634a70640 1 -- 172.21.15.78:0/2091364355 learned_addr learned my addr 172.21.15.78:0/2091364355 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:28:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.272+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9630120df0 con 0x7f9630109450 2024-09-16T00:28:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.272+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9630120c70 con 0x7f9630109830 2024-09-16T00:28:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.272+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9630120f70 con 0x7f963011b140 2024-09-16T00:28:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.273+0000 7f9634a70640 1 -- 172.21.15.78:0/2091364355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963011b140 msgr2=0x7f9630113cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:28:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.273+0000 7f9634a70640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963011b140 0x7f9630113cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.273+0000 7f9634a70640 1 -- 172.21.15.78:0/2091364355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9630109830 msgr2=0x7f9630113770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:28:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.273+0000 7f9634a70640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9630109830 0x7f9630113770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.273+0000 7f9634a70640 1 -- 172.21.15.78:0/2091364355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9630114270 con 0x7f9630109450 2024-09-16T00:28:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.274+0000 7f9634a70640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 0x7f963011a6e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f9620005ab0 tx=0x7f9620005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.274+0000 7f962dffb640 1 -- 172.21.15.78:0/2091364355 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620040030 con 0x7f9630109450 2024-09-16T00:28:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.274+0000 7f962dffb640 1 -- 172.21.15.78:0/2091364355 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9620005480 con 0x7f9630109450 2024-09-16T00:28:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.274+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9630114500 con 0x7f9630109450 2024-09-16T00:28:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.275+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96301183e0 con 0x7f9630109450 2024-09-16T00:28:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.275+0000 7f962dffb640 1 -- 172.21.15.78:0/2091364355 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962003c7c0 con 0x7f9630109450 2024-09-16T00:28:57.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.279+0000 7f962dffb640 1 -- 172.21.15.78:0/2091364355 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9620043030 con 0x7f9630109450 2024-09-16T00:28:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.279+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c005290 con 0x7f9630109450 2024-09-16T00:28:57.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.280+0000 7f962dffb640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96180a3410 0x7f96180a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.281+0000 7f962dffb640 1 -- 172.21.15.78:0/2091364355 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f96200f0290 con 0x7f9630109450 2024-09-16T00:28:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.281+0000 7f962ffff640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96180a3410 0x7f96180a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:57.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.282+0000 7f962ffff640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96180a3410 0x7f96180a58d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9630114f30 tx=0x7f962400b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:57.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.291+0000 7f962dffb640 1 -- 172.21.15.78:0/2091364355 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96200bcbf0 con 0x7f9630109450 2024-09-16T00:28:57.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.659+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f961c005580 con 0x7f9630109450 2024-09-16T00:28:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.664+0000 7f962dffb640 1 -- 172.21.15.78:0/2091364355 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f96200bc4f0 con 0x7f9630109450 2024-09-16T00:28:57.665 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:28:57.665 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:28:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:28:57.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.673+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96180a3410 msgr2=0x7f96180a58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:57.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.673+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96180a3410 0x7f96180a58d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9630114f30 tx=0x7f962400b370 comp rx=0 tx=0).stop 2024-09-16T00:28:57.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.673+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 msgr2=0x7f963011a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:57.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.673+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 0x7f963011a6e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f9620005ab0 tx=0x7f9620005080 comp rx=0 tx=0).stop 2024-09-16T00:28:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.678+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 shutdown_connections 2024-09-16T00:28:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.678+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96180a3410 0x7f96180a58d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.678+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f963011b140 0x7f9630113cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.678+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9630109830 0x7f9630113770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.678+0000 7f9636cd4640 1 --2- 172.21.15.78:0/2091364355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9630109450 0x7f963011a6e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.678+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 >> 172.21.15.78:0/2091364355 conn(0x7f963006e140 msgr2=0x7f9630072350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.678+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 shutdown_connections 2024-09-16T00:28:57.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:28:57.678+0000 7f9636cd4640 1 -- 172.21.15.78:0/2091364355 wait complete. 2024-09-16T00:28:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:57 smithi148 ceph-mon[35599]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 674 KiB/s wr, 82 op/s 2024-09-16T00:28:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:57 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2091364355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:58.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:57 smithi078 ceph-mon[31157]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 674 KiB/s wr, 82 op/s 2024-09-16T00:28:58.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:57 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2091364355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:58.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:57 smithi193 ceph-mon[35484]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 674 KiB/s wr, 82 op/s 2024-09-16T00:28:58.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:57 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2091364355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:28:59.059 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-16T00:28:59.059 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-16T00:28:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:00.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:28:59 smithi078 ceph-mon[31157]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 537 KiB/s wr, 60 op/s 2024-09-16T00:29:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:28:59 smithi148 ceph-mon[35599]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 537 KiB/s wr, 60 op/s 2024-09-16T00:29:00.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.227+0000 7f1f3ffff640 1 -- 172.21.15.78:0/3646883287 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3802f300 con 0x7f1f48100e20 2024-09-16T00:29:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3646883287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 msgr2=0x7f1f48101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3646883287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 0x7f1f48101200 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009860 tx=0x7f1f3802a3d0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3646883287 shutdown_connections 2024-09-16T00:29:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3646883287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f48102ef0 0x7f1f4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3646883287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f48102570 0x7f1f481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3646883287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 0x7f1f48101200 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3646883287 >> 172.21.15.78:0/3646883287 conn(0x7f1f480fc7d0 msgr2=0x7f1f480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3646883287 shutdown_connections 2024-09-16T00:29:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.228+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3646883287 wait complete. 2024-09-16T00:29:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.229+0000 7f1f4f9c2640 1 Processor -- start 2024-09-16T00:29:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.229+0000 7f1f4f9c2640 1 -- start start 2024-09-16T00:29:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.230+0000 7f1f4f9c2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 0x7f1f48109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.230+0000 7f1f4f9c2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f48102570 0x7f1f4810a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.230+0000 7f1f4f9c2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f48102ef0 0x7f1f4810a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.230+0000 7f1f4f9c2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810feb0 con 0x7f1f48100e20 2024-09-16T00:29:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.230+0000 7f1f4f9c2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4810fd30 con 0x7f1f48102ef0 2024-09-16T00:29:00.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.230+0000 7f1f4f9c2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48110030 con 0x7f1f48102570 2024-09-16T00:29:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.232+0000 7f1f4cf5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f48102570 0x7f1f4810a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.232+0000 7f1f4df5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f48102ef0 0x7f1f4810a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.232+0000 7f1f4df5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f48102ef0 0x7f1f4810a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:38712/0 (socket says 172.21.15.78:38712) 2024-09-16T00:29:00.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.232+0000 7f1f4df5f640 1 -- 172.21.15.78:0/3125287212 learned_addr learned my addr 172.21.15.78:0/3125287212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:00.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.234+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 0x7f1f48109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.236+0000 7f1f4d75e640 1 -- 172.21.15.78:0/3125287212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f48102570 msgr2=0x7f1f4810a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.236+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f48102570 0x7f1f4810a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.236+0000 7f1f4d75e640 1 -- 172.21.15.78:0/3125287212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f48102ef0 msgr2=0x7f1f4810a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.236+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f48102ef0 0x7f1f4810a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.236+0000 7f1f4d75e640 1 -- 172.21.15.78:0/3125287212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f481aadf0 con 0x7f1f48100e20 2024-09-16T00:29:00.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.236+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 0x7f1f48109c40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1f38000c00 tx=0x7f1f38030710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.242+0000 7f1f3e7fc640 1 -- 172.21.15.78:0/3125287212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38005460 con 0x7f1f48100e20 2024-09-16T00:29:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.242+0000 7f1f3e7fc640 1 -- 172.21.15.78:0/3125287212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f38030900 con 0x7f1f48100e20 2024-09-16T00:29:00.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.242+0000 7f1f3e7fc640 1 -- 172.21.15.78:0/3125287212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38004d20 con 0x7f1f48100e20 2024-09-16T00:29:00.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.243+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3125287212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f481ab080 con 0x7f1f48100e20 2024-09-16T00:29:00.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.247+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3125287212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481ab510 con 0x7f1f48100e20 2024-09-16T00:29:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.258+0000 7f1f51d9b740 1 Processor -- start 2024-09-16T00:29:00.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.258+0000 7f1f3e7fc640 1 -- 172.21.15.78:0/3125287212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1f38039040 con 0x7f1f48100e20 2024-09-16T00:29:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f51d9b740 1 -- start start 2024-09-16T00:29:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d6760 0x55d61d4d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d7080 0x55d61d4e1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d61d43c2a0 con 0x55d61d4d7080 2024-09-16T00:29:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d61d3e3b10 con 0x55d61d4d6760 2024-09-16T00:29:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d61d438260 con 0x55d61d4e1940 2024-09-16T00:29:00.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f4cf5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f4cf5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4e3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:52586/0 (socket says 172.21.15.78:52586) 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.265+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/1566534612 learned_addr learned my addr 172.21.15.78:0/1566534612 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.266+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/1566534612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d6760 msgr2=0x55d61d4d6b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.266+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/1566534612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d6760 0x55d61d4d6b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.266+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/1566534612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d7080 msgr2=0x55d61d4e1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.266+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/1566534612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d7080 0x55d61d4e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.266+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/1566534612 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d61d1adb90 con 0x55d61d4e1940 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.266+0000 7f1f3e7fc640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f340a3390 0x7f1f340a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.266+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/1566534612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4e3d30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400dc50 tx=0x7f1f4400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f650c85c8a09fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:00.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.266+0000 7f1f3e7fc640 1 -- 172.21.15.78:0/3125287212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1f38031460 con 0x7f1f48100e20 2024-09-16T00:29:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.270+0000 7f1f337fe640 1 -- 172.21.15.78:0/1566534612 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44015070 con 0x55d61d4e1940 2024-09-16T00:29:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.270+0000 7f1f337fe640 1 -- 172.21.15.78:0/1566534612 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f440035f0 con 0x55d61d4e1940 2024-09-16T00:29:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.271+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1566534612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 msgr2=0x55d61d4e3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.271+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1566534612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4e3d30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400dc50 tx=0x7f1f4400f730 comp rx=0 tx=0).stop 2024-09-16T00:29:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.272+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1566534612 shutdown_connections 2024-09-16T00:29:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.272+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1566534612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4e3d30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.272+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1566534612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d7080 0x55d61d4e1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.272+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1566534612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d6760 0x55d61d4d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.272+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1566534612 >> 172.21.15.78:0/1566534612 conn(0x55d61d4d2cf0 msgr2=0x55d61d4d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:00.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.273+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1566534612 shutdown_connections 2024-09-16T00:29:00.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.274+0000 7f1f4df5f640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f340a3390 0x7f1f340a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.274+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1566534612 wait complete. 2024-09-16T00:29:00.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.274+0000 7f1f51d9b740 1 -- start start 2024-09-16T00:29:00.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.274+0000 7f1f51d9b740 1 Processor -- start 2024-09-16T00:29:00.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.275+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d6760 0x55d61d4de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.275+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4d7080 0x55d61d4decd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.275+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4e1940 0x55d61d4df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.275+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d61d4f5570 con 0x55d61d4e1940 2024-09-16T00:29:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.275+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d61d3c46e0 con 0x55d61d4d6760 2024-09-16T00:29:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.275+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d61d4f56f0 con 0x55d61d4d7080 2024-09-16T00:29:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.276+0000 7f1f51d9b740 10 client.0 _ll_register_callbacks cb 0x55d61d3e8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:00.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.277+0000 7f1f4df5f640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f340a3390 0x7f1f340a5850 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1f40005c40 tx=0x7f1f400095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.277+0000 7f1f4df5f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4e1940 0x55d61d4df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.277+0000 7f1f4df5f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4e1940 0x55d61d4df210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57848/0 (socket says 172.21.15.78:57848) 2024-09-16T00:29:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.277+0000 7f1f4df5f640 1 -- 172.21.15.78:0/1398992947 learned_addr learned my addr 172.21.15.78:0/1398992947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:00.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.277+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4d7080 0x55d61d4decd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/1398992947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d6760 msgr2=0x55d61d4de790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:29:00.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d6760 0x55d61d4de790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/1398992947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4e1940 msgr2=0x55d61d4df210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:00.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4e1940 0x55d61d4df210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/1398992947 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d61d50fa40 con 0x55d61d4d7080 2024-09-16T00:29:00.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4d7080 0x55d61d4decd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55d61d4d4070 tx=0x7f1f4400d880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbbe865163d1e98a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:00.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:29:00.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f30ff9640 1 -- 172.21.15.78:0/1398992947 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44015070 con 0x55d61d4d7080 2024-09-16T00:29:00.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.278+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1398992947 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f440035f0 con 0x55d61d4d7080 2024-09-16T00:29:00.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.279+0000 7f1f30ff9640 1 -- 172.21.15.78:0/1398992947 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1f44003a70 con 0x55d61d4d7080 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.280+0000 7f1f30ff9640 1 -- 172.21.15.78:0/1398992947 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1f44003fb0 con 0x55d61d4d7080 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.280+0000 7f1f51d9b740 10 client.27172 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.280+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1398992947 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d61d5123b0 con 0x55d61d4d7080 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.281+0000 7f1f30ff9640 1 -- 172.21.15.78:0/1398992947 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f440035f0 con 0x55d61d4d7080 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.281+0000 7f1f30ff9640 1 -- 172.21.15.78:0/1398992947 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1f440196e0 con 0x55d61d4d7080 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.281+0000 7f1f51d9b740 10 client.27172 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.281+0000 7f1f51d9b740 10 client.27172 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.281+0000 7f1f51d9b740 20 client.27172 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.281+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d61d513bc0 0x55d61d516080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.281+0000 7f1f51d9b740 4 client.27172 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.281+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1398992947 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d61d4499d0 con 0x55d61d513bc0 2024-09-16T00:29:00.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.282+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d61d513bc0 0x55d61d516080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.284+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d61d513bc0 0x55d61d516080 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e91ec64a238691 server_cookie=1818a4b8136a1f8e in_seq=0 out_seq=0 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.284+0000 7f1f30ff9640 10 client.27172 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.285+0000 7f1f30ff9640 1 -- 172.21.15.78:0/1398992947 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d61d4499d0 con 0x55d61d513bc0 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.285+0000 7f1f30ff9640 10 client.27172 handle_command_reply: tid=0 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.285+0000 7f1f51d9b740 1 client.27172 shutdown 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.285+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1398992947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4d7080 msgr2=0x55d61d4decd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.285+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4d7080 0x55d61d4decd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55d61d4d4070 tx=0x7f1f4400d880 comp rx=0 tx=0).stop 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.286+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1398992947 shutdown_connections 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.286+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d61d513bc0 0x55d61d516080 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.286+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4e1940 0x55d61d4df210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.286+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4d7080 0x55d61d4decd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.286+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/1398992947 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d6760 0x55d61d4de790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.286+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1398992947 >> 172.21.15.78:0/1398992947 conn(0x55d61d4d2cf0 msgr2=0x55d61d4d8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:00.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.286+0000 7f1f30ff9640 0 client.27172 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:00.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.291+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1398992947 shutdown_connections 2024-09-16T00:29:00.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.293+0000 7f1f51d9b740 1 -- 172.21.15.78:0/1398992947 wait complete. 2024-09-16T00:29:00.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.293+0000 7f1f51d9b740 20 client.27172 trim_cache size 0 max 16384 2024-09-16T00:29:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.329+0000 7f1f51d9b740 1 Processor -- start 2024-09-16T00:29:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.329+0000 7f1f51d9b740 1 -- start start 2024-09-16T00:29:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d51b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d51bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4806ce70 con 0x55d61d4d6760 2024-09-16T00:29:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4806e9b0 con 0x55d61d4d7080 2024-09-16T00:29:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4806d540 con 0x55d61d4e1940 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f4cf5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d51b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f4cf5d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d51b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:38724/0 (socket says 172.21.15.78:38724) 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/4150832731 learned_addr learned my addr 172.21.15.78:0/4150832731 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f4df5f640 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d51bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.330+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.331+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/4150832731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 msgr2=0x55d61d51bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.331+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d51bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.331+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/4150832731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 msgr2=0x55d61d4df080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.331+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4df080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.331+0000 7f1f4cf5d640 1 -- 172.21.15.78:0/4150832731 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d61d4d2b20 con 0x55d61d4d7080 2024-09-16T00:29:00.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.331+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d51b990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f44019ca0 tx=0x7f1f4400fd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e23322bf8e69a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.331+0000 7f1f32ffd640 1 -- 172.21.15.78:0/4150832731 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44015040 con 0x55d61d4d7080 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.331+0000 7f1f32ffd640 1 -- 172.21.15.78:0/4150832731 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f44003a70 con 0x55d61d4d7080 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.332+0000 7f1f32ffd640 1 -- 172.21.15.78:0/4150832731 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44011aa0 con 0x55d61d4d7080 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.332+0000 7f1f51d9b740 1 -- 172.21.15.78:0/4150832731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 msgr2=0x55d61d51b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.332+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d51b990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f44019ca0 tx=0x7f1f4400fd60 comp rx=0 tx=0).stop 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.333+0000 7f1f51d9b740 1 -- 172.21.15.78:0/4150832731 shutdown_connections 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.333+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d51bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.333+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d51b990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.333+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/4150832731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4df080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.333+0000 7f1f51d9b740 1 -- 172.21.15.78:0/4150832731 >> 172.21.15.78:0/4150832731 conn(0x55d61d4d2cf0 msgr2=0x55d61d4e0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.333+0000 7f1f51d9b740 1 -- 172.21.15.78:0/4150832731 shutdown_connections 2024-09-16T00:29:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.333+0000 7f1f51d9b740 1 -- 172.21.15.78:0/4150832731 wait complete. 2024-09-16T00:29:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.334+0000 7f1f51d9b740 1 -- start start 2024-09-16T00:29:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.334+0000 7f1f51d9b740 1 Processor -- start 2024-09-16T00:29:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d4f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f51d9b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4806e2e0 con 0x55d61d4d6760 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4806dc10 con 0x55d61d4d7080 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f51d9b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4811d300 con 0x55d61d4e1940 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f51d9b740 10 client.0 _ll_register_callbacks cb 0x7f1f44003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f4d75e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f4d75e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4fb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57876/0 (socket says 172.21.15.78:57876) 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.335+0000 7f1f4d75e640 1 -- 172.21.15.78:0/86118629 learned_addr learned my addr 172.21.15.78:0/86118629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.336+0000 7f1f4df5f640 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.336+0000 7f1f4d75e640 1 -- 172.21.15.78:0/86118629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 msgr2=0x55d61d4f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:00.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.336+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.336+0000 7f1f4d75e640 1 -- 172.21.15.78:0/86118629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 msgr2=0x55d61d4f6340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:29:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.336+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d4f6340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.336+0000 7f1f4d75e640 1 -- 172.21.15.78:0/86118629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d61d4ce260 con 0x55d61d4d6760 2024-09-16T00:29:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.337+0000 7f1f4d75e640 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4fb010 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1f380ba4d0 tx=0x7f1f38005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd12c91c645b1a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.337+0000 7f1f30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.337+0000 7f1f30ff9640 1 -- 172.21.15.78:0/86118629 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f380bde40 con 0x55d61d4d6760 2024-09-16T00:29:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.337+0000 7f1f30ff9640 1 -- 172.21.15.78:0/86118629 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1f380bc370 con 0x55d61d4d6760 2024-09-16T00:29:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.338+0000 7f1f51d9b740 1 -- 172.21.15.78:0/86118629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f44003a70 con 0x55d61d4d6760 2024-09-16T00:29:00.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.338+0000 7f1f30ff9640 1 -- 172.21.15.78:0/86118629 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1f380be070 con 0x55d61d4d6760 2024-09-16T00:29:00.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.338+0000 7f1f51d9b740 10 client.18198 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.338+0000 7f1f51d9b740 1 -- 172.21.15.78:0/86118629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d61d57e710 con 0x55d61d4d6760 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f30ff9640 1 -- 172.21.15.78:0/86118629 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f44003a70 con 0x55d61d4d6760 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f30ff9640 1 -- 172.21.15.78:0/86118629 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1f38031890 con 0x55d61d4d6760 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f51d9b740 10 client.18198 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f51d9b740 10 client.18198 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f51d9b740 20 client.18198 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d61d57ee00 0x55d61d581210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f51d9b740 4 client.18198 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f51d9b740 1 -- 172.21.15.78:0/86118629 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x55d61d443d20 con 0x55d61d57ee00 2024-09-16T00:29:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.339+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d61d57ee00 0x55d61d581210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:00.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.346+0000 7f1f4cf5d640 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d61d57ee00 0x55d61d581210 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab09a34edf5d5e78 server_cookie=3da825b2dde64149 in_seq=0 out_seq=0 2024-09-16T00:29:00.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.346+0000 7f1f30ff9640 10 client.18198 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:00.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:28:59 smithi193 ceph-mon[35484]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 537 KiB/s wr, 60 op/s 2024-09-16T00:29:00.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f30ff9640 1 -- 172.21.15.78:0/86118629 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d61d443d20 con 0x55d61d57ee00 2024-09-16T00:29:00.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f30ff9640 10 client.18198 handle_command_reply: tid=0 2024-09-16T00:29:00.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 client.18198 shutdown 2024-09-16T00:29:00.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 -- 172.21.15.78:0/86118629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 msgr2=0x55d61d4fb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4fb010 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1f380ba4d0 tx=0x7f1f38005f90 comp rx=0 tx=0).stop 2024-09-16T00:29:00.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 -- 172.21.15.78:0/86118629 shutdown_connections 2024-09-16T00:29:00.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55d61d57ee00 0x55d61d581210 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d61d4e1940 0x55d61d4f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f30ff9640 0 client.18198 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:00.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d61d4d7080 0x55d61d4f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 --2- 172.21.15.78:0/86118629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d61d4d6760 0x55d61d4fb010 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 -- 172.21.15.78:0/86118629 >> 172.21.15.78:0/86118629 conn(0x55d61d4d2cf0 msgr2=0x55d61d4e0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 -- 172.21.15.78:0/86118629 shutdown_connections 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.581+0000 7f1f51d9b740 1 -- 172.21.15.78:0/86118629 wait complete. 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f51d9b740 20 client.18198 trim_cache size 0 max 16384 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 58, 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:29:00.584 INFO:teuthology.orchestra.run.smithi078.stdout: "5c7e811e": { 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 58, 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 137.7, 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "QUIESCED", 2024-09-16T00:29:00.585 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:29:00.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:00.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:00.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:00.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:00.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:00.586 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f340a3390 msgr2=0x7f1f340a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f340a3390 0x7f1f340a5850 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1f40005c40 tx=0x7f1f400095c0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 msgr2=0x7f1f48109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 0x7f1f48109c40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1f38000c00 tx=0x7f1f38030710 comp rx=0 tx=0).stop 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3125287212 shutdown_connections 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f1f340a3390 0x7f1f340a5850 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1f48102ef0 0x7f1f4810a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f48102570 0x7f1f4810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 --2- 172.21.15.78:0/3125287212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f48100e20 0x7f1f48109c40 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3125287212 >> 172.21.15.78:0/3125287212 conn(0x7f1f480fc7d0 msgr2=0x7f1f480fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3125287212 shutdown_connections 2024-09-16T00:29:00.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:00.582+0000 7f1f4f9c2640 1 -- 172.21.15.78:0/3125287212 wait complete. 2024-09-16T00:29:00.858 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 900.0 s, 399.8 tps, lat 372.571 ms stddev 1470.020 2024-09-16T00:29:01.325 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5c7e811e', quiesce took 5.2 seconds. Will release after: 12.5 2024-09-16T00:29:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:01 smithi148 ceph-mon[35599]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-09-16T00:29:02.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:01 smithi078 ceph-mon[31157]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-09-16T00:29:02.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:01 smithi193 ceph-mon[35484]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.8 MiB/s wr, 159 op/s 2024-09-16T00:29:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:03 smithi148 ceph-mon[35599]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-09-16T00:29:04.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:03 smithi078 ceph-mon[31157]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-09-16T00:29:04.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:03 smithi193 ceph-mon[35484]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.3 MiB/s wr, 236 op/s 2024-09-16T00:29:05.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 905.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:29:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:05 smithi148 ceph-mon[35599]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-09-16T00:29:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:05 smithi078 ceph-mon[31157]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-09-16T00:29:06.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:05 smithi193 ceph-mon[35484]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 3.4 MiB/s wr, 244 op/s 2024-09-16T00:29:08.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:07 smithi148 ceph-mon[35599]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.6 MiB/s wr, 263 op/s 2024-09-16T00:29:08.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:07 smithi078 ceph-mon[31157]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.6 MiB/s wr, 263 op/s 2024-09-16T00:29:08.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:07 smithi193 ceph-mon[35484]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.6 MiB/s wr, 263 op/s 2024-09-16T00:29:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:09 smithi148 ceph-mon[35599]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.5 MiB/s wr, 253 op/s 2024-09-16T00:29:10.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:09 smithi078 ceph-mon[31157]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.5 MiB/s wr, 253 op/s 2024-09-16T00:29:10.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:09 smithi193 ceph-mon[35484]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 3.5 MiB/s wr, 253 op/s 2024-09-16T00:29:10.863 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 910.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:29:11.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:29:11.695 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:12.093 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:11 smithi078 ceph-mon[31157]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 -- 172.21.15.78:0/4248543644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 msgr2=0x7f23f41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 --2- 172.21.15.78:0/4248543644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 0x7f23f41021b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009900 tx=0x7f23e402a410 comp rx=0 tx=0).stop 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 -- 172.21.15.78:0/4248543644 shutdown_connections 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 --2- 172.21.15.78:0/4248543644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23f4106f00 0x7f23f410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 --2- 172.21.15.78:0/4248543644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4102780 0x7f23f41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 --2- 172.21.15.78:0/4248543644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 0x7f23f41021b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 -- 172.21.15.78:0/4248543644 >> 172.21.15.78:0/4248543644 conn(0x7f23f40fd8b0 msgr2=0x7f23f40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 -- 172.21.15.78:0/4248543644 shutdown_connections 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.176+0000 7f23f9619640 1 -- 172.21.15.78:0/4248543644 wait complete. 2024-09-16T00:29:12.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.177+0000 7f23f9619640 1 Processor -- start 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.177+0000 7f23f9619640 1 -- start start 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.177+0000 7f23f9619640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 0x7f23f4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.177+0000 7f23f9619640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4102780 0x7f23f4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.177+0000 7f23f9619640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23f4106f00 0x7f23f419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.177+0000 7f23f9619640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4111f60 con 0x7f23f4101dd0 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.177+0000 7f23f9619640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4111de0 con 0x7f23f4106f00 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.177+0000 7f23f9619640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41120e0 con 0x7f23f4102780 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 0x7f23f4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 0x7f23f4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48316/0 (socket says 172.21.15.78:48316) 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1444444759 learned_addr learned my addr 172.21.15.78:0/1444444759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f27fc640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4102780 0x7f23f4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f37fe640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23f4106f00 0x7f23f419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1444444759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4102780 msgr2=0x7f23f4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4102780 0x7f23f4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1444444759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23f4106f00 msgr2=0x7f23f419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23f4106f00 0x7f23f419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1444444759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f419d3b0 con 0x7f23f4101dd0 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f37fe640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23f4106f00 0x7f23f419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 0x7f23f4198400 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f23e4006010 tx=0x7f23e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23dbfff640 1 -- 172.21.15.78:0/1444444759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e403f030 con 0x7f23f4101dd0 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f9619640 1 -- 172.21.15.78:0/1444444759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f419d640 con 0x7f23f4101dd0 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23dbfff640 1 -- 172.21.15.78:0/1444444759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e4003ce0 con 0x7f23f4101dd0 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23dbfff640 1 -- 172.21.15.78:0/1444444759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4035070 con 0x7f23f4101dd0 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.178+0000 7f23f9619640 1 -- 172.21.15.78:0/1444444759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f41a4f80 con 0x7f23f4101dd0 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.180+0000 7f23dbfff640 1 -- 172.21.15.78:0/1444444759 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f23e4042030 con 0x7f23f4101dd0 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23fb9f2740 1 Processor -- start 2024-09-16T00:29:12.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23fb9f2740 1 -- start start 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23dbfff640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f23dc0a3240 0x7f23dc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1f4a0 0x560f3db1f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db2a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23dbfff640 1 -- 172.21.15.78:0/1444444759 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f23e40f01e0 con 0x7f23f4101dd0 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 0x560f3db2ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560f3da852e0 con 0x560f3db1fdc0 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23f37fe640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f23dc0a3240 0x7f23dc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560f3da2cce0 con 0x560f3db2a680 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560f3da812a0 con 0x560f3db1f4a0 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db2a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db2a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48318/0 (socket says 172.21.15.78:48318) 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23f27fc640 1 -- 172.21.15.78:0/2618930935 learned_addr learned my addr 172.21.15.78:0/2618930935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:12.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.181+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/2618930935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1f4a0 0x560f3db1f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23f27fc640 1 -- 172.21.15.78:0/2618930935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1f4a0 msgr2=0x560f3db1f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23f27fc640 1 --2- 172.21.15.78:0/2618930935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1f4a0 0x560f3db1f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23f27fc640 1 -- 172.21.15.78:0/2618930935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 msgr2=0x560f3db2ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23f27fc640 1 --2- 172.21.15.78:0/2618930935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 0x560f3db2ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23f27fc640 1 -- 172.21.15.78:0/2618930935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3da11440 con 0x560f3db1fdc0 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23f37fe640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f23dc0a3240 0x7f23dc0a5700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009370 tx=0x7f23ec00f730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23f27fc640 1 --2- 172.21.15.78:0/2618930935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db2a140 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f23e800abf0 tx=0x7f23e800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc32801a3e66af6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23d97fa640 1 -- 172.21.15.78:0/2618930935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e801d070 con 0x560f3db1fdc0 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23d97fa640 1 -- 172.21.15.78:0/2618930935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e8018b20 con 0x560f3db1fdc0 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.182+0000 7f23d97fa640 1 -- 172.21.15.78:0/2618930935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8010a30 con 0x560f3db1fdc0 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- 172.21.15.78:0/2618930935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 msgr2=0x560f3db2a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/2618930935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db2a140 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f23e800abf0 tx=0x7f23e800f790 comp rx=0 tx=0).stop 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- 172.21.15.78:0/2618930935 shutdown_connections 2024-09-16T00:29:12.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/2618930935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 0x560f3db2ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/2618930935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db2a140 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/2618930935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1f4a0 0x560f3db1f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- 172.21.15.78:0/2618930935 >> 172.21.15.78:0/2618930935 conn(0x560f3db1ba30 msgr2=0x560f3db1be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- 172.21.15.78:0/2618930935 shutdown_connections 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- 172.21.15.78:0/2618930935 wait complete. 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- start start 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 Processor -- start 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1f4a0 0x560f3db2a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db27880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x560f3db2d250 con 0x560f3db1fdc0 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560f3da40050 con 0x560f3db1f4a0 2024-09-16T00:29:12.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560f3db2d3d0 con 0x560f3db2a680 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.183+0000 7f23fb9f2740 10 client.0 _ll_register_callbacks cb 0x560f3da80160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db272e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48330/0 (socket says 172.21.15.78:48330) 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 -- 172.21.15.78:0/3512329934 learned_addr learned my addr 172.21.15.78:0/3512329934 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f37fe640 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db27880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1f4a0 0x560f3db2a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 -- 172.21.15.78:0/3512329934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 msgr2=0x560f3db27880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db27880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 -- 172.21.15.78:0/3512329934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1f4a0 msgr2=0x560f3db2a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1f4a0 0x560f3db2a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 -- 172.21.15.78:0/3512329934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3d7f63c0 con 0x560f3db1fdc0 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f37fe640 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db27880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:12.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23f27fc640 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db272e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f23e8010690 tx=0x7f23e80186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dad514f44bb33e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23d6ff5640 1 -- 172.21.15.78:0/3512329934 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8026020 con 0x560f3db1fdc0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23d6ff5640 1 -- 172.21.15.78:0/3512329934 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f23e8005250 con 0x560f3db1fdc0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23fb9f2740 1 -- 172.21.15.78:0/3512329934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23e8018b20 con 0x560f3db1fdc0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23d6ff5640 1 -- 172.21.15.78:0/3512329934 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f23e8031050 con 0x560f3db1fdc0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23fb9f2740 10 client.18210 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.184+0000 7f23fb9f2740 1 -- 172.21.15.78:0/3512329934 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560f3db38c60 con 0x560f3db1fdc0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23d6ff5640 1 -- 172.21.15.78:0/3512329934 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e8018b20 con 0x560f3db1fdc0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23d6ff5640 1 -- 172.21.15.78:0/3512329934 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f23e8004750 con 0x560f3db1fdc0 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23fb9f2740 10 client.18210 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23fb9f2740 10 client.18210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23fb9f2740 20 client.18210 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f3db3a120 0x560f3db3c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23fb9f2740 4 client.18210 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:12.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23fb9f2740 1 -- 172.21.15.78:0/3512329934 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3da92850 con 0x560f3db3a120 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.185+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f3db3a120 0x560f3db3c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23d6ff5640 10 client.18210 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f3db3a120 0x560f3db3c510 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=393a024939f103bf server_cookie=df1f7b7c9b4e1e7e in_seq=0 out_seq=0 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23d6ff5640 1 -- 172.21.15.78:0/3512329934 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f3da92850 con 0x560f3db3a120 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23d6ff5640 10 client.18210 handle_command_reply: tid=0 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 client.18210 shutdown 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 -- 172.21.15.78:0/3512329934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 msgr2=0x560f3db272e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db272e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f23e8010690 tx=0x7f23e80186f0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 -- 172.21.15.78:0/3512329934 shutdown_connections 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f3db3a120 0x560f3db3c510 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db27880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1fdc0 0x560f3db272e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/3512329934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1f4a0 0x560f3db2a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 -- 172.21.15.78:0/3512329934 >> 172.21.15.78:0/3512329934 conn(0x560f3db1ba30 msgr2=0x560f3db21710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23d6ff5640 0 client.18210 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.186+0000 7f23fb9f2740 1 -- 172.21.15.78:0/3512329934 shutdown_connections 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.187+0000 7f23fb9f2740 1 -- 172.21.15.78:0/3512329934 wait complete. 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.187+0000 7f23fb9f2740 20 client.18210 trim_cache size 0 max 16384 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.203+0000 7f23fb9f2740 1 Processor -- start 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.203+0000 7f23fb9f2740 1 -- start start 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1fdc0 0x560f3db262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db41b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41aebc0 con 0x560f3db1f4a0 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41b0700 con 0x560f3db1fdc0 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41af290 con 0x560f3db2a680 2024-09-16T00:29:12.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48338/0 (socket says 172.21.15.78:48338) 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1336910656 learned_addr learned my addr 172.21.15.78:0/1336910656 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f37fe640 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db41b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f27fc640 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1fdc0 0x560f3db262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1336910656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 msgr2=0x560f3db41b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db41b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1336910656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1fdc0 msgr2=0x560f3db262a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1fdc0 0x560f3db262a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1336910656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3db1cbb0 con 0x560f3db1f4a0 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db364b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f23e40b8960 tx=0x7f23e4003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8bfc379889fb49 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:12.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23d8ff9640 1 -- 172.21.15.78:0/1336910656 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40d9070 con 0x560f3db1f4a0 2024-09-16T00:29:12.221 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:29:12.221 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:29:12.221 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:29:12.221 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:29:12.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23d8ff9640 1 -- 172.21.15.78:0/1336910656 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e403b580 con 0x560f3db1f4a0 2024-09-16T00:29:12.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.204+0000 7f23d8ff9640 1 -- 172.21.15.78:0/1336910656 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4049040 con 0x560f3db1f4a0 2024-09-16T00:29:12.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1336910656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 msgr2=0x560f3db364b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db364b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f23e40b8960 tx=0x7f23e4003e90 comp rx=0 tx=0).stop 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1336910656 shutdown_connections 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db2a680 0x560f3db41b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db1fdc0 0x560f3db262a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1336910656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db364b0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1336910656 >> 172.21.15.78:0/1336910656 conn(0x560f3db1ba30 msgr2=0x560f3db228b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1336910656 shutdown_connections 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1336910656 wait complete. 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 -- start start 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.205+0000 7f23fb9f2740 1 Processor -- start 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db2e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1fdc0 0x560f3db2e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 0x560f3db2ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41b0030 con 0x560f3db1f4a0 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41af960 con 0x560f3db2a680 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f23f41acba0 con 0x560f3db1fdc0 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 10 client.0 _ll_register_callbacks cb 0x7f23e801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db2e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db2e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48350/0 (socket says 172.21.15.78:48350) 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1516203488 learned_addr learned my addr 172.21.15.78:0/1516203488 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f27fc640 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1fdc0 0x560f3db2e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f37fe640 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 0x560f3db2ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1516203488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1fdc0 msgr2=0x560f3db2e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1fdc0 0x560f3db2e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1516203488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 msgr2=0x560f3db2ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:12.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 0x560f3db2ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 -- 172.21.15.78:0/1516203488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3dba3b40 con 0x560f3db1f4a0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23f2ffd640 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db2e380 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f23e404a000 tx=0x7f23e40bc580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935d8bdb779ea60 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23d6ff5640 1 -- 172.21.15.78:0/1516203488 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40ab060 con 0x560f3db1f4a0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23d6ff5640 1 -- 172.21.15.78:0/1516203488 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f23e40ba800 con 0x560f3db1f4a0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1516203488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f3da8cd60 con 0x560f3db1f4a0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23d6ff5640 1 -- 172.21.15.78:0/1516203488 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f23e40d9070 con 0x560f3db1f4a0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.206+0000 7f23fb9f2740 10 client.18216 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1516203488 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560f3dba4e30 con 0x560f3db1f4a0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23d6ff5640 1 -- 172.21.15.78:0/1516203488 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560f3da8cd60 con 0x560f3db1f4a0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23d6ff5640 1 -- 172.21.15.78:0/1516203488 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f23e8019ce0 con 0x560f3db1f4a0 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23fb9f2740 10 client.18216 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23fb9f2740 10 client.18216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:12.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23fb9f2740 20 client.18216 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f3dba53f0 0x560f3dba7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23fb9f2740 4 client.18216 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1516203488 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f3da09210 con 0x560f3dba53f0 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.207+0000 7f23f27fc640 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f3dba53f0 0x560f3dba7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23d6ff5640 10 client.18216 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23f27fc640 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f3dba53f0 0x560f3dba7810 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9508d6929d0dab server_cookie=fb30d21c17a52589 in_seq=0 out_seq=0 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23d6ff5640 1 -- 172.21.15.78:0/1516203488 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f3da09210 con 0x560f3dba53f0 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23d6ff5640 10 client.18216 handle_command_reply: tid=0 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 client.18216 shutdown 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1516203488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 msgr2=0x560f3db2e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db2e380 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f23e404a000 tx=0x7f23e40bc580 comp rx=0 tx=0).stop 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1516203488 shutdown_connections 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x560f3dba53f0 0x560f3dba7810 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560f3db2a680 0x560f3db2ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f3db1fdc0 0x560f3db2e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 --2- 172.21.15.78:0/1516203488 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x560f3db1f4a0 0x560f3db2e380 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1516203488 >> 172.21.15.78:0/1516203488 conn(0x560f3db1ba30 msgr2=0x560f3db262a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23d6ff5640 0 client.18216 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1516203488 shutdown_connections 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 1 -- 172.21.15.78:0/1516203488 wait complete. 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.208+0000 7f23fb9f2740 20 client.18216 trim_cache size 0 max 16384 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 -- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f23dc0a3240 msgr2=0x7f23dc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f23dc0a3240 0x7f23dc0a5700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009370 tx=0x7f23ec00f730 comp rx=0 tx=0).stop 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 -- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 msgr2=0x7f23f4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 0x7f23f4198400 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f23e4006010 tx=0x7f23e40042f0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 -- 172.21.15.78:0/1444444759 shutdown_connections 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f23dc0a3240 0x7f23dc0a5700 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f23f4106f00 0x7f23f419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23f4102780 0x7f23f4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 --2- 172.21.15.78:0/1444444759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f23f4101dd0 0x7f23f4198400 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:12.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 -- 172.21.15.78:0/1444444759 >> 172.21.15.78:0/1444444759 conn(0x7f23f40fd8b0 msgr2=0x7f23f40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:12.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 -- 172.21.15.78:0/1444444759 shutdown_connections 2024-09-16T00:29:12.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:12.209+0000 7f23f9619640 1 -- 172.21.15.78:0/1444444759 wait complete. 2024-09-16T00:29:12.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:11 smithi193 ceph-mon[35484]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-16T00:29:12.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:11 smithi148 ceph-mon[35599]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.1 MiB/s wr, 216 op/s 2024-09-16T00:29:12.978 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:29:12.978 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:13.224 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.715+0000 7fe19136e640 1 -- 172.21.15.78:0/1285938993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 msgr2=0x7fe18c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.715+0000 7fe19136e640 1 --2- 172.21.15.78:0/1285938993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 0x7fe18c101200 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009ca0 tx=0x7fe17c02a950 comp rx=0 tx=0).stop 2024-09-16T00:29:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.716+0000 7fe19136e640 1 -- 172.21.15.78:0/1285938993 shutdown_connections 2024-09-16T00:29:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.716+0000 7fe19136e640 1 --2- 172.21.15.78:0/1285938993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18c102ef0 0x7fe18c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.716+0000 7fe19136e640 1 --2- 172.21.15.78:0/1285938993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe18c102570 0x7fe18c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.716+0000 7fe19136e640 1 --2- 172.21.15.78:0/1285938993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 0x7fe18c101200 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.716+0000 7fe19136e640 1 -- 172.21.15.78:0/1285938993 >> 172.21.15.78:0/1285938993 conn(0x7fe18c0fc7d0 msgr2=0x7fe18c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.716+0000 7fe19136e640 1 -- 172.21.15.78:0/1285938993 shutdown_connections 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.716+0000 7fe19136e640 1 -- 172.21.15.78:0/1285938993 wait complete. 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.716+0000 7fe19136e640 1 Processor -- start 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe19136e640 1 -- start start 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe19136e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 0x7fe18c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe19136e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe18c102570 0x7fe18c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe19136e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18c102ef0 0x7fe18c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe19136e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c110fa0 con 0x7fe18c100e20 2024-09-16T00:29:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe19136e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c110e20 con 0x7fe18c102ef0 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe19136e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c111120 con 0x7fe18c102570 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 0x7fe18c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 0x7fe18c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48370/0 (socket says 172.21.15.78:48370) 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 -- 172.21.15.78:0/1536873175 learned_addr learned my addr 172.21.15.78:0/1536873175 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18b7fe640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18c102ef0 0x7fe18c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18a7fc640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe18c102570 0x7fe18c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 -- 172.21.15.78:0/1536873175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe18c102570 msgr2=0x7fe18c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe18c102570 0x7fe18c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 -- 172.21.15.78:0/1536873175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18c102ef0 msgr2=0x7fe18c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18c102ef0 0x7fe18c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 -- 172.21.15.78:0/1536873175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c19d540 con 0x7fe18c100e20 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18b7fe640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18c102ef0 0x7fe18c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18a7fc640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe18c102570 0x7fe18c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.717+0000 7fe18affd640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 0x7fe18c198590 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe17c005cc0 tx=0x7fe17c02f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.718+0000 7fe183fff640 1 -- 172.21.15.78:0/1536873175 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c034040 con 0x7fe18c100e20 2024-09-16T00:29:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.718+0000 7fe183fff640 1 -- 172.21.15.78:0/1536873175 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe17c003e40 con 0x7fe18c100e20 2024-09-16T00:29:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.718+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c19d7d0 con 0x7fe18c100e20 2024-09-16T00:29:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.718+0000 7fe183fff640 1 -- 172.21.15.78:0/1536873175 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c045480 con 0x7fe18c100e20 2024-09-16T00:29:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.718+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c1a5110 con 0x7fe18c100e20 2024-09-16T00:29:13.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.719+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18c1021b0 con 0x7fe18c100e20 2024-09-16T00:29:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.720+0000 7fe183fff640 1 -- 172.21.15.78:0/1536873175 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe17c037020 con 0x7fe18c100e20 2024-09-16T00:29:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.721+0000 7fe183fff640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1780a3240 0x7fe1780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.721+0000 7fe18a7fc640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1780a3240 0x7fe1780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.721+0000 7fe183fff640 1 -- 172.21.15.78:0/1536873175 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe17c0f0ef0 con 0x7fe18c100e20 2024-09-16T00:29:13.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.721+0000 7fe18a7fc640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1780a3240 0x7fe1780a5700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0fffc0 tx=0x7fe174008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:13.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.725+0000 7fe183fff640 1 -- 172.21.15.78:0/1536873175 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe17c0bd8b0 con 0x7fe18c100e20 2024-09-16T00:29:13.827 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:13.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.930+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe18c199920 con 0x7fe18c100e20 2024-09-16T00:29:13.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.932+0000 7fe183fff640 1 -- 172.21.15.78:0/1536873175 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7fe18c199920 con 0x7fe18c100e20 2024-09-16T00:29:13.933 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:13.933 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:29:13.933 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:29:13.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1780a3240 msgr2=0x7fe1780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:13.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1780a3240 0x7fe1780a5700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0fffc0 tx=0x7fe174008040 comp rx=0 tx=0).stop 2024-09-16T00:29:13.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 msgr2=0x7fe18c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:13.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 0x7fe18c198590 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe17c005cc0 tx=0x7fe17c02f6c0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 shutdown_connections 2024-09-16T00:29:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1780a3240 0x7fe1780a5700 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe18c102ef0 0x7fe18c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe18c102570 0x7fe18c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 --2- 172.21.15.78:0/1536873175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe18c100e20 0x7fe18c198590 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.934+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 >> 172.21.15.78:0/1536873175 conn(0x7fe18c0fc7d0 msgr2=0x7fe18c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.935+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 shutdown_connections 2024-09-16T00:29:13.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:13.935+0000 7fe19136e640 1 -- 172.21.15.78:0/1536873175 wait complete. 2024-09-16T00:29:14.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:13 smithi078 ceph-mon[31157]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-16T00:29:14.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:13 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1536873175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:14.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:13 smithi193 ceph-mon[35484]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-16T00:29:14.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:13 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1536873175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:14.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:13 smithi148 ceph-mon[35599]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-16T00:29:14.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:13 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1536873175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:14.642 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:14.669 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 -- 172.21.15.78:0/2104693845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8100e20 msgr2=0x7f09e8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 --2- 172.21.15.78:0/2104693845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8100e20 0x7f09e8101200 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009900 tx=0x7f09e402a480 comp rx=0 tx=0).stop 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 -- 172.21.15.78:0/2104693845 shutdown_connections 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 --2- 172.21.15.78:0/2104693845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09e8102ef0 0x7f09e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 --2- 172.21.15.78:0/2104693845 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e8102570 0x7f09e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 --2- 172.21.15.78:0/2104693845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8100e20 0x7f09e8101200 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 -- 172.21.15.78:0/2104693845 >> 172.21.15.78:0/2104693845 conn(0x7f09e80fc7d0 msgr2=0x7f09e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 -- 172.21.15.78:0/2104693845 shutdown_connections 2024-09-16T00:29:15.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.160+0000 7f09f048e640 1 -- 172.21.15.78:0/2104693845 wait complete. 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09f048e640 1 Processor -- start 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09f048e640 1 -- start start 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09f048e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09e8100e20 0x7f09e8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09f048e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8102570 0x7f09e8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09f048e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e8102ef0 0x7f09e819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09f048e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09e81120a0 con 0x7f09e8102570 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09f048e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8111f20 con 0x7f09e8102ef0 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09f048e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8112220 con 0x7f09e8100e20 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8102570 0x7f09e8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8102570 0x7f09e8198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48384/0 (socket says 172.21.15.78:48384) 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 -- 172.21.15.78:0/2863694816 learned_addr learned my addr 172.21.15.78:0/2863694816 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eea2b640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e8102ef0 0x7f09e819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09ee22a640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09e8100e20 0x7f09e8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:15.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 -- 172.21.15.78:0/2863694816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09e8100e20 msgr2=0x7f09e8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09e8100e20 0x7f09e8198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 -- 172.21.15.78:0/2863694816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e8102ef0 msgr2=0x7f09e819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e8102ef0 0x7f09e819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 -- 172.21.15.78:0/2863694816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e819d4f0 con 0x7f09e8102570 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.161+0000 7f09eda29640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8102570 0x7f09e8198a80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f09d800d950 tx=0x7f09d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.162+0000 7f09df7fe640 1 -- 172.21.15.78:0/2863694816 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8017070 con 0x7f09e8102570 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.162+0000 7f09df7fe640 1 -- 172.21.15.78:0/2863694816 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d8009c30 con 0x7f09e8102570 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.162+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e819d7e0 con 0x7f09e8102570 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.162+0000 7f09df7fe640 1 -- 172.21.15.78:0/2863694816 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d800f6b0 con 0x7f09e8102570 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.162+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e81a50c0 con 0x7f09e8102570 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.163+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e8101200 con 0x7f09e8102570 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.164+0000 7f09df7fe640 1 -- 172.21.15.78:0/2863694816 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f09d80109d0 con 0x7f09e8102570 2024-09-16T00:29:15.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.164+0000 7f09df7fe640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09d40a3240 0x7f09d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:15.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.165+0000 7f09ee22a640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09d40a3240 0x7f09d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:15.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.165+0000 7f09df7fe640 1 -- 172.21.15.78:0/2863694816 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f09d80cf910 con 0x7f09e8102570 2024-09-16T00:29:15.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.165+0000 7f09ee22a640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09d40a3240 0x7f09d40a5700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f09e4005ab0 tx=0x7f09e4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:15.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.168+0000 7f09df7fe640 1 -- 172.21.15.78:0/2863694816 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09d8009de0 con 0x7f09e8102570 2024-09-16T00:29:15.206 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:15.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.383+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09e8109090 con 0x7f09e8102570 2024-09-16T00:29:15.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.383+0000 7f09df7fe640 1 -- 172.21.15.78:0/2863694816 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11986 (secure 0 0 0) 0x7f09d809c270 con 0x7f09e8102570 2024-09-16T00:29:15.384 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:15.384 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"btime":"2024-09-16T00:16:54:575602+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:16:53.846180+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24421,24427,24440,24445,24470]},"id":1}]} 2024-09-16T00:29:15.384 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 34 2024-09-16T00:29:15.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09d40a3240 msgr2=0x7f09d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09d40a3240 0x7f09d40a5700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f09e4005ab0 tx=0x7f09e4005a20 comp rx=0 tx=0).stop 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8102570 msgr2=0x7f09e8198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8102570 0x7f09e8198a80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f09d800d950 tx=0x7f09d800de10 comp rx=0 tx=0).stop 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 shutdown_connections 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f09d40a3240 0x7f09d40a5700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f09e8102ef0 0x7f09e819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09e8102570 0x7f09e8198a80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 --2- 172.21.15.78:0/2863694816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09e8100e20 0x7f09e8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 >> 172.21.15.78:0/2863694816 conn(0x7f09e80fc7d0 msgr2=0x7f09e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 shutdown_connections 2024-09-16T00:29:15.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:15.386+0000 7f09f048e640 1 -- 172.21.15.78:0/2863694816 wait complete. 2024-09-16T00:29:15.865 INFO:teuthology.orchestra.run.smithi078.stderr:progress: 915.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-16T00:29:16.121 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:15 smithi078 ceph-mon[31157]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 313 KiB/s wr, 38 op/s 2024-09-16T00:29:16.121 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:15 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2863694816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:16.149 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 5c7e811e --release --await' 2024-09-16T00:29:16.149 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 5c7e811e --release --await 2024-09-16T00:29:16.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 -- 172.21.15.78:0/3239730617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4106f00 msgr2=0x7f41b410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/3239730617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4106f00 0x7f41b410ea80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009ca0 tx=0x7f41ac02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 -- 172.21.15.78:0/3239730617 shutdown_connections 2024-09-16T00:29:16.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/3239730617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4106f00 0x7f41b410ea80 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/3239730617 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b4102780 0x7f41b41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/3239730617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b4101dd0 0x7f41b41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 -- 172.21.15.78:0/3239730617 >> 172.21.15.78:0/3239730617 conn(0x7f41b40fd8b0 msgr2=0x7f41b40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 -- 172.21.15.78:0/3239730617 shutdown_connections 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.339+0000 7f41b9cb2640 1 -- 172.21.15.78:0/3239730617 wait complete. 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b9cb2640 1 Processor -- start 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b9cb2640 1 -- start start 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b9cb2640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4101dd0 0x7f41b41a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b9cb2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b4102780 0x7f41b41a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b9cb2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b4106f00 0x7f41b419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b9cb2640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4112000 con 0x7f41b4101dd0 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b9cb2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4111e80 con 0x7f41b4106f00 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b9cb2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4112180 con 0x7f41b4102780 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4101dd0 0x7f41b41a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4101dd0 0x7f41b41a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33048/0 (socket says 172.21.15.78:33048) 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.340+0000 7f41b37fe640 1 -- 172.21.15.78:0/750156765 learned_addr learned my addr 172.21.15.78:0/750156765 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b4102780 0x7f41b41a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b37fe640 1 -- 172.21.15.78:0/750156765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b4102780 msgr2=0x7f41b41a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b37fe640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b4102780 0x7f41b41a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b37fe640 1 -- 172.21.15.78:0/750156765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b4106f00 msgr2=0x7f41b419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b3fff640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b4106f00 0x7f41b419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b37fe640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b4106f00 0x7f41b419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b37fe640 1 -- 172.21.15.78:0/750156765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b419acc0 con 0x7f41b4101dd0 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b3fff640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b4106f00 0x7f41b419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b37fe640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4101dd0 0x7f41b41a03d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f41a400b4a0 tx=0x7f41a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b0ff9640 1 -- 172.21.15.78:0/750156765 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4014070 con 0x7f41b4101dd0 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b9cb2640 1 -- 172.21.15.78:0/750156765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b419afb0 con 0x7f41b4101dd0 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b0ff9640 1 -- 172.21.15.78:0/750156765 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4010af0 con 0x7f41b4101dd0 2024-09-16T00:29:16.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b0ff9640 1 -- 172.21.15.78:0/750156765 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4017d10 con 0x7f41b4101dd0 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.341+0000 7f41b9cb2640 1 -- 172.21.15.78:0/750156765 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b41a72a0 con 0x7f41b4101dd0 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.343+0000 7f41b0ff9640 1 -- 172.21.15.78:0/750156765 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f41a4017070 con 0x7f41b4101dd0 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41bc08b740 1 Processor -- start 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41bc08b740 1 -- start start 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x558508912090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x558508912a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x55850891f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41bc08b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5585088782f0 con 0x55850891d370 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41bc08b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55850881fcf0 con 0x5585089125d0 2024-09-16T00:29:16.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41bc08b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5585088742b0 con 0x558508913b40 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41b0ff9640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f419c0a32c0 0x7f419c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41b2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x55850891f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41b2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x55850891f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33058/0 (socket says 172.21.15.78:33058) 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41b2ffd640 1 -- 172.21.15.78:0/3931321161 learned_addr learned my addr 172.21.15.78:0/3931321161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.344+0000 7f41b3fff640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f419c0a32c0 0x7f419c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b0ff9640 1 -- 172.21.15.78:0/750156765 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f41a40cf070 con 0x7f41b4101dd0 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b37fe640 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x558508912a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b3fff640 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x558508912090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b2ffd640 1 -- 172.21.15.78:0/3931321161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 msgr2=0x558508912090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x558508912090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b2ffd640 1 -- 172.21.15.78:0/3931321161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 msgr2=0x558508912a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x558508912a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b2ffd640 1 -- 172.21.15.78:0/3931321161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585088043a0 con 0x55850891d370 2024-09-16T00:29:16.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b3fff640 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x558508912090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b37fe640 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x558508912a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x55850891f760 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f41a800b7c0 tx=0x7f41a800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3cadc50c78c6c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f41b3fff640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f419c0a32c0 0x7f419c0a5780 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f41ac006010 tx=0x7f41ac005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f4199ffb640 1 -- 172.21.15.78:0/3931321161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a800d950 con 0x55850891d370 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f4199ffb640 1 -- 172.21.15.78:0/3931321161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a8018070 con 0x55850891d370 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.345+0000 7f4199ffb640 1 -- 172.21.15.78:0/3931321161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a80123e0 con 0x55850891d370 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 -- 172.21.15.78:0/3931321161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 msgr2=0x55850891f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x55850891f760 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f41a800b7c0 tx=0x7f41a800bc80 comp rx=0 tx=0).stop 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 -- 172.21.15.78:0/3931321161 shutdown_connections 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x55850891f760 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x558508912a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 --2- 172.21.15.78:0/3931321161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x558508912090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 -- 172.21.15.78:0/3931321161 >> 172.21.15.78:0/3931321161 conn(0x5585088db310 msgr2=0x55850890ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 -- 172.21.15.78:0/3931321161 shutdown_connections 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 -- 172.21.15.78:0/3931321161 wait complete. 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.346+0000 7f41bc08b740 1 -- start start 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41bc08b740 1 Processor -- start 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x55850891cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558508913b40 0x558508919f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850891d370 0x55850891a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41bc08b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558508920070 con 0x558508913b40 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41bc08b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558508833060 con 0x5585089125d0 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41bc08b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5585089201f0 con 0x55850891d370 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41bc08b740 10 client.0 _ll_register_callbacks cb 0x5585085868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558508913b40 0x558508919f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b2ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558508913b40 0x558508919f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33068/0 (socket says 172.21.15.78:33068) 2024-09-16T00:29:16.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b2ffd640 1 -- 172.21.15.78:0/2620616907 learned_addr learned my addr 172.21.15.78:0/2620616907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b37fe640 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x55850891cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b2ffd640 1 -- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850891d370 msgr2=0x55850891a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b3fff640 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850891d370 0x55850891a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850891d370 0x55850891a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b2ffd640 1 -- 172.21.15.78:0/2620616907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 msgr2=0x55850891cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x55850891cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b2ffd640 1 -- 172.21.15.78:0/2620616907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585085e93c0 con 0x558508913b40 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.347+0000 7f41b3fff640 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850891d370 0x55850891a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558508913b40 0x558508919f40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f41a8010040 tx=0x7f41a8013a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6799608feec14c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f4192ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f4192ffd640 1 -- 172.21.15.78:0/2620616907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a800db50 con 0x558508913b40 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f4192ffd640 1 -- 172.21.15.78:0/2620616907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f41a800dcf0 con 0x558508913b40 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 1 -- 172.21.15.78:0/2620616907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a8018070 con 0x558508913b40 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f4192ffd640 1 -- 172.21.15.78:0/2620616907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f41a802e050 con 0x558508913b40 2024-09-16T00:29:16.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 10 client.18240 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 1 -- 172.21.15.78:0/2620616907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55850892b9d0 con 0x558508913b40 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f4192ffd640 1 -- 172.21.15.78:0/2620616907 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a8018070 con 0x558508913b40 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f4192ffd640 1 -- 172.21.15.78:0/2620616907 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f41a801c580 con 0x558508913b40 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 10 client.18240 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 10 client.18240 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 20 client.18240 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55850892cdd0 0x55850892f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 4 client.18240 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.348+0000 7f41bc08b740 1 -- 172.21.15.78:0/2620616907 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558508885860 con 0x55850892cdd0 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.349+0000 7f41b37fe640 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55850892cdd0 0x55850892f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.349+0000 7f41b37fe640 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55850892cdd0 0x55850892f1c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dee57ba1dc9d17f4 server_cookie=bc80b07cd785c9ae in_seq=0 out_seq=0 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.349+0000 7f4192ffd640 10 client.18240 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f4192ffd640 1 -- 172.21.15.78:0/2620616907 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558508885860 con 0x55850892cdd0 2024-09-16T00:29:16.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f4192ffd640 10 client.18240 handle_command_reply: tid=0 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 client.18240 shutdown 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 -- 172.21.15.78:0/2620616907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558508913b40 msgr2=0x558508919f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558508913b40 0x558508919f40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f41a8010040 tx=0x7f41a8013a60 comp rx=0 tx=0).stop 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 -- 172.21.15.78:0/2620616907 shutdown_connections 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x55850892cdd0 0x55850892f1c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850891d370 0x55850891a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558508913b40 0x558508919f40 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f4192ffd640 0 client.18240 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2620616907 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5585089125d0 0x55850891cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 -- 172.21.15.78:0/2620616907 >> 172.21.15.78:0/2620616907 conn(0x5585088db310 msgr2=0x558508914960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 -- 172.21.15.78:0/2620616907 shutdown_connections 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 1 -- 172.21.15.78:0/2620616907 wait complete. 2024-09-16T00:29:16.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.350+0000 7f41bc08b740 20 client.18240 trim_cache size 0 max 16384 2024-09-16T00:29:16.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.363+0000 7f41bc08b740 1 Processor -- start 2024-09-16T00:29:16.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.363+0000 7f41bc08b740 1 -- start start 2024-09-16T00:29:16.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585089125d0 0x558508929340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x55850891e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55850891d370 0x55850891eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41bc08b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41b41b0fb0 con 0x5585089125d0 2024-09-16T00:29:16.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41bc08b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41b41b2af0 con 0x55850891d370 2024-09-16T00:29:16.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41bc08b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41b41b1680 con 0x558508913b40 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585089125d0 0x558508929340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585089125d0 0x558508929340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33078/0 (socket says 172.21.15.78:33078) 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 -- 172.21.15.78:0/2306932543 learned_addr learned my addr 172.21.15.78:0/2306932543 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/2306932543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x55850891e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 -- 172.21.15.78:0/2306932543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 msgr2=0x55850891e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 --2- 172.21.15.78:0/2306932543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x55850891e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 -- 172.21.15.78:0/2306932543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55850891d370 msgr2=0x55850891eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 --2- 172.21.15.78:0/2306932543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55850891d370 0x55850891eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 -- 172.21.15.78:0/2306932543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850890f960 con 0x5585089125d0 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.364+0000 7f41b37fe640 1 --2- 172.21.15.78:0/2306932543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585089125d0 0x558508929340 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f41a4017fd0 tx=0x7f41a40bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947c3b4e7e47c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41997fa640 1 -- 172.21.15.78:0/2306932543 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40b3070 con 0x5585089125d0 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41997fa640 1 -- 172.21.15.78:0/2306932543 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4099370 con 0x5585089125d0 2024-09-16T00:29:16.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41997fa640 1 -- 172.21.15.78:0/2306932543 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a401e040 con 0x5585089125d0 2024-09-16T00:29:16.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41bc08b740 1 -- 172.21.15.78:0/2306932543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585089125d0 msgr2=0x558508929340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2306932543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585089125d0 0x558508929340 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f41a4017fd0 tx=0x7f41a40bb040 comp rx=0 tx=0).stop 2024-09-16T00:29:16.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41bc08b740 1 -- 172.21.15.78:0/2306932543 shutdown_connections 2024-09-16T00:29:16.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2306932543 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55850891d370 0x55850891eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2306932543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558508913b40 0x55850891e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41bc08b740 1 --2- 172.21.15.78:0/2306932543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5585089125d0 0x558508929340 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41bc08b740 1 -- 172.21.15.78:0/2306932543 >> 172.21.15.78:0/2306932543 conn(0x5585088db310 msgr2=0x558508919f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.365+0000 7f41bc08b740 1 -- 172.21.15.78:0/2306932543 shutdown_connections 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 -- 172.21.15.78:0/2306932543 wait complete. 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 -- start start 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 Processor -- start 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585089125d0 0x558508933970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558508913b40 0x558508933eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x5585089244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f41b41b2420 con 0x55850891d370 2024-09-16T00:29:16.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f41b41b1d50 con 0x558508913b40 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41b41aef90 con 0x5585089125d0 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.366+0000 7f41bc08b740 10 client.0 _ll_register_callbacks cb 0x7f41a800b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x5585089244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x5585089244b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33086/0 (socket says 172.21.15.78:33086) 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 -- 172.21.15.78:0/218275645 learned_addr learned my addr 172.21.15.78:0/218275645 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b37fe640 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585089125d0 0x558508933970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b2ffd640 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558508913b40 0x558508933eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 -- 172.21.15.78:0/218275645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585089125d0 msgr2=0x558508933970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585089125d0 0x558508933970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 -- 172.21.15.78:0/218275645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558508913b40 msgr2=0x558508933eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558508913b40 0x558508933eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 -- 172.21.15.78:0/218275645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585089249f0 con 0x55850891d370 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41b3fff640 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x5585089244b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f41ac035000 tx=0x7f41ac02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f83cb494d295f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:16.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f4192ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f4192ffd640 1 -- 172.21.15.78:0/218275645 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac04d070 con 0x55850891d370 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f41bc08b740 1 -- 172.21.15.78:0/218275645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a4099370 con 0x55850891d370 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.367+0000 7f4192ffd640 1 -- 172.21.15.78:0/218275645 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f41ac0046c0 con 0x55850891d370 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f4192ffd640 1 -- 172.21.15.78:0/218275645 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f41ac059050 con 0x55850891d370 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f41bc08b740 10 client.18246 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f41bc08b740 1 -- 172.21.15.78:0/218275645 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558508925d40 con 0x55850891d370 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f4192ffd640 1 -- 172.21.15.78:0/218275645 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4099370 con 0x55850891d370 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f4192ffd640 1 -- 172.21.15.78:0/218275645 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f41ac048040 con 0x55850891d370 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f41bc08b740 10 client.18246 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f41bc08b740 10 client.18246 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f41bc08b740 20 client.18246 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f41bc08b740 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558508997840 0x558508999c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f41bc08b740 4 client.18246 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:29:16.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.368+0000 7f41bc08b740 1 -- 172.21.15.78:0/218275645 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55850887fd70 con 0x558508997840 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41b37fe640 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558508997840 0x558508999c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f4192ffd640 10 client.18246 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41b37fe640 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558508997840 0x558508999c60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b1174fc79e92e902 server_cookie=5a524c40b8d0b109 in_seq=0 out_seq=0 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f4192ffd640 1 -- 172.21.15.78:0/218275645 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55850887fd70 con 0x558508997840 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f4192ffd640 10 client.18246 handle_command_reply: tid=0 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 client.18246 shutdown 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 -- 172.21.15.78:0/218275645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 msgr2=0x5585089244b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x5585089244b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f41ac035000 tx=0x7f41ac02f310 comp rx=0 tx=0).stop 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 -- 172.21.15.78:0/218275645 shutdown_connections 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55850891d370 0x5585089244b0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558508913b40 0x558508933eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x558508997840 0x558508999c60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 --2- 172.21.15.78:0/218275645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5585089125d0 0x558508933970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f41bc08b740 1 -- 172.21.15.78:0/218275645 >> 172.21.15.78:0/218275645 conn(0x5585088db310 msgr2=0x5585089186e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:16.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.369+0000 7f4192ffd640 0 client.18246 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:29:16.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41bc08b740 1 -- 172.21.15.78:0/218275645 shutdown_connections 2024-09-16T00:29:16.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41bc08b740 1 -- 172.21.15.78:0/218275645 wait complete. 2024-09-16T00:29:16.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41bc08b740 20 client.18246 trim_cache size 0 max 16384 2024-09-16T00:29:16.376 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:29:16.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 -- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f419c0a32c0 msgr2=0x7f419c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f419c0a32c0 0x7f419c0a5780 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f41ac006010 tx=0x7f41ac005ef0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 -- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4101dd0 msgr2=0x7f41b41a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4101dd0 0x7f41b41a03d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f41a400b4a0 tx=0x7f41a400b960 comp rx=0 tx=0).stop 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 -- 172.21.15.78:0/750156765 shutdown_connections 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f419c0a32c0 0x7f419c0a5780 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f41b4106f00 0x7f41b419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41b4102780 0x7f41b41a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 --2- 172.21.15.78:0/750156765 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41b4101dd0 0x7f41b41a03d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.370+0000 7f41b9cb2640 1 -- 172.21.15.78:0/750156765 >> 172.21.15.78:0/750156765 conn(0x7f41b40fd8b0 msgr2=0x7f41b40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.371+0000 7f41b9cb2640 1 -- 172.21.15.78:0/750156765 shutdown_connections 2024-09-16T00:29:16.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:16.371+0000 7f41b9cb2640 1 -- 172.21.15.78:0/750156765 wait complete. 2024-09-16T00:29:16.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:15 smithi193 ceph-mon[35484]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 313 KiB/s wr, 38 op/s 2024-09-16T00:29:16.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:15 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2863694816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:16.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:15 smithi148 ceph-mon[35599]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 313 KiB/s wr, 38 op/s 2024-09-16T00:29:16.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:15 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2863694816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:17.188 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:17.197 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:29:17.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.710+0000 7fbf27f34640 1 -- 172.21.15.78:0/4096031465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 msgr2=0x7fbf20105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.710+0000 7fbf27f34640 1 --2- 172.21.15.78:0/4096031465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 0x7fbf20105930 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009900 tx=0x7fbf1002a470 comp rx=0 tx=0).stop 2024-09-16T00:29:17.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.710+0000 7fbf27f34640 1 -- 172.21.15.78:0/4096031465 shutdown_connections 2024-09-16T00:29:17.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.710+0000 7fbf27f34640 1 --2- 172.21.15.78:0/4096031465 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf20105f50 0x7fbf201063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.710+0000 7fbf27f34640 1 --2- 172.21.15.78:0/4096031465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 0x7fbf20105930 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.710+0000 7fbf27f34640 1 --2- 172.21.15.78:0/4096031465 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf20100b40 0x7fbf20100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.710+0000 7fbf27f34640 1 -- 172.21.15.78:0/4096031465 >> 172.21.15.78:0/4096031465 conn(0x7fbf200fc7d0 msgr2=0x7fbf200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:17.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.710+0000 7fbf27f34640 1 -- 172.21.15.78:0/4096031465 shutdown_connections 2024-09-16T00:29:17.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.711+0000 7fbf27f34640 1 -- 172.21.15.78:0/4096031465 wait complete. 2024-09-16T00:29:17.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.711+0000 7fbf27f34640 1 Processor -- start 2024-09-16T00:29:17.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.711+0000 7fbf27f34640 1 -- start start 2024-09-16T00:29:17.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf27f34640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf20100b40 0x7fbf201a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf27f34640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 0x7fbf201a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf27f34640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf20105f50 0x7fbf2019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf27f34640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201185d0 con 0x7fbf20101460 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf27f34640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf20118450 con 0x7fbf20100b40 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf27f34640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf20118750 con 0x7fbf20105f50 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf254cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 0x7fbf201a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf25cd0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf20100b40 0x7fbf201a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf254cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 0x7fbf201a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33100/0 (socket says 172.21.15.78:33100) 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf254cf640 1 -- 172.21.15.78:0/3742277618 learned_addr learned my addr 172.21.15.78:0/3742277618 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf254cf640 1 -- 172.21.15.78:0/3742277618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf20105f50 msgr2=0x7fbf2019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf254cf640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf20105f50 0x7fbf2019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf254cf640 1 -- 172.21.15.78:0/3742277618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf20100b40 msgr2=0x7fbf201a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf254cf640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf20100b40 0x7fbf201a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.712+0000 7fbf254cf640 1 -- 172.21.15.78:0/3742277618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2019f040 con 0x7fbf20101460 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.713+0000 7fbf254cf640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 0x7fbf201a4c90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005ab0 tx=0x7fbf100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:17.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.713+0000 7fbf16ffd640 1 -- 172.21.15.78:0/3742277618 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1003f030 con 0x7fbf20101460 2024-09-16T00:29:17.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.713+0000 7fbf27f34640 1 -- 172.21.15.78:0/3742277618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2019f2d0 con 0x7fbf20101460 2024-09-16T00:29:17.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.713+0000 7fbf16ffd640 1 -- 172.21.15.78:0/3742277618 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf10003ce0 con 0x7fbf20101460 2024-09-16T00:29:17.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.713+0000 7fbf16ffd640 1 -- 172.21.15.78:0/3742277618 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10035070 con 0x7fbf20101460 2024-09-16T00:29:17.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.713+0000 7fbf27f34640 1 -- 172.21.15.78:0/3742277618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf201ab620 con 0x7fbf20101460 2024-09-16T00:29:17.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.715+0000 7fbf16ffd640 1 -- 172.21.15.78:0/3742277618 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fbf10042030 con 0x7fbf20101460 2024-09-16T00:29:17.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.716+0000 7fbf16ffd640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbf0c0a3240 0x7fbf0c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.716+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbf0c0a3240 0x7fbf0c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.716+0000 7fbf2a30d740 1 Processor -- start 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.716+0000 7fbf16ffd640 1 -- 172.21.15.78:0/3742277618 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbf100f0480 con 0x7fbf20101460 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf2a30d740 1 -- start start 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1060 0x5638258a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258ac240 0x5638258ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563825806620 con 0x5638258a1980 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5638257ade50 con 0x5638258ac240 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5638258025e0 con 0x5638258a1060 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbf0c0a3240 0x7fbf0c0a5700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbf2019fa50 tx=0x7fbf1c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33104/0 (socket says 172.21.15.78:33104) 2024-09-16T00:29:17.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 -- 172.21.15.78:0/877961162 learned_addr learned my addr 172.21.15.78:0/877961162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1060 0x5638258a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf264d1640 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258ac240 0x5638258ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 -- 172.21.15.78:0/877961162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1060 msgr2=0x5638258a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1060 0x5638258a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 -- 172.21.15.78:0/877961162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258ac240 msgr2=0x5638258ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258ac240 0x5638258ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 -- 172.21.15.78:0/877961162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382579ed40 con 0x5638258a1980 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1060 0x5638258a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf264d1640 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258ac240 0x5638258ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:17.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.717+0000 7fbf254cf640 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258abd00 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbf100c4e80 tx=0x7fbf1003b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a67a3280a8836c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.718+0000 7fbf0bfff640 1 -- 172.21.15.78:0/877961162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10049070 con 0x5638258a1980 2024-09-16T00:29:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.718+0000 7fbf0bfff640 1 -- 172.21.15.78:0/877961162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf100f0070 con 0x5638258a1980 2024-09-16T00:29:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.718+0000 7fbf0bfff640 1 -- 172.21.15.78:0/877961162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100baa40 con 0x5638258a1980 2024-09-16T00:29:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.718+0000 7fbf2a30d740 1 -- 172.21.15.78:0/877961162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 msgr2=0x5638258abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.718+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258abd00 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbf100c4e80 tx=0x7fbf1003b720 comp rx=0 tx=0).stop 2024-09-16T00:29:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 -- 172.21.15.78:0/877961162 shutdown_connections 2024-09-16T00:29:17.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258ac240 0x5638258ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258abd00 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/877961162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1060 0x5638258a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 -- 172.21.15.78:0/877961162 >> 172.21.15.78:0/877961162 conn(0x56382589d5f0 msgr2=0x56382589d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 -- 172.21.15.78:0/877961162 shutdown_connections 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 -- 172.21.15.78:0/877961162 wait complete. 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 -- start start 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.719+0000 7fbf2a30d740 1 Processor -- start 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1060 0x5638258a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1980 0x5638258a9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5638258aebd0 con 0x5638258a1060 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56382578ef10 con 0x5638258a1980 2024-09-16T00:29:17.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5638258aed50 con 0x5638258ac240 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf2a30d740 10 client.0 _ll_register_callbacks cb 0x5638257b2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf25cd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1060 0x5638258a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf25cd0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1060 0x5638258a91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33112/0 (socket says 172.21.15.78:33112) 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf25cd0640 1 -- 172.21.15.78:0/759470331 learned_addr learned my addr 172.21.15.78:0/759470331 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf264d1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf254cf640 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1980 0x5638258a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf25cd0640 1 -- 172.21.15.78:0/759470331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 msgr2=0x5638258a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf25cd0640 1 -- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1980 msgr2=0x5638258a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1980 0x5638258a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf25cd0640 1 -- 172.21.15.78:0/759470331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563825515580 con 0x5638258a1060 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf254cf640 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1980 0x5638258a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:17.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.720+0000 7fbf264d1640 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1060 0x5638258a91f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00e7c0 tx=0x7fbf1c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fd70ac83ec841f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf097fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf097fa640 1 -- 172.21.15.78:0/759470331 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c025020 con 0x5638258a1060 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf097fa640 1 -- 172.21.15.78:0/759470331 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbf1c0139d0 con 0x5638258a1060 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf2a30d740 1 -- 172.21.15.78:0/759470331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf100f0070 con 0x5638258a1060 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf097fa640 1 -- 172.21.15.78:0/759470331 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbf1c030050 con 0x5638258a1060 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf2a30d740 10 client.18258 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf2a30d740 1 -- 172.21.15.78:0/759470331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5638255df960 con 0x5638258a1060 2024-09-16T00:29:17.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.721+0000 7fbf097fa640 1 -- 172.21.15.78:0/759470331 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf100f0070 con 0x5638258a1060 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.722+0000 7fbf097fa640 1 -- 172.21.15.78:0/759470331 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbf1c021cc0 con 0x5638258a1060 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.722+0000 7fbf2a30d740 10 client.18258 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.722+0000 7fbf2a30d740 10 client.18258 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.722+0000 7fbf2a30d740 20 client.18258 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.722+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5638258bbc50 0x5638258be040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.722+0000 7fbf2a30d740 4 client.18258 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.722+0000 7fbf2a30d740 1 -- 172.21.15.78:0/759470331 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563825813b90 con 0x5638258bbc50 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.722+0000 7fbf254cf640 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5638258bbc50 0x5638258be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.723+0000 7fbf097fa640 10 client.18258 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.723+0000 7fbf254cf640 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5638258bbc50 0x5638258be040 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fc2d462815b07 server_cookie=edac0765bb86d9c4 in_seq=0 out_seq=0 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.723+0000 7fbf097fa640 1 -- 172.21.15.78:0/759470331 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563825813b90 con 0x5638258bbc50 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.723+0000 7fbf097fa640 10 client.18258 handle_command_reply: tid=0 2024-09-16T00:29:17.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 client.18258 shutdown 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 -- 172.21.15.78:0/759470331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1060 msgr2=0x5638258a91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1060 0x5638258a91f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00e7c0 tx=0x7fbf1c0047d0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 -- 172.21.15.78:0/759470331 shutdown_connections 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5638258bbc50 0x5638258be040 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf097fa640 0 client.18258 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1980 0x5638258a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/759470331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1060 0x5638258a91f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 -- 172.21.15.78:0/759470331 >> 172.21.15.78:0/759470331 conn(0x56382589d5f0 msgr2=0x5638258a3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 -- 172.21.15.78:0/759470331 shutdown_connections 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 1 -- 172.21.15.78:0/759470331 wait complete. 2024-09-16T00:29:17.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.724+0000 7fbf2a30d740 20 client.18258 trim_cache size 0 max 16384 2024-09-16T00:29:17.738 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.742+0000 7fbf2a30d740 1 Processor -- start 2024-09-16T00:29:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.742+0000 7fbf2a30d740 1 -- start start 2024-09-16T00:29:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.742+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.742+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1980 0x5638258c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.742+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258ac240 0x5638258a6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.742+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201b5280 con 0x5638258ac240 2024-09-16T00:29:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.742+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201b6dc0 con 0x5638258a1060 2024-09-16T00:29:17.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201b5950 con 0x5638258a1980 2024-09-16T00:29:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258ac240 0x5638258a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258ac240 0x5638258a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33118/0 (socket says 172.21.15.78:33118) 2024-09-16T00:29:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 -- 172.21.15.78:0/1029803358 learned_addr learned my addr 172.21.15.78:0/1029803358 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf254cf640 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1980 0x5638258c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 -- 172.21.15.78:0/1029803358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1980 msgr2=0x5638258c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1980 0x5638258c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 -- 172.21.15.78:0/1029803358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 msgr2=0x5638258c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 -- 172.21.15.78:0/1029803358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56382589e280 con 0x5638258ac240 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf264d1640 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258ac240 0x5638258a6490 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbf18011540 tx=0x7fbf18011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcefc2880ddd8813 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf0b7fe640 1 -- 172.21.15.78:0/1029803358 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18004d20 con 0x5638258ac240 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf0b7fe640 1 -- 172.21.15.78:0/1029803358 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf1800fc50 con 0x5638258ac240 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.743+0000 7fbf0b7fe640 1 -- 172.21.15.78:0/1029803358 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180059a0 con 0x5638258ac240 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 -- 172.21.15.78:0/1029803358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258ac240 msgr2=0x5638258a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258ac240 0x5638258a6490 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbf18011540 tx=0x7fbf18011940 comp rx=0 tx=0).stop 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 -- 172.21.15.78:0/1029803358 shutdown_connections 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258ac240 0x5638258a6490 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258a1980 0x5638258c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/1029803358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 -- 172.21.15.78:0/1029803358 >> 172.21.15.78:0/1029803358 conn(0x56382589d5f0 msgr2=0x5638258a99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 -- 172.21.15.78:0/1029803358 shutdown_connections 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.744+0000 7fbf2a30d740 1 -- 172.21.15.78:0/1029803358 wait complete. 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 1 -- start start 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 1 Processor -- start 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201b66f0 con 0x5638258a1980 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201b6020 con 0x5638258a1060 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201b3260 con 0x5638258ac240 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf2a30d740 10 client.0 _ll_register_callbacks cb 0x7fbf1c0156b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf254cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf25cd0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf254cf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33124/0 (socket says 172.21.15.78:33124) 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf254cf640 1 -- 172.21.15.78:0/3929637161 learned_addr learned my addr 172.21.15.78:0/3929637161 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.745+0000 7fbf264d1640 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf254cf640 1 -- 172.21.15.78:0/3929637161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 msgr2=0x5638258b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf254cf640 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf254cf640 1 -- 172.21.15.78:0/3929637161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 msgr2=0x5638258b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf254cf640 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf254cf640 1 -- 172.21.15.78:0/3929637161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638258b1370 con 0x5638258a1980 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf264d1640 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf254cf640 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258b43a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbf100edc70 tx=0x7fbf1002a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef403b6086f752f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf097fa640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf097fa640 1 -- 172.21.15.78:0/3929637161 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100d9070 con 0x5638258a1980 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf097fa640 1 -- 172.21.15.78:0/3929637161 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fbf100f01f0 con 0x5638258a1980 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf2a30d740 1 -- 172.21.15.78:0/3929637161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf1800fc50 con 0x5638258a1980 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf097fa640 1 -- 172.21.15.78:0/3929637161 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbf10049040 con 0x5638258a1980 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf2a30d740 10 client.18264 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.746+0000 7fbf2a30d740 1 -- 172.21.15.78:0/3929637161 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5638259270c0 con 0x5638258a1980 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf097fa640 1 -- 172.21.15.78:0/3929637161 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf1800fc50 con 0x5638258a1980 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf097fa640 1 -- 172.21.15.78:0/3929637161 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fbf100b8db0 con 0x5638258a1980 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf2a30d740 10 client.18264 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf2a30d740 10 client.18264 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:29:17.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf2a30d740 20 client.18264 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5638258bbc50 0x563825929660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf2a30d740 4 client.18264 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5c7e811e", "release": true, "await": true}] 2024-09-16T00:29:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf2a30d740 1 -- 172.21.15.78:0/3929637161 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5c7e811e", "release": true, "await": true}) -- 0x56382580e0a0 con 0x5638258bbc50 2024-09-16T00:29:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5638258bbc50 0x563825929660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf25cd0640 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5638258bbc50 0x563825929660 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d012168366dea2d server_cookie=2788fe9407dffea2 in_seq=0 out_seq=0 2024-09-16T00:29:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.747+0000 7fbf097fa640 10 client.18264 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:17.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.811+0000 7fbf097fa640 1 -- 172.21.15.78:0/3929637161 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56382580e0a0 con 0x5638258bbc50 2024-09-16T00:29:17.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.811+0000 7fbf097fa640 10 client.18264 handle_command_reply: tid=0 2024-09-16T00:29:17.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.811+0000 7fbf2a30d740 1 client.18264 shutdown 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 -- 172.21.15.78:0/3929637161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 msgr2=0x5638258b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258b43a0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbf100edc70 tx=0x7fbf1002a470 comp rx=0 tx=0).stop 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 -- 172.21.15.78:0/3929637161 shutdown_connections 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5638258bbc50 0x563825929660 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5638258ac240 0x5638258b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf097fa640 0 client.18264 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5638258a1980 0x5638258b43a0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 --2- 172.21.15.78:0/3929637161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5638258a1060 0x5638258b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 -- 172.21.15.78:0/3929637161 >> 172.21.15.78:0/3929637161 conn(0x56382589d5f0 msgr2=0x5638258a99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:17.814 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:29:17.814 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 34, 2024-09-16T00:29:17.814 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:29:17.814 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 60, 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": { 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "5c7e811e": { 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "version": 60, 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "age_ref": 0.0, 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "timeout": 90.0, 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "expiration": 137.7, 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "members": { 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "file:/": { 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "excluded": false, 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "state": { 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RELEASED", 2024-09-16T00:29:17.815 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 -- 172.21.15.78:0/3929637161 shutdown_connections 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 1 -- 172.21.15.78:0/3929637161 wait complete. 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.812+0000 7fbf2a30d740 20 client.18264 trim_cache size 0 max 16384 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.813+0000 7fbf27f34640 1 -- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbf0c0a3240 msgr2=0x7fbf0c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.813+0000 7fbf27f34640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbf0c0a3240 0x7fbf0c0a5700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fbf2019fa50 tx=0x7fbf1c009110 comp rx=0 tx=0).stop 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.813+0000 7fbf27f34640 1 -- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 msgr2=0x7fbf201a4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:17.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.813+0000 7fbf27f34640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 0x7fbf201a4c90 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbf10005ab0 tx=0x7fbf100042f0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.813+0000 7fbf27f34640 1 -- 172.21.15.78:0/3742277618 shutdown_connections 2024-09-16T00:29:17.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.814+0000 7fbf27f34640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fbf0c0a3240 0x7fbf0c0a5700 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.814+0000 7fbf27f34640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf20105f50 0x7fbf2019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.814+0000 7fbf27f34640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf20101460 0x7fbf201a4c90 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.814+0000 7fbf27f34640 1 --2- 172.21.15.78:0/3742277618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf20100b40 0x7fbf201a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:17.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.814+0000 7fbf27f34640 1 -- 172.21.15.78:0/3742277618 >> 172.21.15.78:0/3742277618 conn(0x7fbf200fc7d0 msgr2=0x7fbf2010f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:17.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.814+0000 7fbf27f34640 1 -- 172.21.15.78:0/3742277618 shutdown_connections 2024-09-16T00:29:17.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:17.814+0000 7fbf27f34640 1 -- 172.21.15.78:0/3742277618 wait complete. 2024-09-16T00:29:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:17 smithi078 ceph-mon[31157]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 251 KiB/s wr, 32 op/s 2024-09-16T00:29:18.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:17 smithi193 ceph-mon[35484]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 251 KiB/s wr, 32 op/s 2024-09-16T00:29:18.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:17 smithi148 ceph-mon[35599]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 251 KiB/s wr, 32 op/s 2024-09-16T00:29:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 -- 172.21.15.78:0/646465949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3844101df0 msgr2=0x7f38441021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 --2- 172.21.15.78:0/646465949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3844101df0 0x7f38441021d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3834009ca0 tx=0x7f383402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 -- 172.21.15.78:0/646465949 shutdown_connections 2024-09-16T00:29:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 --2- 172.21.15.78:0/646465949 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3844106f20 0x7f384410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 --2- 172.21.15.78:0/646465949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38441027a0 0x7f38441067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 --2- 172.21.15.78:0/646465949 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3844101df0 0x7f38441021d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 -- 172.21.15.78:0/646465949 >> 172.21.15.78:0/646465949 conn(0x7f38440fd8f0 msgr2=0x7f38440ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 -- 172.21.15.78:0/646465949 shutdown_connections 2024-09-16T00:29:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.518+0000 7f38490cc640 1 -- 172.21.15.78:0/646465949 wait complete. 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.520+0000 7f38490cc640 1 Processor -- start 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.520+0000 7f38490cc640 1 -- start start 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.520+0000 7f38490cc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3844101df0 0x7f3844198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.520+0000 7f38490cc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38441027a0 0x7f3844198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f38490cc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3844106f20 0x7f384419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f38490cc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3844111fd0 con 0x7f38441027a0 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f38490cc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3844111e50 con 0x7f3844106f20 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f38490cc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3844112150 con 0x7f3844101df0 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f3842d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3844101df0 0x7f3844198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f3842d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3844101df0 0x7f3844198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:41208/0 (socket says 172.21.15.78:41208) 2024-09-16T00:29:18.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f384259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38441027a0 0x7f3844198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f3842d9d640 1 -- 172.21.15.78:0/2991499846 learned_addr learned my addr 172.21.15.78:0/2991499846 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f384359e640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3844106f20 0x7f384419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f3842d9d640 1 -- 172.21.15.78:0/2991499846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3844106f20 msgr2=0x7f384419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f3842d9d640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3844106f20 0x7f384419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f3842d9d640 1 -- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38441027a0 msgr2=0x7f3844198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f3842d9d640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38441027a0 0x7f3844198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f3842d9d640 1 -- 172.21.15.78:0/2991499846 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384419d3e0 con 0x7f3844101df0 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f384259c640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38441027a0 0x7f3844198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.521+0000 7f384359e640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3844106f20 0x7f384419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:18.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.522+0000 7f3842d9d640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3844101df0 0x7f3844198430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3834005e30 tx=0x7f3834003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:18.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.522+0000 7f382bfff640 1 -- 172.21.15.78:0/2991499846 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834031650 con 0x7f3844101df0 2024-09-16T00:29:18.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.522+0000 7f38490cc640 1 -- 172.21.15.78:0/2991499846 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384419d670 con 0x7f3844101df0 2024-09-16T00:29:18.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.522+0000 7f382bfff640 1 -- 172.21.15.78:0/2991499846 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38340317f0 con 0x7f3844101df0 2024-09-16T00:29:18.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.522+0000 7f382bfff640 1 -- 172.21.15.78:0/2991499846 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383402fa30 con 0x7f3844101df0 2024-09-16T00:29:18.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.522+0000 7f38490cc640 1 -- 172.21.15.78:0/2991499846 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38441a4fb0 con 0x7f3844101df0 2024-09-16T00:29:18.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.525+0000 7f382bfff640 1 -- 172.21.15.78:0/2991499846 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f383402fc30 con 0x7f3844101df0 2024-09-16T00:29:18.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.525+0000 7f382bfff640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f382c0a3240 0x7f382c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384b4a5740 1 Processor -- start 2024-09-16T00:29:18.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f382bfff640 1 -- 172.21.15.78:0/2991499846 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f38340f0ff0 con 0x7f3844101df0 2024-09-16T00:29:18.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384259c640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f382c0a3240 0x7f382c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384b4a5740 1 -- start start 2024-09-16T00:29:18.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185957e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c1859ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384b4a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c184f12f0 con 0x557c18595db0 2024-09-16T00:29:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384b4a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c18498cf0 con 0x557c1859a970 2024-09-16T00:29:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384b4a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557c184ed2b0 con 0x557c18595400 2024-09-16T00:29:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.526+0000 7f384259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33130/0 (socket says 172.21.15.78:33130) 2024-09-16T00:29:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 -- 172.21.15.78:0/713098663 learned_addr learned my addr 172.21.15.78:0/713098663 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f3842d9d640 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f382c0a3240 0x7f382c0a5700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3844108950 tx=0x7f3838005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384359e640 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c1859ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 -- 172.21.15.78:0/713098663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 msgr2=0x557c185957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 -- 172.21.15.78:0/713098663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 msgr2=0x557c1859ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c1859ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 -- 172.21.15.78:0/713098663 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c1847d3a0 con 0x557c18595db0 2024-09-16T00:29:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f384259c640 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859a350 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f38380042b0 tx=0x7f3838014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64be45c67ce1002 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:18.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f38297fa640 1 -- 172.21.15.78:0/713098663 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383800f480 con 0x557c18595db0 2024-09-16T00:29:18.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f38297fa640 1 -- 172.21.15.78:0/713098663 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383800fb00 con 0x557c18595db0 2024-09-16T00:29:18.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.527+0000 7f38297fa640 1 -- 172.21.15.78:0/713098663 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383801db80 con 0x557c18595db0 2024-09-16T00:29:18.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 -- 172.21.15.78:0/713098663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 msgr2=0x557c1859a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859a350 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f38380042b0 tx=0x7f3838014000 comp rx=0 tx=0).stop 2024-09-16T00:29:18.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 -- 172.21.15.78:0/713098663 shutdown_connections 2024-09-16T00:29:18.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c1859ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859a350 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 --2- 172.21.15.78:0/713098663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 -- 172.21.15.78:0/713098663 >> 172.21.15.78:0/713098663 conn(0x557c18554310 msgr2=0x557c18587660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:18.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 -- 172.21.15.78:0/713098663 shutdown_connections 2024-09-16T00:29:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.528+0000 7f384b4a5740 1 -- 172.21.15.78:0/713098663 wait complete. 2024-09-16T00:29:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.529+0000 7f384b4a5740 1 -- start start 2024-09-16T00:29:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.529+0000 7f384b4a5740 1 Processor -- start 2024-09-16T00:29:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.529+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.529+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c1859e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384b4a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557c185a1890 con 0x557c18595db0 2024-09-16T00:29:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384b4a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557c184ac060 con 0x557c1859a970 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384b4a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557c185a1a10 con 0x557c18595400 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f3842d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f3842d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:41220/0 (socket says 172.21.15.78:41220) 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f3842d9d640 1 -- 172.21.15.78:0/1602337308 learned_addr learned my addr 172.21.15.78:0/1602337308 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384b4a5740 10 client.0 _ll_register_callbacks cb 0x557c181ff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384359e640 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c1859e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384259c640 1 -- 172.21.15.78:0/1602337308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 msgr2=0x557c185a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:18.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384259c640 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384259c640 1 -- 172.21.15.78:0/1602337308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 msgr2=0x557c1859e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:18.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384259c640 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c1859e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384259c640 1 -- 172.21.15.78:0/1602337308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c182623c0 con 0x557c18595db0 2024-09-16T00:29:18.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.530+0000 7f384259c640 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859e040 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3838011ca0 tx=0x7f3838004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98711a20711296ce server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:18.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.531+0000 7f3826ff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:18.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.531+0000 7f3826ff5640 1 -- 172.21.15.78:0/1602337308 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383800fd70 con 0x557c18595db0 2024-09-16T00:29:18.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.531+0000 7f3826ff5640 1 -- 172.21.15.78:0/1602337308 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f383802b040 con 0x557c18595db0 2024-09-16T00:29:18.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.531+0000 7f384b4a5740 1 -- 172.21.15.78:0/1602337308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383800fb00 con 0x557c18595db0 2024-09-16T00:29:18.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.532+0000 7f3826ff5640 1 -- 172.21.15.78:0/1602337308 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f383800f480 con 0x557c18595db0 2024-09-16T00:29:18.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.532+0000 7f384b4a5740 10 client.18276 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:18.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.532+0000 7f384b4a5740 1 -- 172.21.15.78:0/1602337308 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557c185ad2c0 con 0x557c18595db0 2024-09-16T00:29:18.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.532+0000 7f3826ff5640 1 -- 172.21.15.78:0/1602337308 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383801d8c0 con 0x557c18595db0 2024-09-16T00:29:18.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.533+0000 7f3826ff5640 1 -- 172.21.15.78:0/1602337308 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f383801a070 con 0x557c18595db0 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.533+0000 7f384b4a5740 10 client.18276 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.533+0000 7f384b4a5740 10 client.18276 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.533+0000 7f384b4a5740 20 client.18276 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.533+0000 7f384b4a5740 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x557c185ae6c0 0x557c185b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.533+0000 7f384b4a5740 4 client.18276 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.533+0000 7f384b4a5740 1 -- 172.21.15.78:0/1602337308 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f383800fb00 con 0x557c185ae6c0 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.534+0000 7f3842d9d640 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x557c185ae6c0 0x557c185b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.535+0000 7f3842d9d640 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x557c185ae6c0 0x557c185b0ab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10de3ffc9afd0771 server_cookie=dc453959985f1473 in_seq=0 out_seq=0 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.535+0000 7f3826ff5640 10 client.18276 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f3826ff5640 1 -- 172.21.15.78:0/1602337308 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f383800fb00 con 0x557c185ae6c0 2024-09-16T00:29:18.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f3826ff5640 10 client.18276 handle_command_reply: tid=0 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 client.18276 shutdown 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 -- 172.21.15.78:0/1602337308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 msgr2=0x557c1859e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859e040 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3838011ca0 tx=0x7f3838004430 comp rx=0 tx=0).stop 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 -- 172.21.15.78:0/1602337308 shutdown_connections 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x557c185ae6c0 0x557c185b0ab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c1859e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c1859e040 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 --2- 172.21.15.78:0/1602337308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 -- 172.21.15.78:0/1602337308 >> 172.21.15.78:0/1602337308 conn(0x557c18554310 msgr2=0x557c18594a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f3826ff5640 0 client.18276 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.536+0000 7f384b4a5740 1 -- 172.21.15.78:0/1602337308 shutdown_connections 2024-09-16T00:29:18.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.537+0000 7f384b4a5740 1 -- 172.21.15.78:0/1602337308 wait complete. 2024-09-16T00:29:18.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.537+0000 7f384b4a5740 20 client.18276 trim_cache size 0 max 16384 2024-09-16T00:29:18.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.555+0000 7f384b4a5740 1 Processor -- start 2024-09-16T00:29:18.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.555+0000 7f384b4a5740 1 -- start start 2024-09-16T00:29:18.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.555+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185991b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.555+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c18595db0 0x557c1859ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.555+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1859a970 0x557c185b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.555+0000 7f384b4a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38441acb10 con 0x557c1859a970 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.555+0000 7f384b4a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38441ae650 con 0x557c18595db0 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.555+0000 7f384b4a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f38441ad1e0 con 0x557c18595400 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1859a970 0x557c185b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384359e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1859a970 0x557c185b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33150/0 (socket says 172.21.15.78:33150) 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384359e640 1 -- 172.21.15.78:0/784371737 learned_addr learned my addr 172.21.15.78:0/784371737 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384259c640 1 --2- 172.21.15.78:0/784371737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c18595db0 0x557c1859ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384359e640 1 -- 172.21.15.78:0/784371737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 msgr2=0x557c185991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384359e640 1 --2- 172.21.15.78:0/784371737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384359e640 1 -- 172.21.15.78:0/784371737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c18595db0 msgr2=0x557c1859ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384359e640 1 --2- 172.21.15.78:0/784371737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c18595db0 0x557c1859ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.556+0000 7f384359e640 1 -- 172.21.15.78:0/784371737 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c185a1100 con 0x557c1859a970 2024-09-16T00:29:18.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.557+0000 7f384359e640 1 --2- 172.21.15.78:0/784371737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1859a970 0x557c185b36f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f383c009d20 tx=0x7f383c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d555501252a6082a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:18.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.557+0000 7f3828ff9640 1 -- 172.21.15.78:0/784371737 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c009e10 con 0x557c1859a970 2024-09-16T00:29:18.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.557+0000 7f3828ff9640 1 -- 172.21.15.78:0/784371737 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c005500 con 0x557c1859a970 2024-09-16T00:29:18.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.557+0000 7f3828ff9640 1 -- 172.21.15.78:0/784371737 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0117b0 con 0x557c1859a970 2024-09-16T00:29:18.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.558+0000 7f384b4a5740 1 -- 172.21.15.78:0/784371737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1859a970 msgr2=0x557c185b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.558+0000 7f384b4a5740 1 --2- 172.21.15.78:0/784371737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1859a970 0x557c185b36f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f383c009d20 tx=0x7f383c013830 comp rx=0 tx=0).stop 2024-09-16T00:29:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.563+0000 7f384b4a5740 1 -- 172.21.15.78:0/784371737 shutdown_connections 2024-09-16T00:29:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.563+0000 7f384b4a5740 1 --2- 172.21.15.78:0/784371737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c1859a970 0x557c185b36f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.563+0000 7f384b4a5740 1 --2- 172.21.15.78:0/784371737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c18595db0 0x557c1859ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.563+0000 7f384b4a5740 1 --2- 172.21.15.78:0/784371737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.563+0000 7f384b4a5740 1 -- 172.21.15.78:0/784371737 >> 172.21.15.78:0/784371737 conn(0x557c18554310 msgr2=0x557c1859f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.564+0000 7f384b4a5740 1 -- 172.21.15.78:0/784371737 shutdown_connections 2024-09-16T00:29:18.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.568+0000 7f384b4a5740 1 -- 172.21.15.78:0/784371737 wait complete. 2024-09-16T00:29:18.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.568+0000 7f384b4a5740 1 -- start start 2024-09-16T00:29:18.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.568+0000 7f384b4a5740 1 Processor -- start 2024-09-16T00:29:18.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.568+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c185b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f384b4a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c185b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f3842d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f3842d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.78:41242/0 (socket says 172.21.15.78:41242) 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f3842d9d640 1 -- 172.21.15.78:0/3747153503 learned_addr learned my addr 172.21.15.78:0/3747153503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38441adf80 con 0x557c18595db0 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38441ad8b0 con 0x557c1859a970 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f38441aaaf0 con 0x557c18595400 2024-09-16T00:29:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f384b4a5740 10 client.0 _ll_register_callbacks cb 0x7f383402ffc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:18.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f3842d9d640 1 -- 172.21.15.78:0/3747153503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 msgr2=0x557c185b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:18.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f3842d9d640 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c185b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f3842d9d640 1 -- 172.21.15.78:0/3747153503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 msgr2=0x557c185b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:18.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f3842d9d640 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c185b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.569+0000 7f3842d9d640 1 -- 172.21.15.78:0/3747153503 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c18592b00 con 0x557c18595400 2024-09-16T00:29:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.572+0000 7f3842d9d640 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185b1510 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3834009a60 tx=0x7f38340048c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfbef4e064852064 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.573+0000 7f3826ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-16T00:29:18.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.576+0000 7f3826ff5640 1 -- 172.21.15.78:0/3747153503 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38340d6060 con 0x557c18595400 2024-09-16T00:29:18.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.576+0000 7f3826ff5640 1 -- 172.21.15.78:0/3747153503 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f38340bb5b0 con 0x557c18595400 2024-09-16T00:29:18.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.576+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383c005500 con 0x557c18595400 2024-09-16T00:29:18.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.577+0000 7f3826ff5640 1 -- 172.21.15.78:0/3747153503 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f38340db050 con 0x557c18595400 2024-09-16T00:29:18.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.578+0000 7f384b4a5740 10 client.27235 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:18.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.578+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557c18593e20 con 0x557c18595400 2024-09-16T00:29:18.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.578+0000 7f3826ff5640 1 -- 172.21.15.78:0/3747153503 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c005500 con 0x557c18595400 2024-09-16T00:29:18.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.578+0000 7f3826ff5640 1 -- 172.21.15.78:0/3747153503 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f38340bd510 con 0x557c18595400 2024-09-16T00:29:18.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.578+0000 7f384b4a5740 10 client.27235 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:18.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.579+0000 7f384b4a5740 10 client.27235 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:29:18.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.579+0000 7f384b4a5740 20 client.27235 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:18.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.579+0000 7f384b4a5740 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x557c18622260 0x557c186246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:18.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.579+0000 7f384b4a5740 4 client.27235 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:29:18.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.579+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c184fe860 con 0x557c18622260 2024-09-16T00:29:18.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.579+0000 7f384259c640 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x557c18622260 0x557c186246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:18.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.580+0000 7f384259c640 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x557c18622260 0x557c186246c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=585aebab10e201a8 server_cookie=429274f7b031c0ed in_seq=0 out_seq=0 2024-09-16T00:29:18.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.580+0000 7f3826ff5640 10 client.27235 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:29:18.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.580+0000 7f3826ff5640 1 -- 172.21.15.78:0/3747153503 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c184fe860 con 0x557c18622260 2024-09-16T00:29:18.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.580+0000 7f3826ff5640 10 client.27235 handle_command_reply: tid=0 2024-09-16T00:29:18.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.580+0000 7f384b4a5740 1 client.27235 shutdown 2024-09-16T00:29:18.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.581+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 msgr2=0x557c185b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.581+0000 7f384b4a5740 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185b1510 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3834009a60 tx=0x7f38340048c0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.581+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 shutdown_connections 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.581+0000 7f384b4a5740 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x557c18622260 0x557c186246c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.581+0000 7f384b4a5740 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557c1859a970 0x557c185b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.581+0000 7f384b4a5740 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557c18595db0 0x557c185b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.581+0000 7f384b4a5740 1 --2- 172.21.15.78:0/3747153503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c18595400 0x557c185b1510 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.582+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 >> 172.21.15.78:0/3747153503 conn(0x557c18554310 msgr2=0x557c18594a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.582+0000 7f3826ff5640 0 client.27235 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.582+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 shutdown_connections 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.582+0000 7f384b4a5740 1 -- 172.21.15.78:0/3747153503 wait complete. 2024-09-16T00:29:18.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.582+0000 7f384b4a5740 20 client.27235 trim_cache size 0 max 16384 2024-09-16T00:29:18.584 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:29:18.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 -- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f382c0a3240 msgr2=0x7f382c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f382c0a3240 0x7f382c0a5700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3844108950 tx=0x7f3838005f90 comp rx=0 tx=0).stop 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 -- 172.21.15.78:0/2991499846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3844101df0 msgr2=0x7f3844198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3844101df0 0x7f3844198430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3834005e30 tx=0x7f3834003f90 comp rx=0 tx=0).stop 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 -- 172.21.15.78:0/2991499846 shutdown_connections 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f382c0a3240 0x7f382c0a5700 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3844106f20 0x7f384419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38441027a0 0x7f3844198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 --2- 172.21.15.78:0/2991499846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3844101df0 0x7f3844198430 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 -- 172.21.15.78:0/2991499846 >> 172.21.15.78:0/2991499846 conn(0x7f38440fd8f0 msgr2=0x7f38440fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 -- 172.21.15.78:0/2991499846 shutdown_connections 2024-09-16T00:29:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:18.584+0000 7f38490cc640 1 -- 172.21.15.78:0/2991499846 wait complete. 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:transaction type: 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:scaling factor: 32 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:query mode: simple 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:number of clients: 100 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:number of threads: 4 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:duration: 900 s 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:number of transactions actually processed: 821292 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:latency average = 111.743 ms 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:latency stddev = 839.129 ms 2024-09-16T00:29:18.858 INFO:teuthology.orchestra.run.smithi078.stdout:tps = 894.653798 (including connections establishing) 2024-09-16T00:29:18.859 INFO:teuthology.orchestra.run.smithi078.stdout:tps = 894.656090 (excluding connections establishing) 2024-09-16T00:29:18.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-16T00:29:19.122 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5c7e811e', total seconds elapsed: 23.0 2024-09-16T00:29:19.971 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:29:20.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:19 smithi078 ceph-mon[31157]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:29:20.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:19 smithi193 ceph-mon[35484]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:29:20.453 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:19 smithi148 ceph-mon[35599]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-16T00:29:20.635 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:21.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.310+0000 7fe1ce00c640 1 -- 172.21.15.78:0/527002883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8106f20 msgr2=0x7fe1c810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.310+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/527002883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8106f20 0x7fe1c810eaa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009ca0 tx=0x7fe1c002a8f0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.310+0000 7fe1ce00c640 1 -- 172.21.15.78:0/527002883 shutdown_connections 2024-09-16T00:29:21.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.310+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/527002883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8106f20 0x7fe1c810eaa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.310+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/527002883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1c81027a0 0x7fe1c81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.310+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/527002883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1c8101df0 0x7fe1c81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.310+0000 7fe1ce00c640 1 -- 172.21.15.78:0/527002883 >> 172.21.15.78:0/527002883 conn(0x7fe1c80fd8f0 msgr2=0x7fe1c80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:21.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.312+0000 7fe1ce00c640 1 -- 172.21.15.78:0/527002883 shutdown_connections 2024-09-16T00:29:21.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.312+0000 7fe1ce00c640 1 -- 172.21.15.78:0/527002883 wait complete. 2024-09-16T00:29:21.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.313+0000 7fe1ce00c640 1 Processor -- start 2024-09-16T00:29:21.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.313+0000 7fe1ce00c640 1 -- start start 2024-09-16T00:29:21.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.313+0000 7fe1ce00c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8101df0 0x7fe1c8117200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.313+0000 7fe1ce00c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1c81027a0 0x7fe1c8117740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.313+0000 7fe1ce00c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1c8106f20 0x7fe1c81112c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.313+0000 7fe1ce00c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c810fdb0 con 0x7fe1c8106f20 2024-09-16T00:29:21.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.313+0000 7fe1ce00c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c810fc30 con 0x7fe1c81027a0 2024-09-16T00:29:21.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.313+0000 7fe1ce00c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c810ff30 con 0x7fe1c8101df0 2024-09-16T00:29:21.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.314+0000 7fe1c7fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1c8106f20 0x7fe1c81112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.314+0000 7fe1c6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1c81027a0 0x7fe1c8117740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.314+0000 7fe1c6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1c81027a0 0x7fe1c8117740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:43322/0 (socket says 172.21.15.78:43322) 2024-09-16T00:29:21.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.314+0000 7fe1c6ffd640 1 -- 172.21.15.78:0/824756585 learned_addr learned my addr 172.21.15.78:0/824756585 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:21.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.314+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8101df0 0x7fe1c8117200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.317+0000 7fe1c6ffd640 1 -- 172.21.15.78:0/824756585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8101df0 msgr2=0x7fe1c8117200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.317+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8101df0 0x7fe1c8117200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.317+0000 7fe1c6ffd640 1 -- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1c8106f20 msgr2=0x7fe1c81112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.317+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1c8106f20 0x7fe1c81112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.317+0000 7fe1c6ffd640 1 -- 172.21.15.78:0/824756585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c8111b80 con 0x7fe1c81027a0 2024-09-16T00:29:21.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.318+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8101df0 0x7fe1c8117200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:21.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.318+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1c81027a0 0x7fe1c8117740 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800e9f0 tx=0x7fe1b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:21.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.320+0000 7fe1c4ff9640 1 -- 172.21.15.78:0/824756585 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800cd80 con 0x7fe1c81027a0 2024-09-16T00:29:21.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.321+0000 7fe1c4ff9640 1 -- 172.21.15.78:0/824756585 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b8010910 con 0x7fe1c81027a0 2024-09-16T00:29:21.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.321+0000 7fe1c4ff9640 1 -- 172.21.15.78:0/824756585 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8018990 con 0x7fe1c81027a0 2024-09-16T00:29:21.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.321+0000 7fe1ce00c640 1 -- 172.21.15.78:0/824756585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c8111e70 con 0x7fe1c81027a0 2024-09-16T00:29:21.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.321+0000 7fe1ce00c640 1 -- 172.21.15.78:0/824756585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c8071e10 con 0x7fe1c81027a0 2024-09-16T00:29:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.325+0000 7fe1d03e5740 1 Processor -- start 2024-09-16T00:29:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.325+0000 7fe1d03e5740 1 -- start start 2024-09-16T00:29:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.333+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d249b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.333+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d249ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.336+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d249ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.336+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d249ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33184/0 (socket says 172.21.15.78:33184) 2024-09-16T00:29:21.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.338+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 0x5584d24a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.338+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584d24012f0 con 0x5584d249b5d0 2024-09-16T00:29:21.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.338+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5584d23a8cf0 con 0x5584d249cb40 2024-09-16T00:29:21.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.338+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5584d23fd2b0 con 0x5584d24a6370 2024-09-16T00:29:21.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.340+0000 7fe1c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d249b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.340+0000 7fe1c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d249b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:43326/0 (socket says 172.21.15.78:43326) 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.340+0000 7fe1c7fff640 1 -- 172.21.15.78:0/1320551164 learned_addr learned my addr 172.21.15.78:0/1320551164 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.341+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/1320551164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 0x5584d24a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.341+0000 7fe1c4ff9640 1 -- 172.21.15.78:0/824756585 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe1b8018b30 con 0x7fe1c81027a0 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.342+0000 7fe1c4ff9640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1b00a3400 0x7fe1b00a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.343+0000 7fe1c4ff9640 1 -- 172.21.15.78:0/824756585 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe1b80cfed0 con 0x7fe1c81027a0 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.347+0000 7fe1c7fff640 1 -- 172.21.15.78:0/1320551164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 msgr2=0x5584d24a8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.347+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/1320551164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 0x5584d24a8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.347+0000 7fe1c7fff640 1 -- 172.21.15.78:0/1320551164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 msgr2=0x5584d249ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.347+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/1320551164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d249ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.347+0000 7fe1c7fff640 1 -- 172.21.15.78:0/1320551164 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d238d3a0 con 0x5584d249cb40 2024-09-16T00:29:21.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.349+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1b00a3400 0x7fe1b00a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.349+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/1320551164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d249b090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0035000 tx=0x7fe1c0001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a3dcbc2555d178 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.355+0000 7fe1adffb640 1 -- 172.21.15.78:0/1320551164 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0042040 con 0x5584d249cb40 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.355+0000 7fe1adffb640 1 -- 172.21.15.78:0/1320551164 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1c0044070 con 0x5584d249cb40 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.356+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1320551164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 msgr2=0x5584d249b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.356+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1320551164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d249b090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0035000 tx=0x7fe1c0001080 comp rx=0 tx=0).stop 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.357+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1320551164 shutdown_connections 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.357+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1320551164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 0x5584d24a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.357+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1320551164 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d249ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.357+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1320551164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d249b090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.357+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1320551164 >> 172.21.15.78:0/1320551164 conn(0x5584d2464310 msgr2=0x5584d2497c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:21.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.359+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1320551164 shutdown_connections 2024-09-16T00:29:21.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.362+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1320551164 wait complete. 2024-09-16T00:29:21.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.362+0000 7fe1d03e5740 1 -- start start 2024-09-16T00:29:21.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.363+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1b00a3400 0x7fe1b00a58c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe1c004d000 tx=0x7fe1c004e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:21.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.363+0000 7fe1d03e5740 1 Processor -- start 2024-09-16T00:29:21.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.365+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d24a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.365+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 0x5584d24a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.365+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d24a6370 0x5584d24a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.365+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5584d24a90c0 con 0x5584d249b5d0 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5584d23bc060 con 0x5584d24a6370 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5584d24a9240 con 0x5584d249cb40 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1d03e5740 10 client.0 _ll_register_callbacks cb 0x5584d210f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d24a6370 0x5584d24a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d24a6370 0x5584d24a3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:43338/0 (socket says 172.21.15.78:43338) 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 -- 172.21.15.78:0/4281311115 learned_addr learned my addr 172.21.15.78:0/4281311115 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 0x5584d24a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d24a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 -- 172.21.15.78:0/4281311115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 msgr2=0x5584d24a2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 0x5584d24a2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 -- 172.21.15.78:0/4281311115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 msgr2=0x5584d24a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d24a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 -- 172.21.15.78:0/4281311115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d21723c0 con 0x5584d24a6370 2024-09-16T00:29:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 0x5584d24a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.366+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d24a6370 0x5584d24a3430 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe1c002a930 tx=0x7fe1c0031a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68b4bb52c01c9af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:21.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.368+0000 7fe1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:29:21.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.373+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/4281311115 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0042070 con 0x5584d24a6370 2024-09-16T00:29:21.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.373+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/4281311115 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe1c00486d0 con 0x5584d24a6370 2024-09-16T00:29:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.375+0000 7fe1d03e5740 1 -- 172.21.15.78:0/4281311115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c0044070 con 0x5584d24a6370 2024-09-16T00:29:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.376+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/4281311115 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe1c00488d0 con 0x5584d24a6370 2024-09-16T00:29:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.376+0000 7fe1d03e5740 10 client.27413 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:21.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.376+0000 7fe1d03e5740 1 -- 172.21.15.78:0/4281311115 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5584d24b4b30 con 0x5584d24a6370 2024-09-16T00:29:21.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.380+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/4281311115 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1c0044070 con 0x5584d24a6370 2024-09-16T00:29:21.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.381+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/4281311115 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe1c0065a10 con 0x5584d24a6370 2024-09-16T00:29:21.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.381+0000 7fe1d03e5740 10 client.27413 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:21.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.381+0000 7fe1d03e5740 10 client.27413 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:29:21.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.381+0000 7fe1d03e5740 20 client.27413 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:21.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.381+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5584d24b6030 0x5584d24b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.381+0000 7fe1d03e5740 4 client.27413 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:21.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.381+0000 7fe1d03e5740 1 -- 172.21.15.78:0/4281311115 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584d240e860 con 0x5584d24b6030 2024-09-16T00:29:21.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.386+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5584d24b6030 0x5584d24b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.396+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5584d24b6030 0x5584d24b8480 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5317d1a8b52d71af server_cookie=eae93e08422dc968 in_seq=0 out_seq=0 2024-09-16T00:29:21.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.396+0000 7fe1a6ffd640 10 client.27413 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:29:21.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.398+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/4281311115 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584d240e860 con 0x5584d24b6030 2024-09-16T00:29:21.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.398+0000 7fe1a6ffd640 10 client.27413 handle_command_reply: tid=0 2024-09-16T00:29:21.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.398+0000 7fe1d03e5740 1 client.27413 shutdown 2024-09-16T00:29:21.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.401+0000 7fe1d03e5740 1 -- 172.21.15.78:0/4281311115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d24a6370 msgr2=0x5584d24a3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.401+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d24a6370 0x5584d24a3430 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe1c002a930 tx=0x7fe1c0031a40 comp rx=0 tx=0).stop 2024-09-16T00:29:21.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.405+0000 7fe1d03e5740 1 -- 172.21.15.78:0/4281311115 shutdown_connections 2024-09-16T00:29:21.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.405+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5584d24b6030 0x5584d24b8480 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.405+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d24a6370 0x5584d24a3430 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.405+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 0x5584d24a2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.405+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/4281311115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d249b5d0 0x5584d24a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.405+0000 7fe1a6ffd640 0 client.27413 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:29:21.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.405+0000 7fe1d03e5740 1 -- 172.21.15.78:0/4281311115 >> 172.21.15.78:0/4281311115 conn(0x5584d2464310 msgr2=0x5584d249d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:21.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.409+0000 7fe1d03e5740 1 -- 172.21.15.78:0/4281311115 shutdown_connections 2024-09-16T00:29:21.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.410+0000 7fe1d03e5740 1 -- 172.21.15.78:0/4281311115 wait complete. 2024-09-16T00:29:21.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.410+0000 7fe1d03e5740 20 client.27413 trim_cache size 0 max 16384 2024-09-16T00:29:21.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1d03e5740 1 Processor -- start 2024-09-16T00:29:21.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1d03e5740 1 -- start start 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249b5d0 0x5584d24a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 0x5584d249e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24a6370 0x5584d249e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249b5d0 0x5584d24a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249b5d0 0x5584d24a5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:43354/0 (socket says 172.21.15.78:43354) 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1c77fe640 1 -- 172.21.15.78:0/2480544214 learned_addr learned my addr 172.21.15.78:0/2480544214 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1d03e5740 1 -- 172.21.15.78:0/2480544214 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b400fa70 con 0x5584d24a6370 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1d03e5740 1 -- 172.21.15.78:0/2480544214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b40115b0 con 0x5584d249b5d0 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.435+0000 7fe1d03e5740 1 -- 172.21.15.78:0/2480544214 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b4010140 con 0x5584d249cb40 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.436+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/2480544214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24a6370 0x5584d249e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.437+0000 7fe1c77fe640 1 -- 172.21.15.78:0/2480544214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 msgr2=0x5584d249e120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.437+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/2480544214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 0x5584d249e120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.437+0000 7fe1c77fe640 1 -- 172.21.15.78:0/2480544214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24a6370 msgr2=0x5584d249e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.437+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/2480544214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24a6370 0x5584d249e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.437+0000 7fe1c77fe640 1 -- 172.21.15.78:0/2480544214 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584d24b8b00 con 0x5584d249b5d0 2024-09-16T00:29:21.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.441+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/2480544214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249b5d0 0x5584d24a5220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009dd0 tx=0x7fe1bc008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e35b1492d3dea531 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:21.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.442+0000 7fe1ad7fa640 1 -- 172.21.15.78:0/2480544214 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc005970 con 0x5584d249b5d0 2024-09-16T00:29:21.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.442+0000 7fe1ad7fa640 1 -- 172.21.15.78:0/2480544214 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1bc021030 con 0x5584d249b5d0 2024-09-16T00:29:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.443+0000 7fe1d03e5740 1 -- 172.21.15.78:0/2480544214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249b5d0 msgr2=0x5584d24a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.443+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/2480544214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249b5d0 0x5584d24a5220 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009dd0 tx=0x7fe1bc008040 comp rx=0 tx=0).stop 2024-09-16T00:29:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.444+0000 7fe1d03e5740 1 -- 172.21.15.78:0/2480544214 shutdown_connections 2024-09-16T00:29:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.444+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/2480544214 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24a6370 0x5584d249e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.444+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/2480544214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d249cb40 0x5584d249e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.444+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/2480544214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249b5d0 0x5584d24a5220 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009dd0 tx=0x7fe1bc008040 comp rx=0 tx=0).stop 2024-09-16T00:29:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.444+0000 7fe1d03e5740 1 -- 172.21.15.78:0/2480544214 >> 172.21.15.78:0/2480544214 conn(0x5584d2464310 msgr2=0x5584d2496d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:21.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.447+0000 7fe1d03e5740 1 -- 172.21.15.78:0/2480544214 shutdown_connections 2024-09-16T00:29:21.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.448+0000 7fe1d03e5740 1 -- 172.21.15.78:0/2480544214 wait complete. 2024-09-16T00:29:21.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.449+0000 7fe1d03e5740 1 -- start start 2024-09-16T00:29:21.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.449+0000 7fe1d03e5740 1 Processor -- start 2024-09-16T00:29:21.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.450+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d24ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.450+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 0x5584d24aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.450+0000 7fe1d03e5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24af030 0x5584d24ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.450+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b4010ee0 con 0x5584d24af030 2024-09-16T00:29:21.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.450+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b4010810 con 0x5584d249cb40 2024-09-16T00:29:21.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.450+0000 7fe1d03e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b400db00 con 0x5584d24a6370 2024-09-16T00:29:21.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.450+0000 7fe1d03e5740 10 client.0 _ll_register_callbacks cb 0x7fe1c0009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:21.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.451+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d24ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.451+0000 7fe1c77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d24ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:43370/0 (socket says 172.21.15.78:43370) 2024-09-16T00:29:21.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.451+0000 7fe1c77fe640 1 -- 172.21.15.78:0/1609019730 learned_addr learned my addr 172.21.15.78:0/1609019730 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:21.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.451+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 0x5584d24aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.452+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24af030 0x5584d24ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.456+0000 7fe1c77fe640 1 -- 172.21.15.78:0/1609019730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 msgr2=0x5584d24aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:21.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.456+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 0x5584d24aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.456+0000 7fe1c77fe640 1 -- 172.21.15.78:0/1609019730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24af030 msgr2=0x5584d24ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:21.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.456+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24af030 0x5584d24ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.456+0000 7fe1c77fe640 1 -- 172.21.15.78:0/1609019730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584d24abdf0 con 0x5584d249cb40 2024-09-16T00:29:21.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.458+0000 7fe1c7fff640 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24af030 0x5584d24ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.461+0000 7fe1c77fe640 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d24ae5b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009840 tx=0x7fe1bc0068a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc09a01b47b92885 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:21.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.465+0000 7fe1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:29:21.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.471+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/1609019730 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc033060 con 0x5584d249cb40 2024-09-16T00:29:21.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.471+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/1609019730 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe1bc01d710 con 0x5584d249cb40 2024-09-16T00:29:21.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.476+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1609019730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1bc021030 con 0x5584d249cb40 2024-09-16T00:29:21.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.478+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/1609019730 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe1bc01d910 con 0x5584d249cb40 2024-09-16T00:29:21.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.479+0000 7fe1d03e5740 10 client.27419 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:21.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.479+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1609019730 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5584d25218d0 con 0x5584d249cb40 2024-09-16T00:29:21.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.480+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/1609019730 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1bc021030 con 0x5584d249cb40 2024-09-16T00:29:21.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.481+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/1609019730 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe1bc0373a0 con 0x5584d249cb40 2024-09-16T00:29:21.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.484+0000 7fe1d03e5740 10 client.27419 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:21.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.484+0000 7fe1d03e5740 10 client.27419 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:29:21.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.484+0000 7fe1d03e5740 20 client.27419 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:21.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.484+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5584d2521e00 0x5584d25241f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:21.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.484+0000 7fe1d03e5740 4 client.27419 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:29:21.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.484+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1609019730 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584d2408d70 con 0x5584d2521e00 2024-09-16T00:29:21.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.489+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5584d2521e00 0x5584d25241f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:21.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.493+0000 7fe1c6ffd640 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5584d2521e00 0x5584d25241f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2f81149ac91671b server_cookie=728d30a7dcb7c64f in_seq=0 out_seq=0 2024-09-16T00:29:21.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.493+0000 7fe1a6ffd640 10 client.27419 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:29:21.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.493+0000 7fe1a6ffd640 1 -- 172.21.15.78:0/1609019730 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584d2408d70 con 0x5584d2521e00 2024-09-16T00:29:21.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.493+0000 7fe1a6ffd640 10 client.27419 handle_command_reply: tid=0 2024-09-16T00:29:21.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 client.27419 shutdown 2024-09-16T00:29:21.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1609019730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 msgr2=0x5584d24ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d24ae5b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009840 tx=0x7fe1bc0068a0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1609019730 shutdown_connections 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5584d24af030 0x5584d24ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584d24a6370 0x5584d24aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x5584d2521e00 0x5584d25241f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 --2- 172.21.15.78:0/1609019730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5584d249cb40 0x5584d24ae5b0 secure :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009840 tx=0x7fe1bc0068a0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1a6ffd640 0 client.27419 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1609019730 >> 172.21.15.78:0/1609019730 conn(0x5584d2464310 msgr2=0x5584d24a2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1609019730 shutdown_connections 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 1 -- 172.21.15.78:0/1609019730 wait complete. 2024-09-16T00:29:21.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.494+0000 7fe1d03e5740 20 client.27419 trim_cache size 0 max 16384 2024-09-16T00:29:21.496 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:29:21.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.499+0000 7fe1ce00c640 1 -- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1b00a3400 msgr2=0x7fe1b00a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.499+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1b00a3400 0x7fe1b00a58c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe1c004d000 tx=0x7fe1c004e000 comp rx=0 tx=0).stop 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.499+0000 7fe1ce00c640 1 -- 172.21.15.78:0/824756585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1c81027a0 msgr2=0x7fe1c8117740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.499+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1c81027a0 0x7fe1c8117740 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800e9f0 tx=0x7fe1b800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.500+0000 7fe1ce00c640 1 -- 172.21.15.78:0/824756585 shutdown_connections 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.500+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe1b00a3400 0x7fe1b00a58c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.500+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1c8106f20 0x7fe1c81112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.500+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe1c81027a0 0x7fe1c8117740 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.500+0000 7fe1ce00c640 1 --2- 172.21.15.78:0/824756585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1c8101df0 0x7fe1c8117200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:21.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.500+0000 7fe1ce00c640 1 -- 172.21.15.78:0/824756585 >> 172.21.15.78:0/824756585 conn(0x7fe1c80fd8f0 msgr2=0x7fe1c80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:21.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.503+0000 7fe1ce00c640 1 -- 172.21.15.78:0/824756585 shutdown_connections 2024-09-16T00:29:21.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:21.503+0000 7fe1ce00c640 1 -- 172.21.15.78:0/824756585 wait complete. 2024-09-16T00:29:22.162 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:21 smithi078 ceph-mon[31157]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 879 KiB/s wr, 106 op/s 2024-09-16T00:29:22.262 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:29:22.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:21 smithi193 ceph-mon[35484]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 879 KiB/s wr, 106 op/s 2024-09-16T00:29:22.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:21 smithi148 ceph-mon[35599]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 879 KiB/s wr, 106 op/s 2024-09-16T00:29:22.848 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:23.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:23 smithi193 ceph-mon[35484]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-16T00:29:23.440 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:23 smithi078 ceph-mon[31157]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-16T00:29:23.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:23 smithi148 ceph-mon[35599]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-16T00:29:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.593+0000 7fef5dff8640 1 -- 172.21.15.78:0/71607883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 msgr2=0x7fef581154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.593+0000 7fef5dff8640 1 --2- 172.21.15.78:0/71607883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 0x7fef581154f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fef50005fd0 tx=0x7fef5002b430 comp rx=0 tx=0).stop 2024-09-16T00:29:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.593+0000 7fef5dff8640 1 -- 172.21.15.78:0/71607883 shutdown_connections 2024-09-16T00:29:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.593+0000 7fef5dff8640 1 --2- 172.21.15.78:0/71607883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 0x7fef581154f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.593+0000 7fef5dff8640 1 --2- 172.21.15.78:0/71607883 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58109e00 0x7fef5810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.593+0000 7fef5dff8640 1 --2- 172.21.15.78:0/71607883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef58109450 0x7fef58109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.593+0000 7fef5dff8640 1 -- 172.21.15.78:0/71607883 >> 172.21.15.78:0/71607883 conn(0x7fef5806e2d0 msgr2=0x7fef5806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:23.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.597+0000 7fef5dff8640 1 -- 172.21.15.78:0/71607883 shutdown_connections 2024-09-16T00:29:23.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.597+0000 7fef5dff8640 1 -- 172.21.15.78:0/71607883 wait complete. 2024-09-16T00:29:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.599+0000 7fef5dff8640 1 Processor -- start 2024-09-16T00:29:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.600+0000 7fef5dff8640 1 -- start start 2024-09-16T00:29:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.600+0000 7fef5dff8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef58109450 0x7fef5810f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.600+0000 7fef5dff8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58109e00 0x7fef5810fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.600+0000 7fef5dff8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 0x7fef58110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.600+0000 7fef5dff8640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef58118660 con 0x7fef5810ea80 2024-09-16T00:29:23.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.600+0000 7fef5dff8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef581184e0 con 0x7fef58109e00 2024-09-16T00:29:23.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.600+0000 7fef5dff8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef581187e0 con 0x7fef58109450 2024-09-16T00:29:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.604+0000 7fef57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 0x7fef58110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.604+0000 7fef57fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 0x7fef58110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33218/0 (socket says 172.21.15.78:33218) 2024-09-16T00:29:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.604+0000 7fef57fff640 1 -- 172.21.15.78:0/4188162810 learned_addr learned my addr 172.21.15.78:0/4188162810 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.608+0000 7fef577fe640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef58109450 0x7fef5810f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.608+0000 7fef56ffd640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58109e00 0x7fef5810fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.610+0000 7fef57fff640 1 -- 172.21.15.78:0/4188162810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef58109450 msgr2=0x7fef5810f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.610+0000 7fef57fff640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef58109450 0x7fef5810f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.610+0000 7fef57fff640 1 -- 172.21.15.78:0/4188162810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58109e00 msgr2=0x7fef5810fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.610+0000 7fef57fff640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58109e00 0x7fef5810fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.610+0000 7fef57fff640 1 -- 172.21.15.78:0/4188162810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef581a2e60 con 0x7fef5810ea80 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.611+0000 7fef57fff640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 0x7fef58110010 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fef5002ba20 tx=0x7fef5000aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.612+0000 7fef54ff9640 1 -- 172.21.15.78:0/4188162810 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50037040 con 0x7fef5810ea80 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.612+0000 7fef54ff9640 1 -- 172.21.15.78:0/4188162810 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef50042070 con 0x7fef5810ea80 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.612+0000 7fef54ff9640 1 -- 172.21.15.78:0/4188162810 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5003dde0 con 0x7fef5810ea80 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.612+0000 7fef5dff8640 1 -- 172.21.15.78:0/4188162810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef581a30f0 con 0x7fef5810ea80 2024-09-16T00:29:23.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.612+0000 7fef5dff8640 1 -- 172.21.15.78:0/4188162810 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581a3580 con 0x7fef5810ea80 2024-09-16T00:29:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.617+0000 7fef603d1740 1 Processor -- start 2024-09-16T00:29:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.618+0000 7fef54ff9640 1 -- 172.21.15.78:0/4188162810 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fef50047460 con 0x7fef5810ea80 2024-09-16T00:29:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.618+0000 7fef603d1740 1 -- start start 2024-09-16T00:29:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.618+0000 7fef603d1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.618+0000 7fef603d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0e5d0 0x55bf8af0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.618+0000 7fef603d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.618+0000 7fef603d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf8ae742f0 con 0x55bf8af0e5d0 2024-09-16T00:29:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.618+0000 7fef603d1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bf8ae1bcf0 con 0x55bf8af0fb40 2024-09-16T00:29:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.618+0000 7fef603d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bf8ae702b0 con 0x55bf8af19370 2024-09-16T00:29:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.619+0000 7fef54ff9640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fef400a3460 0x7fef400a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.619+0000 7fef54ff9640 1 -- 172.21.15.78:0/4188162810 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fef500f1190 con 0x7fef5810ea80 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0e5d0 0x55bf8af0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0e5d0 0x55bf8af0ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33228/0 (socket says 172.21.15.78:33228) 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef577fe640 1 -- 172.21.15.78:0/3780969245 learned_addr learned my addr 172.21.15.78:0/3780969245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef56ffd640 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef56ffd640 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef577fe640 1 -- 172.21.15.78:0/3780969245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 msgr2=0x55bf8af1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef577fe640 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af1b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef577fe640 1 -- 172.21.15.78:0/3780969245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 msgr2=0x55bf8af0e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef577fe640 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af0e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.623+0000 7fef577fe640 1 -- 172.21.15.78:0/3780969245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8ae003a0 con 0x55bf8af0e5d0 2024-09-16T00:29:23.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.624+0000 7fef57fff640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fef400a3460 0x7fef400a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.625+0000 7fef577fe640 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0e5d0 0x55bf8af0ea10 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b820 tx=0x7fef4800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc5e344748114a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.627+0000 7fef3dffb640 1 -- 172.21.15.78:0/3780969245 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4800d950 con 0x55bf8af0e5d0 2024-09-16T00:29:23.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.627+0000 7fef3dffb640 1 -- 172.21.15.78:0/3780969245 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48019070 con 0x55bf8af0e5d0 2024-09-16T00:29:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef603d1740 1 -- 172.21.15.78:0/3780969245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0e5d0 msgr2=0x55bf8af0ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef603d1740 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0e5d0 0x55bf8af0ea10 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b820 tx=0x7fef4800bce0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef57fff640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fef400a3460 0x7fef400a5920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fef500c5b30 tx=0x7fef5000a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef603d1740 1 -- 172.21.15.78:0/3780969245 shutdown_connections 2024-09-16T00:29:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef603d1740 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef603d1740 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0e5d0 0x55bf8af0ea10 secure :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b820 tx=0x7fef4800bce0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef603d1740 1 --2- 172.21.15.78:0/3780969245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef603d1740 1 -- 172.21.15.78:0/3780969245 >> 172.21.15.78:0/3780969245 conn(0x55bf8aed7310 msgr2=0x55bf8af0ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:23.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.628+0000 7fef603d1740 1 -- 172.21.15.78:0/3780969245 shutdown_connections 2024-09-16T00:29:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.632+0000 7fef603d1740 1 -- 172.21.15.78:0/3780969245 wait complete. 2024-09-16T00:29:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.632+0000 7fef603d1740 1 -- start start 2024-09-16T00:29:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.632+0000 7fef603d1740 1 Processor -- start 2024-09-16T00:29:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.633+0000 7fef603d1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.633+0000 7fef603d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af19370 0x55bf8af15d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.633+0000 7fef603d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af162d0 0x55bf8af16750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.633+0000 7fef603d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55bf8af1c0c0 con 0x55bf8af19370 2024-09-16T00:29:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.633+0000 7fef603d1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bf8ae2f060 con 0x55bf8af0fb40 2024-09-16T00:29:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.633+0000 7fef603d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bf8af1c240 con 0x55bf8af162d0 2024-09-16T00:29:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.633+0000 7fef603d1740 10 client.0 _ll_register_callbacks cb 0x55bf8ab828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:23.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.635+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.635+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af18ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.78:43384/0 (socket says 172.21.15.78:43384) 2024-09-16T00:29:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.635+0000 7fef56ffd640 1 -- 172.21.15.78:0/3340014035 learned_addr learned my addr 172.21.15.78:0/3340014035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:23.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.635+0000 7fef56ffd640 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af162d0 0x55bf8af16750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.637+0000 7fef577fe640 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af19370 0x55bf8af15d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.640+0000 7fef577fe640 1 -- 172.21.15.78:0/3340014035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af162d0 msgr2=0x55bf8af16750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:23.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.640+0000 7fef577fe640 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af162d0 0x55bf8af16750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.640+0000 7fef577fe640 1 -- 172.21.15.78:0/3340014035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 msgr2=0x55bf8af18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.640+0000 7fef577fe640 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.640+0000 7fef577fe640 1 -- 172.21.15.78:0/3340014035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf8abe53c0 con 0x55bf8af19370 2024-09-16T00:29:23.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.642+0000 7fef577fe640 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af19370 0x55bf8af15d90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fef48006e90 tx=0x7fef48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5899fe4869a3fbf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.642+0000 7fef36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.644+0000 7fef36ffd640 1 -- 172.21.15.78:0/3340014035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4800db50 con 0x55bf8af19370 2024-09-16T00:29:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.644+0000 7fef603d1740 1 -- 172.21.15.78:0/3340014035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf8ae81860 con 0x55bf8af19370 2024-09-16T00:29:23.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.644+0000 7fef36ffd640 1 -- 172.21.15.78:0/3340014035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fef4800dcf0 con 0x55bf8af19370 2024-09-16T00:29:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.646+0000 7fef36ffd640 1 -- 172.21.15.78:0/3340014035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fef4802d050 con 0x55bf8af19370 2024-09-16T00:29:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.646+0000 7fef603d1740 10 client.18309 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:23.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.646+0000 7fef603d1740 1 -- 172.21.15.78:0/3340014035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bf8af49e40 con 0x55bf8af19370 2024-09-16T00:29:23.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.648+0000 7fef36ffd640 1 -- 172.21.15.78:0/3340014035 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bf8ae81860 con 0x55bf8af19370 2024-09-16T00:29:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.649+0000 7fef36ffd640 1 -- 172.21.15.78:0/3340014035 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fef48013070 con 0x55bf8af19370 2024-09-16T00:29:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.649+0000 7fef603d1740 10 client.18309 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.649+0000 7fef603d1740 10 client.18309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:29:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.649+0000 7fef603d1740 20 client.18309 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.649+0000 7fef603d1740 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55bf8af4b580 0x55bf8af4da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.649+0000 7fef603d1740 4 client.18309 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:23.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.649+0000 7fef603d1740 1 -- 172.21.15.78:0/3340014035 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf8ae7bd70 con 0x55bf8af4b580 2024-09-16T00:29:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.654+0000 7fef56ffd640 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55bf8af4b580 0x55bf8af4da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.665+0000 7fef36ffd640 10 client.18309 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:29:23.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.665+0000 7fef56ffd640 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55bf8af4b580 0x55bf8af4da00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=168cb61105d7acb2 server_cookie=c5be323c7cb2d650 in_seq=0 out_seq=0 2024-09-16T00:29:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.670+0000 7fef36ffd640 1 -- 172.21.15.78:0/3340014035 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf8ae7bd70 con 0x55bf8af4b580 2024-09-16T00:29:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.670+0000 7fef36ffd640 10 client.18309 handle_command_reply: tid=0 2024-09-16T00:29:23.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.670+0000 7fef603d1740 1 client.18309 shutdown 2024-09-16T00:29:23.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.675+0000 7fef603d1740 1 -- 172.21.15.78:0/3340014035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af19370 msgr2=0x55bf8af15d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.675+0000 7fef603d1740 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af19370 0x55bf8af15d90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fef48006e90 tx=0x7fef48004270 comp rx=0 tx=0).stop 2024-09-16T00:29:23.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.676+0000 7fef603d1740 1 -- 172.21.15.78:0/3340014035 shutdown_connections 2024-09-16T00:29:23.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.676+0000 7fef603d1740 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55bf8af4b580 0x55bf8af4da00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.676+0000 7fef603d1740 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af162d0 0x55bf8af16750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.676+0000 7fef603d1740 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af19370 0x55bf8af15d90 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.676+0000 7fef603d1740 1 --2- 172.21.15.78:0/3340014035 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.676+0000 7fef603d1740 1 -- 172.21.15.78:0/3340014035 >> 172.21.15.78:0/3340014035 conn(0x55bf8aed7310 msgr2=0x55bf8af10a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.680+0000 7fef36ffd640 0 client.18309 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:29:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.680+0000 7fef603d1740 1 -- 172.21.15.78:0/3340014035 shutdown_connections 2024-09-16T00:29:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.680+0000 7fef603d1740 1 -- 172.21.15.78:0/3340014035 wait complete. 2024-09-16T00:29:23.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.681+0000 7fef603d1740 20 client.18309 trim_cache size 0 max 16384 2024-09-16T00:29:23.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.710+0000 7fef603d1740 1 Processor -- start 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.710+0000 7fef603d1740 1 -- start start 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.710+0000 7fef603d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0fb40 0x55bf8af15650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.710+0000 7fef603d1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af162d0 0x55bf8af15b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.710+0000 7fef603d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af53310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.710+0000 7fef603d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef4400fa60 con 0x55bf8af0fb40 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.710+0000 7fef603d1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef440115a0 con 0x55bf8af162d0 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0fb40 0x55bf8af15650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef56ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0fb40 0x55bf8af15650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33244/0 (socket says 172.21.15.78:33244) 2024-09-16T00:29:23.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef56ffd640 1 -- 172.21.15.78:0/3309051853 learned_addr learned my addr 172.21.15.78:0/3309051853 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef603d1740 1 -- 172.21.15.78:0/3309051853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef44010130 con 0x55bf8af19370 2024-09-16T00:29:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef577fe640 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af162d0 0x55bf8af15b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef56ffd640 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af53310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef577fe640 1 -- 172.21.15.78:0/3309051853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 msgr2=0x55bf8af53310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef577fe640 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af53310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef577fe640 1 -- 172.21.15.78:0/3309051853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0fb40 msgr2=0x55bf8af15650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef577fe640 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0fb40 0x55bf8af15650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef577fe640 1 -- 172.21.15.78:0/3309051853 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8af0b960 con 0x55bf8af162d0 2024-09-16T00:29:23.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.711+0000 7fef577fe640 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af162d0 0x55bf8af15b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef48001f70 tx=0x7fef4800b570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce856daa79b3571d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.713+0000 7fef3d7fa640 1 -- 172.21.15.78:0/3309051853 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef480036b0 con 0x55bf8af162d0 2024-09-16T00:29:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.713+0000 7fef3d7fa640 1 -- 172.21.15.78:0/3309051853 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48024070 con 0x55bf8af162d0 2024-09-16T00:29:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.713+0000 7fef3d7fa640 1 -- 172.21.15.78:0/3309051853 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48014830 con 0x55bf8af162d0 2024-09-16T00:29:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.715+0000 7fef603d1740 1 -- 172.21.15.78:0/3309051853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af162d0 msgr2=0x55bf8af15b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.715+0000 7fef603d1740 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af162d0 0x55bf8af15b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef48001f70 tx=0x7fef4800b570 comp rx=0 tx=0).stop 2024-09-16T00:29:23.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.719+0000 7fef603d1740 1 -- 172.21.15.78:0/3309051853 shutdown_connections 2024-09-16T00:29:23.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.719+0000 7fef603d1740 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af53310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.719+0000 7fef603d1740 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af162d0 0x55bf8af15b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.719+0000 7fef603d1740 1 --2- 172.21.15.78:0/3309051853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af0fb40 0x55bf8af15650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.719+0000 7fef603d1740 1 -- 172.21.15.78:0/3309051853 >> 172.21.15.78:0/3309051853 conn(0x55bf8aed7310 msgr2=0x55bf8af0e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:23.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.726+0000 7fef603d1740 1 -- 172.21.15.78:0/3309051853 shutdown_connections 2024-09-16T00:29:23.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.729+0000 7fef603d1740 1 -- 172.21.15.78:0/3309051853 wait complete. 2024-09-16T00:29:23.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.730+0000 7fef603d1740 1 -- start start 2024-09-16T00:29:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.730+0000 7fef603d1740 1 Processor -- start 2024-09-16T00:29:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.730+0000 7fef603d1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af21790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.730+0000 7fef603d1740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af162d0 0x55bf8af21cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.731+0000 7fef603d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af12020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.731+0000 7fef603d1740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef44010ed0 con 0x55bf8af162d0 2024-09-16T00:29:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.731+0000 7fef603d1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fef44010800 con 0x55bf8af0fb40 2024-09-16T00:29:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.731+0000 7fef603d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef4400daf0 con 0x55bf8af19370 2024-09-16T00:29:23.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.731+0000 7fef603d1740 10 client.0 _ll_register_callbacks cb 0x7fef4c00e660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.737+0000 7fef577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af162d0 0x55bf8af21cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.737+0000 7fef577fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af162d0 0x55bf8af21cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33254/0 (socket says 172.21.15.78:33254) 2024-09-16T00:29:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.737+0000 7fef577fe640 1 -- 172.21.15.78:0/2126188148 learned_addr learned my addr 172.21.15.78:0/2126188148 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.737+0000 7fef56ffd640 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af21790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.737+0000 7fef56ffd640 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af12020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.749+0000 7fef577fe640 1 -- 172.21.15.78:0/2126188148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 msgr2=0x55bf8af12020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.749+0000 7fef577fe640 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af12020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.749+0000 7fef577fe640 1 -- 172.21.15.78:0/2126188148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 msgr2=0x55bf8af21790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.749+0000 7fef577fe640 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af21790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.749+0000 7fef577fe640 1 -- 172.21.15.78:0/2126188148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf8af11120 con 0x55bf8af162d0 2024-09-16T00:29:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.755+0000 7fef577fe640 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af162d0 0x55bf8af21cd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x55bf8af16e80 tx=0x7fef48003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fa63587937ae6a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:23.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.755+0000 7fef36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:23.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.758+0000 7fef36ffd640 1 -- 172.21.15.78:0/2126188148 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48036070 con 0x55bf8af162d0 2024-09-16T00:29:23.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.758+0000 7fef36ffd640 1 -- 172.21.15.78:0/2126188148 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fef4800dd60 con 0x55bf8af162d0 2024-09-16T00:29:23.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.758+0000 7fef603d1740 1 -- 172.21.15.78:0/2126188148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef48024070 con 0x55bf8af162d0 2024-09-16T00:29:23.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.763+0000 7fef36ffd640 1 -- 172.21.15.78:0/2126188148 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fef48040050 con 0x55bf8af162d0 2024-09-16T00:29:23.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.765+0000 7fef603d1740 10 client.18315 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:23.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.765+0000 7fef603d1740 1 -- 172.21.15.78:0/2126188148 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bf8af13980 con 0x55bf8af162d0 2024-09-16T00:29:23.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.770+0000 7fef36ffd640 1 -- 172.21.15.78:0/2126188148 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48024070 con 0x55bf8af162d0 2024-09-16T00:29:23.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.770+0000 7fef36ffd640 1 -- 172.21.15.78:0/2126188148 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fef4801f8c0 con 0x55bf8af162d0 2024-09-16T00:29:23.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.770+0000 7fef603d1740 10 client.18315 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:23.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.770+0000 7fef603d1740 10 client.18315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:29:23.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.770+0000 7fef603d1740 20 client.18315 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:23.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.770+0000 7fef603d1740 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55bf8afb53a0 0x55bf8afb77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:23.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.770+0000 7fef603d1740 4 client.18315 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:29:23.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.770+0000 7fef603d1740 1 -- 172.21.15.78:0/2126188148 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf8adf8170 con 0x55bf8afb53a0 2024-09-16T00:29:23.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.774+0000 7fef56ffd640 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55bf8afb53a0 0x55bf8afb77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.789+0000 7fef56ffd640 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55bf8afb53a0 0x55bf8afb77c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=645221a006f44766 server_cookie=4e65b0c232d9b517 in_seq=0 out_seq=0 2024-09-16T00:29:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.789+0000 7fef36ffd640 10 client.18315 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:29:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.790+0000 7fef36ffd640 1 -- 172.21.15.78:0/2126188148 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf8adf8170 con 0x55bf8afb53a0 2024-09-16T00:29:23.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.790+0000 7fef36ffd640 10 client.18315 handle_command_reply: tid=0 2024-09-16T00:29:23.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.790+0000 7fef603d1740 1 client.18315 shutdown 2024-09-16T00:29:23.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 -- 172.21.15.78:0/2126188148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af162d0 msgr2=0x55bf8af21cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af162d0 0x55bf8af21cd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x55bf8af16e80 tx=0x7fef48003ed0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 -- 172.21.15.78:0/2126188148 shutdown_connections 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x55bf8afb53a0 0x55bf8afb77c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf8af19370 0x55bf8af12020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55bf8af162d0 0x55bf8af21cd0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 --2- 172.21.15.78:0/2126188148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf8af0fb40 0x55bf8af21790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 -- 172.21.15.78:0/2126188148 >> 172.21.15.78:0/2126188148 conn(0x55bf8aed7310 msgr2=0x55bf8af0e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef36ffd640 0 client.18315 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 -- 172.21.15.78:0/2126188148 shutdown_connections 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 1 -- 172.21.15.78:0/2126188148 wait complete. 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.795+0000 7fef603d1740 20 client.18315 trim_cache size 0 max 16384 2024-09-16T00:29:23.796 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 -- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fef400a3460 msgr2=0x7fef400a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fef400a3460 0x7fef400a5920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fef500c5b30 tx=0x7fef5000a040 comp rx=0 tx=0).stop 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 -- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 msgr2=0x7fef58110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 0x7fef58110010 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fef5002ba20 tx=0x7fef5000aad0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 -- 172.21.15.78:0/4188162810 shutdown_connections 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fef400a3460 0x7fef400a5920 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef5810ea80 0x7fef58110010 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fef58109e00 0x7fef5810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 --2- 172.21.15.78:0/4188162810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef58109450 0x7fef5810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 -- 172.21.15.78:0/4188162810 >> 172.21.15.78:0/4188162810 conn(0x7fef5806e2d0 msgr2=0x7fef580708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:23.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 -- 172.21.15.78:0/4188162810 shutdown_connections 2024-09-16T00:29:23.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:23.800+0000 7fef5dff8640 1 -- 172.21.15.78:0/4188162810 wait complete. 2024-09-16T00:29:26.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:25 smithi193 ceph-mon[35484]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-09-16T00:29:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:25 smithi148 ceph-mon[35599]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-09-16T00:29:26.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:25 smithi078 ceph-mon[31157]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-09-16T00:29:26.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:29:26.624 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:27.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 -- 172.21.15.78:0/2553943930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 msgr2=0x7f2c9810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/2553943930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 0x7f2c9810eac0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ca0 tx=0x7f2c9002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 -- 172.21.15.78:0/2553943930 shutdown_connections 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/2553943930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 0x7f2c9810eac0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/2553943930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981027c0 0x7f2c98106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/2553943930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98101e10 0x7f2c981021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 -- 172.21.15.78:0/2553943930 >> 172.21.15.78:0/2553943930 conn(0x7f2c980fd8d0 msgr2=0x7f2c980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 -- 172.21.15.78:0/2553943930 shutdown_connections 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 -- 172.21.15.78:0/2553943930 wait complete. 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 Processor -- start 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.160+0000 7f2c9eccc640 1 -- start start 2024-09-16T00:29:27.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9eccc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98101e10 0x7f2c980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9eccc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981027c0 0x7f2c98074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9eccc640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 0x7f2c98074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9eccc640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98111fa0 con 0x7f2c98106f40 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9eccc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98111e20 con 0x7f2c98101e10 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9eccc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98112120 con 0x7f2c981027c0 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9d269640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 0x7f2c98074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9d269640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 0x7f2c98074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49256/0 (socket says 172.21.15.78:49256) 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9d269640 1 -- 172.21.15.78:0/3879347902 learned_addr learned my addr 172.21.15.78:0/3879347902 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98101e10 0x7f2c980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c97fff640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981027c0 0x7f2c98074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9ca68640 1 -- 172.21.15.78:0/3879347902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981027c0 msgr2=0x7f2c98074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981027c0 0x7f2c98074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9ca68640 1 -- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 msgr2=0x7f2c98074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 0x7f2c98074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9ca68640 1 -- 172.21.15.78:0/3879347902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c981a68a0 con 0x7f2c98101e10 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9d269640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 0x7f2c98074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.161+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98101e10 0x7f2c980744f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00e9f0 tx=0x7f2c8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.162+0000 7f2c95ffb640 1 -- 172.21.15.78:0/3879347902 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00ccc0 con 0x7f2c98101e10 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.162+0000 7f2c9eccc640 1 -- 172.21.15.78:0/3879347902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c981a6b90 con 0x7f2c98101e10 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.162+0000 7f2c95ffb640 1 -- 172.21.15.78:0/3879347902 <== mon.2 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8c010910 con 0x7f2c98101e10 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.162+0000 7f2c95ffb640 1 -- 172.21.15.78:0/3879347902 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c018960 con 0x7f2c98101e10 2024-09-16T00:29:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.162+0000 7f2c9eccc640 1 -- 172.21.15.78:0/3879347902 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981a7120 con 0x7f2c98101e10 2024-09-16T00:29:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.165+0000 7f2c95ffb640 1 -- 172.21.15.78:0/3879347902 <== mon.2 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2c8c010ac0 con 0x7f2c98101e10 2024-09-16T00:29:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.165+0000 7f2ca10a5740 1 Processor -- start 2024-09-16T00:29:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.165+0000 7f2ca10a5740 1 -- start start 2024-09-16T00:29:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdd2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcdd2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcddf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fbcd382f0 con 0x558fbcdd25d0 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558fbccdfcf0 con 0x558fbcdd3b40 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558fbcd342b0 con 0x558fbcddd370 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c95ffb640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c880a3240 0x7f2c880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcdd2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c97fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcddf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcdd2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49270/0 (socket says 172.21.15.78:49270) 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 -- 172.21.15.78:0/603720151 learned_addr learned my addr 172.21.15.78:0/603720151 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9d269640 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9d269640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c880a3240 0x7f2c880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c95ffb640 1 -- 172.21.15.78:0/3879347902 <== mon.2 v2:172.21.15.148:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2c8c0cffe0 con 0x7f2c98101e10 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 -- 172.21.15.78:0/603720151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 msgr2=0x558fbcddf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcddf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 -- 172.21.15.78:0/603720151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 msgr2=0x558fbcdd2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdd2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 -- 172.21.15.78:0/603720151 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbccc43a0 con 0x558fbcdd25d0 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c97fff640 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcddf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9d269640 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdd2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.166+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcdd2a10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c098d20 tx=0x7f2c8c09c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b1940fc61b5ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2c82ffd640 1 -- 172.21.15.78:0/603720151 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c09c350 con 0x558fbcdd25d0 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2c9d269640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c880a3240 0x7f2c880a5700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009a60 tx=0x7f2c90004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2c82ffd640 1 -- 172.21.15.78:0/603720151 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8c0a0650 con 0x558fbcdd25d0 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2c82ffd640 1 -- 172.21.15.78:0/603720151 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0a7f20 con 0x558fbcdd25d0 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 -- 172.21.15.78:0/603720151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 msgr2=0x558fbcdd2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcdd2a10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c098d20 tx=0x7f2c8c09c250 comp rx=0 tx=0).stop 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 -- 172.21.15.78:0/603720151 shutdown_connections 2024-09-16T00:29:27.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcddf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcdd2a10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/603720151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdd2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 -- 172.21.15.78:0/603720151 >> 172.21.15.78:0/603720151 conn(0x558fbcd9b310 msgr2=0x558fbcdcec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 -- 172.21.15.78:0/603720151 shutdown_connections 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.167+0000 7f2ca10a5740 1 -- 172.21.15.78:0/603720151 wait complete. 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 1 -- start start 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 1 Processor -- start 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd25d0 0x558fbcddcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdda050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcddd370 0x558fbcdda7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fbcde0010 con 0x558fbcddd370 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558fbccf3060 con 0x558fbcdd3b40 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558fbcde0190 con 0x558fbcdd25d0 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2ca10a5740 10 client.0 _ll_register_callbacks cb 0x558fbca468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.168+0000 7f2c9d269640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcddd370 0x558fbcdda7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c97fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdda050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9d269640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcddd370 0x558fbcdda7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49282/0 (socket says 172.21.15.78:49282) 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9d269640 1 -- 172.21.15.78:0/883778031 learned_addr learned my addr 172.21.15.78:0/883778031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd25d0 0x558fbcddcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9d269640 1 -- 172.21.15.78:0/883778031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd25d0 msgr2=0x558fbcddcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9d269640 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd25d0 0x558fbcddcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9d269640 1 -- 172.21.15.78:0/883778031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 msgr2=0x558fbcdda050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:27.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9d269640 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdda050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9d269640 1 -- 172.21.15.78:0/883778031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbcaa93c0 con 0x558fbcddd370 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c9d269640 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcddd370 0x558fbcdda7e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f2c90006010 tx=0x7f2c900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a509be24b1205a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c77fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c77fff640 1 -- 172.21.15.78:0/883778031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9004e030 con 0x558fbcddd370 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.169+0000 7f2c77fff640 1 -- 172.21.15.78:0/883778031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2c90045070 con 0x558fbcddd370 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2ca10a5740 1 -- 172.21.15.78:0/883778031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c8c0a0650 con 0x558fbcddd370 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2c77fff640 1 -- 172.21.15.78:0/883778031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2c90030ac0 con 0x558fbcddd370 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2ca10a5740 10 client.18327 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2ca10a5740 1 -- 172.21.15.78:0/883778031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558fbcdebb20 con 0x558fbcddd370 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2c77fff640 1 -- 172.21.15.78:0/883778031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c8c0a0650 con 0x558fbcddd370 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2c77fff640 1 -- 172.21.15.78:0/883778031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2c90048040 con 0x558fbcddd370 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2ca10a5740 10 client.18327 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2ca10a5740 10 client.18327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:27.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2ca10a5740 20 client.18327 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:27.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.170+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558fbcded020 0x558fbcdef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.171+0000 7f2ca10a5740 4 client.18327 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:27.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.171+0000 7f2ca10a5740 1 -- 172.21.15.78:0/883778031 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fbcd45860 con 0x558fbcded020 2024-09-16T00:29:27.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.171+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558fbcded020 0x558fbcdef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.171+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558fbcded020 0x558fbcdef460 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67fdae96dfef58 server_cookie=c88e6275d58df4ac in_seq=0 out_seq=0 2024-09-16T00:29:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.171+0000 7f2c77fff640 10 client.18327 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2c77fff640 1 -- 172.21.15.78:0/883778031 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fbcd45860 con 0x558fbcded020 2024-09-16T00:29:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2c77fff640 10 client.18327 handle_command_reply: tid=0 2024-09-16T00:29:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 client.18327 shutdown 2024-09-16T00:29:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 -- 172.21.15.78:0/883778031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcddd370 msgr2=0x558fbcdda7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcddd370 0x558fbcdda7e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f2c90006010 tx=0x7f2c900095c0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 -- 172.21.15.78:0/883778031 shutdown_connections 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558fbcded020 0x558fbcdef460 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2c77fff640 0 client.18327 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcddd370 0x558fbcdda7e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd3b40 0x558fbcdda050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/883778031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd25d0 0x558fbcddcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 -- 172.21.15.78:0/883778031 >> 172.21.15.78:0/883778031 conn(0x558fbcd9b310 msgr2=0x558fbcdd4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 -- 172.21.15.78:0/883778031 shutdown_connections 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 1 -- 172.21.15.78:0/883778031 wait complete. 2024-09-16T00:29:27.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.172+0000 7f2ca10a5740 20 client.18327 trim_cache size 0 max 16384 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.187+0000 7f2ca10a5740 1 Processor -- start 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.187+0000 7f2ca10a5740 1 -- start start 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.187+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd25d0 0x558fbcdd9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.187+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd3b40 0x558fbcddbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.187+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcdf4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.187+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b0d90 con 0x558fbcdd3b40 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.187+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b28d0 con 0x558fbcdd25d0 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.187+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b1460 con 0x558fbcddd370 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd3b40 0x558fbcddbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd3b40 0x558fbcddbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49294/0 (socket says 172.21.15.78:49294) 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 -- 172.21.15.78:0/2281055349 learned_addr learned my addr 172.21.15.78:0/2281055349 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c9ca68640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd25d0 0x558fbcdd9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c9d269640 1 --2- 172.21.15.78:0/2281055349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcdf4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 -- 172.21.15.78:0/2281055349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 msgr2=0x558fbcdf4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 --2- 172.21.15.78:0/2281055349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcdf4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 -- 172.21.15.78:0/2281055349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd25d0 msgr2=0x558fbcdd9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 --2- 172.21.15.78:0/2281055349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd25d0 0x558fbcdd9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 -- 172.21.15.78:0/2281055349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbcdd22d0 con 0x558fbcdd3b40 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c97fff640 1 --2- 172.21.15.78:0/2281055349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd3b40 0x558fbcddbfc0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005fd0 tx=0x7f2c8400bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4f0d784ec15aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c827fc640 1 -- 172.21.15.78:0/2281055349 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84003b50 con 0x558fbcdd3b40 2024-09-16T00:29:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c827fc640 1 -- 172.21.15.78:0/2281055349 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84005500 con 0x558fbcdd3b40 2024-09-16T00:29:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.188+0000 7f2c827fc640 1 -- 172.21.15.78:0/2281055349 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84011440 con 0x558fbcdd3b40 2024-09-16T00:29:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 -- 172.21.15.78:0/2281055349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd3b40 msgr2=0x558fbcddbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/2281055349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd3b40 0x558fbcddbfc0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2c84005fd0 tx=0x7f2c8400bc90 comp rx=0 tx=0).stop 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 -- 172.21.15.78:0/2281055349 shutdown_connections 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/2281055349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcddd370 0x558fbcdf4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/2281055349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd3b40 0x558fbcddbfc0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/2281055349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcdd25d0 0x558fbcdd9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 -- 172.21.15.78:0/2281055349 >> 172.21.15.78:0/2281055349 conn(0x558fbcd9b310 msgr2=0x558fbcddb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 -- 172.21.15.78:0/2281055349 shutdown_connections 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.189+0000 7f2ca10a5740 1 -- 172.21.15.78:0/2281055349 wait complete. 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 1 -- start start 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 1 Processor -- start 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcde1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd3b40 0x558fbcde1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcddd370 0x558fbcde1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b2200 con 0x558fbcdd25d0 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981b1b30 con 0x558fbcddd370 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981aed70 con 0x558fbcdd3b40 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2ca10a5740 10 client.0 _ll_register_callbacks cb 0x7f2c90031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2c9ca68640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcde1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2c9ca68640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcde1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49296/0 (socket says 172.21.15.78:49296) 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2c9ca68640 1 -- 172.21.15.78:0/3418298182 learned_addr learned my addr 172.21.15.78:0/3418298182 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2c97fff640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd3b40 0x558fbcde1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.190+0000 7f2c9d269640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcddd370 0x558fbcde1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c9ca68640 1 -- 172.21.15.78:0/3418298182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd3b40 msgr2=0x558fbcde1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd3b40 0x558fbcde1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c9ca68640 1 -- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcddd370 msgr2=0x558fbcde1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcddd370 0x558fbcde1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c9ca68640 1 -- 172.21.15.78:0/3418298182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbce56df0 con 0x558fbcdd25d0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c97fff640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd3b40 0x558fbcde1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c9d269640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcddd370 0x558fbcde1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c9ca68640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcde1240 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c09c2e0 tx=0x7f2c8c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458cb4dd9b3aee77 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c77fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c77fff640 1 -- 172.21.15.78:0/3418298182 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0b4070 con 0x558fbcdd25d0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c77fff640 1 -- 172.21.15.78:0/3418298182 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f2c8c0989c0 con 0x558fbcdd25d0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2ca10a5740 1 -- 172.21.15.78:0/3418298182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c84005500 con 0x558fbcdd25d0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c77fff640 1 -- 172.21.15.78:0/3418298182 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2c8c0b5260 con 0x558fbcdd25d0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2ca10a5740 10 client.18333 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2ca10a5740 1 -- 172.21.15.78:0/3418298182 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558fbce580b0 con 0x558fbcdd25d0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.191+0000 7f2c77fff640 1 -- 172.21.15.78:0/3418298182 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84005500 con 0x558fbcdd25d0 2024-09-16T00:29:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2c77fff640 1 -- 172.21.15.78:0/3418298182 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2c8c0a4b10 con 0x558fbcdd25d0 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2ca10a5740 10 client.18333 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2ca10a5740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2ca10a5740 20 client.18333 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558fbcded020 0x558fbce5a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2ca10a5740 4 client.18333 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2ca10a5740 1 -- 172.21.15.78:0/3418298182 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558fbcd3fd70 con 0x558fbcded020 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2c97fff640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558fbcded020 0x558fbce5a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2c77fff640 10 client.18333 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2c97fff640 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558fbcded020 0x558fbce5a350 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1bf10bac6f1c6 server_cookie=97f7dc12c345bdeb in_seq=0 out_seq=0 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2c77fff640 1 -- 172.21.15.78:0/3418298182 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558fbcd3fd70 con 0x558fbcded020 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2c77fff640 10 client.18333 handle_command_reply: tid=0 2024-09-16T00:29:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.192+0000 7f2ca10a5740 1 client.18333 shutdown 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 -- 172.21.15.78:0/3418298182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 msgr2=0x558fbcde1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcde1240 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c09c2e0 tx=0x7f2c8c00cf60 comp rx=0 tx=0).stop 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 -- 172.21.15.78:0/3418298182 shutdown_connections 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x558fbcded020 0x558fbce5a350 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558fbcddd370 0x558fbcde1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbcdd3b40 0x558fbcde1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2c77fff640 0 client.18333 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 --2- 172.21.15.78:0/3418298182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fbcdd25d0 0x558fbcde1240 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 -- 172.21.15.78:0/3418298182 >> 172.21.15.78:0/3418298182 conn(0x558fbcd9b310 msgr2=0x558fbcddb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 -- 172.21.15.78:0/3418298182 shutdown_connections 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 1 -- 172.21.15.78:0/3418298182 wait complete. 2024-09-16T00:29:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2ca10a5740 20 client.18333 trim_cache size 0 max 16384 2024-09-16T00:29:27.200 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:29:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2c9eccc640 1 -- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c880a3240 msgr2=0x7f2c880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c880a3240 0x7f2c880a5700 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009a60 tx=0x7f2c90004a60 comp rx=0 tx=0).stop 2024-09-16T00:29:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2c9eccc640 1 -- 172.21.15.78:0/3879347902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98101e10 msgr2=0x7f2c980744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.193+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98101e10 0x7f2c980744f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00e9f0 tx=0x7f2c8c00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.194+0000 7f2c9eccc640 1 -- 172.21.15.78:0/3879347902 shutdown_connections 2024-09-16T00:29:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.194+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f2c880a3240 0x7f2c880a5700 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.194+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c98106f40 0x7f2c98074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.194+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c981027c0 0x7f2c98074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.194+0000 7f2c9eccc640 1 --2- 172.21.15.78:0/3879347902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2c98101e10 0x7f2c980744f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.194+0000 7f2c9eccc640 1 -- 172.21.15.78:0/3879347902 >> 172.21.15.78:0/3879347902 conn(0x7f2c980fd8d0 msgr2=0x7f2c980fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.194+0000 7f2c9eccc640 1 -- 172.21.15.78:0/3879347902 shutdown_connections 2024-09-16T00:29:27.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:27.194+0000 7f2c9eccc640 1 -- 172.21.15.78:0/3879347902 wait complete. 2024-09-16T00:29:28.113 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:27 smithi078 ceph-mon[31157]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-09-16T00:29:28.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:27 smithi193 ceph-mon[35484]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-09-16T00:29:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:27 smithi148 ceph-mon[35599]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-09-16T00:29:29.166 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:29:29.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:29:29.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:28 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:29.206 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:28 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:29.236 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:28 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:29.660 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:29.974 DEBUG:teuthology.orchestra.run.smithi078:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-16T00:29:30.060 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/: 2024-09-16T00:29:30.060 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.060 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:29:30.061 INFO:teuthology.orchestra.run.smithi078.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 .. 2024-09-16T00:29:30.061 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 data 2024-09-16T00:29:30.061 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.061 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data: 2024-09-16T00:29:30.061 INFO:teuthology.orchestra.run.smithi078.stdout:total 36 2024-09-16T00:29:30.062 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 . 2024-09-16T00:29:30.062 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 .. 2024-09-16T00:29:30.062 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 16 00:13 base 2024-09-16T00:29:30.062 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 16 00:13 current_logfiles 2024-09-16T00:29:30.062 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 16 00:13 global 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 log 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_commit_ts 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_dynshmem 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 16 00:13 pg_hba.conf 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 16 00:13 pg_ident.conf 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:29 pg_logical 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 pg_multixact 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_notify 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_replslot 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_serial 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_snapshots 2024-09-16T00:29:30.063 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:29 pg_stat 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:29 pg_stat_tmp 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:29 pg_subtrans 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_tblspc 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 pg_twophase 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 PG_VERSION 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 61 Sep 16 00:29 pg_wal 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 pg_xact 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 16 00:13 postgresql.auto.conf 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 16 00:13 postgresql.conf 2024-09-16T00:29:30.064 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 16 00:13 postmaster.opts 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base: 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 16 00:13 . 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 1 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 13493 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 16 00:20 13494 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:14 pgsql_tmp 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.065 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:total 7777 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 . 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 16 00:13 .. 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 112 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 113 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 1247 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1247_fsm 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1247_vm 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 16 00:13 1249 2024-09-16T00:29:30.066 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1249_fsm 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1249_vm 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 16 00:13 1255 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1255_fsm 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1255_vm 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 16 00:13 1259 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1259_fsm 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1259_vm 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 16 00:13 13344 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13344_fsm 2024-09-16T00:29:30.067 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13344_vm 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13346 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13348 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13349_fsm 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349_vm 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13351 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13353 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13354_fsm 2024-09-16T00:29:30.068 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354_vm 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13356 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13358 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13359_fsm 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359_vm 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13361 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13363 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1417 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1418 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 174 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 175 2024-09-16T00:29:30.069 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2187 2024-09-16T00:29:30.070 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2224 2024-09-16T00:29:30.070 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2328 2024-09-16T00:29:30.070 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2336 2024-09-16T00:29:30.070 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2337 2024-09-16T00:29:30.070 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2579 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2600 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2600_fsm 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2600_vm 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2601_fsm 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601_vm 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2602 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2602_fsm 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2602_vm 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2603 2024-09-16T00:29:30.071 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2603_fsm 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2603_vm 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2604 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2605 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2605_fsm 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2605_vm 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2606_fsm 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606_vm 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2607 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2607_fsm 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2607_vm 2024-09-16T00:29:30.072 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 16 00:13 2608 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2608_fsm 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2608_vm 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 16 00:13 2609 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2609_fsm 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2609_vm 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2610 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2610_fsm 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2610_vm 2024-09-16T00:29:30.073 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2611 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2612_fsm 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612_vm 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2613 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2615_fsm 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615_vm 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2616 2024-09-16T00:29:30.074 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2616_fsm 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2616_vm 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 16 00:13 2617 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2617_fsm 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2617_vm 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 16 00:13 2618 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2618_fsm 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2618_vm 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 16 00:13 2619 2024-09-16T00:29:30.075 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2619_fsm 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2619_vm 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2620 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2650 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2651 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2652 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2653 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2654 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2655 2024-09-16T00:29:30.076 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2656 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2657 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 16 00:13 2658 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 2659 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2660 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2661 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2662 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2663 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2664 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2665 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2666 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2667 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2668 2024-09-16T00:29:30.077 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2669 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2670 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 16 00:13 2673 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 16 00:13 2674 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 16 00:13 2675 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2678 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2679 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2680 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2681 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2682 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2683 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2684 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2685 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2686 2024-09-16T00:29:30.078 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2687 2024-09-16T00:29:30.079 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2688 2024-09-16T00:29:30.079 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2689 2024-09-16T00:29:30.079 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 16 00:13 2690 2024-09-16T00:29:30.079 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 16 00:13 2691 2024-09-16T00:29:30.079 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2692 2024-09-16T00:29:30.079 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2693 2024-09-16T00:29:30.079 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2696 2024-09-16T00:29:30.079 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2699 2024-09-16T00:29:30.080 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2701 2024-09-16T00:29:30.080 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2702 2024-09-16T00:29:30.080 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2703 2024-09-16T00:29:30.080 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2704 2024-09-16T00:29:30.080 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2753 2024-09-16T00:29:30.081 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2753_fsm 2024-09-16T00:29:30.081 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2753_vm 2024-09-16T00:29:30.081 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2754 2024-09-16T00:29:30.081 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2755 2024-09-16T00:29:30.081 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2756 2024-09-16T00:29:30.081 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2757 2024-09-16T00:29:30.082 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2830 2024-09-16T00:29:30.082 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2831 2024-09-16T00:29:30.082 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2832 2024-09-16T00:29:30.082 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2833 2024-09-16T00:29:30.082 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2834 2024-09-16T00:29:30.082 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2835 2024-09-16T00:29:30.082 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2836 2024-09-16T00:29:30.082 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2837 2024-09-16T00:29:30.083 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 16 00:13 2838 2024-09-16T00:29:30.083 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2838_fsm 2024-09-16T00:29:30.083 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2838_vm 2024-09-16T00:29:30.083 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2839 2024-09-16T00:29:30.083 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840 2024-09-16T00:29:30.083 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840_fsm 2024-09-16T00:29:30.084 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2840_vm 2024-09-16T00:29:30.084 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2841 2024-09-16T00:29:30.084 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2995 2024-09-16T00:29:30.084 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2996 2024-09-16T00:29:30.084 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079 2024-09-16T00:29:30.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3079_fsm 2024-09-16T00:29:30.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079_vm 2024-09-16T00:29:30.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3080 2024-09-16T00:29:30.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3081 2024-09-16T00:29:30.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3085 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3118 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3119 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 16 00:13 3164 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3256 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3257 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3258 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3350 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3351 2024-09-16T00:29:30.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3379 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3380 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3381 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3394 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3394_fsm 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3394_vm 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3395 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3429 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3430 2024-09-16T00:29:30.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3431 2024-09-16T00:29:30.088 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3433 2024-09-16T00:29:30.088 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3439 2024-09-16T00:29:30.088 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3440 2024-09-16T00:29:30.088 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3455 2024-09-16T00:29:30.088 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 16 00:13 3456 2024-09-16T00:29:30.088 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3456_fsm 2024-09-16T00:29:30.089 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3456_vm 2024-09-16T00:29:30.089 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3466 2024-09-16T00:29:30.089 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3467 2024-09-16T00:29:30.089 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3468 2024-09-16T00:29:30.089 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3501 2024-09-16T00:29:30.089 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3502 2024-09-16T00:29:30.089 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3503 2024-09-16T00:29:30.090 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3534 2024-09-16T00:29:30.090 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541 2024-09-16T00:29:30.090 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3541_fsm 2024-09-16T00:29:30.090 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541_vm 2024-09-16T00:29:30.090 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3542 2024-09-16T00:29:30.090 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3574 2024-09-16T00:29:30.090 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3575 2024-09-16T00:29:30.091 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3576 2024-09-16T00:29:30.091 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3596 2024-09-16T00:29:30.091 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3597 2024-09-16T00:29:30.091 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3598 2024-09-16T00:29:30.091 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3599 2024-09-16T00:29:30.091 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600 2024-09-16T00:29:30.091 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3600_fsm 2024-09-16T00:29:30.092 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600_vm 2024-09-16T00:29:30.092 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601 2024-09-16T00:29:30.092 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3601_fsm 2024-09-16T00:29:30.092 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601_vm 2024-09-16T00:29:30.092 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602 2024-09-16T00:29:30.092 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3602_fsm 2024-09-16T00:29:30.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602_vm 2024-09-16T00:29:30.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603 2024-09-16T00:29:30.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603_fsm 2024-09-16T00:29:30.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3603_vm 2024-09-16T00:29:30.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3604 2024-09-16T00:29:30.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3605 2024-09-16T00:29:30.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3606 2024-09-16T00:29:30.093 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3607 2024-09-16T00:29:30.094 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3608 2024-09-16T00:29:30.094 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 3609 2024-09-16T00:29:30.094 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3712 2024-09-16T00:29:30.094 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764 2024-09-16T00:29:30.094 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3764_fsm 2024-09-16T00:29:30.094 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764_vm 2024-09-16T00:29:30.094 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3766 2024-09-16T00:29:30.095 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3767 2024-09-16T00:29:30.095 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3997 2024-09-16T00:29:30.095 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4143 2024-09-16T00:29:30.095 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4144 2024-09-16T00:29:30.095 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4145 2024-09-16T00:29:30.095 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4146 2024-09-16T00:29:30.095 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4147 2024-09-16T00:29:30.096 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4148 2024-09-16T00:29:30.096 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4149 2024-09-16T00:29:30.096 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4150 2024-09-16T00:29:30.096 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4151 2024-09-16T00:29:30.096 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4152 2024-09-16T00:29:30.096 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4153 2024-09-16T00:29:30.096 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4154 2024-09-16T00:29:30.097 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4155 2024-09-16T00:29:30.097 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4156 2024-09-16T00:29:30.097 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4157 2024-09-16T00:29:30.097 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4158 2024-09-16T00:29:30.097 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4159 2024-09-16T00:29:30.097 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4160 2024-09-16T00:29:30.097 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4161 2024-09-16T00:29:30.098 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4162 2024-09-16T00:29:30.102 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4163 2024-09-16T00:29:30.102 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4164 2024-09-16T00:29:30.102 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4165 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4166 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4167 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4168 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4169 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4170 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4171 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4172 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4173 2024-09-16T00:29:30.103 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4174 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 5002 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 548 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 549 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6102 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6104 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6106 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6110 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6111 2024-09-16T00:29:30.104 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6112 2024-09-16T00:29:30.105 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6113 2024-09-16T00:29:30.105 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6117 2024-09-16T00:29:30.105 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 826 2024-09-16T00:29:30.105 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 827 2024-09-16T00:29:30.105 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 828 2024-09-16T00:29:30.105 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 16 00:13 pg_filenode.map 2024-09-16T00:29:30.105 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 PG_VERSION 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:total 7777 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 16 00:13 . 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 16 00:13 .. 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 112 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 113 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 1247 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1247_fsm 2024-09-16T00:29:30.106 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1247_vm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 16 00:13 1249 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1249_fsm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1249_vm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 16 00:13 1255 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1255_fsm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1255_vm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 16 00:13 1259 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1259_fsm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1259_vm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 16 00:13 13344 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13344_fsm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13344_vm 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13346 2024-09-16T00:29:30.107 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13348 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13349_fsm 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349_vm 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13351 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13353 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13354_fsm 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354_vm 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13356 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13358 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13359_fsm 2024-09-16T00:29:30.108 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359_vm 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13361 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13363 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1417 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1418 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 174 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 175 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2187 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2224 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2328 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2336 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2337 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2579 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2600 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2600_fsm 2024-09-16T00:29:30.109 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2600_vm 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2601_fsm 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601_vm 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2602 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2602_fsm 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2602_vm 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2603 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2603_fsm 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2603_vm 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2604 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2605 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2605_fsm 2024-09-16T00:29:30.110 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2605_vm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2606_fsm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2606_vm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2607 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2607_fsm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2607_vm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 16 00:13 2608 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2608_fsm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2608_vm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 16 00:13 2609 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2609_fsm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2609_vm 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2610 2024-09-16T00:29:30.111 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2610_fsm 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2610_vm 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2611 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2612_fsm 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612_vm 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2613 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2615_fsm 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615_vm 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2616 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2616_fsm 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2616_vm 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 16 00:13 2617 2024-09-16T00:29:30.112 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2617_fsm 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2617_vm 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 16 00:13 2618 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2618_fsm 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2618_vm 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 16 00:13 2619 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2619_fsm 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2619_vm 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2620 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2650 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2651 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2652 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2653 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2654 2024-09-16T00:29:30.113 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2655 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2656 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2657 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 16 00:13 2658 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:13 2659 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2660 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2661 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2662 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2663 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2664 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2665 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2666 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2667 2024-09-16T00:29:30.114 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2668 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2669 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2670 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 16 00:13 2673 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 16 00:13 2674 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 16 00:13 2675 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2678 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2679 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2680 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2681 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2682 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2683 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2684 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2685 2024-09-16T00:29:30.115 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2686 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2687 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2688 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2689 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 16 00:13 2690 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 16 00:13 2691 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2692 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2693 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2696 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2699 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2701 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2702 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2703 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2704 2024-09-16T00:29:30.116 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2753 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2753_fsm 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2753_vm 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2754 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2755 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2756 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2757 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2830 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2831 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2832 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2833 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2834 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2835 2024-09-16T00:29:30.117 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2836 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2837 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 16 00:13 2838 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2838_fsm 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2838_vm 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2839 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840_fsm 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2840_vm 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2841 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2995 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2996 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3079_fsm 2024-09-16T00:29:30.118 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079_vm 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3080 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3081 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3085 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3118 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3119 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 16 00:13 3164 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3256 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3257 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3258 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3350 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3351 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3379 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3380 2024-09-16T00:29:30.119 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3381 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3394 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3394_fsm 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3394_vm 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3395 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3429 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3430 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3431 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3433 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3439 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3440 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3455 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 16 00:13 3456 2024-09-16T00:29:30.120 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3456_fsm 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3456_vm 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3466 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3467 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3468 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3501 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3502 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3503 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3534 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3541_fsm 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541_vm 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3542 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3574 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3575 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3576 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3596 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3597 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3598 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3599 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3600_fsm 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600_vm 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3601_fsm 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601_vm 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3602_fsm 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602_vm 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603_fsm 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3603_vm 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3604 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3605 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3606 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3607 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3608 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 3609 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3712 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3764_fsm 2024-09-16T00:29:30.123 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764_vm 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3766 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3767 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3997 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4143 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4144 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4145 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4146 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4147 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4148 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4149 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4150 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4151 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4152 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4153 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4154 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4155 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4156 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4157 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4158 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4159 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4160 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4161 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4162 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4163 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4164 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4165 2024-09-16T00:29:30.126 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4166 2024-09-16T00:29:30.126 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4167 2024-09-16T00:29:30.126 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4168 2024-09-16T00:29:30.126 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4169 2024-09-16T00:29:30.126 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4170 2024-09-16T00:29:30.126 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4171 2024-09-16T00:29:30.126 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4172 2024-09-16T00:29:30.126 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4173 2024-09-16T00:29:30.127 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4174 2024-09-16T00:29:30.127 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 5002 2024-09-16T00:29:30.127 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 548 2024-09-16T00:29:30.127 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 549 2024-09-16T00:29:30.127 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6102 2024-09-16T00:29:30.127 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6104 2024-09-16T00:29:30.128 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6106 2024-09-16T00:29:30.128 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6110 2024-09-16T00:29:30.128 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6111 2024-09-16T00:29:30.128 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6112 2024-09-16T00:29:30.128 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6113 2024-09-16T00:29:30.128 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6117 2024-09-16T00:29:30.128 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 826 2024-09-16T00:29:30.129 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 827 2024-09-16T00:29:30.129 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 828 2024-09-16T00:29:30.129 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 16 00:13 pg_filenode.map 2024-09-16T00:29:30.129 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 PG_VERSION 2024-09-16T00:29:30.129 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.129 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-16T00:29:30.129 INFO:teuthology.orchestra.run.smithi078.stdout:total 552574 2024-09-16T00:29:30.130 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 16 00:20 . 2024-09-16T00:29:30.130 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 16 00:13 .. 2024-09-16T00:29:30.130 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 112 2024-09-16T00:29:30.130 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 113 2024-09-16T00:29:30.130 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:14 1247 2024-09-16T00:29:30.130 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1247_fsm 2024-09-16T00:29:30.130 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:14 1247_vm 2024-09-16T00:29:30.131 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 16 00:14 1249 2024-09-16T00:29:30.131 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:14 1249_fsm 2024-09-16T00:29:30.131 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:14 1249_vm 2024-09-16T00:29:30.131 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 16 00:13 1255 2024-09-16T00:29:30.131 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1255_fsm 2024-09-16T00:29:30.131 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1255_vm 2024-09-16T00:29:30.132 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 16 00:29 1259 2024-09-16T00:29:30.132 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1259_fsm 2024-09-16T00:29:30.132 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:14 1259_vm 2024-09-16T00:29:30.132 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 16 00:13 13344 2024-09-16T00:29:30.132 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13344_fsm 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13344_vm 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13346 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13348 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13349_fsm 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13349_vm 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13351 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13353 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354 2024-09-16T00:29:30.133 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13354_fsm 2024-09-16T00:29:30.134 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13354_vm 2024-09-16T00:29:30.134 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13356 2024-09-16T00:29:30.134 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13358 2024-09-16T00:29:30.134 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359 2024-09-16T00:29:30.134 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 13359_fsm 2024-09-16T00:29:30.134 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13359_vm 2024-09-16T00:29:30.134 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 13361 2024-09-16T00:29:30.135 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 13363 2024-09-16T00:29:30.135 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1417 2024-09-16T00:29:30.135 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 1418 2024-09-16T00:29:30.135 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 440287232 Sep 16 00:29 16396 2024-09-16T00:29:30.135 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 16 00:29 16396_fsm 2024-09-16T00:29:30.135 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:21 16396_vm 2024-09-16T00:29:30.135 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 16 00:29 16397 2024-09-16T00:29:30.136 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:29 16397_fsm 2024-09-16T00:29:30.136 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:29 16397_vm 2024-09-16T00:29:30.136 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1286144 Sep 16 00:29 16399 2024-09-16T00:29:30.136 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:29 16399_fsm 2024-09-16T00:29:30.136 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:26 16399_vm 2024-09-16T00:29:30.136 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:29 16400 2024-09-16T00:29:30.136 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 16 00:29 16402 2024-09-16T00:29:30.137 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 16 00:29 16404 2024-09-16T00:29:30.137 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 43376640 Sep 16 00:29 16406 2024-09-16T00:29:30.137 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:29 16406_fsm 2024-09-16T00:29:30.137 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:27 16406_vm 2024-09-16T00:29:30.137 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 174 2024-09-16T00:29:30.137 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 175 2024-09-16T00:29:30.137 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2187 2024-09-16T00:29:30.138 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2224 2024-09-16T00:29:30.138 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2328 2024-09-16T00:29:30.138 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2336 2024-09-16T00:29:30.138 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2337 2024-09-16T00:29:30.138 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:14 2579 2024-09-16T00:29:30.138 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2600 2024-09-16T00:29:30.138 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2600_fsm 2024-09-16T00:29:30.139 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2600_vm 2024-09-16T00:29:30.139 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601 2024-09-16T00:29:30.139 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2601_fsm 2024-09-16T00:29:30.139 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2601_vm 2024-09-16T00:29:30.139 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2602 2024-09-16T00:29:30.139 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2602_fsm 2024-09-16T00:29:30.139 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2602_vm 2024-09-16T00:29:30.140 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2603 2024-09-16T00:29:30.140 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2603_fsm 2024-09-16T00:29:30.140 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2603_vm 2024-09-16T00:29:30.140 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2604 2024-09-16T00:29:30.140 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2605 2024-09-16T00:29:30.140 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2605_fsm 2024-09-16T00:29:30.140 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2605_vm 2024-09-16T00:29:30.141 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:14 2606 2024-09-16T00:29:30.141 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2606_fsm 2024-09-16T00:29:30.141 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:14 2606_vm 2024-09-16T00:29:30.141 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2607 2024-09-16T00:29:30.141 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2607_fsm 2024-09-16T00:29:30.141 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2607_vm 2024-09-16T00:29:30.141 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 16 00:14 2608 2024-09-16T00:29:30.142 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2608_fsm 2024-09-16T00:29:30.142 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:14 2608_vm 2024-09-16T00:29:30.142 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 16 00:13 2609 2024-09-16T00:29:30.142 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2609_fsm 2024-09-16T00:29:30.142 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2609_vm 2024-09-16T00:29:30.142 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:14 2610 2024-09-16T00:29:30.142 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2610_fsm 2024-09-16T00:29:30.142 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:14 2610_vm 2024-09-16T00:29:30.143 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2611 2024-09-16T00:29:30.143 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612 2024-09-16T00:29:30.143 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2612_fsm 2024-09-16T00:29:30.143 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2612_vm 2024-09-16T00:29:30.143 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2613 2024-09-16T00:29:30.143 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615 2024-09-16T00:29:30.143 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2615_fsm 2024-09-16T00:29:30.144 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2615_vm 2024-09-16T00:29:30.144 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2616 2024-09-16T00:29:30.144 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2616_fsm 2024-09-16T00:29:30.144 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2616_vm 2024-09-16T00:29:30.144 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 16 00:13 2617 2024-09-16T00:29:30.144 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2617_fsm 2024-09-16T00:29:30.144 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2617_vm 2024-09-16T00:29:30.145 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 16 00:13 2618 2024-09-16T00:29:30.145 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2618_fsm 2024-09-16T00:29:30.145 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2618_vm 2024-09-16T00:29:30.145 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Sep 16 00:29 2619 2024-09-16T00:29:30.145 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:27 2619_fsm 2024-09-16T00:29:30.145 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:23 2619_vm 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2620 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2650 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2651 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2652 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2653 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2654 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2655 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2656 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2657 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 16 00:14 2658 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 16 00:14 2659 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2660 2024-09-16T00:29:30.146 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2661 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:14 2662 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:14 2663 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:14 2664 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:14 2665 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:14 2666 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:14 2667 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2668 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2669 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2670 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 16 00:14 2673 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 16 00:14 2674 2024-09-16T00:29:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 16 00:13 2675 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:14 2678 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:14 2679 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2680 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2681 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2682 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2683 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2684 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2685 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2686 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2687 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 2688 2024-09-16T00:29:30.148 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 16 00:13 2689 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 16 00:13 2690 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 16 00:13 2691 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2692 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2693 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:28 2696 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2699 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2701 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2702 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:14 2703 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:14 2704 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2753 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2753_fsm 2024-09-16T00:29:30.149 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2753_vm 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2754 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2755 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2756 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 2757 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2830 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2831 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2832 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2833 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2834 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2835 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2836 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2837 2024-09-16T00:29:30.150 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 16 00:13 2838 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2838_fsm 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2838_vm 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2839 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2840_fsm 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2840_vm 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2841 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2995 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2996 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3079_fsm 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3079_vm 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3080 2024-09-16T00:29:30.151 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3081 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:13 3085 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3118 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3119 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 16 00:13 3164 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3256 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3257 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3258 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3350 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3351 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3379 2024-09-16T00:29:30.152 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3380 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3381 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3394 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3394_fsm 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3394_vm 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3395 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3429 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3430 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3431 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3433 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3439 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3440 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 16 00:14 3455 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 16 00:13 3456 2024-09-16T00:29:30.153 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3456_fsm 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3456_vm 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3466 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3467 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3468 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3501 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3502 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3503 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3534 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3541_fsm 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3541_vm 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3542 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3574 2024-09-16T00:29:30.154 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3575 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3576 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3596 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3597 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3598 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3599 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3600_fsm 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3600_vm 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3601_fsm 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3601_vm 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602 2024-09-16T00:29:30.155 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3602_fsm 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3602_vm 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3603_fsm 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3603_vm 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3604 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3605 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3606 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3607 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3608 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 16 00:13 3609 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3712 2024-09-16T00:29:30.156 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 3764_fsm 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3764_vm 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3766 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 3767 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3997 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4143 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4144 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4145 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4146 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4147 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4148 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4149 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4150 2024-09-16T00:29:30.157 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4151 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4152 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4153 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4154 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4155 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4156 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4157 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4158 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4159 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4160 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4161 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4162 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4163 2024-09-16T00:29:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4164 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4165 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4166 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4167 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4168 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4169 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4170 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4171 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4172 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4173 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4174 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 5002 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 548 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 549 2024-09-16T00:29:30.159 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6102 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6104 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6106 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6110 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6111 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6112 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6113 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6117 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 826 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 827 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 828 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 16 00:13 pg_filenode.map 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 16 00:13 pg_internal.init 2024-09-16T00:29:30.160 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:13 PG_VERSION 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:14 . 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 16 00:13 .. 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/global: 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:total 560 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 16 00:13 . 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1213 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1213_fsm 2024-09-16T00:29:30.161 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1213_vm 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1214 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1214_fsm 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1214_vm 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 1232 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 1233 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1260 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1260_fsm 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1260_vm 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1261 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1261_fsm 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1261_vm 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1262 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 1262_fsm 2024-09-16T00:29:30.162 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 1262_vm 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2396 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 16 00:13 2396_fsm 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2396_vm 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2397 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2671 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2672 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2676 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2677 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2694 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2695 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2697 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 16 00:13 2698 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2846 2024-09-16T00:29:30.163 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2847 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2964 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2965 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 2966 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 2967 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 3592 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 3593 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4060 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4061 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4175 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4176 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4177 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4178 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4181 2024-09-16T00:29:30.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4182 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4183 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4184 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 4185 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 4186 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6000 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6001 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6002 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 6100 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6114 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 6115 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:29 pg_control 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 16 00:13 pg_filenode.map 2024-09-16T00:29:30.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 16 00:13 pg_internal.init 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/log: 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:total 2 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 Sep 16 00:29 postgresql-Mon.log 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-16T00:29:30.166 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:total 1 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:29 . 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 mappings 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 16 00:29 replorigin_checkpoint 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 snapshots 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.167 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:29 .. 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:29 .. 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-16T00:29:30.168 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 . 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 members 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 offsets 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:total 8 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 .. 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:13 0000 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-16T00:29:30.169 INFO:teuthology.orchestra.run.smithi078.stdout:total 8 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 16 00:13 .. 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 16 00:18 0000 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.170 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:total 15 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 16 00:29 . 2024-09-16T00:29:30.171 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 16 00:29 db_0.stat 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 16 00:29 db_13494.stat 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 16 00:29 global.stat 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:29 . 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:total 144 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:29 . 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 16 00:29 000C 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-16T00:29:30.173 INFO:teuthology.orchestra.run.smithi078.stdout:total 983040 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 61 Sep 16 00:29 . 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:29 000000010000000000000064 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:23 000000010000000000000065 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:18 000000010000000000000066 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 000000010000000000000067 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000068 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000069 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 00000001000000000000006A 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 00000001000000000000006B 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 00000001000000000000006C 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 00000001000000000000006D 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 00000001000000000000006E 2024-09-16T00:29:30.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:21 00000001000000000000006F 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:21 000000010000000000000070 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 000000010000000000000071 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000072 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 000000010000000000000073 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 000000010000000000000074 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 000000010000000000000075 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000076 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000077 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000078 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000079 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:22 00000001000000000000007A 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 00000001000000000000007B 2024-09-16T00:29:30.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 00000001000000000000007C 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 00000001000000000000007D 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:18 00000001000000000000007E 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:20 00000001000000000000007F 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:23 000000010000000000000080 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000081 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000082 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:19 000000010000000000000083 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 000000010000000000000084 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:25 000000010000000000000085 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:25 000000010000000000000086 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:26 000000010000000000000087 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:26 000000010000000000000088 2024-09-16T00:29:30.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:28 000000010000000000000089 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 00000001000000000000008A 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:25 00000001000000000000008B 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:28 00000001000000000000008C 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:26 00000001000000000000008D 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:25 00000001000000000000008E 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:27 00000001000000000000008F 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 000000010000000000000090 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:26 000000010000000000000091 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:23 000000010000000000000092 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:25 000000010000000000000093 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 000000010000000000000094 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:25 000000010000000000000095 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:26 000000010000000000000096 2024-09-16T00:29:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 000000010000000000000097 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 000000010000000000000098 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 000000010000000000000099 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 00000001000000000000009A 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:27 00000001000000000000009B 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:25 00000001000000000000009C 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:24 00000001000000000000009D 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:23 00000001000000000000009E 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 16 00:26 00000001000000000000009F 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 archive_status 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:total 0 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 16 00:13 . 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 61 Sep 16 00:29 .. 2024-09-16T00:29:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:30.179 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-16T00:29:30.179 INFO:teuthology.orchestra.run.smithi078.stdout:total 208 2024-09-16T00:29:30.179 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 16 00:13 . 2024-09-16T00:29:30.179 INFO:teuthology.orchestra.run.smithi078.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 16 00:29 .. 2024-09-16T00:29:30.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Sep 16 00:29 0000 2024-09-16T00:29:30.179 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:29 smithi078 ceph-mon[31157]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-09-16T00:29:30.180 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-16T00:29:30.191 INFO:tasks.quiescer:joining Quiescers 2024-09-16T00:29:30.191 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-16T00:29:30.192 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:30.192 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:29 smithi193 ceph-mon[35484]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-09-16T00:29:30.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:29 smithi148 ceph-mon[35599]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-09-16T00:29:31.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:31.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:31.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:29:31.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:31.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:31.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:29:31.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:31.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:29:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:31 smithi148 ceph-mon[35599]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-09-16T00:29:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:31 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:29:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:31 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:29:32.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:31 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:32.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:31 smithi078 ceph-mon[31157]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-09-16T00:29:32.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:31 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:29:32.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:31 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:29:32.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:31 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:32.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 -- 172.21.15.78:0/20129539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac100e20 msgr2=0x7f33ac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 --2- 172.21.15.78:0/20129539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac100e20 0x7f33ac101200 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3394009ca0 tx=0x7f339402a950 comp rx=0 tx=0).stop 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 -- 172.21.15.78:0/20129539 shutdown_connections 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 --2- 172.21.15.78:0/20129539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac102ef0 0x7f33ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 --2- 172.21.15.78:0/20129539 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ac102570 0x7f33ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 --2- 172.21.15.78:0/20129539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac100e20 0x7f33ac101200 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 -- 172.21.15.78:0/20129539 >> 172.21.15.78:0/20129539 conn(0x7f33ac0fc7d0 msgr2=0x7f33ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 -- 172.21.15.78:0/20129539 shutdown_connections 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 -- 172.21.15.78:0/20129539 wait complete. 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 Processor -- start 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.105+0000 7f33b304c640 1 -- start start 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b304c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ac100e20 0x7f33ac1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b304c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac102570 0x7f33ac1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b304c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102ef0 0x7f33ac19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b304c640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1120a0 con 0x7f33ac102ef0 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b304c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac111f20 con 0x7f33ac100e20 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b304c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac112220 con 0x7f33ac102570 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102ef0 0x7f33ac19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102ef0 0x7f33ac19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49328/0 (socket says 172.21.15.78:49328) 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 -- 172.21.15.78:0/2102092929 learned_addr learned my addr 172.21.15.78:0/2102092929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33abfff640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac102570 0x7f33ac1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b0de8640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ac100e20 0x7f33ac1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 -- 172.21.15.78:0/2102092929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac102570 msgr2=0x7f33ac1a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac102570 0x7f33ac1a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 -- 172.21.15.78:0/2102092929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ac100e20 msgr2=0x7f33ac1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ac100e20 0x7f33ac1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 -- 172.21.15.78:0/2102092929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac19acc0 con 0x7f33ac102ef0 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33abfff640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac102570 0x7f33ac1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b15e9640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102ef0 0x7f33ac19a590 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009fd0 tx=0x7f33a400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33a9ffb640 1 -- 172.21.15.78:0/2102092929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4011830 con 0x7f33ac102ef0 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33a9ffb640 1 -- 172.21.15.78:0/2102092929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a4016070 con 0x7f33ac102ef0 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33b304c640 1 -- 172.21.15.78:0/2102092929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac19afb0 con 0x7f33ac102ef0 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.106+0000 7f33a9ffb640 1 -- 172.21.15.78:0/2102092929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a400f2e0 con 0x7f33ac102ef0 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.107+0000 7f33b304c640 1 -- 172.21.15.78:0/2102092929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac1a7320 con 0x7f33ac102ef0 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.109+0000 7f33a9ffb640 1 -- 172.21.15.78:0/2102092929 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f33a40119d0 con 0x7f33ac102ef0 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.109+0000 7f33b5425740 1 Processor -- start 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.109+0000 7f33b5425740 1 -- start start 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33a9ffb640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33980a32c0 0x7f33980a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b5425740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74233b60 0x562e74233f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b5425740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b5425740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e7423ed40 0x562e74241130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b5425740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e74199280 con 0x562e74233b60 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b5425740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562e74140b10 con 0x562e74234480 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b5425740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e74195240 con 0x562e7423ed40 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33abfff640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33980a32c0 0x7f33980a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33a9ffb640 1 -- 172.21.15.78:0/2102092929 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f33a40cf330 con 0x7f33ac102ef0 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74233b60 0x562e74233f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33abfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74233b60 0x562e74233f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49330/0 (socket says 172.21.15.78:49330) 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 -- 172.21.15.78:0/3883176129 learned_addr learned my addr 172.21.15.78:0/3883176129 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 -- 172.21.15.78:0/3883176129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e7423ed40 msgr2=0x562e74241130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b15e9640 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e7423ed40 0x562e74241130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e7423ed40 0x562e74241130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 -- 172.21.15.78:0/3883176129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 msgr2=0x562e7423e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 -- 172.21.15.78:0/3883176129 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e7414d1f0 con 0x562e74233b60 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b15e9640 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e7423ed40 0x562e74241130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33abfff640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33980a32c0 0x7f33980a5780 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f33ac105650 tx=0x7f339c00bbf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.110+0000 7f33b0de8640 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74233b60 0x562e74233f40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3394030020 tx=0x7f339402f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7ef5ddea861737 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33a2ffd640 1 -- 172.21.15.78:0/3883176129 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394034040 con 0x562e74233b60 2024-09-16T00:29:32.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33a2ffd640 1 -- 172.21.15.78:0/3883176129 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3394003970 con 0x562e74233b60 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33a2ffd640 1 -- 172.21.15.78:0/3883176129 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394004640 con 0x562e74233b60 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 -- 172.21.15.78:0/3883176129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74233b60 msgr2=0x562e74233f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74233b60 0x562e74233f40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3394030020 tx=0x7f339402f930 comp rx=0 tx=0).stop 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 -- 172.21.15.78:0/3883176129 shutdown_connections 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e7423ed40 0x562e74241130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 --2- 172.21.15.78:0/3883176129 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74233b60 0x562e74233f40 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 -- 172.21.15.78:0/3883176129 >> 172.21.15.78:0/3883176129 conn(0x562e742300f0 msgr2=0x562e742304d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 -- 172.21.15.78:0/3883176129 shutdown_connections 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.111+0000 7f33b5425740 1 -- 172.21.15.78:0/3883176129 wait complete. 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 1 -- start start 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 1 Processor -- start 2024-09-16T00:29:32.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e7423bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e7423c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x562e74241910 con 0x562e7423ed40 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562e74153f10 con 0x562e74234480 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e74241a90 con 0x562e74233b60 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b5425740 10 client.0 _ll_register_callbacks cb 0x562e7411ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b15e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e7423c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b15e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e7423c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49336/0 (socket says 172.21.15.78:49336) 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.112+0000 7f33b15e9640 1 -- 172.21.15.78:0/2251130523 learned_addr learned my addr 172.21.15.78:0/2251130523 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b0de8640 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e7423bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33abfff640 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b15e9640 1 -- 172.21.15.78:0/2251130523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 msgr2=0x562e7423bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b15e9640 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e7423bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b15e9640 1 -- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 msgr2=0x562e7423c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b15e9640 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b15e9640 1 -- 172.21.15.78:0/2251130523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e73f0a6a0 con 0x562e7423ed40 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33abfff640 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b15e9640 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e7423c640 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000c00 tx=0x7f33a409b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31ae2fb27ef662c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f338bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f338bfff640 1 -- 172.21.15.78:0/2251130523 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a409b590 con 0x562e7423ed40 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f338bfff640 1 -- 172.21.15.78:0/2251130523 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f33a4010510 con 0x562e7423ed40 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b5425740 1 -- 172.21.15.78:0/2251130523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3394003970 con 0x562e7423ed40 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f338bfff640 1 -- 172.21.15.78:0/2251130523 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f33a4055020 con 0x562e7423ed40 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b5425740 10 client.18345 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f33b5425740 1 -- 172.21.15.78:0/2251130523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e7424d4c0 con 0x562e7423ed40 2024-09-16T00:29:32.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.113+0000 7f338bfff640 1 -- 172.21.15.78:0/2251130523 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3394003970 con 0x562e7423ed40 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f338bfff640 1 -- 172.21.15.78:0/2251130523 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f33a4097c00 con 0x562e7423ed40 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f33b5425740 10 client.18345 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f33b5425740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f33b5425740 20 client.18345 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f33b5425740 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562e7424e920 0x562e74250d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f33b5425740 4 client.18345 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f33b5425740 1 -- 172.21.15.78:0/2251130523 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e741a67f0 con 0x562e7424e920 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f33b0de8640 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562e7424e920 0x562e74250d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f338bfff640 10 client.18345 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.114+0000 7f33b0de8640 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562e7424e920 0x562e74250d80 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e738af5ad89bd0 server_cookie=cfe223b0cfa549e4 in_seq=0 out_seq=0 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f338bfff640 1 -- 172.21.15.78:0/2251130523 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e741a67f0 con 0x562e7424e920 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f338bfff640 10 client.18345 handle_command_reply: tid=0 2024-09-16T00:29:32.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 client.18345 shutdown 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 -- 172.21.15.78:0/2251130523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 msgr2=0x562e7423c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e7423c640 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000c00 tx=0x7f33a409b470 comp rx=0 tx=0).stop 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 -- 172.21.15.78:0/2251130523 shutdown_connections 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562e7424e920 0x562e74250d80 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e7423c640 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e7423c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 --2- 172.21.15.78:0/2251130523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e7423bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f338bfff640 0 client.18345 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.115+0000 7f33b5425740 1 -- 172.21.15.78:0/2251130523 >> 172.21.15.78:0/2251130523 conn(0x562e742300f0 msgr2=0x562e74235e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.116+0000 7f33b5425740 1 -- 172.21.15.78:0/2251130523 shutdown_connections 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.116+0000 7f33b5425740 1 -- 172.21.15.78:0/2251130523 wait complete. 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.116+0000 7f33b5425740 20 client.18345 trim_cache size 0 max 16384 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b5425740 1 Processor -- start 2024-09-16T00:29:32.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b5425740 1 -- start start 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b5425740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e7423da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b5425740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74234480 0x562e74256650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b5425740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e7423ed40 0x562e74256b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b5425740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1b0ea0 con 0x562e74234480 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b5425740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1b29e0 con 0x562e7423ed40 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b5425740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1b1570 con 0x562e74233b60 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74234480 0x562e74256650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33abfff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74234480 0x562e74256650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49342/0 (socket says 172.21.15.78:49342) 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33abfff640 1 -- 172.21.15.78:0/1767014226 learned_addr learned my addr 172.21.15.78:0/1767014226 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b15e9640 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e7423ed40 0x562e74256b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33abfff640 1 -- 172.21.15.78:0/1767014226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 msgr2=0x562e7423da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33abfff640 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e7423da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33abfff640 1 -- 172.21.15.78:0/1767014226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e7423ed40 msgr2=0x562e74256b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33abfff640 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e7423ed40 0x562e74256b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33abfff640 1 -- 172.21.15.78:0/1767014226 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e7422b830 con 0x562e74234480 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.133+0000 7f33b15e9640 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e7423ed40 0x562e74256b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33abfff640 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74234480 0x562e74256650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f339c0058c0 tx=0x7f339c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2864312ef2532de server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33a27fc640 1 -- 172.21.15.78:0/1767014226 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c011cd0 con 0x562e74234480 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33a27fc640 1 -- 172.21.15.78:0/1767014226 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c011e70 con 0x562e74234480 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33a27fc640 1 -- 172.21.15.78:0/1767014226 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0218f0 con 0x562e74234480 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 -- 172.21.15.78:0/1767014226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74234480 msgr2=0x562e74256650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74234480 0x562e74256650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f339c0058c0 tx=0x7f339c005850 comp rx=0 tx=0).stop 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 -- 172.21.15.78:0/1767014226 shutdown_connections 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e7423ed40 0x562e74256b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e74234480 0x562e74256650 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 --2- 172.21.15.78:0/1767014226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e7423da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 -- 172.21.15.78:0/1767014226 >> 172.21.15.78:0/1767014226 conn(0x562e742300f0 msgr2=0x562e7423d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 -- 172.21.15.78:0/1767014226 shutdown_connections 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.134+0000 7f33b5425740 1 -- 172.21.15.78:0/1767014226 wait complete. 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 1 -- start start 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 1 Processor -- start 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e74242b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e742430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e74243600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1b2310 con 0x562e7423ed40 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1b1c40 con 0x562e74234480 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac1aee80 con 0x562e74233b60 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b5425740 10 client.0 _ll_register_callbacks cb 0x7f33a4097170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b15e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e74243600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.135+0000 7f33b15e9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e74243600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49354/0 (socket says 172.21.15.78:49354) 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b15e9640 1 -- 172.21.15.78:0/3570221368 learned_addr learned my addr 172.21.15.78:0/3570221368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33abfff640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e742430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b0de8640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e74242b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b15e9640 1 -- 172.21.15.78:0/3570221368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 msgr2=0x562e74242b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b15e9640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e74242b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b15e9640 1 -- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 msgr2=0x562e742430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b15e9640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e742430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b15e9640 1 -- 172.21.15.78:0/3570221368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e742b8780 con 0x562e7423ed40 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33abfff640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e742430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b0de8640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e74242b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b15e9640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e74243600 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f33a4097f60 tx=0x7f33a4010400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf07ac5ec56fd550 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f338bfff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:32.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f338bfff640 1 -- 172.21.15.78:0/3570221368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4097830 con 0x562e7423ed40 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f338bfff640 1 -- 172.21.15.78:0/3570221368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f33a4097c00 con 0x562e7423ed40 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b5425740 1 -- 172.21.15.78:0/3570221368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339c011e70 con 0x562e7423ed40 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f338bfff640 1 -- 172.21.15.78:0/3570221368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f33a409aef0 con 0x562e7423ed40 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b5425740 10 client.18351 fetch_fsmap learned FSMap version 34 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f33b5425740 1 -- 172.21.15.78:0/3570221368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e742b99b0 con 0x562e7423ed40 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.136+0000 7f338bfff640 1 -- 172.21.15.78:0/3570221368 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c011e70 con 0x562e7423ed40 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f338bfff640 1 -- 172.21.15.78:0/3570221368 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f33a409a500 con 0x562e7423ed40 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f33b5425740 10 client.18351 fetch_fsmap finished waiting for FSMap version 34 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f33b5425740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:32.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f33b5425740 20 client.18351 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f33b5425740 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562e742ba070 0x562e742bc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f33b5425740 4 client.18351 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f33b5425740 1 -- 172.21.15.78:0/3570221368 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e741a0d00 con 0x562e742ba070 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f33b0de8640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562e742ba070 0x562e742bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f33b0de8640 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562e742ba070 0x562e742bc490 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910bfa250e799e05 server_cookie=adc76167427f2cc in_seq=0 out_seq=0 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.137+0000 7f338bfff640 10 client.18351 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f338bfff640 1 -- 172.21.15.78:0/3570221368 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e741a0d00 con 0x562e742ba070 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f338bfff640 10 client.18351 handle_command_reply: tid=0 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 client.18351 shutdown 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 -- 172.21.15.78:0/3570221368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 msgr2=0x562e74243600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e74243600 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f33a4097f60 tx=0x7f33a4010400 comp rx=0 tx=0).stop 2024-09-16T00:29:32.364 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:29:32.364 INFO:teuthology.orchestra.run.smithi078.stdout: "return_code": 0, 2024-09-16T00:29:32.364 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_tag": "9f7ada4e-ae07-419b-ad78-5b773241d74a", 2024-09-16T00:29:32.364 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": "asynchronous" 2024-09-16T00:29:32.364 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 -- 172.21.15.78:0/3570221368 shutdown_connections 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x562e742ba070 0x562e742bc490 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x562e7423ed40 0x562e74243600 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562e74234480 0x562e742430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f338bfff640 0 client.18351 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 --2- 172.21.15.78:0/3570221368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e74233b60 0x562e74242b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.138+0000 7f33b5425740 1 -- 172.21.15.78:0/3570221368 >> 172.21.15.78:0/3570221368 conn(0x562e742300f0 msgr2=0x562e7423c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b5425740 1 -- 172.21.15.78:0/3570221368 shutdown_connections 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b5425740 1 -- 172.21.15.78:0/3570221368 wait complete. 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b5425740 20 client.18351 trim_cache size 0 max 16384 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 -- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33980a32c0 msgr2=0x7f33980a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33980a32c0 0x7f33980a5780 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f33ac105650 tx=0x7f339c00bbf0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 -- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102ef0 msgr2=0x7f33ac19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102ef0 0x7f33ac19a590 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009fd0 tx=0x7f33a400b8e0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 -- 172.21.15.78:0/2102092929 shutdown_connections 2024-09-16T00:29:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f33980a32c0 0x7f33980a5780 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33ac102ef0 0x7f33ac19a590 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33ac102570 0x7f33ac1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 --2- 172.21.15.78:0/2102092929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f33ac100e20 0x7f33ac1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 -- 172.21.15.78:0/2102092929 >> 172.21.15.78:0/2102092929 conn(0x7f33ac0fc7d0 msgr2=0x7f33ac105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 -- 172.21.15.78:0/2102092929 shutdown_connections 2024-09-16T00:29:32.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:31.139+0000 7f33b304c640 1 -- 172.21.15.78:0/2102092929 wait complete. 2024-09-16T00:29:32.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:31 smithi193 ceph-mon[35484]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.0 MiB/s wr, 675 op/s 2024-09-16T00:29:32.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:31 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:29:32.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:31 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:29:32.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:31 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:32 smithi148 ceph-mon[35599]: scrub queued for path: / 2024-09-16T00:29:33.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:32 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:32 smithi148 ceph-mon[35599]: scrub summary: active paths [/] 2024-09-16T00:29:33.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:32 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:32 smithi148 ceph-mon[35599]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:32 smithi078 ceph-mon[31157]: scrub queued for path: / 2024-09-16T00:29:33.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:32 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:32 smithi078 ceph-mon[31157]: scrub summary: active paths [/] 2024-09-16T00:29:33.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:32 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:32 smithi078 ceph-mon[31157]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:32 smithi193 ceph-mon[35484]: scrub queued for path: / 2024-09-16T00:29:33.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:32 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:32 smithi193 ceph-mon[35484]: scrub summary: active paths [/] 2024-09-16T00:29:33.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:32 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:33.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:32 smithi193 ceph-mon[35484]: scrub summary: idle+waiting paths [/] 2024-09-16T00:29:34.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:33 smithi078 ceph-mon[31157]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 640 op/s 2024-09-16T00:29:34.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:33 smithi193 ceph-mon[35484]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 640 op/s 2024-09-16T00:29:34.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:33 smithi148 ceph-mon[35599]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 MiB/s wr, 640 op/s 2024-09-16T00:29:34.475 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:29:34.511 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:35.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.055+0000 7f3c78373640 1 -- 172.21.15.78:0/3047777133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102570 msgr2=0x7f3c701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.055+0000 7f3c78373640 1 --2- 172.21.15.78:0/3047777133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102570 0x7f3c701029b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009900 tx=0x7f3c6c02a480 comp rx=0 tx=0).stop 2024-09-16T00:29:35.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 -- 172.21.15.78:0/3047777133 shutdown_connections 2024-09-16T00:29:35.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 --2- 172.21.15.78:0/3047777133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c70102ef0 0x7f3c7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 --2- 172.21.15.78:0/3047777133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102570 0x7f3c701029b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 --2- 172.21.15.78:0/3047777133 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c70100e20 0x7f3c70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 -- 172.21.15.78:0/3047777133 >> 172.21.15.78:0/3047777133 conn(0x7f3c700fc7d0 msgr2=0x7f3c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 -- 172.21.15.78:0/3047777133 shutdown_connections 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 -- 172.21.15.78:0/3047777133 wait complete. 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 Processor -- start 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.056+0000 7f3c78373640 1 -- start start 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c78373640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c70100e20 0x7f3c701983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c78373640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c70102570 0x7f3c70198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c78373640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c78373640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701120a0 con 0x7f3c70102ef0 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c78373640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70111f20 con 0x7f3c70100e20 2024-09-16T00:29:35.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c78373640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70112220 con 0x7f3c70102570 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49378/0 (socket says 172.21.15.78:49378) 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 -- 172.21.15.78:0/4253403029 learned_addr learned my addr 172.21.15.78:0/4253403029 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c7610f640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c70100e20 0x7f3c701983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 -- 172.21.15.78:0/4253403029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c70102570 msgr2=0x7f3c70198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c70102570 0x7f3c70198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 -- 172.21.15.78:0/4253403029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c70100e20 msgr2=0x7f3c701983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c70100e20 0x7f3c701983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 -- 172.21.15.78:0/4253403029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7019d390 con 0x7f3c70102ef0 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c76910640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ccb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800cce0 tx=0x7f3c6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c637fe640 1 -- 172.21.15.78:0/4253403029 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800f450 con 0x7f3c70102ef0 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c637fe640 1 -- 172.21.15.78:0/4253403029 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6800fad0 con 0x7f3c70102ef0 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7019d680 con 0x7f3c70102ef0 2024-09-16T00:29:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c637fe640 1 -- 172.21.15.78:0/4253403029 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68018b60 con 0x7f3c70102ef0 2024-09-16T00:29:35.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.057+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701a4f60 con 0x7f3c70102ef0 2024-09-16T00:29:35.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.058+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70101200 con 0x7f3c70102ef0 2024-09-16T00:29:35.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.059+0000 7f3c637fe640 1 -- 172.21.15.78:0/4253403029 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3c6800fc80 con 0x7f3c70102ef0 2024-09-16T00:29:35.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.060+0000 7f3c637fe640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c5c0a3240 0x7f3c5c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.060+0000 7f3c7610f640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c5c0a3240 0x7f3c5c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.060+0000 7f3c637fe640 1 -- 172.21.15.78:0/4253403029 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f3c680d0200 con 0x7f3c70102ef0 2024-09-16T00:29:35.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.061+0000 7f3c7610f640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c5c0a3240 0x7f3c5c0a5700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400abe0 tx=0x7f3c6400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.064+0000 7f3c637fe640 1 -- 172.21.15.78:0/4253403029 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c68014030 con 0x7f3c70102ef0 2024-09-16T00:29:35.079 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:34 smithi078 ceph-mon[31157]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:29:35.141 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:35.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.287+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c701a5160 con 0x7f3c70102ef0 2024-09-16T00:29:35.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.288+0000 7f3c637fe640 1 -- 172.21.15.78:0/4253403029 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11987 (secure 0 0 0) 0x7f3c6809cb60 con 0x7f3c70102ef0 2024-09-16T00:29:35.289 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:29:35.289 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 35 2024-09-16T00:29:35.289 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":35,"btime":"2024-09-16T00:29:33:947759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:29:33.868463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:29:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c5c0a3240 msgr2=0x7f3c5c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c5c0a3240 0x7f3c5c0a5700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400abe0 tx=0x7f3c6400a2d0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102ef0 msgr2=0x7f3c7019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ccb0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800cce0 tx=0x7f3c6800a620 comp rx=0 tx=0).stop 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 shutdown_connections 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3c5c0a3240 0x7f3c5c0a5700 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c70102ef0 0x7f3c7019ccb0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3c70102570 0x7f3c70198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 --2- 172.21.15.78:0/4253403029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3c70100e20 0x7f3c701983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 >> 172.21.15.78:0/4253403029 conn(0x7f3c700fc7d0 msgr2=0x7f3c70106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 shutdown_connections 2024-09-16T00:29:35.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:35.291+0000 7f3c78373640 1 -- 172.21.15.78:0/4253403029 wait complete. 2024-09-16T00:29:35.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:34 smithi193 ceph-mon[35484]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:29:35.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:34 smithi148 ceph-mon[35599]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-16T00:29:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:35 smithi078 ceph-mon[31157]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-09-16T00:29:36.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:35 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4253403029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:36.174 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --cancel --all' 2024-09-16T00:29:36.174 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --cancel --all 2024-09-16T00:29:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 -- 172.21.15.78:0/112384118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f010c180 msgr2=0x7f65f010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 --2- 172.21.15.78:0/112384118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f010c180 0x7f65f010e620 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009ca0 tx=0x7f65e802a950 comp rx=0 tx=0).stop 2024-09-16T00:29:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 -- 172.21.15.78:0/112384118 shutdown_connections 2024-09-16T00:29:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 --2- 172.21.15.78:0/112384118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f010c180 0x7f65f010e620 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 --2- 172.21.15.78:0/112384118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f0103670 0x7f65f010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 --2- 172.21.15.78:0/112384118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f0102d50 0x7f65f0103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 -- 172.21.15.78:0/112384118 >> 172.21.15.78:0/112384118 conn(0x7f65f00fc7d0 msgr2=0x7f65f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:36.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 -- 172.21.15.78:0/112384118 shutdown_connections 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.404+0000 7f65f6f43640 1 -- 172.21.15.78:0/112384118 wait complete. 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f6f43640 1 Processor -- start 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f6f43640 1 -- start start 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f6f43640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f0102d50 0x7f65f0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f6f43640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f0103670 0x7f65f0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f6f43640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f010c180 0x7f65f019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f6f43640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65f0111f80 con 0x7f65f0103670 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f6f43640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65f0111e00 con 0x7f65f010c180 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f6f43640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65f0112100 con 0x7f65f0102d50 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f0103670 0x7f65f0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f0103670 0x7f65f0198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58036/0 (socket says 172.21.15.78:58036) 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65effff640 1 -- 172.21.15.78:0/289784475 learned_addr learned my addr 172.21.15.78:0/289784475 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f54e0640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f010c180 0x7f65f019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f0102d50 0x7f65f0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65effff640 1 -- 172.21.15.78:0/289784475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f0102d50 msgr2=0x7f65f0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65effff640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f0102d50 0x7f65f0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65effff640 1 -- 172.21.15.78:0/289784475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f010c180 msgr2=0x7f65f019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65effff640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f010c180 0x7f65f019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.405+0000 7f65effff640 1 -- 172.21.15.78:0/289784475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f019d3d0 con 0x7f65f0103670 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.406+0000 7f65effff640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f0103670 0x7f65f0198960 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00c970 tx=0x7f65dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.406+0000 7f65edffb640 1 -- 172.21.15.78:0/289784475 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc007bb0 con 0x7f65f0103670 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.406+0000 7f65f6f43640 1 -- 172.21.15.78:0/289784475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f019d6c0 con 0x7f65f0103670 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.406+0000 7f65edffb640 1 -- 172.21.15.78:0/289784475 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65dc007d50 con 0x7f65f0103670 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.406+0000 7f65edffb640 1 -- 172.21.15.78:0/289784475 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc018820 con 0x7f65f0103670 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.406+0000 7f65f6f43640 1 -- 172.21.15.78:0/289784475 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f01a4fa0 con 0x7f65f0103670 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.408+0000 7f65edffb640 1 -- 172.21.15.78:0/289784475 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f65dc0189c0 con 0x7f65f0103670 2024-09-16T00:29:36.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f931c740 1 Processor -- start 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65edffb640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65e00a3240 0x7f65e00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f931c740 1 -- start start 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f54e0640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65e00a3240 0x7f65e00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65edffb640 1 -- 172.21.15.78:0/289784475 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f65dc0cfc50 con 0x7f65f0103670 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f931c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f931c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5ded140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f931c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5ded680 0x5647c5defa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f931c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647c5d482e0 con 0x5647c5de24a0 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f931c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5647c5cefce0 con 0x5647c5de2dc0 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f931c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5647c5d442a0 con 0x5647c5ded680 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f4cdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f4cdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58040/0 (socket says 172.21.15.78:58040) 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f4cdf640 1 -- 172.21.15.78:0/1730670386 learned_addr learned my addr 172.21.15.78:0/1730670386 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:36.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65effff640 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5ded140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f4cdf640 1 -- 172.21.15.78:0/1730670386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5ded680 msgr2=0x5647c5defa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5ded680 0x5647c5defa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f4cdf640 1 -- 172.21.15.78:0/1730670386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 msgr2=0x5647c5ded140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5ded140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f4cdf640 1 -- 172.21.15.78:0/1730670386 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c5cd4440 con 0x5647c5de24a0 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65effff640 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5ded140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.409+0000 7f65f54e0640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65e00a3240 0x7f65e00a5700 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f65e8005e30 tx=0x7f65e8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de2880 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f65e400b760 tx=0x7f65e400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac83cb274f3afe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65daffd640 1 -- 172.21.15.78:0/1730670386 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4003d50 con 0x5647c5de24a0 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65daffd640 1 -- 172.21.15.78:0/1730670386 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4015070 con 0x5647c5de24a0 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65daffd640 1 -- 172.21.15.78:0/1730670386 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004e00 con 0x5647c5de24a0 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65f931c740 1 -- 172.21.15.78:0/1730670386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 msgr2=0x5647c5de2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65f931c740 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de2880 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f65e400b760 tx=0x7f65e400bc20 comp rx=0 tx=0).stop 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65f931c740 1 -- 172.21.15.78:0/1730670386 shutdown_connections 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65f931c740 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5ded680 0x5647c5defa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65f931c740 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5ded140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65f931c740 1 --2- 172.21.15.78:0/1730670386 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de2880 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.410+0000 7f65f931c740 1 -- 172.21.15.78:0/1730670386 >> 172.21.15.78:0/1730670386 conn(0x5647c5ddea30 msgr2=0x5647c5ddee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 -- 172.21.15.78:0/1730670386 shutdown_connections 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 -- 172.21.15.78:0/1730670386 wait complete. 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 -- start start 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 Processor -- start 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 0x5647c5decf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5dea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5ded680 0x5647c5dea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5647c5df0250 con 0x5647c5ded680 2024-09-16T00:29:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5647c5d03050 con 0x5647c5de2dc0 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5647c5df03d0 con 0x5647c5de24a0 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f931c740 10 client.0 _ll_register_callbacks cb 0x5647c5d43160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f54e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5ded680 0x5647c5dea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f54e0640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5ded680 0x5647c5dea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58044/0 (socket says 172.21.15.78:58044) 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5dea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.411+0000 7f65f54e0640 1 -- 172.21.15.78:0/2776380015 learned_addr learned my addr 172.21.15.78:0/2776380015 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 0x5647c5decf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65effff640 1 -- 172.21.15.78:0/2776380015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 msgr2=0x5647c5decf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65effff640 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 0x5647c5decf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65effff640 1 -- 172.21.15.78:0/2776380015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5ded680 msgr2=0x5647c5dea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65effff640 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5ded680 0x5647c5dea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65effff640 1 -- 172.21.15.78:0/2776380015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c5ab93c0 con 0x5647c5de2dc0 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65f54e0640 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5ded680 0x5647c5dea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65effff640 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5dea240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f65dc09be40 tx=0x7f65dc018cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4477d2269b2dbf75 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65cffff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65cffff640 1 -- 172.21.15.78:0/2776380015 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0a0260 con 0x5647c5de2dc0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65f931c740 1 -- 172.21.15.78:0/2776380015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e4015070 con 0x5647c5de2dc0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65cffff640 1 -- 172.21.15.78:0/2776380015 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f65dc00fac0 con 0x5647c5de2dc0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65cffff640 1 -- 172.21.15.78:0/2776380015 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f65dc081050 con 0x5647c5de2dc0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65f931c740 10 client.27476 fetch_fsmap learned FSMap version 35 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.412+0000 7f65f931c740 1 -- 172.21.15.78:0/2776380015 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5647c5e00090 con 0x5647c5de2dc0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65cffff640 1 -- 172.21.15.78:0/2776380015 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4015070 con 0x5647c5de2dc0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65cffff640 1 -- 172.21.15.78:0/2776380015 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f65dc09b390 con 0x5647c5de2dc0 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65f931c740 10 client.27476 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65f931c740 10 client.27476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65f931c740 20 client.27476 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65f931c740 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5647c5e014a0 0x5647c5e038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65f931c740 4 client.27476 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65f931c740 1 -- 172.21.15.78:0/2776380015 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647c5d55850 con 0x5647c5e014a0 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5647c5e014a0 0x5647c5e038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65cffff640 10 client.27476 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.413+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5647c5e014a0 0x5647c5e038b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9dbe92841e4f1a3 server_cookie=66dd7be8926f7c77 in_seq=0 out_seq=0 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65cffff640 1 -- 172.21.15.78:0/2776380015 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647c5d55850 con 0x5647c5e014a0 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65cffff640 10 client.27476 handle_command_reply: tid=0 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 client.27476 shutdown 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 -- 172.21.15.78:0/2776380015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 msgr2=0x5647c5dea240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5dea240 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f65dc09be40 tx=0x7f65dc018cb0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 -- 172.21.15.78:0/2776380015 shutdown_connections 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5647c5e014a0 0x5647c5e038b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5ded680 0x5647c5dea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5de2dc0 0x5647c5dea240 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65cffff640 0 client.27476 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 --2- 172.21.15.78:0/2776380015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 0x5647c5decf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 -- 172.21.15.78:0/2776380015 >> 172.21.15.78:0/2776380015 conn(0x5647c5ddea30 msgr2=0x5647c5de4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.414+0000 7f65f931c740 1 -- 172.21.15.78:0/2776380015 shutdown_connections 2024-09-16T00:29:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.415+0000 7f65f931c740 1 -- 172.21.15.78:0/2776380015 wait complete. 2024-09-16T00:29:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.415+0000 7f65f931c740 20 client.27476 trim_cache size 0 max 16384 2024-09-16T00:29:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:35 smithi193 ceph-mon[35484]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-09-16T00:29:36.433 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:35 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4253403029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.433+0000 7f65f931c740 1 Processor -- start 2024-09-16T00:29:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.433+0000 7f65f931c740 1 -- start start 2024-09-16T00:29:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f931c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f931c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de2dc0 0x5647c5e09180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f931c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 0x5647c5e096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f931c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01ac8f0 con 0x5647c5de24a0 2024-09-16T00:29:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f931c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01ae460 con 0x5647c5ded680 2024-09-16T00:29:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f931c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01acfc0 con 0x5647c5de2dc0 2024-09-16T00:29:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58048/0 (socket says 172.21.15.78:58048) 2024-09-16T00:29:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 -- 172.21.15.78:0/3009276394 learned_addr learned my addr 172.21.15.78:0/3009276394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f54e0640 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 0x5647c5e096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65effff640 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de2dc0 0x5647c5e09180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 -- 172.21.15.78:0/3009276394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de2dc0 msgr2=0x5647c5e09180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de2dc0 0x5647c5e09180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 -- 172.21.15.78:0/3009276394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 msgr2=0x5647c5e096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 0x5647c5e096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 -- 172.21.15.78:0/3009276394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c5de2bf0 con 0x5647c5de24a0 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.434+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de57f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f65e40049b0 tx=0x7f65e40049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bb1b4d1510ab42 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65da7fc640 1 -- 172.21.15.78:0/3009276394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4005e50 con 0x5647c5de24a0 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65da7fc640 1 -- 172.21.15.78:0/3009276394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4015040 con 0x5647c5de24a0 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65da7fc640 1 -- 172.21.15.78:0/3009276394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4019680 con 0x5647c5de24a0 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 -- 172.21.15.78:0/3009276394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 msgr2=0x5647c5de57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de57f0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f65e40049b0 tx=0x7f65e40049e0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 -- 172.21.15.78:0/3009276394 shutdown_connections 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 0x5647c5e096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de2dc0 0x5647c5e09180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 --2- 172.21.15.78:0/3009276394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de24a0 0x5647c5de57f0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 -- 172.21.15.78:0/3009276394 >> 172.21.15.78:0/3009276394 conn(0x5647c5ddea30 msgr2=0x5647c5de50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:36.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 -- 172.21.15.78:0/3009276394 shutdown_connections 2024-09-16T00:29:36.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.435+0000 7f65f931c740 1 -- 172.21.15.78:0/3009276394 wait complete. 2024-09-16T00:29:36.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 1 -- start start 2024-09-16T00:29:36.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 1 Processor -- start 2024-09-16T00:29:36.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 0x5647c5df5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de2dc0 0x5647c5df0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 0x5647c5df1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01add60 con 0x5647c5de2dc0 2024-09-16T00:29:36.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01ad690 con 0x5647c5ded680 2024-09-16T00:29:36.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65f01aa8d0 con 0x5647c5de24a0 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65f931c740 10 client.0 _ll_register_callbacks cb 0x7f65dc00a4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de2dc0 0x5647c5df0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65effff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de2dc0 0x5647c5df0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58052/0 (socket says 172.21.15.78:58052) 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.436+0000 7f65effff640 1 -- 172.21.15.78:0/3712238635 learned_addr learned my addr 172.21.15.78:0/3712238635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65f54e0640 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 0x5647c5df1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 0x5647c5df5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65effff640 1 -- 172.21.15.78:0/3712238635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 msgr2=0x5647c5df5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65effff640 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 0x5647c5df5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65effff640 1 -- 172.21.15.78:0/3712238635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 msgr2=0x5647c5df1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65effff640 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 0x5647c5df1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65effff640 1 -- 172.21.15.78:0/3712238635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c5df1890 con 0x5647c5de2dc0 2024-09-16T00:29:36.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65effff640 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de2dc0 0x5647c5df0e10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f65dc098310 tx=0x7f65dc07ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30838d40574f8b35 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65cffff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65cffff640 1 -- 172.21.15.78:0/3712238635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0a9060 con 0x5647c5de2dc0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65cffff640 1 -- 172.21.15.78:0/3712238635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f65dc09b300 con 0x5647c5de2dc0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65f931c740 1 -- 172.21.15.78:0/3712238635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e4015040 con 0x5647c5de2dc0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65cffff640 1 -- 172.21.15.78:0/3712238635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f65dc0b0050 con 0x5647c5de2dc0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65f931c740 10 client.18375 fetch_fsmap learned FSMap version 35 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.437+0000 7f65f931c740 1 -- 172.21.15.78:0/3712238635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5647c5e6be30 con 0x5647c5de2dc0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65cffff640 1 -- 172.21.15.78:0/3712238635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4015040 con 0x5647c5de2dc0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65cffff640 1 -- 172.21.15.78:0/3712238635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f65dc0a4690 con 0x5647c5de2dc0 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65f931c740 10 client.18375 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65f931c740 10 client.18375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65f931c740 20 client.18375 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65f931c740 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5647c5e6c550 0x5647c5e6e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:36.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65f931c740 4 client.18375 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65f931c740 1 -- 172.21.15.78:0/3712238635 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5647c5d4fd60 con 0x5647c5e6c550 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.438+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5647c5e6c550 0x5647c5e6e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65cffff640 10 client.18375 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f4cdf640 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5647c5e6c550 0x5647c5e6e970 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f90391fc4ec03a0d server_cookie=43b6a6b08a19cc93 in_seq=0 out_seq=0 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65cffff640 1 -- 172.21.15.78:0/3712238635 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5647c5d4fd60 con 0x5647c5e6c550 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65cffff640 10 client.18375 handle_command_reply: tid=0 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 client.18375 shutdown 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 -- 172.21.15.78:0/3712238635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de2dc0 msgr2=0x5647c5df0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de2dc0 0x5647c5df0e10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f65dc098310 tx=0x7f65dc07ab60 comp rx=0 tx=0).stop 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 -- 172.21.15.78:0/3712238635 shutdown_connections 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5647c5e6c550 0x5647c5e6e970 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5647c5ded680 0x5647c5df1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65cffff640 0 client.18375 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5647c5de2dc0 0x5647c5df0e10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 --2- 172.21.15.78:0/3712238635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5647c5de24a0 0x5647c5df5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 -- 172.21.15.78:0/3712238635 >> 172.21.15.78:0/3712238635 conn(0x5647c5ddea30 msgr2=0x5647c5de86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 -- 172.21.15.78:0/3712238635 shutdown_connections 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 1 -- 172.21.15.78:0/3712238635 wait complete. 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.439+0000 7f65f931c740 20 client.18375 trim_cache size 0 max 16384 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:29:36.444 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": { 2024-09-16T00:29:36.445 INFO:teuthology.orchestra.run.smithi078.stdout: "9f7ada4e-ae07-419b-ad78-5b773241d74a": { 2024-09-16T00:29:36.445 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/", 2024-09-16T00:29:36.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tag": "9f7ada4e-ae07-419b-ad78-5b773241d74a", 2024-09-16T00:29:36.445 INFO:teuthology.orchestra.run.smithi078.stdout: "options": "recursive,force" 2024-09-16T00:29:36.445 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:36.445 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:29:36.445 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:29:36.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 -- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65e00a3240 msgr2=0x7f65e00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65e00a3240 0x7f65e00a5700 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f65e8005e30 tx=0x7f65e8005c50 comp rx=0 tx=0).stop 2024-09-16T00:29:36.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 -- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f0103670 msgr2=0x7f65f0198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:36.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f0103670 0x7f65f0198960 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00c970 tx=0x7f65dc00ce30 comp rx=0 tx=0).stop 2024-09-16T00:29:36.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 -- 172.21.15.78:0/289784475 shutdown_connections 2024-09-16T00:29:36.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f65e00a3240 0x7f65e00a5700 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f65f010c180 0x7f65f019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65f0103670 0x7f65f0198960 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 --2- 172.21.15.78:0/289784475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f0102d50 0x7f65f0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:36.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 -- 172.21.15.78:0/289784475 >> 172.21.15.78:0/289784475 conn(0x7f65f00fc7d0 msgr2=0x7f65f00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:36.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 -- 172.21.15.78:0/289784475 shutdown_connections 2024-09-16T00:29:36.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:36.440+0000 7f65f6f43640 1 -- 172.21.15.78:0/289784475 wait complete. 2024-09-16T00:29:36.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:35 smithi148 ceph-mon[35599]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.2 MiB/s wr, 483 op/s 2024-09-16T00:29:36.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:35 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4253403029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:36.699 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:37.194 INFO:tasks.cephfs.filesystem:scrub status for tag:9f7ada4e-ae07-419b-ad78-5b773241d74a - {'path': '/', 'tag': '9f7ada4e-ae07-419b-ad78-5b773241d74a', 'options': 'recursive,force'} 2024-09-16T00:29:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 -- 172.21.15.78:0/3072729073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4101420 msgr2=0x7fcfd410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 --2- 172.21.15.78:0/3072729073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4101420 0x7fcfd410d950 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009cd0 tx=0x7fcfcc02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 -- 172.21.15.78:0/3072729073 shutdown_connections 2024-09-16T00:29:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 --2- 172.21.15.78:0/3072729073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4101420 0x7fcfd410d950 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 --2- 172.21.15.78:0/3072729073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd4100a80 0x7fcfd4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 --2- 172.21.15.78:0/3072729073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd4106a80 0x7fcfd4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 -- 172.21.15.78:0/3072729073 >> 172.21.15.78:0/3072729073 conn(0x7fcfd40fc7a0 msgr2=0x7fcfd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 -- 172.21.15.78:0/3072729073 shutdown_connections 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.748+0000 7fcfdabca640 1 -- 172.21.15.78:0/3072729073 wait complete. 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfdabca640 1 Processor -- start 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfdabca640 1 -- start start 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfdabca640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4100a80 0x7fcfd4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfdabca640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd4101420 0x7fcfd4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfdabca640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd4106a80 0x7fcfd419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfdabca640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4110e80 con 0x7fcfd4100a80 2024-09-16T00:29:37.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfdabca640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4110d00 con 0x7fcfd4106a80 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfdabca640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4111000 con 0x7fcfd4101420 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd8966640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4100a80 0x7fcfd4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd8966640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4100a80 0x7fcfd4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58070/0 (socket says 172.21.15.78:58070) 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd8966640 1 -- 172.21.15.78:0/170777195 learned_addr learned my addr 172.21.15.78:0/170777195 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd4101420 0x7fcfd4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd9167640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd4106a80 0x7fcfd419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd8966640 1 -- 172.21.15.78:0/170777195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd4101420 msgr2=0x7fcfd4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd8966640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd4101420 0x7fcfd4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd8966640 1 -- 172.21.15.78:0/170777195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd4106a80 msgr2=0x7fcfd419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd8966640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd4106a80 0x7fcfd419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd8966640 1 -- 172.21.15.78:0/170777195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd419c2e0 con 0x7fcfd4100a80 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd4101420 0x7fcfd4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.749+0000 7fcfd9167640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd4106a80 0x7fcfd419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:37.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.750+0000 7fcfd8966640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4100a80 0x7fcfd4197330 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c890 tx=0x7fcfc800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.750+0000 7fcfd1ffb640 1 -- 172.21.15.78:0/170777195 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8004220 con 0x7fcfd4100a80 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.750+0000 7fcfd1ffb640 1 -- 172.21.15.78:0/170777195 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc80043c0 con 0x7fcfd4100a80 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.750+0000 7fcfd1ffb640 1 -- 172.21.15.78:0/170777195 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80049e0 con 0x7fcfd4100a80 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.750+0000 7fcfdabca640 1 -- 172.21.15.78:0/170777195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd419c5d0 con 0x7fcfd4100a80 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.750+0000 7fcfdabca640 1 -- 172.21.15.78:0/170777195 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd41a3eb0 con 0x7fcfd4100a80 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.752+0000 7fcfd1ffb640 1 -- 172.21.15.78:0/170777195 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcfc80079b0 con 0x7fcfd4100a80 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfdcfa3740 1 Processor -- start 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfdcfa3740 1 -- start start 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfd1ffb640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcfc40a32c0 0x7fcfc40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e6719de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e67246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfd9167640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcfc40a32c0 0x7fcfc40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6724be0 0x55e3e6726fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e3e667e200 con 0x55e3e671a320 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e3e6624dc0 con 0x55e3e6724be0 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e3e667a1c0 con 0x55e3e6719a00 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfd1ffb640 1 -- 172.21.15.78:0/170777195 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcfc80cfc80 con 0x7fcfd4100a80 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e67246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfd3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e67246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58086/0 (socket says 172.21.15.78:58086) 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.753+0000 7fcfd3fff640 1 -- 172.21.15.78:0/32584554 learned_addr learned my addr 172.21.15.78:0/32584554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd8966640 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e6719de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd9167640 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6724be0 0x55e3e6726fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd3fff640 1 -- 172.21.15.78:0/32584554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 msgr2=0x55e3e6719de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e6719de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd3fff640 1 -- 172.21.15.78:0/32584554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6724be0 msgr2=0x55e3e6726fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6724be0 0x55e3e6726fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd3fff640 1 -- 172.21.15.78:0/32584554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e63ef620 con 0x55e3e671a320 2024-09-16T00:29:37.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd9167640 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6724be0 0x55e3e6726fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd9167640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcfc40a32c0 0x7fcfc40a5780 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009ca0 tx=0x7fcfcc0099e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e67246a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000bd30 tx=0x7fcfc000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc1c0b47fa0e182 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfbeffd640 1 -- 172.21.15.78:0/32584554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0015070 con 0x55e3e671a320 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfbeffd640 1 -- 172.21.15.78:0/32584554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc000be20 con 0x55e3e671a320 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.754+0000 7fcfbeffd640 1 -- 172.21.15.78:0/32584554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0011a80 con 0x55e3e671a320 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/32584554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 msgr2=0x55e3e67246a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e67246a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000bd30 tx=0x7fcfc000f730 comp rx=0 tx=0).stop 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/32584554 shutdown_connections 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6724be0 0x55e3e6726fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e67246a0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/32584554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e6719de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/32584554 >> 172.21.15.78:0/32584554 conn(0x55e3e6715f90 msgr2=0x55e3e6716370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/32584554 shutdown_connections 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/32584554 wait complete. 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- start start 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 Processor -- start 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e67244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e671a320 0x55e3e67217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6724be0 0x55e3e6721ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e3e67277b0 con 0x55e3e6724be0 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e3e6618c50 con 0x55e3e671a320 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.755+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e3e6727930 con 0x55e3e6719a00 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfdcfa3740 10 client.0 _ll_register_callbacks cb 0x55e3e662aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6724be0 0x55e3e6721ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6724be0 0x55e3e6721ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58092/0 (socket says 172.21.15.78:58092) 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 -- 172.21.15.78:0/1434832916 learned_addr learned my addr 172.21.15.78:0/1434832916 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:37.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd8966640 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e67244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e671a320 0x55e3e67217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 -- 172.21.15.78:0/1434832916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 msgr2=0x55e3e67244f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e67244f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 -- 172.21.15.78:0/1434832916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e671a320 msgr2=0x55e3e67217a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e671a320 0x55e3e67217a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 -- 172.21.15.78:0/1434832916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e67224e0 con 0x55e3e6724be0 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd8966640 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e67244f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfd9167640 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6724be0 0x55e3e6721ce0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc031d60 tx=0x7fcfcc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ec9283f95e4d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:29:37.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfb3fff640 1 -- 172.21.15.78:0/1434832916 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc04d070 con 0x55e3e6724be0 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfb3fff640 1 -- 172.21.15.78:0/1434832916 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcfcc0305a0 con 0x55e3e6724be0 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/1434832916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc000be20 con 0x55e3e6724be0 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfb3fff640 1 -- 172.21.15.78:0/1434832916 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfcc0307a0 con 0x55e3e6724be0 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfdcfa3740 10 client.18387 fetch_fsmap learned FSMap version 35 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.756+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/1434832916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3e67332a0 con 0x55e3e6724be0 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfb3fff640 1 -- 172.21.15.78:0/1434832916 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc000be20 con 0x55e3e6724be0 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfb3fff640 1 -- 172.21.15.78:0/1434832916 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fcfcc030070 con 0x55e3e6724be0 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfdcfa3740 10 client.18387 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfdcfa3740 10 client.18387 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfdcfa3740 20 client.18387 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e3e6734700 0x55e3e6736b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfdcfa3740 4 client.18387 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/1434832916 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3e668b770 con 0x55e3e6734700 2024-09-16T00:29:37.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfd8966640 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e3e6734700 0x55e3e6736b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfb3fff640 10 client.18387 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.757+0000 7fcfd8966640 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e3e6734700 0x55e3e6736b30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0b1274a5a538f88 server_cookie=ea96e2e005b9f784 in_seq=0 out_seq=0 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfb3fff640 1 -- 172.21.15.78:0/1434832916 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3e668b770 con 0x55e3e6734700 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfb3fff640 10 client.18387 handle_command_reply: tid=0 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 client.18387 shutdown 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/1434832916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6724be0 msgr2=0x55e3e6721ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6724be0 0x55e3e6721ce0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc031d60 tx=0x7fcfcc02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/1434832916 shutdown_connections 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e3e6734700 0x55e3e6736b30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6724be0 0x55e3e6721ce0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e671a320 0x55e3e67217a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/1434832916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e67244f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfb3fff640 0 client.18387 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.758+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/1434832916 >> 172.21.15.78:0/1434832916 conn(0x55e3e6715f90 msgr2=0x55e3e67151d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.759+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/1434832916 shutdown_connections 2024-09-16T00:29:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.759+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/1434832916 wait complete. 2024-09-16T00:29:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.759+0000 7fcfdcfa3740 20 client.18387 trim_cache size 0 max 16384 2024-09-16T00:29:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfdcfa3740 1 Processor -- start 2024-09-16T00:29:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfdcfa3740 1 -- start start 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6715f90 0x55e3e671cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6719a00 0x55e3e6720070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e671a320 0x55e3e6730a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41adb70 con 0x55e3e6715f90 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41af6e0 con 0x55e3e6719a00 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41ae240 con 0x55e3e671a320 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfd8966640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6715f90 0x55e3e671cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfd8966640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6715f90 0x55e3e671cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58094/0 (socket says 172.21.15.78:58094) 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.775+0000 7fcfd8966640 1 -- 172.21.15.78:0/454717929 learned_addr learned my addr 172.21.15.78:0/454717929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6719a00 0x55e3e6720070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd9167640 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e671a320 0x55e3e6730a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd8966640 1 -- 172.21.15.78:0/454717929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e671a320 msgr2=0x55e3e6730a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd8966640 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e671a320 0x55e3e6730a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd8966640 1 -- 172.21.15.78:0/454717929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6719a00 msgr2=0x55e3e6720070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd8966640 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6719a00 0x55e3e6720070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd8966640 1 -- 172.21.15.78:0/454717929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e6711950 con 0x55e3e6715f90 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6719a00 0x55e3e6720070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfd8966640 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6715f90 0x55e3e671cbd0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80cd470 tx=0x7fcfc80054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9feb0df50219190c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfbe7fc640 1 -- 172.21.15.78:0/454717929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc801d040 con 0x55e3e6715f90 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfbe7fc640 1 -- 172.21.15.78:0/454717929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc80a4bb0 con 0x55e3e6715f90 2024-09-16T00:29:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.776+0000 7fcfbe7fc640 1 -- 172.21.15.78:0/454717929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80a4720 con 0x55e3e6715f90 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/454717929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6715f90 msgr2=0x55e3e671cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6715f90 0x55e3e671cbd0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80cd470 tx=0x7fcfc80054a0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/454717929 shutdown_connections 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e671a320 0x55e3e6730a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6719a00 0x55e3e6720070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/454717929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e6715f90 0x55e3e671cbd0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/454717929 >> 172.21.15.78:0/454717929 conn(0x55e3e6721e40 msgr2=0x55e3e67151d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/454717929 shutdown_connections 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/454717929 wait complete. 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 -- start start 2024-09-16T00:29:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.777+0000 7fcfdcfa3740 1 Processor -- start 2024-09-16T00:29:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6715f90 0x55e3e671f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e671d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfdcfa3740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e671e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41aefe0 con 0x55e3e671a320 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41ae910 con 0x55e3e6715f90 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfdcfa3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41abb50 con 0x55e3e6719a00 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfdcfa3740 10 client.0 _ll_register_callbacks cb 0x7fcfcc031c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd9167640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e671e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd8966640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6715f90 0x55e3e671f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd9167640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e671e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:58098/0 (socket says 172.21.15.78:58098) 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd9167640 1 -- 172.21.15.78:0/776447892 learned_addr learned my addr 172.21.15.78:0/776447892 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e671d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd8966640 1 -- 172.21.15.78:0/776447892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 msgr2=0x55e3e671d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd8966640 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e671d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd8966640 1 -- 172.21.15.78:0/776447892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 msgr2=0x55e3e671e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd8966640 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e671e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd8966640 1 -- 172.21.15.78:0/776447892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e67a1400 con 0x55e3e6715f90 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd9167640 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e671e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.778+0000 7fcfd8966640 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6715f90 0x55e3e671f460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005940 tx=0x7fcfc809c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c62efed3561fd0d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfb3fff640 1 -- 172.21.15.78:0/776447892 <== mon.2 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc801d070 con 0x55e3e6715f90 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfb3fff640 1 -- 172.21.15.78:0/776447892 <== mon.2 v2:172.21.15.148:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fcfc809a240 con 0x55e3e6715f90 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/776447892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc80a4bb0 con 0x55e3e6715f90 2024-09-16T00:29:37.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfb3fff640 1 -- 172.21.15.78:0/776447892 <== mon.2 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfc80985b0 con 0x55e3e6715f90 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 10 client.27494 fetch_fsmap learned FSMap version 35 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/776447892 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e3e67a25e0 con 0x55e3e6715f90 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfb3fff640 1 -- 172.21.15.78:0/776447892 <== mon.2 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc80a4bb0 con 0x55e3e6715f90 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfb3fff640 1 -- 172.21.15.78:0/776447892 <== mon.2 v2:172.21.15.148:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fcfc80bab80 con 0x55e3e6715f90 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 10 client.27494 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 10 client.27494 resolve_mds: validated gid 24470 aka daemon mds.b 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 20 client.27494 populate_metadata read hostname 'smithi078' 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e3e6724be0 0x55e3e67a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 4 client.27494 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.779+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/776447892 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55e3e6685c80 con 0x55e3e6724be0 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.780+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e3e6724be0 0x55e3e67a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.780+0000 7fcfd3fff640 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e3e6724be0 0x55e3e67a4cb0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95b38b18264a740 server_cookie=eda8d13095dc0f6f in_seq=0 out_seq=0 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.780+0000 7fcfb3fff640 10 client.27494 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:37.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.780+0000 7fcfb3fff640 1 -- 172.21.15.78:0/776447892 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55e3e6685c80 con 0x55e3e6724be0 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.780+0000 7fcfb3fff640 10 client.27494 handle_command_reply: tid=0 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.780+0000 7fcfdcfa3740 1 client.27494 shutdown 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/776447892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6715f90 msgr2=0x55e3e671f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6715f90 0x55e3e671f460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005940 tx=0x7fcfc809c640 comp rx=0 tx=0).stop 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/776447892 shutdown_connections 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55e3e6724be0 0x55e3e67a4cb0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e3e671a320 0x55e3e671e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3e6719a00 0x55e3e671d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 --2- 172.21.15.78:0/776447892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3e6715f90 0x55e3e671f460 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfb3fff640 0 client.27494 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/776447892 >> 172.21.15.78:0/776447892 conn(0x55e3e6721e40 msgr2=0x55e3e6723000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/776447892 shutdown_connections 2024-09-16T00:29:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 1 -- 172.21.15.78:0/776447892 wait complete. 2024-09-16T00:29:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdcfa3740 20 client.27494 trim_cache size 0 max 16384 2024-09-16T00:29:37.788 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:29:37.788 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 35, 2024-09-16T00:29:37.788 INFO:teuthology.orchestra.run.smithi078.stdout: "leader": 24470, 2024-09-16T00:29:37.788 INFO:teuthology.orchestra.run.smithi078.stdout: "set_version": 60, 2024-09-16T00:29:37.788 INFO:teuthology.orchestra.run.smithi078.stdout: "sets": {} 2024-09-16T00:29:37.788 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:29:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdabca640 1 -- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcfc40a32c0 msgr2=0x7fcfc40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdabca640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcfc40a32c0 0x7fcfc40a5780 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009ca0 tx=0x7fcfcc0099e0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdabca640 1 -- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4100a80 msgr2=0x7fcfd4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdabca640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4100a80 0x7fcfd4197330 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800c890 tx=0x7fcfc800cd50 comp rx=0 tx=0).stop 2024-09-16T00:29:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.781+0000 7fcfdabca640 1 -- 172.21.15.78:0/170777195 shutdown_connections 2024-09-16T00:29:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.782+0000 7fcfdabca640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fcfc40a32c0 0x7fcfc40a5780 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.782+0000 7fcfdabca640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfd4106a80 0x7fcfd419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.782+0000 7fcfdabca640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfd4101420 0x7fcfd4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.782+0000 7fcfdabca640 1 --2- 172.21.15.78:0/170777195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfd4100a80 0x7fcfd4197330 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.782+0000 7fcfdabca640 1 -- 172.21.15.78:0/170777195 >> 172.21.15.78:0/170777195 conn(0x7fcfd40fc7a0 msgr2=0x7fcfd40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.782+0000 7fcfdabca640 1 -- 172.21.15.78:0/170777195 shutdown_connections 2024-09-16T00:29:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:29:37.782+0000 7fcfdabca640 1 -- 172.21.15.78:0/170777195 wait complete. 2024-09-16T00:29:38.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:37 smithi078 ceph-mon[31157]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-09-16T00:29:38.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:37 smithi193 ceph-mon[35484]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-09-16T00:29:38.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:37 smithi148 ceph-mon[35599]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-09-16T00:29:38.624 INFO:tasks.quiescer:done joining Quiescers 2024-09-16T00:29:38.624 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-16T00:29:38.634 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-16T00:29:40.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:39 smithi078 ceph-mon[31157]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.0 MiB/s wr, 251 op/s 2024-09-16T00:29:40.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:39 smithi193 ceph-mon[35484]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.0 MiB/s wr, 251 op/s 2024-09-16T00:29:40.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:39 smithi148 ceph-mon[35599]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 2.0 MiB/s wr, 251 op/s 2024-09-16T00:29:42.231 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:41 smithi193 ceph-mon[35484]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-16T00:29:42.256 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:41 smithi148 ceph-mon[35599]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-16T00:29:42.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:41 smithi078 ceph-mon[31157]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-16T00:29:44.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:43 smithi078 ceph-mon[31157]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 551 KiB/s wr, 80 op/s 2024-09-16T00:29:44.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:43 smithi193 ceph-mon[35484]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 551 KiB/s wr, 80 op/s 2024-09-16T00:29:44.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:43 smithi148 ceph-mon[35599]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 551 KiB/s wr, 80 op/s 2024-09-16T00:29:45.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:44 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:29:45.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:44 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:29:45.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:44 smithi078 ceph-mon[31157]: scrub summary: idle 2024-09-16T00:29:45.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:44 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:29:45.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:44 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:29:45.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:44 smithi193 ceph-mon[35484]: scrub summary: idle 2024-09-16T00:29:45.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:44 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:29:45.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:44 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:29:45.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:44 smithi148 ceph-mon[35599]: scrub summary: idle 2024-09-16T00:29:46.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:45 smithi078 ceph-mon[31157]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-16T00:29:46.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:45 smithi193 ceph-mon[35484]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-16T00:29:46.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:45 smithi148 ceph-mon[35599]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-16T00:29:48.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:47 smithi193 ceph-mon[35484]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-09-16T00:29:48.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:47 smithi148 ceph-mon[35599]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-09-16T00:29:48.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:47 smithi078 ceph-mon[31157]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-09-16T00:29:50.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:49 smithi193 ceph-mon[35484]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-16T00:29:50.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:49 smithi148 ceph-mon[35599]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-16T00:29:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:49 smithi078 ceph-mon[31157]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-16T00:29:52.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:51 smithi193 ceph-mon[35484]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-16T00:29:52.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:51 smithi148 ceph-mon[35599]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-16T00:29:52.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:51 smithi078 ceph-mon[31157]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-16T00:29:54.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:53 smithi193 ceph-mon[35484]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:29:54.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:53 smithi148 ceph-mon[35599]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:29:54.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:53 smithi078 ceph-mon[31157]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-16T00:29:56.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:55 smithi193 ceph-mon[35484]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-16T00:29:56.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:55 smithi148 ceph-mon[35599]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-16T00:29:56.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:55 smithi078 ceph-mon[31157]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-16T00:29:58.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:57 smithi193 ceph-mon[35484]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-16T00:29:58.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:57 smithi148 ceph-mon[35599]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-16T00:29:58.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:57 smithi078 ceph-mon[31157]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-16T00:30:00.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:29:59 smithi193 ceph-mon[35484]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-16T00:30:00.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:29:59 smithi148 ceph-mon[35599]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-16T00:30:00.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:29:59 smithi078 ceph-mon[31157]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-09-16T00:30:01.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:00 smithi193 ceph-mon[35484]: overall HEALTH_OK 2024-09-16T00:30:01.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:00 smithi148 ceph-mon[35599]: overall HEALTH_OK 2024-09-16T00:30:01.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:00 smithi078 ceph-mon[31157]: overall HEALTH_OK 2024-09-16T00:30:02.171 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:01 smithi078 ceph-mon[31157]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T00:30:02.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:01 smithi193 ceph-mon[35484]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T00:30:02.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:01 smithi148 ceph-mon[35599]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-16T00:30:04.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:03 smithi193 ceph-mon[35484]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-16T00:30:04.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:03 smithi148 ceph-mon[35599]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-16T00:30:04.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:03 smithi078 ceph-mon[31157]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-16T00:30:06.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:05 smithi193 ceph-mon[35484]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-09-16T00:30:06.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:05 smithi148 ceph-mon[35599]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-09-16T00:30:06.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:05 smithi078 ceph-mon[31157]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-09-16T00:30:07.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:30:07.475 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:07.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 -- 172.21.15.78:0/828041872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c100e20 msgr2=0x7f052c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 --2- 172.21.15.78:0/828041872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c100e20 0x7f052c101200 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f051c009900 tx=0x7f051c02a480 comp rx=0 tx=0).stop 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 -- 172.21.15.78:0/828041872 shutdown_connections 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 --2- 172.21.15.78:0/828041872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f052c102ef0 0x7f052c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 --2- 172.21.15.78:0/828041872 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f052c102570 0x7f052c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 --2- 172.21.15.78:0/828041872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c100e20 0x7f052c101200 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 -- 172.21.15.78:0/828041872 >> 172.21.15.78:0/828041872 conn(0x7f052c0fc7d0 msgr2=0x7f052c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 -- 172.21.15.78:0/828041872 shutdown_connections 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.950+0000 7f0531067640 1 -- 172.21.15.78:0/828041872 wait complete. 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.951+0000 7f0531067640 1 Processor -- start 2024-09-16T00:30:07.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.951+0000 7f0531067640 1 -- start start 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.951+0000 7f0531067640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f052c100e20 0x7f052c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.951+0000 7f0531067640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f052c102570 0x7f052c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.951+0000 7f0531067640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c102ef0 0x7f052c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.951+0000 7f0531067640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f052c110f80 con 0x7f052c102ef0 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.951+0000 7f0531067640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f052c110e00 con 0x7f052c100e20 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.951+0000 7f0531067640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f052c111100 con 0x7f052c102570 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c102ef0 0x7f052c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c102ef0 0x7f052c19bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57020/0 (socket says 172.21.15.78:57020) 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 -- 172.21.15.78:0/4235541751 learned_addr learned my addr 172.21.15.78:0/4235541751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052a59c640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f052c102570 0x7f052c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052ad9d640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f052c100e20 0x7f052c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 -- 172.21.15.78:0/4235541751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f052c102570 msgr2=0x7f052c197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f052c102570 0x7f052c197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 -- 172.21.15.78:0/4235541751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f052c100e20 msgr2=0x7f052c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f052c100e20 0x7f052c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 -- 172.21.15.78:0/4235541751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052c19c370 con 0x7f052c102ef0 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f052b59e640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c102ef0 0x7f052c19bc90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f052400eb10 tx=0x7f052400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f0513fff640 1 -- 172.21.15.78:0/4235541751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524004b20 con 0x7f052c102ef0 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f0531067640 1 -- 172.21.15.78:0/4235541751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052c19c660 con 0x7f052c102ef0 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f0513fff640 1 -- 172.21.15.78:0/4235541751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0524004cc0 con 0x7f052c102ef0 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f0513fff640 1 -- 172.21.15.78:0/4235541751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524010760 con 0x7f052c102ef0 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.952+0000 7f0531067640 1 -- 172.21.15.78:0/4235541751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052c1a3f40 con 0x7f052c102ef0 2024-09-16T00:30:07.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.954+0000 7f0513fff640 1 -- 172.21.15.78:0/4235541751 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0524005020 con 0x7f052c102ef0 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0533440740 1 Processor -- start 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0533440740 1 -- start start 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0533440740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e839c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0533440740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839cdc0 0x55c4e83a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0533440740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0533440740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c4e83022e0 con 0x55c4e839c4a0 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0533440740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c4e82a9ce0 con 0x55c4e83a7680 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0533440740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4e82fe2a0 con 0x55c4e839cdc0 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0513fff640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05140a3240 0x7f05140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f052a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e839c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f052a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e839c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57024/0 (socket says 172.21.15.78:57024) 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f052a59c640 1 -- 172.21.15.78:0/2435281109 learned_addr learned my addr 172.21.15.78:0/2435281109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:07.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f052b59e640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05140a3240 0x7f05140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f052ad9d640 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839cdc0 0x55c4e83a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f052a59c640 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.955+0000 7f0513fff640 1 -- 172.21.15.78:0/4235541751 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f05240d0320 con 0x7f052c102ef0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f052a59c640 1 -- 172.21.15.78:0/2435281109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839cdc0 msgr2=0x55c4e83a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f052a59c640 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839cdc0 0x55c4e83a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f052a59c640 1 -- 172.21.15.78:0/2435281109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 msgr2=0x55c4e83a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f052a59c640 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f052a59c640 1 -- 172.21.15.78:0/2435281109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e828e440 con 0x55c4e839c4a0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f052b59e640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05140a3240 0x7f05140a5700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0524099050 tx=0x7f052406b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f052a59c640 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e839c880 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0520005bc0 tx=0x7f0520009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c53966d7624e974 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f05117fa640 1 -- 172.21.15.78:0/2435281109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052000e8b0 con 0x55c4e839c4a0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f05117fa640 1 -- 172.21.15.78:0/2435281109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052001b040 con 0x55c4e839c4a0 2024-09-16T00:30:07.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.956+0000 7f05117fa640 1 -- 172.21.15.78:0/2435281109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052001fe40 con 0x55c4e839c4a0 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 -- 172.21.15.78:0/2435281109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 msgr2=0x55c4e839c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e839c880 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0520005bc0 tx=0x7f0520009950 comp rx=0 tx=0).stop 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 -- 172.21.15.78:0/2435281109 shutdown_connections 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839cdc0 0x55c4e83a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 --2- 172.21.15.78:0/2435281109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e839c880 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 -- 172.21.15.78:0/2435281109 >> 172.21.15.78:0/2435281109 conn(0x55c4e8398a30 msgr2=0x55c4e8398e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 -- 172.21.15.78:0/2435281109 shutdown_connections 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 -- 172.21.15.78:0/2435281109 wait complete. 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 -- start start 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.957+0000 7f0533440740 1 Processor -- start 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f0533440740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 0x55c4e83a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f0533440740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83a4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f0533440740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f0533440740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55c4e83aa250 con 0x55c4e839cdc0 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f0533440740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c4e82bd050 con 0x55c4e83a7680 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f0533440740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4e83aa3d0 con 0x55c4e839c4a0 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f0533440740 10 client.0 _ll_register_callbacks cb 0x55c4e82fd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57034/0 (socket says 172.21.15.78:57034) 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 -- 172.21.15.78:0/1577556239 learned_addr learned my addr 172.21.15.78:0/1577556239 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052a59c640 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 0x55c4e83a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052a59c640 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 -- 172.21.15.78:0/1577556239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 msgr2=0x55c4e83a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 0x55c4e83a6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 -- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 msgr2=0x55c4e83a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 -- 172.21.15.78:0/1577556239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e80733c0 con 0x55c4e839cdc0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052a59c640 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f052ad9d640 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83a4240 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f051c005c40 tx=0x7f051c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a814946f1dea6eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f050eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f050eff5640 1 -- 172.21.15.78:0/1577556239 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c035040 con 0x55c4e839cdc0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f050eff5640 1 -- 172.21.15.78:0/1577556239 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f051c0048c0 con 0x55c4e839cdc0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.958+0000 7f0533440740 1 -- 172.21.15.78:0/1577556239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f052001b040 con 0x55c4e839cdc0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f050eff5640 1 -- 172.21.15.78:0/1577556239 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f051c005d50 con 0x55c4e839cdc0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f0533440740 10 client.18405 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f0533440740 1 -- 172.21.15.78:0/1577556239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c4e83b5bd0 con 0x55c4e839cdc0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f050eff5640 1 -- 172.21.15.78:0/1577556239 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052001b040 con 0x55c4e839cdc0 2024-09-16T00:30:07.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f050eff5640 1 -- 172.21.15.78:0/1577556239 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f051c031540 con 0x55c4e839cdc0 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f0533440740 10 client.18405 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f0533440740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f0533440740 20 client.18405 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f0533440740 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c4e83b70e0 0x55c4e83b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f0533440740 4 client.18405 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f0533440740 1 -- 172.21.15.78:0/1577556239 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4e830f850 con 0x55c4e83b70e0 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.959+0000 7f052a59c640 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c4e83b70e0 0x55c4e83b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.960+0000 7f052a59c640 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c4e83b70e0 0x55c4e83b9500 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a40bf1c189a345 server_cookie=a1d743f3f001f215 in_seq=0 out_seq=0 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.960+0000 7f050eff5640 10 client.18405 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.960+0000 7f050eff5640 1 -- 172.21.15.78:0/1577556239 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4e830f850 con 0x55c4e83b70e0 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.960+0000 7f050eff5640 10 client.18405 handle_command_reply: tid=0 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 client.18405 shutdown 2024-09-16T00:30:07.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 -- 172.21.15.78:0/1577556239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 msgr2=0x55c4e83a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83a4240 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f051c005c40 tx=0x7f051c005270 comp rx=0 tx=0).stop 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 -- 172.21.15.78:0/1577556239 shutdown_connections 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c4e83b70e0 0x55c4e83b9500 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83a4240 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f050eff5640 0 client.18405 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 --2- 172.21.15.78:0/1577556239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 0x55c4e83a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 -- 172.21.15.78:0/1577556239 >> 172.21.15.78:0/1577556239 conn(0x55c4e8398a30 msgr2=0x55c4e839e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 -- 172.21.15.78:0/1577556239 shutdown_connections 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 1 -- 172.21.15.78:0/1577556239 wait complete. 2024-09-16T00:30:07.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.961+0000 7f0533440740 20 client.18405 trim_cache size 0 max 16384 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f0533440740 1 Processor -- start 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f0533440740 1 -- start start 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f0533440740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e83b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f0533440740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e839cdc0 0x55c4e83bed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f0533440740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e83a7680 0x55c4e83bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f0533440740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1ac9c0 con 0x55c4e839c4a0 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f0533440740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1ae530 con 0x55c4e839cdc0 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f0533440740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1ad090 con 0x55c4e83a7680 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e83b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052a59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e83b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57048/0 (socket says 172.21.15.78:57048) 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052a59c640 1 -- 172.21.15.78:0/3113705146 learned_addr learned my addr 172.21.15.78:0/3113705146 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052ad9d640 1 --2- 172.21.15.78:0/3113705146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e839cdc0 0x55c4e83bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052a59c640 1 -- 172.21.15.78:0/3113705146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e83a7680 msgr2=0x55c4e83bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052a59c640 1 --2- 172.21.15.78:0/3113705146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e83a7680 0x55c4e83bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052a59c640 1 -- 172.21.15.78:0/3113705146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e839cdc0 msgr2=0x55c4e83bed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052a59c640 1 --2- 172.21.15.78:0/3113705146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e839cdc0 0x55c4e83bed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.978+0000 7f052a59c640 1 -- 172.21.15.78:0/3113705146 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e83a6bb0 con 0x55c4e839c4a0 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f052a59c640 1 --2- 172.21.15.78:0/3113705146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e83b34b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f052000e780 tx=0x7f0520005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2731f3e7e440e491 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0510ff9640 1 -- 172.21.15.78:0/3113705146 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052001fb50 con 0x55c4e839c4a0 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0510ff9640 1 -- 172.21.15.78:0/3113705146 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052001fcf0 con 0x55c4e839c4a0 2024-09-16T00:30:07.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0510ff9640 1 -- 172.21.15.78:0/3113705146 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052000e970 con 0x55c4e839c4a0 2024-09-16T00:30:07.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0533440740 1 -- 172.21.15.78:0/3113705146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 msgr2=0x55c4e83b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0533440740 1 --2- 172.21.15.78:0/3113705146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e83b34b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f052000e780 tx=0x7f0520005f90 comp rx=0 tx=0).stop 2024-09-16T00:30:07.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0533440740 1 -- 172.21.15.78:0/3113705146 shutdown_connections 2024-09-16T00:30:07.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0533440740 1 --2- 172.21.15.78:0/3113705146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e83a7680 0x55c4e83bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0533440740 1 --2- 172.21.15.78:0/3113705146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e839cdc0 0x55c4e83bed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0533440740 1 --2- 172.21.15.78:0/3113705146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839c4a0 0x55c4e83b34b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.979+0000 7f0533440740 1 -- 172.21.15.78:0/3113705146 >> 172.21.15.78:0/3113705146 conn(0x55c4e8398a30 msgr2=0x55c4e839f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 -- 172.21.15.78:0/3113705146 shutdown_connections 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 -- 172.21.15.78:0/3113705146 wait complete. 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 -- start start 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 Processor -- start 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 0x55c4e83afec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1ade30 con 0x55c4e839cdc0 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1ad760 con 0x55c4e83a7680 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1aa9a0 con 0x55c4e839c4a0 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f0533440740 10 client.0 _ll_register_callbacks cb 0x7f05240093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:07.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f052ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f052ad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83ab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57060/0 (socket says 172.21.15.78:57060) 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f052ad9d640 1 -- 172.21.15.78:0/1090083659 learned_addr learned my addr 172.21.15.78:0/1090083659 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.980+0000 7f052a59c640 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f052a59c640 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 0x55c4e83afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f052ad9d640 1 -- 172.21.15.78:0/1090083659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 msgr2=0x55c4e83afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f052ad9d640 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 0x55c4e83afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f052ad9d640 1 -- 172.21.15.78:0/1090083659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 msgr2=0x55c4e83ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f052ad9d640 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f052ad9d640 1 -- 172.21.15.78:0/1090083659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e8420d00 con 0x55c4e839cdc0 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f052ad9d640 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83ab240 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f051c02a4c0 tx=0x7f051c031d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fd688d8fc2fa7d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f050eff5640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f050eff5640 1 -- 172.21.15.78:0/1090083659 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c035070 con 0x55c4e839cdc0 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f050eff5640 1 -- 172.21.15.78:0/1090083659 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f051c0048d0 con 0x55c4e839cdc0 2024-09-16T00:30:07.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f0533440740 1 -- 172.21.15.78:0/1090083659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f052001fcf0 con 0x55c4e839cdc0 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f050eff5640 1 -- 172.21.15.78:0/1090083659 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f051c009780 con 0x55c4e839cdc0 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f0533440740 10 client.18411 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f0533440740 1 -- 172.21.15.78:0/1090083659 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c4e8421e60 con 0x55c4e839cdc0 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f050eff5640 1 -- 172.21.15.78:0/1090083659 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f052001fcf0 con 0x55c4e839cdc0 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f050eff5640 1 -- 172.21.15.78:0/1090083659 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f051c003a40 con 0x55c4e839cdc0 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f0533440740 10 client.18411 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f0533440740 10 client.18411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f0533440740 20 client.18411 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.981+0000 7f0533440740 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c4e84225d0 0x55c4e84249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 4 client.18411 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 -- 172.21.15.78:0/1090083659 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c4e8309d60 con 0x55c4e84225d0 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f052a59c640 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c4e84225d0 0x55c4e84249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f052a59c640 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c4e84225d0 0x55c4e84249f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c5feb772943090 server_cookie=677f28b5fc989904 in_seq=0 out_seq=0 2024-09-16T00:30:07.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f050eff5640 10 client.18411 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f050eff5640 1 -- 172.21.15.78:0/1090083659 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c4e8309d60 con 0x55c4e84225d0 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f050eff5640 10 client.18411 handle_command_reply: tid=0 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 client.18411 shutdown 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 -- 172.21.15.78:0/1090083659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 msgr2=0x55c4e83ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83ab240 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f051c02a4c0 tx=0x7f051c031d80 comp rx=0 tx=0).stop 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 -- 172.21.15.78:0/1090083659 shutdown_connections 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x55c4e84225d0 0x55c4e84249f0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c4e83a7680 0x55c4e83ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55c4e839cdc0 0x55c4e83ab240 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 --2- 172.21.15.78:0/1090083659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4e839c4a0 0x55c4e83afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f0533440740 1 -- 172.21.15.78:0/1090083659 >> 172.21.15.78:0/1090083659 conn(0x55c4e8398a30 msgr2=0x55c4e8397350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.982+0000 7f050eff5640 0 client.18411 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:30:07.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0533440740 1 -- 172.21.15.78:0/1090083659 shutdown_connections 2024-09-16T00:30:07.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0533440740 1 -- 172.21.15.78:0/1090083659 wait complete. 2024-09-16T00:30:07.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0533440740 20 client.18411 trim_cache size 0 max 16384 2024-09-16T00:30:07.989 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:30:07.989 INFO:teuthology.orchestra.run.smithi078.stdout: "status": "no active scrubs running", 2024-09-16T00:30:07.989 INFO:teuthology.orchestra.run.smithi078.stdout: "scrubs": {} 2024-09-16T00:30:07.989 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:30:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 -- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05140a3240 msgr2=0x7f05140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05140a3240 0x7f05140a5700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0524099050 tx=0x7f052406b3c0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 -- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c102ef0 msgr2=0x7f052c19bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c102ef0 0x7f052c19bc90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f052400eb10 tx=0x7f052400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 -- 172.21.15.78:0/4235541751 shutdown_connections 2024-09-16T00:30:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f05140a3240 0x7f05140a5700 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f052c102ef0 0x7f052c19bc90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f052c102570 0x7f052c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 --2- 172.21.15.78:0/4235541751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f052c100e20 0x7f052c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:07.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 -- 172.21.15.78:0/4235541751 >> 172.21.15.78:0/4235541751 conn(0x7f052c0fc7d0 msgr2=0x7f052c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:07.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 -- 172.21.15.78:0/4235541751 shutdown_connections 2024-09-16T00:30:07.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:07.983+0000 7f0531067640 1 -- 172.21.15.78:0/4235541751 wait complete. 2024-09-16T00:30:08.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:07 smithi193 ceph-mon[35484]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-16T00:30:08.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:07 smithi148 ceph-mon[35599]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-16T00:30:08.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:07 smithi078 ceph-mon[31157]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-16T00:30:08.875 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:30:08.876 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:09.126 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:09.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 -- 172.21.15.78:0/1281230427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c100e20 msgr2=0x7f0a3c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 --2- 172.21.15.78:0/1281230427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c100e20 0x7f0a3c101200 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009900 tx=0x7f0a2c02a480 comp rx=0 tx=0).stop 2024-09-16T00:30:09.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 -- 172.21.15.78:0/1281230427 shutdown_connections 2024-09-16T00:30:09.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 --2- 172.21.15.78:0/1281230427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c102ef0 0x7f0a3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 --2- 172.21.15.78:0/1281230427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a3c102570 0x7f0a3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 --2- 172.21.15.78:0/1281230427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c100e20 0x7f0a3c101200 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 -- 172.21.15.78:0/1281230427 >> 172.21.15.78:0/1281230427 conn(0x7f0a3c0fc7d0 msgr2=0x7f0a3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 -- 172.21.15.78:0/1281230427 shutdown_connections 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 -- 172.21.15.78:0/1281230427 wait complete. 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 Processor -- start 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.665+0000 7f0a427b1640 1 -- start start 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a427b1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a3c100e20 0x7f0a3c1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a427b1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c102570 0x7f0a3c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a427b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c102ef0 0x7f0a3c1a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a427b1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c111020 con 0x7f0a3c102570 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a427b1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c110ea0 con 0x7f0a3c102ef0 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a427b1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1111a0 con 0x7f0a3c100e20 2024-09-16T00:30:09.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c102570 0x7f0a3c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c102570 0x7f0a3c199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57084/0 (socket says 172.21.15.78:57084) 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 -- 172.21.15.78:0/3959551137 learned_addr learned my addr 172.21.15.78:0/3959551137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3bfff640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a3c100e20 0x7f0a3c1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a40d4e640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c102ef0 0x7f0a3c1a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 -- 172.21.15.78:0/3959551137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a3c100e20 msgr2=0x7f0a3c1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a3c100e20 0x7f0a3c1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 -- 172.21.15.78:0/3959551137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c102ef0 msgr2=0x7f0a3c1a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c102ef0 0x7f0a3c1a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 -- 172.21.15.78:0/3959551137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1a4ab0 con 0x7f0a3c102570 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a3b7fe640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c102570 0x7f0a3c199a30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000b4a0 tx=0x7f0a3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a397fa640 1 -- 172.21.15.78:0/3959551137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30014070 con 0x7f0a3c102570 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a397fa640 1 -- 172.21.15.78:0/3959551137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a30010af0 con 0x7f0a3c102570 2024-09-16T00:30:09.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1a4c80 con 0x7f0a3c102570 2024-09-16T00:30:09.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.666+0000 7f0a397fa640 1 -- 172.21.15.78:0/3959551137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30017d10 con 0x7f0a3c102570 2024-09-16T00:30:09.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.667+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1a5190 con 0x7f0a3c102570 2024-09-16T00:30:09.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.668+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c101280 con 0x7f0a3c102570 2024-09-16T00:30:09.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.669+0000 7f0a397fa640 1 -- 172.21.15.78:0/3959551137 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0a30017070 con 0x7f0a3c102570 2024-09-16T00:30:09.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.669+0000 7f0a397fa640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a240a32c0 0x7f0a240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.669+0000 7f0a3bfff640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a240a32c0 0x7f0a240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.669+0000 7f0a397fa640 1 -- 172.21.15.78:0/3959551137 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f0a300cf080 con 0x7f0a3c102570 2024-09-16T00:30:09.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.670+0000 7f0a3bfff640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a240a32c0 0x7f0a240a5780 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005ab0 tx=0x7f0a2c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.674+0000 7f0a397fa640 1 -- 172.21.15.78:0/3959551137 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a3009ba40 con 0x7f0a3c102570 2024-09-16T00:30:09.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.874+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a3c06a560 con 0x7f0a3c102570 2024-09-16T00:30:09.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.875+0000 7f0a397fa640 1 -- 172.21.15.78:0/3959551137 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11987 (secure 0 0 0) 0x7f0a3c06a560 con 0x7f0a3c102570 2024-09-16T00:30:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:30:09.876 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":35,"btime":"2024-09-16T00:29:33:947759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:29:33.868463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:30:09.877 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 35 2024-09-16T00:30:09.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a240a32c0 msgr2=0x7f0a240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a240a32c0 0x7f0a240a5780 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005ab0 tx=0x7f0a2c005a20 comp rx=0 tx=0).stop 2024-09-16T00:30:09.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c102570 msgr2=0x7f0a3c199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c102570 0x7f0a3c199a30 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000b4a0 tx=0x7f0a3000b960 comp rx=0 tx=0).stop 2024-09-16T00:30:09.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 shutdown_connections 2024-09-16T00:30:09.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f0a240a32c0 0x7f0a240a5780 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0a3c102ef0 0x7f0a3c1a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a3c102570 0x7f0a3c199a30 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 --2- 172.21.15.78:0/3959551137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a3c100e20 0x7f0a3c1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 >> 172.21.15.78:0/3959551137 conn(0x7f0a3c0fc7d0 msgr2=0x7f0a3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 shutdown_connections 2024-09-16T00:30:09.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:09.878+0000 7f0a427b1640 1 -- 172.21.15.78:0/3959551137 wait complete. 2024-09-16T00:30:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:09 smithi193 ceph-mon[35484]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-16T00:30:10.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:09 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3959551137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:09 smithi148 ceph-mon[35599]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-16T00:30:10.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:09 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3959551137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:10.239 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:09 smithi078 ceph-mon[31157]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-16T00:30:10.240 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:09 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3959551137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:10.579 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-16T00:30:10.823 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:11.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 -- 172.21.15.78:0/1568758661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 msgr2=0x7f1180100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 --2- 172.21.15.78:0/1568758661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 0x7f1180100ee0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f1168009930 tx=0x7f116802a420 comp rx=0 tx=0).stop 2024-09-16T00:30:11.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 -- 172.21.15.78:0/1568758661 shutdown_connections 2024-09-16T00:30:11.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 --2- 172.21.15.78:0/1568758661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1180101420 0x7f118010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 --2- 172.21.15.78:0/1568758661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 0x7f1180100ee0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 --2- 172.21.15.78:0/1568758661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1180106a80 0x7f1180106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 -- 172.21.15.78:0/1568758661 >> 172.21.15.78:0/1568758661 conn(0x7f11800fc7a0 msgr2=0x7f11800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 -- 172.21.15.78:0/1568758661 shutdown_connections 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.336+0000 7f1184d5e640 1 -- 172.21.15.78:0/1568758661 wait complete. 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1184d5e640 1 Processor -- start 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1184d5e640 1 -- start start 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1184d5e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 0x7f11801973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1184d5e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1180101420 0x7f1180197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1184d5e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1180106a80 0x7f118019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1184d5e640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1180110ea0 con 0x7f1180100a80 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1184d5e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1180110d20 con 0x7f1180101420 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1184d5e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1180111020 con 0x7f1180106a80 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f117e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 0x7f11801973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f117e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 0x7f11801973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57100/0 (socket says 172.21.15.78:57100) 2024-09-16T00:30:11.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f117e59c640 1 -- 172.21.15.78:0/2983030838 learned_addr learned my addr 172.21.15.78:0/2983030838 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1177fff640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1180101420 0x7f1180197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f117e59c640 1 -- 172.21.15.78:0/2983030838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1180106a80 msgr2=0x7f118019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f117e59c640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1180106a80 0x7f118019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f117e59c640 1 -- 172.21.15.78:0/2983030838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1180101420 msgr2=0x7f1180197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f117e59c640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1180101420 0x7f1180197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f117e59c640 1 -- 172.21.15.78:0/2983030838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118019c370 con 0x7f1180100a80 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.337+0000 7f1177fff640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1180101420 0x7f1180197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.338+0000 7f117e59c640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 0x7f11801973c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f117000c890 tx=0x7f117000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.338+0000 7f11777fe640 1 -- 172.21.15.78:0/2983030838 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170004220 con 0x7f1180100a80 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.338+0000 7f11777fe640 1 -- 172.21.15.78:0/2983030838 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11700043c0 con 0x7f1180100a80 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.338+0000 7f1184d5e640 1 -- 172.21.15.78:0/2983030838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118019c660 con 0x7f1180100a80 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.338+0000 7f11777fe640 1 -- 172.21.15.78:0/2983030838 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11700049e0 con 0x7f1180100a80 2024-09-16T00:30:11.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.338+0000 7f1184d5e640 1 -- 172.21.15.78:0/2983030838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11801a3f40 con 0x7f1180100a80 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f11777fe640 1 -- 172.21.15.78:0/2983030838 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f11700079b0 con 0x7f1180100a80 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f1187137740 1 Processor -- start 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f1187137740 1 -- start start 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f1187137740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 0x563bc5f22090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f1187137740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f225d0 0x563bc5f22a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f1187137740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f31970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f1187137740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563bc5e882f0 con 0x563bc5f2f580 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f1187137740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563bc5e2fcf0 con 0x563bc5f23b40 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.341+0000 7f1187137740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563bc5e842b0 con 0x563bc5f225d0 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f31970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f31970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57102/0 (socket says 172.21.15.78:57102) 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 -- 172.21.15.78:0/2792411144 learned_addr learned my addr 172.21.15.78:0/2792411144 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f117e59c640 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f225d0 0x563bc5f22a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f117ed9d640 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 0x563bc5f22090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f11777fe640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f116c0a3240 0x7f116c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 -- 172.21.15.78:0/2792411144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f225d0 msgr2=0x563bc5f22a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f225d0 0x563bc5f22a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 -- 172.21.15.78:0/2792411144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 msgr2=0x563bc5f22090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 0x563bc5f22090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 -- 172.21.15.78:0/2792411144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc5e143a0 con 0x563bc5f2f580 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f117ed9d640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f116c0a3240 0x7f116c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f11777fe640 1 -- 172.21.15.78:0/2983030838 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f11700cfd60 con 0x7f1180100a80 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1177fff640 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f31970 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f116802a940 tx=0x7f11680316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8259fffbbcc664 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1174ff9640 1 -- 172.21.15.78:0/2792411144 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116803f070 con 0x563bc5f2f580 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1174ff9640 1 -- 172.21.15.78:0/2792411144 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11680035f0 con 0x563bc5f2f580 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.342+0000 7f1174ff9640 1 -- 172.21.15.78:0/2792411144 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168035070 con 0x563bc5f2f580 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 -- 172.21.15.78:0/2792411144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 msgr2=0x563bc5f31970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f117ed9d640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f116c0a3240 0x7f116c0a5700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1178009fa0 tx=0x7f1178004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f31970 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f116802a940 tx=0x7f11680316e0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 -- 172.21.15.78:0/2792411144 shutdown_connections 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f31970 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f225d0 0x563bc5f22a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 --2- 172.21.15.78:0/2792411144 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 0x563bc5f22090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 -- 172.21.15.78:0/2792411144 >> 172.21.15.78:0/2792411144 conn(0x563bc5eeb310 msgr2=0x563bc5f1ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 -- 172.21.15.78:0/2792411144 shutdown_connections 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.343+0000 7f1187137740 1 -- 172.21.15.78:0/2792411144 wait complete. 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 1 -- start start 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 1 Processor -- start 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 0x563bc5f2ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f23b40 0x563bc5f2c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f2c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x563bc5f32220 con 0x563bc5f2f580 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563bc5e43060 con 0x563bc5f225d0 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563bc5f323a0 con 0x563bc5f23b40 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1187137740 10 client.0 _ll_register_callbacks cb 0x563bc5b968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f2c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117ed9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f2c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57112/0 (socket says 172.21.15.78:57112) 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117ed9d640 1 -- 172.21.15.78:0/543789763 learned_addr learned my addr 172.21.15.78:0/543789763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117e59c640 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 0x563bc5f2ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f1177fff640 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f23b40 0x563bc5f2c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117ed9d640 1 -- 172.21.15.78:0/543789763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f23b40 msgr2=0x563bc5f2c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117ed9d640 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f23b40 0x563bc5f2c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117ed9d640 1 -- 172.21.15.78:0/543789763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 msgr2=0x563bc5f2ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117ed9d640 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 0x563bc5f2ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.344+0000 7f117ed9d640 1 -- 172.21.15.78:0/543789763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc5bf93c0 con 0x563bc5f2f580 2024-09-16T00:30:11.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f117ed9d640 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f2c810 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1178017e10 tx=0x7f1178004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae81728faf0d4c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1165ffb640 1 -- 172.21.15.78:0/543789763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178017220 con 0x563bc5f2f580 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1165ffb640 1 -- 172.21.15.78:0/543789763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1178016450 con 0x563bc5f2f580 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 1 -- 172.21.15.78:0/543789763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11680035f0 con 0x563bc5f2f580 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1165ffb640 1 -- 172.21.15.78:0/543789763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f117802d050 con 0x563bc5f2f580 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 10 client.18429 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 1 -- 172.21.15.78:0/543789763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563bc5f3dd20 con 0x563bc5f2f580 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1165ffb640 1 -- 172.21.15.78:0/543789763 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11680035f0 con 0x563bc5f2f580 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1165ffb640 1 -- 172.21.15.78:0/543789763 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f11780226a0 con 0x563bc5f2f580 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 10 client.18429 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 10 client.18429 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 20 client.18429 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x563bc5f3f1e0 0x563bc5f41670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 4 client.18429 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.345+0000 7f1187137740 1 -- 172.21.15.78:0/543789763 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bc5e95860 con 0x563bc5f3f1e0 2024-09-16T00:30:11.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.346+0000 7f117e59c640 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x563bc5f3f1e0 0x563bc5f41670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.346+0000 7f1165ffb640 10 client.18429 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.346+0000 7f117e59c640 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x563bc5f3f1e0 0x563bc5f41670 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=54d8ac9a66f651ca server_cookie=c323fd98532f6e97 in_seq=0 out_seq=0 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1165ffb640 1 -- 172.21.15.78:0/543789763 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563bc5e95860 con 0x563bc5f3f1e0 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1165ffb640 10 client.18429 handle_command_reply: tid=0 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 client.18429 shutdown 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 -- 172.21.15.78:0/543789763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 msgr2=0x563bc5f2c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f2c810 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f1178017e10 tx=0x7f1178004bc0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 -- 172.21.15.78:0/543789763 shutdown_connections 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f2f580 0x563bc5f2c810 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x563bc5f3f1e0 0x563bc5f41670 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f23b40 0x563bc5f2c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 --2- 172.21.15.78:0/543789763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 0x563bc5f2ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1165ffb640 0 client.18429 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:30:11.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 -- 172.21.15.78:0/543789763 >> 172.21.15.78:0/543789763 conn(0x563bc5eeb310 msgr2=0x563bc5f24960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 -- 172.21.15.78:0/543789763 shutdown_connections 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 1 -- 172.21.15.78:0/543789763 wait complete. 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.347+0000 7f1187137740 20 client.18429 trim_cache size 0 max 16384 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.359+0000 7f1187137740 1 Processor -- start 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1187137740 1 -- start start 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1187137740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f225d0 0x563bc5f1ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1187137740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 0x563bc5f3b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1187137740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 0x563bc5f46f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1187137740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11801acac0 con 0x563bc5f225d0 2024-09-16T00:30:11.380 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1187137740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f11801ae630 con 0x563bc5f23b40 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1187137740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11801ad190 con 0x563bc5f2f580 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f225d0 0x563bc5f1ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f225d0 0x563bc5f1ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57118/0 (socket says 172.21.15.78:57118) 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 -- 172.21.15.78:0/2791619538 learned_addr learned my addr 172.21.15.78:0/2791619538 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1177fff640 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 0x563bc5f3b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117ed9d640 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 0x563bc5f46f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 -- 172.21.15.78:0/2791619538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 msgr2=0x563bc5f46f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:11.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 0x563bc5f46f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 -- 172.21.15.78:0/2791619538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 msgr2=0x563bc5f3b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 0x563bc5f3b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 -- 172.21.15.78:0/2791619538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc5f2f2e0 con 0x563bc5f225d0 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f117e59c640 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f225d0 0x563bc5f1ee10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f1170005840 tx=0x7f11700cd3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8831033872d53c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1167fff640 1 -- 172.21.15.78:0/2791619538 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117001d070 con 0x563bc5f225d0 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1167fff640 1 -- 172.21.15.78:0/2791619538 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f117009c720 con 0x563bc5f225d0 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.360+0000 7f1167fff640 1 -- 172.21.15.78:0/2791619538 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117009a0c0 con 0x563bc5f225d0 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 -- 172.21.15.78:0/2791619538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f225d0 msgr2=0x563bc5f1ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f225d0 0x563bc5f1ee10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f1170005840 tx=0x7f11700cd3d0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 -- 172.21.15.78:0/2791619538 shutdown_connections 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 0x563bc5f46f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f23b40 0x563bc5f3b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 --2- 172.21.15.78:0/2791619538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f225d0 0x563bc5f1ee10 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 -- 172.21.15.78:0/2791619538 >> 172.21.15.78:0/2791619538 conn(0x563bc5eeb310 msgr2=0x563bc5f1e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 -- 172.21.15.78:0/2791619538 shutdown_connections 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 -- 172.21.15.78:0/2791619538 wait complete. 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 -- start start 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 Processor -- start 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 0x563bc5f37630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.361+0000 7f1187137740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f23b40 0x563bc5f37b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1187137740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 0x563bc5f34600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1187137740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11801adf30 con 0x563bc5f23b40 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1187137740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f11801ad860 con 0x563bc5f225d0 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1187137740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11801aaaa0 con 0x563bc5f2f580 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1187137740 10 client.0 _ll_register_callbacks cb 0x7f1178009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f23b40 0x563bc5f37b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f23b40 0x563bc5f37b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57128/0 (socket says 172.21.15.78:57128) 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 -- 172.21.15.78:0/4177347812 learned_addr learned my addr 172.21.15.78:0/4177347812 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f117ed9d640 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 0x563bc5f34600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f117e59c640 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 0x563bc5f37630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 -- 172.21.15.78:0/4177347812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 msgr2=0x563bc5f34600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 0x563bc5f34600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 -- 172.21.15.78:0/4177347812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 msgr2=0x563bc5f37630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 0x563bc5f37630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 -- 172.21.15.78:0/4177347812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc5f34c40 con 0x563bc5f23b40 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1177fff640 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f23b40 0x563bc5f37b70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1168009930 tx=0x7f11680045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f8f9b62dc0a6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1165ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1165ffb640 1 -- 172.21.15.78:0/4177347812 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168035040 con 0x563bc5f23b40 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1165ffb640 1 -- 172.21.15.78:0/4177347812 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f1168005220 con 0x563bc5f23b40 2024-09-16T00:30:11.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1187137740 1 -- 172.21.15.78:0/4177347812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f117009c720 con 0x563bc5f23b40 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1165ffb640 1 -- 172.21.15.78:0/4177347812 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1168056050 con 0x563bc5f23b40 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1187137740 10 client.18435 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.362+0000 7f1187137740 1 -- 172.21.15.78:0/4177347812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563bc5faa8b0 con 0x563bc5f23b40 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1165ffb640 1 -- 172.21.15.78:0/4177347812 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f117009c720 con 0x563bc5f23b40 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1165ffb640 1 -- 172.21.15.78:0/4177347812 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f11680038c0 con 0x563bc5f23b40 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1187137740 10 client.18435 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1187137740 10 client.18435 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1187137740 20 client.18435 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1187137740 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x563bc5faaca0 0x563bc5fad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1187137740 4 client.18435 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1187137740 1 -- 172.21.15.78:0/4177347812 --> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563bc5e8fd70 con 0x563bc5faaca0 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f117e59c640 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x563bc5faaca0 0x563bc5fad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f1165ffb640 10 client.18435 ms_handle_connect on v2:172.21.15.193:6834/2292206006 2024-09-16T00:30:11.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.363+0000 7f117e59c640 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x563bc5faaca0 0x563bc5fad090 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d54610d600e1cb1d server_cookie=169579430442af2f in_seq=0 out_seq=0 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1165ffb640 1 -- 172.21.15.78:0/4177347812 <== mds.4 v2:172.21.15.193:6834/2292206006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563bc5e8fd70 con 0x563bc5faaca0 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1165ffb640 10 client.18435 handle_command_reply: tid=0 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 client.18435 shutdown 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 -- 172.21.15.78:0/4177347812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f23b40 msgr2=0x563bc5f37b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f23b40 0x563bc5f37b70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1168009930 tx=0x7f11680045c0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 -- 172.21.15.78:0/4177347812 shutdown_connections 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.193:6834/2292206006,v1:172.21.15.193:6835/2292206006] conn(0x563bc5faaca0 0x563bc5fad090 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bc5f2f580 0x563bc5f34600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1165ffb640 0 client.18435 ms_handle_reset on v2:172.21.15.193:6834/2292206006 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x563bc5f23b40 0x563bc5f37b70 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 --2- 172.21.15.78:0/4177347812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563bc5f225d0 0x563bc5f37630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 -- 172.21.15.78:0/4177347812 >> 172.21.15.78:0/4177347812 conn(0x563bc5eeb310 msgr2=0x563bc5f2da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:11.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 -- 172.21.15.78:0/4177347812 shutdown_connections 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 1 -- 172.21.15.78:0/4177347812 wait complete. 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1187137740 20 client.18435 trim_cache size 0 max 16384 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1184d5e640 1 -- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f116c0a3240 msgr2=0x7f116c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1184d5e640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f116c0a3240 0x7f116c0a5700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1178009fa0 tx=0x7f1178004b50 comp rx=0 tx=0).stop 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1184d5e640 1 -- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 msgr2=0x7f11801973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.364+0000 7f1184d5e640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 0x7f11801973c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f117000c890 tx=0x7f117000cd50 comp rx=0 tx=0).stop 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.365+0000 7f1184d5e640 1 -- 172.21.15.78:0/2983030838 shutdown_connections 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.365+0000 7f1184d5e640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f116c0a3240 0x7f116c0a5700 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.365+0000 7f1184d5e640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1180106a80 0x7f118019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.365+0000 7f1184d5e640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1180101420 0x7f1180197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.365+0000 7f1184d5e640 1 --2- 172.21.15.78:0/2983030838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1180100a80 0x7f11801973c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.365+0000 7f1184d5e640 1 -- 172.21.15.78:0/2983030838 >> 172.21.15.78:0/2983030838 conn(0x7f11800fc7a0 msgr2=0x7f11800fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.365+0000 7f1184d5e640 1 -- 172.21.15.78:0/2983030838 shutdown_connections 2024-09-16T00:30:11.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:11.365+0000 7f1184d5e640 1 -- 172.21.15.78:0/2983030838 wait complete. 2024-09-16T00:30:12.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:11 smithi078 ceph-mon[31157]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-16T00:30:12.079 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-16T00:30:12.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:11 smithi193 ceph-mon[35484]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-16T00:30:12.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:11 smithi148 ceph-mon[35599]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-16T00:30:12.324 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:12.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.836+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/549511798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 msgr2=0x7f5c08103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.836+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/549511798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 0x7f5c08103110 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009930 tx=0x7f5bf802a420 comp rx=0 tx=0).stop 2024-09-16T00:30:12.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.836+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/549511798 shutdown_connections 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.836+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/549511798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c081082a0 0x7f5c0810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.836+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/549511798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c081036e0 0x7f5c08107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.836+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/549511798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 0x7f5c08103110 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.837+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/549511798 >> 172.21.15.78:0/549511798 conn(0x7f5c080fc7d0 msgr2=0x7f5c080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.837+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/549511798 shutdown_connections 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.837+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/549511798 wait complete. 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.837+0000 7f5c0e0fe640 1 Processor -- start 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.837+0000 7f5c0e0fe640 1 -- start start 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c0e0fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 0x7f5c081985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c0e0fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c081036e0 0x7f5c08198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c0e0fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c081082a0 0x7f5c0819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c0e0fe640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081120a0 con 0x7f5c08102d30 2024-09-16T00:30:12.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c0e0fe640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08111f20 con 0x7f5c081036e0 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c0e0fe640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c08112220 con 0x7f5c081082a0 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 0x7f5c081985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 0x7f5c081985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57144/0 (socket says 172.21.15.78:57144) 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 -- 172.21.15.78:0/3054058248 learned_addr learned my addr 172.21.15.78:0/3054058248 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5bfeffd640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c081036e0 0x7f5c08198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c07fff640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c081082a0 0x7f5c0819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 -- 172.21.15.78:0/3054058248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c081082a0 msgr2=0x7f5c0819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c081082a0 0x7f5c0819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 -- 172.21.15.78:0/3054058248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c081036e0 msgr2=0x7f5c08198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c081036e0 0x7f5c08198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 -- 172.21.15.78:0/3054058248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0819d550 con 0x7f5c08102d30 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5bfeffd640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c081036e0 0x7f5c08198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c07fff640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c081082a0 0x7f5c0819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:12.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c077fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 0x7f5c081985a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009230 tx=0x7f5bf8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c057fa640 1 -- 172.21.15.78:0/3054058248 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf803f030 con 0x7f5c08102d30 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c057fa640 1 -- 172.21.15.78:0/3054058248 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf8003ad0 con 0x7f5c08102d30 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/3054058248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0819d7e0 con 0x7f5c08102d30 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c057fa640 1 -- 172.21.15.78:0/3054058248 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8035070 con 0x7f5c08102d30 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.838+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/3054058248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c081a5120 con 0x7f5c08102d30 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c057fa640 1 -- 172.21.15.78:0/3054058248 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5bf8042030 con 0x7f5c08102d30 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c104d7740 1 Processor -- start 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c104d7740 1 -- start start 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b971a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b971aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 0x5636b9727760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c104d7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5636b96802f0 con 0x5636b971a5d0 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c104d7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5636b9627cf0 con 0x5636b971bb40 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c104d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5636b967c2b0 con 0x5636b9725370 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b971aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b971aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57152/0 (socket says 172.21.15.78:57152) 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c077fe640 1 -- 172.21.15.78:0/2410586249 learned_addr learned my addr 172.21.15.78:0/2410586249 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c07fff640 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b971a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c077fe640 1 -- 172.21.15.78:0/2410586249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 msgr2=0x5636b9727760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c077fe640 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 0x5636b9727760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c077fe640 1 -- 172.21.15.78:0/2410586249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 msgr2=0x5636b971a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.841+0000 7f5c077fe640 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b971a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5c077fe640 1 -- 172.21.15.78:0/2410586249 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b960c3a0 con 0x5636b971a5d0 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5c07fff640 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b971a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5c077fe640 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b971aa10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80f8480 tx=0x7f5bf80f8570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85d963d8ef2a6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5bfdffb640 1 -- 172.21.15.78:0/2410586249 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80f22d0 con 0x5636b971a5d0 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5bfdffb640 1 -- 172.21.15.78:0/2410586249 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf80fd030 con 0x5636b971a5d0 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5bfdffb640 1 -- 172.21.15.78:0/2410586249 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8101800 con 0x5636b971a5d0 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5c057fa640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5bf40a32c0 0x7f5bf40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5c07fff640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5bf40a32c0 0x7f5bf40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.842+0000 7f5c057fa640 1 -- 172.21.15.78:0/3054058248 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5bf80f01a0 con 0x7f5c08102d30 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 -- 172.21.15.78:0/2410586249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 msgr2=0x5636b971aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c07fff640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5bf40a32c0 0x7f5bf40a5780 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5c000092f0 tx=0x7f5c00009460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b971aa10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80f8480 tx=0x7f5bf80f8570 comp rx=0 tx=0).stop 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 -- 172.21.15.78:0/2410586249 shutdown_connections 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 0x5636b9727760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b971aa10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2410586249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b971a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 -- 172.21.15.78:0/2410586249 >> 172.21.15.78:0/2410586249 conn(0x5636b96e3310 msgr2=0x5636b9716c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 -- 172.21.15.78:0/2410586249 shutdown_connections 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 -- 172.21.15.78:0/2410586249 wait complete. 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 -- start start 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 Processor -- start 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b9724cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.843+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b9722020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 0x5636b9722580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c104d7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5636b9727e30 con 0x5636b971a5d0 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c104d7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5636b963b060 con 0x5636b971bb40 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c104d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5636b9727fb0 con 0x5636b9725370 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c104d7740 10 client.0 _ll_register_callbacks cb 0x5636b938e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b9724cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b9724cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57162/0 (socket says 172.21.15.78:57162) 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 -- 172.21.15.78:0/2287766606 learned_addr learned my addr 172.21.15.78:0/2287766606 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5bfeffd640 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b9722020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 -- 172.21.15.78:0/2287766606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 msgr2=0x5636b9722580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c07fff640 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 0x5636b9722580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 0x5636b9722580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 -- 172.21.15.78:0/2287766606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 msgr2=0x5636b9722020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b9722020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 -- 172.21.15.78:0/2287766606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b93f13c0 con 0x5636b971a5d0 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5bfeffd640 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b9722020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c077fe640 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b9724cd0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80b88c0 tx=0x7f5bf80f3f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7855cbdeb056612 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5be6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5be6ffd640 1 -- 172.21.15.78:0/2287766606 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8049070 con 0x5636b971a5d0 2024-09-16T00:30:12.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5be6ffd640 1 -- 172.21.15.78:0/2287766606 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5bf803b460 con 0x5636b971a5d0 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c104d7740 1 -- 172.21.15.78:0/2287766606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bf80fd030 con 0x5636b971a5d0 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5be6ffd640 1 -- 172.21.15.78:0/2287766606 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5bf80b8700 con 0x5636b971a5d0 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c104d7740 10 client.18447 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.844+0000 7f5c104d7740 1 -- 172.21.15.78:0/2287766606 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5636b9727e30 con 0x5636b971a5d0 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5be6ffd640 1 -- 172.21.15.78:0/2287766606 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf80fd030 con 0x5636b971a5d0 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5be6ffd640 1 -- 172.21.15.78:0/2287766606 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f5bf80b8ad0 con 0x5636b971a5d0 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5c104d7740 10 client.18447 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5c104d7740 10 client.18447 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5c104d7740 20 client.18447 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5636b97349c0 0x5636b9736df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5c104d7740 4 client.18447 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5c104d7740 1 -- 172.21.15.78:0/2287766606 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b968d860 con 0x5636b97349c0 2024-09-16T00:30:12.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5bfeffd640 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5636b97349c0 0x5636b9736df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5be6ffd640 10 client.18447 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.845+0000 7f5bfeffd640 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5636b97349c0 0x5636b9736df0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=102a18bdb6d47182 server_cookie=3564a880715330b6 in_seq=0 out_seq=0 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5be6ffd640 1 -- 172.21.15.78:0/2287766606 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636b968d860 con 0x5636b97349c0 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5be6ffd640 10 client.18447 handle_command_reply: tid=0 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 client.18447 shutdown 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 -- 172.21.15.78:0/2287766606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 msgr2=0x5636b9724cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b9724cd0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80b88c0 tx=0x7f5bf80f3f50 comp rx=0 tx=0).stop 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 -- 172.21.15.78:0/2287766606 shutdown_connections 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b9725370 0x5636b9722580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5636b97349c0 0x5636b9736df0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971bb40 0x5636b9722020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 --2- 172.21.15.78:0/2287766606 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b971a5d0 0x5636b9724cd0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5be6ffd640 0 client.18447 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:30:12.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 -- 172.21.15.78:0/2287766606 >> 172.21.15.78:0/2287766606 conn(0x5636b96e3310 msgr2=0x5636b971c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:12.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 -- 172.21.15.78:0/2287766606 shutdown_connections 2024-09-16T00:30:12.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.846+0000 7f5c104d7740 1 -- 172.21.15.78:0/2287766606 wait complete. 2024-09-16T00:30:12.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.847+0000 7f5c104d7740 20 client.18447 trim_cache size 0 max 16384 2024-09-16T00:30:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c104d7740 1 Processor -- start 2024-09-16T00:30:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c104d7740 1 -- start start 2024-09-16T00:30:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 0x5636b971d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 0x5636b971d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b973c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c104d7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081aec90 con 0x5636b9725370 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c104d7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081b0800 con 0x5636b971a5d0 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c104d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081af360 con 0x5636b971bb40 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b973c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b973c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57178/0 (socket says 172.21.15.78:57178) 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 -- 172.21.15.78:0/1421623419 learned_addr learned my addr 172.21.15.78:0/1421623419 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c077fe640 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 0x5636b971d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5bfeffd640 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 0x5636b971d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 -- 172.21.15.78:0/1421623419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 msgr2=0x5636b971d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 0x5636b971d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 -- 172.21.15.78:0/1421623419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 msgr2=0x5636b971d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 0x5636b971d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 -- 172.21.15.78:0/1421623419 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b9717960 con 0x5636b9725370 2024-09-16T00:30:12.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.860+0000 7f5c07fff640 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b973c660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5c00011630 tx=0x7f5c00013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f30b1fa4f39ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5bfd7fa640 1 -- 172.21.15.78:0/1421623419 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00026430 con 0x5636b9725370 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5bfd7fa640 1 -- 172.21.15.78:0/1421623419 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0001d8f0 con 0x5636b9725370 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5bfd7fa640 1 -- 172.21.15.78:0/1421623419 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00025900 con 0x5636b9725370 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5c104d7740 1 -- 172.21.15.78:0/1421623419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 msgr2=0x5636b973c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5c104d7740 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b973c660 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5c00011630 tx=0x7f5c00013000 comp rx=0 tx=0).stop 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5c104d7740 1 -- 172.21.15.78:0/1421623419 shutdown_connections 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5c104d7740 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b973c660 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5c104d7740 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 0x5636b971d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5c104d7740 1 --2- 172.21.15.78:0/1421623419 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 0x5636b971d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.861+0000 7f5c104d7740 1 -- 172.21.15.78:0/1421623419 >> 172.21.15.78:0/1421623419 conn(0x5636b96e3310 msgr2=0x5636b9723f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 -- 172.21.15.78:0/1421623419 shutdown_connections 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 -- 172.21.15.78:0/1421623419 wait complete. 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 -- start start 2024-09-16T00:30:12.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 Processor -- start 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 0x5636b97292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 0x5636b9729810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b97a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081b0100 con 0x5636b9725370 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081afa30 con 0x5636b971a5d0 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c081acc70 con 0x5636b971bb40 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.862+0000 7f5c104d7740 10 client.0 _ll_register_callbacks cb 0x7f5bf80f8300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b97a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b97a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57186/0 (socket says 172.21.15.78:57186) 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 -- 172.21.15.78:0/4117234112 learned_addr learned my addr 172.21.15.78:0/4117234112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c077fe640 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 0x5636b97292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5bfeffd640 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 0x5636b9729810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 -- 172.21.15.78:0/4117234112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 msgr2=0x5636b9729810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:12.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 0x5636b9729810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 -- 172.21.15.78:0/4117234112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 msgr2=0x5636b97292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 0x5636b97292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 -- 172.21.15.78:0/4117234112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b972a0f0 con 0x5636b9725370 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c07fff640 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b97a2a40 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5c00011a30 tx=0x7f5c00011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05cddfe0073e784 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5be6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5be6ffd640 1 -- 172.21.15.78:0/4117234112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00033020 con 0x5636b9725370 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5be6ffd640 1 -- 172.21.15.78:0/4117234112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f5c0001d650 con 0x5636b9725370 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c104d7740 1 -- 172.21.15.78:0/4117234112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c0001d8f0 con 0x5636b9725370 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5be6ffd640 1 -- 172.21.15.78:0/4117234112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5c0003e050 con 0x5636b9725370 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c104d7740 10 client.18453 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.863+0000 7f5c104d7740 1 -- 172.21.15.78:0/4117234112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5636b97a4090 con 0x5636b9725370 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5be6ffd640 1 -- 172.21.15.78:0/4117234112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0001d8f0 con 0x5636b9725370 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5be6ffd640 1 -- 172.21.15.78:0/4117234112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f5c00025950 con 0x5636b9725370 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5c104d7740 10 client.18453 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:12.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5c104d7740 10 client.18453 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5c104d7740 20 client.18453 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5c104d7740 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5636b97a47a0 0x5636b97a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5c104d7740 4 client.18453 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5c104d7740 1 -- 172.21.15.78:0/4117234112 --> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636b9687d70 con 0x5636b97a47a0 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.864+0000 7f5c077fe640 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5636b97a47a0 0x5636b97a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c077fe640 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5636b97a47a0 0x5636b97a6b90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ca90bf301fb03425 server_cookie=50747426d91ae166 in_seq=0 out_seq=0 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5be6ffd640 10 client.18453 ms_handle_connect on v2:172.21.15.193:6836/1139945107 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5be6ffd640 1 -- 172.21.15.78:0/4117234112 <== mds.3 v2:172.21.15.193:6836/1139945107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636b9687d70 con 0x5636b97a47a0 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5be6ffd640 10 client.18453 handle_command_reply: tid=0 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 client.18453 shutdown 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 -- 172.21.15.78:0/4117234112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 msgr2=0x5636b97a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b97a2a40 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5c00011a30 tx=0x7f5c00011f40 comp rx=0 tx=0).stop 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 -- 172.21.15.78:0/4117234112 shutdown_connections 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5636b9725370 0x5636b97a2a40 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636b971bb40 0x5636b9729810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.193:6836/1139945107,v1:172.21.15.193:6837/1139945107] conn(0x5636b97a47a0 0x5636b97a6b90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 --2- 172.21.15.78:0/4117234112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5636b971a5d0 0x5636b97292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 -- 172.21.15.78:0/4117234112 >> 172.21.15.78:0/4117234112 conn(0x5636b96e3310 msgr2=0x5636b97154e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:12.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5be6ffd640 0 client.18453 ms_handle_reset on v2:172.21.15.193:6836/1139945107 2024-09-16T00:30:12.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 -- 172.21.15.78:0/4117234112 shutdown_connections 2024-09-16T00:30:12.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 1 -- 172.21.15.78:0/4117234112 wait complete. 2024-09-16T00:30:12.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.865+0000 7f5c104d7740 20 client.18453 trim_cache size 0 max 16384 2024-09-16T00:30:12.876 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5bf40a32c0 msgr2=0x7f5bf40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5bf40a32c0 0x7f5bf40a5780 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5c000092f0 tx=0x7f5c00009460 comp rx=0 tx=0).stop 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 msgr2=0x7f5c081985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 0x7f5c081985a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009230 tx=0x7f5bf8004270 comp rx=0 tx=0).stop 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/3054058248 shutdown_connections 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f5bf40a32c0 0x7f5bf40a5780 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c081082a0 0x7f5c0819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5c081036e0 0x7f5c08198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 --2- 172.21.15.78:0/3054058248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c08102d30 0x7f5c081985a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:12.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/3054058248 >> 172.21.15.78:0/3054058248 conn(0x7f5c080fc7d0 msgr2=0x7f5c08105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:12.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/3054058248 shutdown_connections 2024-09-16T00:30:12.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:12.866+0000 7f5c0e0fe640 1 -- 172.21.15.78:0/3054058248 wait complete. 2024-09-16T00:30:13.807 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:30:14.058 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:14.179 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:13 smithi078 ceph-mon[31157]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-16T00:30:14.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:13 smithi193 ceph-mon[35484]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-16T00:30:14.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:13 smithi148 ceph-mon[35599]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 -- 172.21.15.78:0/720035758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4100ca0 msgr2=0x7f96b4106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/720035758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4100ca0 0x7f96b4106840 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f969c009930 tx=0x7f969c02a420 comp rx=0 tx=0).stop 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 -- 172.21.15.78:0/720035758 shutdown_connections 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/720035758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96b4106f50 0x7f96b410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/720035758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4100ca0 0x7f96b4106840 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/720035758 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96b41002f0 0x7f96b41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 -- 172.21.15.78:0/720035758 >> 172.21.15.78:0/720035758 conn(0x7f96b4077210 msgr2=0x7f96b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 -- 172.21.15.78:0/720035758 shutdown_connections 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.566+0000 7f96b8a2f640 1 -- 172.21.15.78:0/720035758 wait complete. 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b8a2f640 1 Processor -- start 2024-09-16T00:30:14.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b8a2f640 1 -- start start 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b8a2f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b41002f0 0x7f96b4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b8a2f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96b4100ca0 0x7f96b41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b8a2f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96b4106f50 0x7f96b419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b8a2f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4111fe0 con 0x7f96b41002f0 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b8a2f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4111e60 con 0x7f96b4100ca0 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b8a2f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4112160 con 0x7f96b4106f50 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b41002f0 0x7f96b4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b41002f0 0x7f96b4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57210/0 (socket says 172.21.15.78:57210) 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b259c640 1 -- 172.21.15.78:0/247147610 learned_addr learned my addr 172.21.15.78:0/247147610 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96b4100ca0 0x7f96b41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96b4106f50 0x7f96b419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b259c640 1 -- 172.21.15.78:0/247147610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96b4106f50 msgr2=0x7f96b419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b259c640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96b4106f50 0x7f96b419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b259c640 1 -- 172.21.15.78:0/247147610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96b4100ca0 msgr2=0x7f96b41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b259c640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96b4100ca0 0x7f96b41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b259c640 1 -- 172.21.15.78:0/247147610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b419d410 con 0x7f96b41002f0 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.567+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96b4106f50 0x7f96b419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.568+0000 7f96b259c640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b41002f0 0x7f96b4198460 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00c920 tx=0x7f96ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.568+0000 7f96ab7fe640 1 -- 172.21.15.78:0/247147610 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac007c70 con 0x7f96b41002f0 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.568+0000 7f96ab7fe640 1 -- 172.21.15.78:0/247147610 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ac007e10 con 0x7f96b41002f0 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.568+0000 7f96b8a2f640 1 -- 172.21.15.78:0/247147610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b419d700 con 0x7f96b41002f0 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.568+0000 7f96ab7fe640 1 -- 172.21.15.78:0/247147610 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac018890 con 0x7f96b41002f0 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.568+0000 7f96b8a2f640 1 -- 172.21.15.78:0/247147610 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b41a4fe0 con 0x7f96b41002f0 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96ab7fe640 1 -- 172.21.15.78:0/247147610 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f96ac018a70 con 0x7f96b41002f0 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96bae08740 1 Processor -- start 2024-09-16T00:30:14.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96bae08740 1 -- start start 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96bae08740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a31090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96bae08740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a31a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96bae08740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a3e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96bae08740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5577319972f0 con 0x557731a3c370 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96bae08740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55773193ecf0 con 0x557731a315d0 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96bae08740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5577319932b0 con 0x557731a32b40 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96b1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a3e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96b1d9b640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a3e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57214/0 (socket says 172.21.15.78:57214) 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.571+0000 7f96b1d9b640 1 -- 172.21.15.78:0/987579939 learned_addr learned my addr 172.21.15.78:0/987579939 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b259c640 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a31a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a31090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b1d9b640 1 -- 172.21.15.78:0/987579939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 msgr2=0x557731a31090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:14.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96ab7fe640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96a00a3240 0x7f96a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a31090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b1d9b640 1 -- 172.21.15.78:0/987579939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 msgr2=0x557731a31a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a31a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b1d9b640 1 -- 172.21.15.78:0/987579939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577319233a0 con 0x557731a3c370 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a31090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96a00a3240 0x7f96a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96ab7fe640 1 -- 172.21.15.78:0/247147610 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f96ac0cfdc0 con 0x7f96b41002f0 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a3e760 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f969c009900 tx=0x7f969c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c3c0c7583c878a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96a8ff9640 1 -- 172.21.15.78:0/987579939 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c040070 con 0x557731a3c370 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96a8ff9640 1 -- 172.21.15.78:0/987579939 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c005020 con 0x557731a3c370 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.572+0000 7f96a8ff9640 1 -- 172.21.15.78:0/987579939 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c035040 con 0x557731a3c370 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 -- 172.21.15.78:0/987579939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 msgr2=0x557731a3e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96a00a3240 0x7f96a00a5700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c420 tx=0x7f96a400e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:14.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a3e760 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f969c009900 tx=0x7f969c0316e0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 -- 172.21.15.78:0/987579939 shutdown_connections 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a3e760 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a31a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 --2- 172.21.15.78:0/987579939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a31090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 -- 172.21.15.78:0/987579939 >> 172.21.15.78:0/987579939 conn(0x5577319fa310 msgr2=0x557731a2dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 -- 172.21.15.78:0/987579939 shutdown_connections 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 -- 172.21.15.78:0/987579939 wait complete. 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 -- start start 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.573+0000 7f96bae08740 1 Processor -- start 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96bae08740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a3bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96bae08740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a38f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96bae08740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96bae08740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x557731a3f010 con 0x557731a3c370 2024-09-16T00:30:14.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96bae08740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557731952060 con 0x557731a315d0 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96bae08740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557731a3f190 con 0x557731a32b40 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96bae08740 10 client.0 _ll_register_callbacks cb 0x5577316a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a394e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57216/0 (socket says 172.21.15.78:57216) 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 -- 172.21.15.78:0/1052772979 learned_addr learned my addr 172.21.15.78:0/1052772979 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b259c640 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a3bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 -- 172.21.15.78:0/1052772979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 msgr2=0x557731a38f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a38f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a38f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 -- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 msgr2=0x557731a3bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a3bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 -- 172.21.15.78:0/1052772979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577317083c0 con 0x557731a3c370 2024-09-16T00:30:14.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b259c640 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a3bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a394e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f96a4016d50 tx=0x7f96a40094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a5a7dadcca0abc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f9699ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.574+0000 7f9699ffb640 1 -- 172.21.15.78:0/1052772979 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4026060 con 0x557731a3c370 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f9699ffb640 1 -- 172.21.15.78:0/1052772979 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f96a4021400 con 0x557731a3c370 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 1 -- 172.21.15.78:0/1052772979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f969c005020 con 0x557731a3c370 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f9699ffb640 1 -- 172.21.15.78:0/1052772979 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f96a402d050 con 0x557731a3c370 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 10 client.18465 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 1 -- 172.21.15.78:0/1052772979 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557731a4a9f0 con 0x557731a3c370 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f9699ffb640 1 -- 172.21.15.78:0/1052772979 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c005020 con 0x557731a3c370 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f9699ffb640 1 -- 172.21.15.78:0/1052772979 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f96a4007370 con 0x557731a3c370 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 10 client.18465 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 10 client.18465 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 20 client.18465 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x557731a4bf00 0x557731a4e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 4 client.18465 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96bae08740 1 -- 172.21.15.78:0/1052772979 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577319a4860 con 0x557731a4bf00 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.575+0000 7f96b259c640 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x557731a4bf00 0x557731a4e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.576+0000 7f96b259c640 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x557731a4bf00 0x557731a4e3a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88287f4e94dc4f66 server_cookie=f126982f750c15ba in_seq=0 out_seq=0 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.576+0000 7f9699ffb640 10 client.18465 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f9699ffb640 1 -- 172.21.15.78:0/1052772979 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577319a4860 con 0x557731a4bf00 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f9699ffb640 10 client.18465 handle_command_reply: tid=0 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 client.18465 shutdown 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 -- 172.21.15.78:0/1052772979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 msgr2=0x557731a394e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a394e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f96a4016d50 tx=0x7f96a40094f0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 -- 172.21.15.78:0/1052772979 shutdown_connections 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a3c370 0x557731a394e0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a32b40 0x557731a38f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x557731a4bf00 0x557731a4e3a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 --2- 172.21.15.78:0/1052772979 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a315d0 0x557731a3bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 -- 172.21.15.78:0/1052772979 >> 172.21.15.78:0/1052772979 conn(0x5577319fa310 msgr2=0x557731a33960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:14.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f9699ffb640 0 client.18465 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:30:14.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 -- 172.21.15.78:0/1052772979 shutdown_connections 2024-09-16T00:30:14.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 1 -- 172.21.15.78:0/1052772979 wait complete. 2024-09-16T00:30:14.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.577+0000 7f96bae08740 20 client.18465 trim_cache size 0 max 16384 2024-09-16T00:30:14.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96bae08740 1 Processor -- start 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96bae08740 1 -- start start 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96bae08740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96bae08740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 0x557731a2d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96bae08740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 0x557731a3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96bae08740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41aeb50 con 0x557731a315d0 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96bae08740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b06c0 con 0x557731a32b40 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96bae08740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41af220 con 0x557731a3c370 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96b259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96b259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57218/0 (socket says 172.21.15.78:57218) 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.590+0000 7f96b259c640 1 -- 172.21.15.78:0/3091231269 learned_addr learned my addr 172.21.15.78:0/3091231269 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/3091231269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 0x557731a2d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f96b259c640 1 -- 172.21.15.78:0/3091231269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 msgr2=0x557731a3d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:14.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f96b259c640 1 --2- 172.21.15.78:0/3091231269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 0x557731a3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f96b259c640 1 -- 172.21.15.78:0/3091231269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 msgr2=0x557731a2d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f96b259c640 1 --2- 172.21.15.78:0/3091231269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 0x557731a2d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f96b259c640 1 -- 172.21.15.78:0/3091231269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557731a2ded0 con 0x557731a315d0 2024-09-16T00:30:14.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f96b259c640 1 --2- 172.21.15.78:0/3091231269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a482b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0cd560 tx=0x7f96ac0cd430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11710dfbe4d65263 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f969bfff640 1 -- 172.21.15.78:0/3091231269 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac09c080 con 0x557731a315d0 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f969bfff640 1 -- 172.21.15.78:0/3091231269 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ac0a0430 con 0x557731a315d0 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.591+0000 7f969bfff640 1 -- 172.21.15.78:0/3091231269 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac0a0110 con 0x557731a315d0 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- 172.21.15.78:0/3091231269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 msgr2=0x557731a482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 --2- 172.21.15.78:0/3091231269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a482b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0cd560 tx=0x7f96ac0cd430 comp rx=0 tx=0).stop 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- 172.21.15.78:0/3091231269 shutdown_connections 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 --2- 172.21.15.78:0/3091231269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 0x557731a3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 --2- 172.21.15.78:0/3091231269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 0x557731a2d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 --2- 172.21.15.78:0/3091231269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a482b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- 172.21.15.78:0/3091231269 >> 172.21.15.78:0/3091231269 conn(0x5577319fa310 msgr2=0x557731a37850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- 172.21.15.78:0/3091231269 shutdown_connections 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- 172.21.15.78:0/3091231269 wait complete. 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- start start 2024-09-16T00:30:14.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 Processor -- start 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a44430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 0x557731a44970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 0x557731a3f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41affc0 con 0x557731a315d0 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41af8f0 con 0x557731a32b40 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41acb30 con 0x557731a3c370 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96bae08740 10 client.0 _ll_register_callbacks cb 0x7f96a4009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96b259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a44430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96b259c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a44430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57224/0 (socket says 172.21.15.78:57224) 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.592+0000 7f96b259c640 1 -- 172.21.15.78:0/1131304103 learned_addr learned my addr 172.21.15.78:0/1131304103 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 0x557731a44970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96b2d9d640 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 0x557731a3f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96b259c640 1 -- 172.21.15.78:0/1131304103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 msgr2=0x557731a3f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96b259c640 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 0x557731a3f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96b259c640 1 -- 172.21.15.78:0/1131304103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 msgr2=0x557731a44970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96b259c640 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 0x557731a44970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96b259c640 1 -- 172.21.15.78:0/1131304103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557731a450b0 con 0x557731a315d0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96b259c640 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a44430 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0cd5b0 tx=0x7f96ac09a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95f6c0749df89f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f9699ffb640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f9699ffb640 1 -- 172.21.15.78:0/1131304103 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac0a4cf0 con 0x557731a315d0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f9699ffb640 1 -- 172.21.15.78:0/1131304103 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f96ac0a0110 con 0x557731a315d0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96bae08740 1 -- 172.21.15.78:0/1131304103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96ac0a0430 con 0x557731a315d0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f9699ffb640 1 -- 172.21.15.78:0/1131304103 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f96ac00fb30 con 0x557731a315d0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96bae08740 10 client.18471 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f96bae08740 1 -- 172.21.15.78:0/1131304103 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557731ab72a0 con 0x557731a315d0 2024-09-16T00:30:14.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.593+0000 7f9699ffb640 1 -- 172.21.15.78:0/1131304103 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ac0a0430 con 0x557731a315d0 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f9699ffb640 1 -- 172.21.15.78:0/1131304103 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f96ac09b4d0 con 0x557731a315d0 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96bae08740 10 client.18471 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96bae08740 10 client.18471 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96bae08740 20 client.18471 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96bae08740 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x557731ab7910 0x557731ab9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96bae08740 4 client.18471 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96bae08740 1 -- 172.21.15.78:0/1131304103 --> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55773199ed70 con 0x557731ab7910 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x557731ab7910 0x557731ab9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96b1d9b640 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x557731ab7910 0x557731ab9d30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0d12f1e2796bb3b server_cookie=fc149de0eac39abf in_seq=0 out_seq=0 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f9699ffb640 10 client.18471 ms_handle_connect on v2:172.21.15.148:6836/3780846668 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f9699ffb640 1 -- 172.21.15.78:0/1131304103 <== mds.2 v2:172.21.15.148:6836/3780846668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55773199ed70 con 0x557731ab7910 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f9699ffb640 10 client.18471 handle_command_reply: tid=0 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.594+0000 7f96bae08740 1 client.18471 shutdown 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 -- 172.21.15.78:0/1131304103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 msgr2=0x557731a44430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a44430 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0cd5b0 tx=0x7f96ac09a420 comp rx=0 tx=0).stop 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 -- 172.21.15.78:0/1131304103 shutdown_connections 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557731a3c370 0x557731a3f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.148:6836/3780846668,v1:172.21.15.148:6837/3780846668] conn(0x557731ab7910 0x557731ab9d30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557731a32b40 0x557731a44970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f9699ffb640 0 client.18471 ms_handle_reset on v2:172.21.15.148:6836/3780846668 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 --2- 172.21.15.78:0/1131304103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x557731a315d0 0x557731a44430 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 -- 172.21.15.78:0/1131304103 >> 172.21.15.78:0/1131304103 conn(0x5577319fa310 msgr2=0x557731a3a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 -- 172.21.15.78:0/1131304103 shutdown_connections 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 1 -- 172.21.15.78:0/1131304103 wait complete. 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96bae08740 20 client.18471 trim_cache size 0 max 16384 2024-09-16T00:30:14.603 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:30:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96b8a2f640 1 -- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96a00a3240 msgr2=0x7f96a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96a00a3240 0x7f96a00a5700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c420 tx=0x7f96a400e040 comp rx=0 tx=0).stop 2024-09-16T00:30:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96b8a2f640 1 -- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b41002f0 msgr2=0x7f96b4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.595+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b41002f0 0x7f96b4198460 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00c920 tx=0x7f96ac00cde0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.596+0000 7f96b8a2f640 1 -- 172.21.15.78:0/247147610 shutdown_connections 2024-09-16T00:30:14.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.596+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f96a00a3240 0x7f96a00a5700 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.596+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96b4106f50 0x7f96b419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.596+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96b4100ca0 0x7f96b41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.596+0000 7f96b8a2f640 1 --2- 172.21.15.78:0/247147610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b41002f0 0x7f96b4198460 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.596+0000 7f96b8a2f640 1 -- 172.21.15.78:0/247147610 >> 172.21.15.78:0/247147610 conn(0x7f96b4077210 msgr2=0x7f96b41057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.596+0000 7f96b8a2f640 1 -- 172.21.15.78:0/247147610 shutdown_connections 2024-09-16T00:30:14.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:14.596+0000 7f96b8a2f640 1 -- 172.21.15.78:0/247147610 wait complete. 2024-09-16T00:30:15.433 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:30:15.678 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:16.084 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:15 smithi078 ceph-mon[31157]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-16T00:30:16.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:15 smithi148 ceph-mon[35599]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-16T00:30:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2412372245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 msgr2=0x7fe7dc106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2412372245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 0x7fe7dc106810 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009900 tx=0x7fe7d002a410 comp rx=0 tx=0).stop 2024-09-16T00:30:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2412372245 shutdown_connections 2024-09-16T00:30:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2412372245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7dc106f40 0x7fe7dc10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2412372245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 0x7fe7dc106810 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2412372245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7dc101e10 0x7fe7dc1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2412372245 >> 172.21.15.78:0/2412372245 conn(0x7fe7dc0fd8d0 msgr2=0x7fe7dc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2412372245 shutdown_connections 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.165+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2412372245 wait complete. 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.166+0000 7fe7e1d28640 1 Processor -- start 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.166+0000 7fe7e1d28640 1 -- start start 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.166+0000 7fe7e1d28640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7dc101e10 0x7fe7dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.166+0000 7fe7e1d28640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 0x7fe7dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.166+0000 7fe7e1d28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7dc106f40 0x7fe7dc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.166+0000 7fe7e1d28640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc111fc0 con 0x7fe7dc1027c0 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.166+0000 7fe7e1d28640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc111e40 con 0x7fe7dc101e10 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.166+0000 7fe7e1d28640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc112140 con 0x7fe7dc106f40 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 0x7fe7dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 0x7fe7dc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57242/0 (socket says 172.21.15.78:57242) 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 -- 172.21.15.78:0/2514197985 learned_addr learned my addr 172.21.15.78:0/2514197985 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:16.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7dbfff640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7dc106f40 0x7fe7dc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7dc101e10 0x7fe7dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 -- 172.21.15.78:0/2514197985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7dc106f40 msgr2=0x7fe7dc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7dc106f40 0x7fe7dc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 -- 172.21.15.78:0/2514197985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7dc101e10 msgr2=0x7fe7dc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7dc101e10 0x7fe7dc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 -- 172.21.15.78:0/2514197985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc1a68c0 con 0x7fe7dc1027c0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7daffd640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 0x7fe7dc074a30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0005ab0 tx=0x7fe7d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7d8ff9640 1 -- 172.21.15.78:0/2514197985 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d003f030 con 0x7fe7dc1027c0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2514197985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc1a6b50 con 0x7fe7dc1027c0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7d8ff9640 1 -- 172.21.15.78:0/2514197985 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d0003ce0 con 0x7fe7dc1027c0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7d8ff9640 1 -- 172.21.15.78:0/2514197985 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0035070 con 0x7fe7dc1027c0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.167+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2514197985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc1a7080 con 0x7fe7dc1027c0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7d8ff9640 1 -- 172.21.15.78:0/2514197985 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe7d0042030 con 0x7fe7dc1027c0 2024-09-16T00:30:16.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7e4101740 1 Processor -- start 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7e4101740 1 -- start start 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 0x5602837eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7e4101740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5602837442f0 con 0x5602837de5d0 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7e4101740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5602836ebcf0 con 0x5602837dfb40 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7e4101740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5602837402b0 con 0x5602837e9370 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57256/0 (socket says 172.21.15.78:57256) 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7db7fe640 1 -- 172.21.15.78:0/1041574086 learned_addr learned my addr 172.21.15.78:0/1041574086 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7dbfff640 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7d8ff9640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7c40a32c0 0x7fe7c40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.170+0000 7fe7daffd640 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 0x5602837eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7dbfff640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7c40a32c0 0x7fe7c40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7db7fe640 1 -- 172.21.15.78:0/1041574086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 msgr2=0x5602837eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 0x5602837eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7db7fe640 1 -- 172.21.15.78:0/1041574086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 msgr2=0x5602837de090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837de090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7db7fe640 1 -- 172.21.15.78:0/1041574086 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602836d03a0 con 0x5602837de5d0 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7d8ff9640 1 -- 172.21.15.78:0/2514197985 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe7d00f05b0 con 0x7fe7dc1027c0 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dea10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00cc60 tx=0x7fe7cc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3987947d022da171 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7dbfff640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7c40a32c0 0x7fe7c40a5780 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009fd0 tx=0x7fe7d4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7c1ffb640 1 -- 172.21.15.78:0/1041574086 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc014070 con 0x5602837de5d0 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7c1ffb640 1 -- 172.21.15.78:0/1041574086 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7cc0047a0 con 0x5602837de5d0 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.171+0000 7fe7c1ffb640 1 -- 172.21.15.78:0/1041574086 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc010730 con 0x5602837de5d0 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 -- 172.21.15.78:0/1041574086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 msgr2=0x5602837dea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dea10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00cc60 tx=0x7fe7cc00e720 comp rx=0 tx=0).stop 2024-09-16T00:30:16.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 -- 172.21.15.78:0/1041574086 shutdown_connections 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 0x5602837eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dea10 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1041574086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837de090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 -- 172.21.15.78:0/1041574086 >> 172.21.15.78:0/1041574086 conn(0x5602837a7310 msgr2=0x5602837dac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 -- 172.21.15.78:0/1041574086 shutdown_connections 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 -- 172.21.15.78:0/1041574086 wait complete. 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 -- start start 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.172+0000 7fe7e4101740 1 Processor -- start 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 0x5602837e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837e6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5602837ec070 con 0x5602837dfb40 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5602836ff060 con 0x5602837e9370 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5602837ec1f0 con 0x5602837de5d0 2024-09-16T00:30:16.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 10 client.0 _ll_register_callbacks cb 0x5602834528c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837e5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57260/0 (socket says 172.21.15.78:57260) 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 -- 172.21.15.78:0/3075950512 learned_addr learned my addr 172.21.15.78:0/3075950512 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 0x5602837e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7dbfff640 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 -- 172.21.15.78:0/3075950512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 msgr2=0x5602837e8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 0x5602837e8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 -- 172.21.15.78:0/3075950512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 msgr2=0x5602837e6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837e6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 -- 172.21.15.78:0/3075950512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602834b53c0 con 0x5602837dfb40 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7dbfff640 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7daffd640 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837e5fc0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006010 tx=0x7fe7d0003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cc32a0f8ef68d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7baffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7baffd640 1 -- 172.21.15.78:0/3075950512 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0049040 con 0x5602837dfb40 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7baffd640 1 -- 172.21.15.78:0/3075950512 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe7d00f0310 con 0x5602837dfb40 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 1 -- 172.21.15.78:0/3075950512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7cc0047a0 con 0x5602837dfb40 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7baffd640 1 -- 172.21.15.78:0/3075950512 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe7d00bc3b0 con 0x5602837dfb40 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 10 client.18483 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.173+0000 7fe7e4101740 1 -- 172.21.15.78:0/3075950512 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5602837f7ac0 con 0x5602837dfb40 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7baffd640 1 -- 172.21.15.78:0/3075950512 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7cc0047a0 con 0x5602837dfb40 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7baffd640 1 -- 172.21.15.78:0/3075950512 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fe7d00b8ee0 con 0x5602837dfb40 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7e4101740 10 client.18483 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7e4101740 10 client.18483 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7e4101740 20 client.18483 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7e4101740 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5602837f8e60 0x5602837fb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7e4101740 4 client.18483 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7e4101740 1 -- 172.21.15.78:0/3075950512 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560283751860 con 0x5602837f8e60 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.174+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5602837f8e60 0x5602837fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.175+0000 7fe7baffd640 10 client.18483 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.175+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5602837f8e60 0x5602837fb250 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5ec0b95e02be328 server_cookie=9e4f49f2d5c5272c in_seq=0 out_seq=0 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.175+0000 7fe7baffd640 1 -- 172.21.15.78:0/3075950512 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560283751860 con 0x5602837f8e60 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.175+0000 7fe7baffd640 10 client.18483 handle_command_reply: tid=0 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.175+0000 7fe7e4101740 1 client.18483 shutdown 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.175+0000 7fe7e4101740 1 -- 172.21.15.78:0/3075950512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 msgr2=0x5602837e5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.175+0000 7fe7e4101740 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837e5fc0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006010 tx=0x7fe7d0003f50 comp rx=0 tx=0).stop 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 1 -- 172.21.15.78:0/3075950512 shutdown_connections 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x5602837f8e60 0x5602837fb250 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837e5fc0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7baffd640 0 client.18483 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 1 --2- 172.21.15.78:0/3075950512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 0x5602837e8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 1 -- 172.21.15.78:0/3075950512 >> 172.21.15.78:0/3075950512 conn(0x5602837a7310 msgr2=0x5602837e0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 1 -- 172.21.15.78:0/3075950512 shutdown_connections 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 1 -- 172.21.15.78:0/3075950512 wait complete. 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.176+0000 7fe7e4101740 20 client.18483 trim_cache size 0 max 16384 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.188+0000 7fe7e4101740 1 Processor -- start 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.188+0000 7fe7e4101740 1 -- start start 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.188+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.188+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837f5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.188+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 0x560283800b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7e4101740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1b0c50 con 0x5602837de5d0 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7e4101740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1b27c0 con 0x5602837dfb40 2024-09-16T00:30:16.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7e4101740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1b1320 con 0x5602837e9370 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57268/0 (socket says 172.21.15.78:57268) 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 -- 172.21.15.78:0/353318563 learned_addr learned my addr 172.21.15.78:0/353318563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7daffd640 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 -- 172.21.15.78:0/353318563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 msgr2=0x560283800b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 0x560283800b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 -- 172.21.15.78:0/353318563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 msgr2=0x5602837f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 -- 172.21.15.78:0/353318563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602837e9080 con 0x5602837de5d0 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7daffd640 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dae10 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00cc60 tx=0x7fe7cc00c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bb94d779660609 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7c17fa640 1 -- 172.21.15.78:0/353318563 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc014050 con 0x5602837de5d0 2024-09-16T00:30:16.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7c17fa640 1 -- 172.21.15.78:0/353318563 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7cc02a070 con 0x5602837de5d0 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.189+0000 7fe7c17fa640 1 -- 172.21.15.78:0/353318563 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc010790 con 0x5602837de5d0 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 -- 172.21.15.78:0/353318563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 msgr2=0x5602837dae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dae10 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00cc60 tx=0x7fe7cc00c550 comp rx=0 tx=0).stop 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 -- 172.21.15.78:0/353318563 shutdown_connections 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837e9370 0x560283800b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837dfb40 0x5602837f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 --2- 172.21.15.78:0/353318563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837de5d0 0x5602837dae10 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 -- 172.21.15.78:0/353318563 >> 172.21.15.78:0/353318563 conn(0x5602837a7310 msgr2=0x5602837da700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 -- 172.21.15.78:0/353318563 shutdown_connections 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 -- 172.21.15.78:0/353318563 wait complete. 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 -- start start 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.190+0000 7fe7e4101740 1 Processor -- start 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 0x5602837ed510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1b20c0 con 0x5602837dfb40 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1b19f0 con 0x5602837e9370 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1aec30 con 0x5602837de5d0 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 10 client.0 _ll_register_callbacks cb 0x7fe7d00b7270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837eda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57284/0 (socket says 172.21.15.78:57284) 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 -- 172.21.15.78:0/1836258583 learned_addr learned my addr 172.21.15.78:0/1836258583 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7dbfff640 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 0x5602837ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 -- 172.21.15.78:0/1836258583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 msgr2=0x5602837ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 0x5602837ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 -- 172.21.15.78:0/1836258583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 msgr2=0x5602837edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 -- 172.21.15.78:0/1836258583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560283862d90 con 0x5602837dfb40 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7dbfff640 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7daffd640 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837eda50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0005940 tx=0x7fe7d003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239c81dfe1b390ee server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7baffd640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7baffd640 1 -- 172.21.15.78:0/1836258583 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0049070 con 0x5602837dfb40 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7baffd640 1 -- 172.21.15.78:0/1836258583 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe7d00c5250 con 0x5602837dfb40 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 1 -- 172.21.15.78:0/1836258583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7cc02a070 con 0x5602837dfb40 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7baffd640 1 -- 172.21.15.78:0/1836258583 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe7d00bc1d0 con 0x5602837dfb40 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 10 client.18489 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.191+0000 7fe7e4101740 1 -- 172.21.15.78:0/1836258583 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560283862d90 con 0x5602837dfb40 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7baffd640 1 -- 172.21.15.78:0/1836258583 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7cc02a070 con 0x5602837dfb40 2024-09-16T00:30:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7baffd640 1 -- 172.21.15.78:0/1836258583 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fe7d00bcad0 con 0x5602837dfb40 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7e4101740 10 client.18489 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7e4101740 10 client.18489 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7e4101740 20 client.18489 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560283864370 0x560283866790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7e4101740 4 client.18489 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7e4101740 1 -- 172.21.15.78:0/1836258583 --> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56028374bd70 con 0x560283864370 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.192+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560283864370 0x560283866790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7baffd640 10 client.18489 ms_handle_connect on v2:172.21.15.193:6838/3313451522 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7db7fe640 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560283864370 0x560283866790 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=adaa885ef716c7ca server_cookie=7dbb728b43ce480a in_seq=0 out_seq=0 2024-09-16T00:30:16.225 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7baffd640 1 -- 172.21.15.78:0/1836258583 <== mds.1 v2:172.21.15.193:6838/3313451522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56028374bd70 con 0x560283864370 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7baffd640 10 client.18489 handle_command_reply: tid=0 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 client.18489 shutdown 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 -- 172.21.15.78:0/1836258583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 msgr2=0x5602837eda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837eda50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0005940 tx=0x7fe7d003b800 comp rx=0 tx=0).stop 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 -- 172.21.15.78:0/1836258583 shutdown_connections 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.193:6838/3313451522,v1:172.21.15.193:6839/3313451522] conn(0x560283864370 0x560283866790 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5602837e9370 0x5602837edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5602837dfb40 0x5602837eda50 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 --2- 172.21.15.78:0/1836258583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602837de5d0 0x5602837ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7e4101740 1 -- 172.21.15.78:0/1836258583 >> 172.21.15.78:0/1836258583 conn(0x5602837a7310 msgr2=0x5602837e0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.193+0000 7fe7baffd640 0 client.18489 ms_handle_reset on v2:172.21.15.193:6838/3313451522 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e4101740 1 -- 172.21.15.78:0/1836258583 shutdown_connections 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e4101740 1 -- 172.21.15.78:0/1836258583 wait complete. 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e4101740 20 client.18489 trim_cache size 0 max 16384 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7c40a32c0 msgr2=0x7fe7c40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7c40a32c0 0x7fe7c40a5780 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009fd0 tx=0x7fe7d4004a40 comp rx=0 tx=0).stop 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 msgr2=0x7fe7dc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 0x7fe7dc074a30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0005ab0 tx=0x7fe7d00042f0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2514197985 shutdown_connections 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe7c40a32c0 0x7fe7c40a5780 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7dc106f40 0x7fe7dc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7dc1027c0 0x7fe7dc074a30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.194+0000 7fe7e1d28640 1 --2- 172.21.15.78:0/2514197985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe7dc101e10 0x7fe7dc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.195+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2514197985 >> 172.21.15.78:0/2514197985 conn(0x7fe7dc0fd8d0 msgr2=0x7fe7dc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.195+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2514197985 shutdown_connections 2024-09-16T00:30:16.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:16.195+0000 7fe7e1d28640 1 -- 172.21.15.78:0/2514197985 wait complete. 2024-09-16T00:30:16.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:15 smithi193 ceph-mon[35484]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-16T00:30:17.086 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:30:17.334 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 -- 172.21.15.78:0/3279516503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 msgr2=0x7f81b4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 --2- 172.21.15.78:0/3279516503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 0x7f81b4100ee0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009930 tx=0x7f81a802a420 comp rx=0 tx=0).stop 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 -- 172.21.15.78:0/3279516503 shutdown_connections 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 --2- 172.21.15.78:0/3279516503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b4101420 0x7f81b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 --2- 172.21.15.78:0/3279516503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 0x7f81b4100ee0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 --2- 172.21.15.78:0/3279516503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b4106a80 0x7f81b4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 -- 172.21.15.78:0/3279516503 >> 172.21.15.78:0/3279516503 conn(0x7f81b40fc7a0 msgr2=0x7f81b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 -- 172.21.15.78:0/3279516503 shutdown_connections 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.872+0000 7f81baa38640 1 -- 172.21.15.78:0/3279516503 wait complete. 2024-09-16T00:30:17.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81baa38640 1 Processor -- start 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81baa38640 1 -- start start 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81baa38640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 0x7f81b41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81baa38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b4101420 0x7f81b4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81baa38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b4106a80 0x7f81b419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81baa38640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4110e40 con 0x7f81b4100a80 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81baa38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4110cc0 con 0x7f81b4101420 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81baa38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4110fc0 con 0x7f81b4106a80 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.873+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 0x7f81b41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 0x7f81b41983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35362/0 (socket says 172.21.15.78:35362) 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b3fff640 1 -- 172.21.15.78:0/3921261743 learned_addr learned my addr 172.21.15.78:0/3921261743 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b37fe640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b4101420 0x7f81b4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b8fd5640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b4106a80 0x7f81b419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b3fff640 1 -- 172.21.15.78:0/3921261743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b4106a80 msgr2=0x7f81b419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b3fff640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b4106a80 0x7f81b419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b3fff640 1 -- 172.21.15.78:0/3921261743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b4101420 msgr2=0x7f81b4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b3fff640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b4101420 0x7f81b4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b3fff640 1 -- 172.21.15.78:0/3921261743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b419d3a0 con 0x7f81b4100a80 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b37fe640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b4101420 0x7f81b4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b3fff640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 0x7f81b41983f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d950 tx=0x7f81a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b17fa640 1 -- 172.21.15.78:0/3921261743 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4017070 con 0x7f81b4100a80 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b17fa640 1 -- 172.21.15.78:0/3921261743 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a4009c30 con 0x7f81b4100a80 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81baa38640 1 -- 172.21.15.78:0/3921261743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b419d690 con 0x7f81b4100a80 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81b17fa640 1 -- 172.21.15.78:0/3921261743 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a400f6b0 con 0x7f81b4100a80 2024-09-16T00:30:17.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.874+0000 7f81baa38640 1 -- 172.21.15.78:0/3921261743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b41a4f70 con 0x7f81b4100a80 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81b17fa640 1 -- 172.21.15.78:0/3921261743 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f81a40109d0 con 0x7f81b4100a80 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81bce11740 1 Processor -- start 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81bce11740 1 -- start start 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81bce11740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81bce11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0eda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81bce11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 0x5581ca0fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81bce11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581ca0532f0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81bce11740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5581c9ffacf0 con 0x5581ca0eeb40 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81bce11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581ca04f2b0 con 0x5581ca0f8370 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0eda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35374/0 (socket says 172.21.15.78:35374) 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81b3fff640 1 -- 172.21.15.78:0/2371406833 learned_addr learned my addr 172.21.15.78:0/2371406833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81b17fa640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f819c0a3240 0x7f819c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.877+0000 7f81b8fd5640 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b37fe640 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 0x5581ca0fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b8fd5640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f819c0a3240 0x7f819c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b17fa640 1 -- 172.21.15.78:0/3921261743 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f81a40cf860 con 0x7f81b4100a80 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b3fff640 1 -- 172.21.15.78:0/2371406833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 msgr2=0x5581ca0fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b3fff640 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 0x5581ca0fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b3fff640 1 -- 172.21.15.78:0/2371406833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 msgr2=0x5581ca0ed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b3fff640 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0ed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b3fff640 1 -- 172.21.15.78:0/2371406833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581c9fdf3a0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b8fd5640 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0ed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:17.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b3fff640 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0eda10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f81a40a4200 tx=0x7f81a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835d5709bc875789 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f81b8fd5640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f819c0a3240 0x7f819c0a5700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005fd0 tx=0x7f81ac009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f819a7fc640 1 -- 172.21.15.78:0/2371406833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4020030 con 0x5581ca0ed5d0 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f819a7fc640 1 -- 172.21.15.78:0/2371406833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a409c1c0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.878+0000 7f819a7fc640 1 -- 172.21.15.78:0/2371406833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a40860f0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 -- 172.21.15.78:0/2371406833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 msgr2=0x5581ca0eda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0eda10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f81a40a4200 tx=0x7f81a4005f90 comp rx=0 tx=0).stop 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 -- 172.21.15.78:0/2371406833 shutdown_connections 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 0x5581ca0fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0eda10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 --2- 172.21.15.78:0/2371406833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0ed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 -- 172.21.15.78:0/2371406833 >> 172.21.15.78:0/2371406833 conn(0x5581ca0b6310 msgr2=0x5581ca0e9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 -- 172.21.15.78:0/2371406833 shutdown_connections 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 -- 172.21.15.78:0/2371406833 wait complete. 2024-09-16T00:30:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.879+0000 7f81bce11740 1 -- start start 2024-09-16T00:30:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81bce11740 1 Processor -- start 2024-09-16T00:30:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81bce11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81bce11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca0f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81bce11740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca0f5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81bce11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5581ca0fafb0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81bce11740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5581ca00e060 con 0x5581ca0f8370 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81bce11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5581ca0fb130 con 0x5581ca0eeb40 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81bce11740 10 client.0 _ll_register_callbacks cb 0x5581c9d618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0f7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35376/0 (socket says 172.21.15.78:35376) 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b3fff640 1 -- 172.21.15.78:0/1177712541 learned_addr learned my addr 172.21.15.78:0/1177712541 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b37fe640 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca0f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b8fd5640 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca0f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b3fff640 1 -- 172.21.15.78:0/1177712541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 msgr2=0x5581ca0f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b3fff640 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca0f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b3fff640 1 -- 172.21.15.78:0/1177712541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 msgr2=0x5581ca0f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b3fff640 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca0f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.880+0000 7f81b3fff640 1 -- 172.21.15.78:0/1177712541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581c9dc43c0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81b3fff640 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0f7d90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f81a4099d70 tx=0x7f81a4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47a999ce80e84b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81937fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81937fe640 1 -- 172.21.15.78:0/1177712541 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4020070 con 0x5581ca0ed5d0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81937fe640 1 -- 172.21.15.78:0/1177712541 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f81a409af10 con 0x5581ca0ed5d0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 1 -- 172.21.15.78:0/1177712541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a409c1c0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81937fe640 1 -- 172.21.15.78:0/1177712541 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f81a409bac0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 10 client.18501 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 1 -- 172.21.15.78:0/1177712541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5581ca106a80 con 0x5581ca0ed5d0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81937fe640 1 -- 172.21.15.78:0/1177712541 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a409c1c0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81937fe640 1 -- 172.21.15.78:0/1177712541 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f81a409a980 con 0x5581ca0ed5d0 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 10 client.18501 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 10 client.18501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 20 client.18501 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581ca107dd0 0x5581ca10a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 4 client.18501 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.881+0000 7f81bce11740 1 -- 172.21.15.78:0/1177712541 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581ca060860 con 0x5581ca107dd0 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.882+0000 7f81b37fe640 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581ca107dd0 0x5581ca10a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.882+0000 7f81937fe640 10 client.18501 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.882+0000 7f81b37fe640 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581ca107dd0 0x5581ca10a1c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=109135021819ab39 server_cookie=843cdc79de6c3ace in_seq=0 out_seq=0 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81937fe640 1 -- 172.21.15.78:0/1177712541 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581ca060860 con 0x5581ca107dd0 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81937fe640 10 client.18501 handle_command_reply: tid=0 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 client.18501 shutdown 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 -- 172.21.15.78:0/1177712541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 msgr2=0x5581ca0f7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0f7d90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f81a4099d70 tx=0x7f81a4010d70 comp rx=0 tx=0).stop 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 -- 172.21.15.78:0/1177712541 shutdown_connections 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581ca107dd0 0x5581ca10a1c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca0f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca0f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 --2- 172.21.15.78:0/1177712541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca0f7d90 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 -- 172.21.15.78:0/1177712541 >> 172.21.15.78:0/1177712541 conn(0x5581ca0b6310 msgr2=0x5581ca0ef930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:17.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81937fe640 0 client.18501 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:30:17.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 -- 172.21.15.78:0/1177712541 shutdown_connections 2024-09-16T00:30:17.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 1 -- 172.21.15.78:0/1177712541 wait complete. 2024-09-16T00:30:17.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.883+0000 7f81bce11740 20 client.18501 trim_cache size 0 max 16384 2024-09-16T00:30:17.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81bce11740 1 Processor -- start 2024-09-16T00:30:17.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81bce11740 1 -- start start 2024-09-16T00:30:17.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81bce11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca1043d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81bce11740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0f0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81bce11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 0x5581ca0f0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81bce11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41aeca0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81bce11740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b0810 con 0x5581ca0eeb40 2024-09-16T00:30:17.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81bce11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41af370 con 0x5581ca0f8370 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca1043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca1043d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35378/0 (socket says 172.21.15.78:35378) 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b3fff640 1 -- 172.21.15.78:0/895418593 learned_addr learned my addr 172.21.15.78:0/895418593 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b37fe640 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b3fff640 1 -- 172.21.15.78:0/895418593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 msgr2=0x5581ca0f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b3fff640 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 0x5581ca0f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b3fff640 1 -- 172.21.15.78:0/895418593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 msgr2=0x5581ca0f0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b3fff640 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0f0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b3fff640 1 -- 172.21.15.78:0/895418593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ca0ea960 con 0x5581ca0ed5d0 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.896+0000 7f81b37fe640 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81b3fff640 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca1043d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f81a409bc80 tx=0x7f81a4009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e59b9b1f00980c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f8199ffb640 1 -- 172.21.15.78:0/895418593 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4020030 con 0x5581ca0ed5d0 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f8199ffb640 1 -- 172.21.15.78:0/895418593 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a409c1c0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f8199ffb640 1 -- 172.21.15.78:0/895418593 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a409a980 con 0x5581ca0ed5d0 2024-09-16T00:30:17.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 -- 172.21.15.78:0/895418593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 msgr2=0x5581ca1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca1043d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f81a409bc80 tx=0x7f81a4009740 comp rx=0 tx=0).stop 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 -- 172.21.15.78:0/895418593 shutdown_connections 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0f8370 0x5581ca0f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0eeb40 0x5581ca0f0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 --2- 172.21.15.78:0/895418593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca1043d0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 -- 172.21.15.78:0/895418593 >> 172.21.15.78:0/895418593 conn(0x5581ca0b6310 msgr2=0x5581ca0f4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 -- 172.21.15.78:0/895418593 shutdown_connections 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.897+0000 7f81bce11740 1 -- 172.21.15.78:0/895418593 wait complete. 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 1 -- start start 2024-09-16T00:30:17.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 1 Processor -- start 2024-09-16T00:30:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca10d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca10aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41b0110 con 0x5581ca0ed5d0 2024-09-16T00:30:17.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41afa40 con 0x5581ca0f8370 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41acc80 con 0x5581ca0eeb40 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81bce11740 10 client.0 _ll_register_callbacks cb 0x7f81a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca10d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca10d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35388/0 (socket says 172.21.15.78:35388) 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81b3fff640 1 -- 172.21.15.78:0/3522558535 learned_addr learned my addr 172.21.15.78:0/3522558535 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.898+0000 7f81b37fe640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b8fd5640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca10aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b3fff640 1 -- 172.21.15.78:0/3522558535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 msgr2=0x5581ca10a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b3fff640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca10a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b3fff640 1 -- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 msgr2=0x5581ca10aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b3fff640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca10aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b3fff640 1 -- 172.21.15.78:0/3522558535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ca10aff0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b8fd5640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:17.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b37fe640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81b3fff640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca10d220 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f81a409ba50 tx=0x7f81a4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd1ad645e58eb9e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81937fe640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81937fe640 1 -- 172.21.15.78:0/3522558535 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4020030 con 0x5581ca0ed5d0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81937fe640 1 -- 172.21.15.78:0/3522558535 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f81a4010460 con 0x5581ca0ed5d0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81bce11740 1 -- 172.21.15.78:0/3522558535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a409c1c0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81937fe640 1 -- 172.21.15.78:0/3522558535 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f81a409b600 con 0x5581ca0ed5d0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81bce11740 10 client.18507 fetch_fsmap learned FSMap version 35 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81bce11740 1 -- 172.21.15.78:0/3522558535 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5581ca10aff0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81937fe640 1 -- 172.21.15.78:0/3522558535 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81a409c1c0 con 0x5581ca0ed5d0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81937fe640 1 -- 172.21.15.78:0/3522558535 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f81a409af10 con 0x5581ca0ed5d0 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81bce11740 10 client.18507 fetch_fsmap finished waiting for FSMap version 35 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.899+0000 7f81bce11740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81bce11740 20 client.18507 populate_metadata read hostname 'smithi078' 2024-09-16T00:30:17.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81bce11740 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581ca176a10 0x5581ca178e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81bce11740 4 client.18507 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81bce11740 1 -- 172.21.15.78:0/3522558535 --> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581ca05ad70 con 0x5581ca176a10 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81b37fe640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581ca176a10 0x5581ca178e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81937fe640 10 client.18507 ms_handle_connect on v2:172.21.15.148:6838/831570873 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81b37fe640 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581ca176a10 0x5581ca178e00 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eefc84084cb424e server_cookie=5850f7be762f79c1 in_seq=0 out_seq=0 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81937fe640 1 -- 172.21.15.78:0/3522558535 <== mds.0 v2:172.21.15.148:6838/831570873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581ca05ad70 con 0x5581ca176a10 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81937fe640 10 client.18507 handle_command_reply: tid=0 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.900+0000 7f81bce11740 1 client.18507 shutdown 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 -- 172.21.15.78:0/3522558535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 msgr2=0x5581ca10d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca10d220 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f81a409ba50 tx=0x7f81a4010d70 comp rx=0 tx=0).stop 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 -- 172.21.15.78:0/3522558535 shutdown_connections 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:6838/831570873,v1:172.21.15.148:6839/831570873] conn(0x5581ca176a10 0x5581ca178e00 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5581ca0f8370 0x5581ca10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5581ca0eeb40 0x5581ca10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81937fe640 0 client.18507 ms_handle_reset on v2:172.21.15.148:6838/831570873 2024-09-16T00:30:17.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 --2- 172.21.15.78:0/3522558535 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5581ca0ed5d0 0x5581ca10d220 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 -- 172.21.15.78:0/3522558535 >> 172.21.15.78:0/3522558535 conn(0x5581ca0b6310 msgr2=0x5581ca0e84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:17.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 -- 172.21.15.78:0/3522558535 shutdown_connections 2024-09-16T00:30:17.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 1 -- 172.21.15.78:0/3522558535 wait complete. 2024-09-16T00:30:17.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81bce11740 20 client.18507 trim_cache size 0 max 16384 2024-09-16T00:30:17.907 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-16T00:30:17.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81baa38640 1 -- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f819c0a3240 msgr2=0x7f819c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81baa38640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f819c0a3240 0x7f819c0a5700 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005fd0 tx=0x7f81ac009600 comp rx=0 tx=0).stop 2024-09-16T00:30:17.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81baa38640 1 -- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 msgr2=0x7f81b41983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.901+0000 7f81baa38640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 0x7f81b41983f0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d950 tx=0x7f81a400de10 comp rx=0 tx=0).stop 2024-09-16T00:30:17.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.902+0000 7f81baa38640 1 -- 172.21.15.78:0/3921261743 shutdown_connections 2024-09-16T00:30:17.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.902+0000 7f81baa38640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f819c0a3240 0x7f819c0a5700 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.902+0000 7f81baa38640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81b4106a80 0x7f81b419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.902+0000 7f81baa38640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f81b4101420 0x7f81b4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.902+0000 7f81baa38640 1 --2- 172.21.15.78:0/3921261743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81b4100a80 0x7f81b41983f0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.902+0000 7f81baa38640 1 -- 172.21.15.78:0/3921261743 >> 172.21.15.78:0/3921261743 conn(0x7f81b40fc7a0 msgr2=0x7f81b406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:17.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.902+0000 7f81baa38640 1 -- 172.21.15.78:0/3921261743 shutdown_connections 2024-09-16T00:30:17.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:17.902+0000 7f81baa38640 1 -- 172.21.15.78:0/3921261743 wait complete. 2024-09-16T00:30:18.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:17 smithi148 ceph-mon[35599]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-16T00:30:18.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:17 smithi078 ceph-mon[31157]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-16T00:30:18.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:17 smithi193 ceph-mon[35484]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-16T00:30:18.857 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-16T00:30:18.858 INFO:tasks.fwd_scrub:done joining 2024-09-16T00:30:18.858 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-16T00:30:18.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:30:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 -- 172.21.15.78:0/2282948582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060104f60 msgr2=0x7f2060107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 --2- 172.21.15.78:0/2282948582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060104f60 0x7f2060107350 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f204c009900 tx=0x7f204c02a410 comp rx=0 tx=0).stop 2024-09-16T00:30:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 -- 172.21.15.78:0/2282948582 shutdown_connections 2024-09-16T00:30:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 --2- 172.21.15.78:0/2282948582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060107890 0x7f2060109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 --2- 172.21.15.78:0/2282948582 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060104f60 0x7f2060107350 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 --2- 172.21.15.78:0/2282948582 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2060102630 0x7f2060104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 -- 172.21.15.78:0/2282948582 >> 172.21.15.78:0/2282948582 conn(0x7f20600fc170 msgr2=0x7f20600fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 -- 172.21.15.78:0/2282948582 shutdown_connections 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.100+0000 7f2066ce6640 1 -- 172.21.15.78:0/2282948582 wait complete. 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2066ce6640 1 Processor -- start 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2066ce6640 1 -- start start 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2066ce6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2060102630 0x7f2060197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2066ce6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060104f60 0x7f20601983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2066ce6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060107890 0x7f206019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2066ce6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f206010d7a0 con 0x7f2060107890 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2066ce6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f206010d620 con 0x7f2060102630 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2066ce6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f206010d920 con 0x7f2060104f60 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060107890 0x7f206019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060107890 0x7f206019f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35396/0 (socket says 172.21.15.78:35396) 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 -- 172.21.15.78:0/645857889 learned_addr learned my addr 172.21.15.78:0/645857889 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:19.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f205ffff640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060104f60 0x7f20601983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2064a82640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2060102630 0x7f2060197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 -- 172.21.15.78:0/645857889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060104f60 msgr2=0x7f20601983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060104f60 0x7f20601983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 -- 172.21.15.78:0/645857889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2060102630 msgr2=0x7f2060197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2060102630 0x7f2060197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 -- 172.21.15.78:0/645857889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206019fad0 con 0x7f2060107890 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2064a82640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2060102630 0x7f2060197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.101+0000 7f2065283640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060107890 0x7f206019f460 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f205800e9d0 tx=0x7f205800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.102+0000 7f205dffb640 1 -- 172.21.15.78:0/645857889 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205800cd80 con 0x7f2060107890 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.102+0000 7f205dffb640 1 -- 172.21.15.78:0/645857889 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2058005020 con 0x7f2060107890 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.102+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206019fdc0 con 0x7f2060107890 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.102+0000 7f205dffb640 1 -- 172.21.15.78:0/645857889 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20580116d0 con 0x7f2060107890 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.102+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20601a0280 con 0x7f2060107890 2024-09-16T00:30:19.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.103+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20600fdc60 con 0x7f2060107890 2024-09-16T00:30:19.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.104+0000 7f205dffb640 1 -- 172.21.15.78:0/645857889 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f20580051d0 con 0x7f2060107890 2024-09-16T00:30:19.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.105+0000 7f205dffb640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20500a3250 0x7f20500a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.105+0000 7f2064a82640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20500a3250 0x7f20500a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.105+0000 7f205dffb640 1 -- 172.21.15.78:0/645857889 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f20580d01d0 con 0x7f2060107890 2024-09-16T00:30:19.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.105+0000 7f2064a82640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20500a3250 0x7f20500a5710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f20540058c0 tx=0x7f2054005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.109+0000 7f205dffb640 1 -- 172.21.15.78:0/645857889 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2058016030 con 0x7f2060107890 2024-09-16T00:30:19.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.280+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2060062e90 con 0x7f2060107890 2024-09-16T00:30:19.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.281+0000 7f205dffb640 1 -- 172.21.15.78:0/645857889 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24757 (secure 0 0 0) 0x7f205809cb90 con 0x7f2060107890 2024-09-16T00:30:19.282 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:30:19.282 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:13:04.600749+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:30:19.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.283+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20500a3250 msgr2=0x7f20500a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.283+0000 7f2066ce6640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20500a3250 0x7f20500a5710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f20540058c0 tx=0x7f2054005850 comp rx=0 tx=0).stop 2024-09-16T00:30:19.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.283+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060107890 msgr2=0x7f206019f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.283+0000 7f2066ce6640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060107890 0x7f206019f460 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f205800e9d0 tx=0x7f205800ee90 comp rx=0 tx=0).stop 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.283+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 shutdown_connections 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.283+0000 7f2066ce6640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f20500a3250 0x7f20500a5710 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.283+0000 7f2066ce6640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2060107890 0x7f206019f460 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.284+0000 7f2066ce6640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2060104f60 0x7f20601983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.284+0000 7f2066ce6640 1 --2- 172.21.15.78:0/645857889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2060102630 0x7f2060197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.284+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 >> 172.21.15.78:0/645857889 conn(0x7f20600fc170 msgr2=0x7f20601030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.284+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 shutdown_connections 2024-09-16T00:30:19.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.284+0000 7f2066ce6640 1 -- 172.21.15.78:0/645857889 wait complete. 2024-09-16T00:30:19.295 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:06:20.608203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:11:01.500267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:11:03.483225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T00:11:09.892287+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T00:30:19.296 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:30:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 -- 172.21.15.78:0/1931140946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 msgr2=0x7fa164103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 --2- 172.21.15.78:0/1931140946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 0x7fa164103240 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa158009900 tx=0x7fa15802a410 comp rx=0 tx=0).stop 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 -- 172.21.15.78:0/1931140946 shutdown_connections 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 --2- 172.21.15.78:0/1931140946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa164103780 0x7fa16410a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 --2- 172.21.15.78:0/1931140946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 0x7fa164103240 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 --2- 172.21.15.78:0/1931140946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa1640ffdf0 0x7fa1641001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 -- 172.21.15.78:0/1931140946 >> 172.21.15.78:0/1931140946 conn(0x7fa1640fb910 msgr2=0x7fa1640fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 -- 172.21.15.78:0/1931140946 shutdown_connections 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 -- 172.21.15.78:0/1931140946 wait complete. 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.398+0000 7fa169af1640 1 Processor -- start 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa169af1640 1 -- start start 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa169af1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1640ffdf0 0x7fa164197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa169af1640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 0x7fa164198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa169af1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164103780 0x7fa16419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa169af1640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa16410d7c0 con 0x7fa164102dc0 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa169af1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa16410d640 con 0x7fa164103780 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa169af1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa16410d940 con 0x7fa1640ffdf0 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 0x7fa164198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 0x7fa164198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35410/0 (socket says 172.21.15.78:35410) 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 -- 172.21.15.78:0/419061296 learned_addr learned my addr 172.21.15.78:0/419061296 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa163fff640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164103780 0x7fa16419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 -- 172.21.15.78:0/419061296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1640ffdf0 msgr2=0x7fa164197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1640ffdf0 0x7fa164197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 -- 172.21.15.78:0/419061296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164103780 msgr2=0x7fa16419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164103780 0x7fa16419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 -- 172.21.15.78:0/419061296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16419fa70 con 0x7fa164102dc0 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.399+0000 7fa162ffd640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 0x7fa164198390 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa158009230 tx=0x7fa158004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.400+0000 7fa160ff9640 1 -- 172.21.15.78:0/419061296 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15803f030 con 0x7fa164102dc0 2024-09-16T00:30:19.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.400+0000 7fa160ff9640 1 -- 172.21.15.78:0/419061296 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa158003ce0 con 0x7fa164102dc0 2024-09-16T00:30:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.400+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16419fd00 con 0x7fa164102dc0 2024-09-16T00:30:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.400+0000 7fa160ff9640 1 -- 172.21.15.78:0/419061296 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158035070 con 0x7fa164102dc0 2024-09-16T00:30:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.400+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1641a01a0 con 0x7fa164102dc0 2024-09-16T00:30:19.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.401+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa164109010 con 0x7fa164102dc0 2024-09-16T00:30:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.402+0000 7fa160ff9640 1 -- 172.21.15.78:0/419061296 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa158042030 con 0x7fa164102dc0 2024-09-16T00:30:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.402+0000 7fa160ff9640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa14c0a3250 0x7fa14c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.403+0000 7fa1637fe640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa14c0a3250 0x7fa14c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.403+0000 7fa160ff9640 1 -- 172.21.15.78:0/419061296 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fa1580f07b0 con 0x7fa164102dc0 2024-09-16T00:30:19.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.403+0000 7fa1637fe640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa14c0a3250 0x7fa14c0a5710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa154005fd0 tx=0x7fa15400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.406+0000 7fa160ff9640 1 -- 172.21.15.78:0/419061296 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1580bd170 con 0x7fa164102dc0 2024-09-16T00:30:19.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.575+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa16402d7c0 con 0x7fa164102dc0 2024-09-16T00:30:19.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.575+0000 7fa160ff9640 1 -- 172.21.15.78:0/419061296 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fa16402d7c0 con 0x7fa164102dc0 2024-09-16T00:30:19.576 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-16T00:30:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa14c0a3250 msgr2=0x7fa14c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa14c0a3250 0x7fa14c0a5710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa154005fd0 tx=0x7fa15400b410 comp rx=0 tx=0).stop 2024-09-16T00:30:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 msgr2=0x7fa164198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 0x7fa164198390 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa158009230 tx=0x7fa158004360 comp rx=0 tx=0).stop 2024-09-16T00:30:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 shutdown_connections 2024-09-16T00:30:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fa14c0a3250 0x7fa14c0a5710 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa164103780 0x7fa16419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa164102dc0 0x7fa164198390 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 --2- 172.21.15.78:0/419061296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1640ffdf0 0x7fa164197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.577+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 >> 172.21.15.78:0/419061296 conn(0x7fa1640fb910 msgr2=0x7fa1641061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.578+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 shutdown_connections 2024-09-16T00:30:19.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.578+0000 7fa169af1640 1 -- 172.21.15.78:0/419061296 wait complete. 2024-09-16T00:30:19.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 -- 172.21.15.78:0/2485627275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a01025b0 msgr2=0x7f37a0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 --2- 172.21.15.78:0/2485627275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a01025b0 0x7f37a0108e40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f3790009ca0 tx=0x7f379002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 -- 172.21.15.78:0/2485627275 shutdown_connections 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 --2- 172.21.15.78:0/2485627275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a01025b0 0x7f37a0108e40 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 --2- 172.21.15.78:0/2485627275 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a0101bf0 0x7f37a0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 --2- 172.21.15.78:0/2485627275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37a0100a00 0x7f37a0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 -- 172.21.15.78:0/2485627275 >> 172.21.15.78:0/2485627275 conn(0x7f37a00fc140 msgr2=0x7f37a00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 -- 172.21.15.78:0/2485627275 shutdown_connections 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.691+0000 7f37a8650640 1 -- 172.21.15.78:0/2485627275 wait complete. 2024-09-16T00:30:19.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a8650640 1 Processor -- start 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a8650640 1 -- start start 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a8650640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37a0100a00 0x7f37a0069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a8650640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a0101bf0 0x7f37a019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a8650640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01025b0 0x7f37a019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a8650640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37a010c680 con 0x7f37a0101bf0 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a8650640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f37a010c500 con 0x7f37a01025b0 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a8650640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37a010c800 con 0x7f37a0100a00 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a0101bf0 0x7f37a019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a0101bf0 0x7f37a019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35424/0 (socket says 172.21.15.78:35424) 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 -- 172.21.15.78:0/3064640068 learned_addr learned my addr 172.21.15.78:0/3064640068 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a63ec640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37a0100a00 0x7f37a0069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a6bed640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01025b0 0x7f37a019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 -- 172.21.15.78:0/3064640068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37a0100a00 msgr2=0x7f37a0069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37a0100a00 0x7f37a0069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 -- 172.21.15.78:0/3064640068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01025b0 msgr2=0x7f37a019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01025b0 0x7f37a019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 -- 172.21.15.78:0/3064640068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a019fab0 con 0x7f37a0101bf0 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a6bed640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01025b0 0x7f37a019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.692+0000 7f37a5beb640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a0101bf0 0x7f37a019cb80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f379400d950 tx=0x7f379400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.693+0000 7f379f7fe640 1 -- 172.21.15.78:0/3064640068 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3794017030 con 0x7f37a0101bf0 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.693+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a019fd40 con 0x7f37a0101bf0 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.693+0000 7f379f7fe640 1 -- 172.21.15.78:0/3064640068 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3794004020 con 0x7f37a0101bf0 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.693+0000 7f379f7fe640 1 -- 172.21.15.78:0/3064640068 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37940047a0 con 0x7f37a0101bf0 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.693+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a01a0280 con 0x7f37a0101bf0 2024-09-16T00:30:19.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.693+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a0107df0 con 0x7f37a0101bf0 2024-09-16T00:30:19.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.695+0000 7f379f7fe640 1 -- 172.21.15.78:0/3064640068 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f37940041d0 con 0x7f37a0101bf0 2024-09-16T00:30:19.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.695+0000 7f379f7fe640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37880a3250 0x7f37880a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.695+0000 7f37a63ec640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37880a3250 0x7f37880a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.695+0000 7f379f7fe640 1 -- 172.21.15.78:0/3064640068 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f37940cf700 con 0x7f37a0101bf0 2024-09-16T00:30:19.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.696+0000 7f37a63ec640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37880a3250 0x7f37880a5710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f37980058c0 tx=0x7f379800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.699+0000 7f379f7fe640 1 -- 172.21.15.78:0/3064640068 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f379409c0c0 con 0x7f37a0101bf0 2024-09-16T00:30:19.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.864+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f37a0062e90 con 0x7f37a0101bf0 2024-09-16T00:30:19.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.864+0000 7f379f7fe640 1 -- 172.21.15.78:0/3064640068 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f379409b9c0 con 0x7f37a0101bf0 2024-09-16T00:30:19.865 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-09-16T00:30:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.866+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37880a3250 msgr2=0x7f37880a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.866+0000 7f37a8650640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37880a3250 0x7f37880a5710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f37980058c0 tx=0x7f379800a4c0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.866+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a0101bf0 msgr2=0x7f37a019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.866+0000 7f37a8650640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a0101bf0 0x7f37a019cb80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f379400d950 tx=0x7f379400de10 comp rx=0 tx=0).stop 2024-09-16T00:30:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.866+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 shutdown_connections 2024-09-16T00:30:19.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.866+0000 7f37a8650640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f37880a3250 0x7f37880a5710 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.867+0000 7f37a8650640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f37a01025b0 0x7f37a019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.867+0000 7f37a8650640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37a0101bf0 0x7f37a019cb80 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.867+0000 7f37a8650640 1 --2- 172.21.15.78:0/3064640068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37a0100a00 0x7f37a0069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.867+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 >> 172.21.15.78:0/3064640068 conn(0x7f37a00fc140 msgr2=0x7f37a0104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.867+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 shutdown_connections 2024-09-16T00:30:19.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.867+0000 7f37a8650640 1 -- 172.21.15.78:0/3064640068 wait complete. 2024-09-16T00:30:19.878 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:30:19.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 -- 172.21.15.78:0/2318286398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81019f0 msgr2=0x7f78d8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 --2- 172.21.15.78:0/2318286398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81019f0 0x7f78d8101df0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f78d4009900 tx=0x7f78d402a410 comp rx=0 tx=0).stop 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 -- 172.21.15.78:0/2318286398 shutdown_connections 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 --2- 172.21.15.78:0/2318286398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d81035a0 0x7f78d8109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 --2- 172.21.15.78:0/2318286398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d8102be0 0x7f78d8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 --2- 172.21.15.78:0/2318286398 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81019f0 0x7f78d8101df0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 -- 172.21.15.78:0/2318286398 >> 172.21.15.78:0/2318286398 conn(0x7f78d80fd190 msgr2=0x7f78d80ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 -- 172.21.15.78:0/2318286398 shutdown_connections 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.982+0000 7f78de79d640 1 -- 172.21.15.78:0/2318286398 wait complete. 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78de79d640 1 Processor -- start 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78de79d640 1 -- start start 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78de79d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d81019f0 0x7f78d8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78de79d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d8102be0 0x7f78d81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78de79d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81035a0 0x7f78d819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78de79d640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78d810d670 con 0x7f78d81035a0 2024-09-16T00:30:19.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78de79d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78d810d4f0 con 0x7f78d81019f0 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78de79d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78d810d7f0 con 0x7f78d8102be0 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81035a0 0x7f78d819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81035a0 0x7f78d819f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35448/0 (socket says 172.21.15.78:35448) 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 -- 172.21.15.78:0/2410933776 learned_addr learned my addr 172.21.15.78:0/2410933776 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78dd79b640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d81019f0 0x7f78d8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 -- 172.21.15.78:0/2410933776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d8102be0 msgr2=0x7f78d81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78dcf9a640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d8102be0 0x7f78d81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d8102be0 0x7f78d81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 -- 172.21.15.78:0/2410933776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d81019f0 msgr2=0x7f78d8197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d81019f0 0x7f78d8197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 -- 172.21.15.78:0/2410933776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d819f980 con 0x7f78d81035a0 2024-09-16T00:30:19.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ddf9c640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81035a0 0x7f78d819f310 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f78d000eb10 tx=0x7f78d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.983+0000 7f78ce7fc640 1 -- 172.21.15.78:0/2410933776 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0004b20 con 0x7f78d81035a0 2024-09-16T00:30:19.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.984+0000 7f78ce7fc640 1 -- 172.21.15.78:0/2410933776 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d0004cc0 con 0x7f78d81035a0 2024-09-16T00:30:19.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.984+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d819fc70 con 0x7f78d81035a0 2024-09-16T00:30:19.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.984+0000 7f78ce7fc640 1 -- 172.21.15.78:0/2410933776 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0010760 con 0x7f78d81035a0 2024-09-16T00:30:19.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.984+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81a01c0 con 0x7f78d81035a0 2024-09-16T00:30:19.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.985+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d8108e30 con 0x7f78d81035a0 2024-09-16T00:30:19.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.986+0000 7f78ce7fc640 1 -- 172.21.15.78:0/2410933776 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f78d0005020 con 0x7f78d81035a0 2024-09-16T00:30:19.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.986+0000 7f78ce7fc640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f78c40a32d0 0x7f78c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.986+0000 7f78dd79b640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f78c40a32d0 0x7f78c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.986+0000 7f78ce7fc640 1 -- 172.21.15.78:0/2410933776 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f78d00d02d0 con 0x7f78d81035a0 2024-09-16T00:30:19.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.987+0000 7f78dd79b640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f78c40a32d0 0x7f78c40a5790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f78d4006010 tx=0x7f78d4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:19.990+0000 7f78ce7fc640 1 -- 172.21.15.78:0/2410933776 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78d009cc90 con 0x7f78d81035a0 2024-09-16T00:30:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.157+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78d8062df0 con 0x7f78d81035a0 2024-09-16T00:30:20.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.157+0000 7f78ce7fc640 1 -- 172.21.15.78:0/2410933776 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f78d009c590 con 0x7f78d81035a0 2024-09-16T00:30:20.158 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-09-16T00:30:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f78c40a32d0 msgr2=0x7f78c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f78c40a32d0 0x7f78c40a5790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f78d4006010 tx=0x7f78d4000bc0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81035a0 msgr2=0x7f78d819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81035a0 0x7f78d819f310 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f78d000eb10 tx=0x7f78d000c2d0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 shutdown_connections 2024-09-16T00:30:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f78c40a32d0 0x7f78c40a5790 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78d81035a0 0x7f78d819f310 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78d8102be0 0x7f78d81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 --2- 172.21.15.78:0/2410933776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78d81019f0 0x7f78d8197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 >> 172.21.15.78:0/2410933776 conn(0x7f78d80fd190 msgr2=0x7f78d8106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 shutdown_connections 2024-09-16T00:30:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.159+0000 7f78de79d640 1 -- 172.21.15.78:0/2410933776 wait complete. 2024-09-16T00:30:20.169 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:19 smithi078 ceph-mon[31157]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-16T00:30:20.169 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:19 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/645857889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:20.169 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:19 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/419061296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:30:20.169 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:19 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3064640068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:30:20.170 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T00:30:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:19 smithi148 ceph-mon[35599]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-16T00:30:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:19 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/645857889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:19 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/419061296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:30:20.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:19 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3064640068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:30:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.273+0000 7faa8259f640 1 -- 172.21.15.78:0/2370325266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c101af0 msgr2=0x7faa7c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.273+0000 7faa8259f640 1 --2- 172.21.15.78:0/2370325266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c101af0 0x7faa7c101f70 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009900 tx=0x7faa6c02a410 comp rx=0 tx=0).stop 2024-09-16T00:30:20.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.273+0000 7faa8259f640 1 -- 172.21.15.78:0/2370325266 shutdown_connections 2024-09-16T00:30:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.273+0000 7faa8259f640 1 --2- 172.21.15.78:0/2370325266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa7c1024b0 0x7faa7c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.273+0000 7faa8259f640 1 --2- 172.21.15.78:0/2370325266 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c101af0 0x7faa7c101f70 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.273+0000 7faa8259f640 1 --2- 172.21.15.78:0/2370325266 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa7c100900 0x7faa7c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.273+0000 7faa8259f640 1 -- 172.21.15.78:0/2370325266 >> 172.21.15.78:0/2370325266 conn(0x7faa7c0fc080 msgr2=0x7faa7c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 -- 172.21.15.78:0/2370325266 shutdown_connections 2024-09-16T00:30:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 -- 172.21.15.78:0/2370325266 wait complete. 2024-09-16T00:30:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 Processor -- start 2024-09-16T00:30:20.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 -- start start 2024-09-16T00:30:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c100900 0x7faa7c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa7c101af0 0x7faa7c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:20.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa7c1024b0 0x7faa7c19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c10c650 con 0x7faa7c100900 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c10c4d0 con 0x7faa7c101af0 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.274+0000 7faa8259f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c10c7d0 con 0x7faa7c1024b0 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c100900 0x7faa7c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c100900 0x7faa7c196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35468/0 (socket says 172.21.15.78:35468) 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 -- 172.21.15.78:0/24265055 learned_addr learned my addr 172.21.15.78:0/24265055 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa80d9c640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa7c101af0 0x7faa7c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa81d9e640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa7c1024b0 0x7faa7c19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 -- 172.21.15.78:0/24265055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa7c1024b0 msgr2=0x7faa7c19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa7c1024b0 0x7faa7c19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 -- 172.21.15.78:0/24265055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa7c101af0 msgr2=0x7faa7c197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa7c101af0 0x7faa7c197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 -- 172.21.15.78:0/24265055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c19e850 con 0x7faa7c100900 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa80d9c640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa7c101af0 0x7faa7c197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:20.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8159d640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c100900 0x7faa7c196c40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7faa7800b450 tx=0x7faa7800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa727fc640 1 -- 172.21.15.78:0/24265055 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78015020 con 0x7faa7c100900 2024-09-16T00:30:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c19eb40 con 0x7faa7c100900 2024-09-16T00:30:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa727fc640 1 -- 172.21.15.78:0/24265055 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa78011d70 con 0x7faa7c100900 2024-09-16T00:30:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa727fc640 1 -- 172.21.15.78:0/24265055 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7800f340 con 0x7faa7c100900 2024-09-16T00:30:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.275+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c19f0e0 con 0x7faa7c100900 2024-09-16T00:30:20.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.276+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7c107c20 con 0x7faa7c100900 2024-09-16T00:30:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.277+0000 7faa727fc640 1 -- 172.21.15.78:0/24265055 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7faa78018020 con 0x7faa7c100900 2024-09-16T00:30:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.278+0000 7faa727fc640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa680a3250 0x7faa680a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.278+0000 7faa80d9c640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa680a3250 0x7faa680a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.278+0000 7faa727fc640 1 -- 172.21.15.78:0/24265055 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7faa780cfd70 con 0x7faa7c100900 2024-09-16T00:30:20.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.279+0000 7faa80d9c640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa680a3250 0x7faa680a5710 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faa6c006010 tx=0x7faa6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:20.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.281+0000 7faa727fc640 1 -- 172.21.15.78:0/24265055 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa7801f020 con 0x7faa7c100900 2024-09-16T00:30:20.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:19 smithi193 ceph-mon[35484]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-16T00:30:20.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:19 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/645857889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:20.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:19 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/419061296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:30:20.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:19 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3064640068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:30:20.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.448+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faa7c02d970 con 0x7faa7c100900 2024-09-16T00:30:20.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.448+0000 7faa727fc640 1 -- 172.21.15.78:0/24265055 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7faa7800d040 con 0x7faa7c100900 2024-09-16T00:30:20.449 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-09-16T00:30:20.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa680a3250 msgr2=0x7faa680a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa680a3250 0x7faa680a5710 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faa6c006010 tx=0x7faa6c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c100900 msgr2=0x7faa7c196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c100900 0x7faa7c196c40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7faa7800b450 tx=0x7faa7800b910 comp rx=0 tx=0).stop 2024-09-16T00:30:20.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 shutdown_connections 2024-09-16T00:30:20.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7faa680a3250 0x7faa680a5710 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa7c1024b0 0x7faa7c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7faa7c101af0 0x7faa7c197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 --2- 172.21.15.78:0/24265055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa7c100900 0x7faa7c196c40 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 >> 172.21.15.78:0/24265055 conn(0x7faa7c0fc080 msgr2=0x7faa7c104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:20.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 shutdown_connections 2024-09-16T00:30:20.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.450+0000 7faa8259f640 1 -- 172.21.15.78:0/24265055 wait complete. 2024-09-16T00:30:20.462 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T00:30:20.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 -- 172.21.15.78:0/2963102426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 msgr2=0x7fe4cc108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2963102426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 0x7fe4cc108e70 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009ca0 tx=0x7fe4c402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 -- 172.21.15.78:0/2963102426 shutdown_connections 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2963102426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 0x7fe4cc108e70 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2963102426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc101c20 0x7fe4cc1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2963102426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc100a30 0x7fe4cc100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 -- 172.21.15.78:0/2963102426 >> 172.21.15.78:0/2963102426 conn(0x7fe4cc0fc190 msgr2=0x7fe4cc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 -- 172.21.15.78:0/2963102426 shutdown_connections 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.560+0000 7fe4d0867640 1 -- 172.21.15.78:0/2963102426 wait complete. 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4d0867640 1 Processor -- start 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4d0867640 1 -- start start 2024-09-16T00:30:20.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4d0867640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc100a30 0x7fe4cc197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4d0867640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc101c20 0x7fe4cc1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4d0867640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 0x7fe4cc19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4d0867640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10c6b0 con 0x7fe4cc1025e0 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4d0867640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10c530 con 0x7fe4cc100a30 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4d0867640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4cc10c830 con 0x7fe4cc101c20 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4cad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 0x7fe4cc19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4cad9d640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 0x7fe4cc19f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:35486/0 (socket says 172.21.15.78:35486) 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4cad9d640 1 -- 172.21.15.78:0/2110882068 learned_addr learned my addr 172.21.15.78:0/2110882068 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4c3fff640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc101c20 0x7fe4cc1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4ca59c640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc100a30 0x7fe4cc197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4cad9d640 1 -- 172.21.15.78:0/2110882068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc101c20 msgr2=0x7fe4cc1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4cad9d640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc101c20 0x7fe4cc1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4cad9d640 1 -- 172.21.15.78:0/2110882068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc100a30 msgr2=0x7fe4cc197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4cad9d640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc100a30 0x7fe4cc197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.561+0000 7fe4cad9d640 1 -- 172.21.15.78:0/2110882068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4cc19fa90 con 0x7fe4cc1025e0 2024-09-16T00:30:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.562+0000 7fe4cad9d640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 0x7fe4cc19f420 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4031ee0 tx=0x7fe4c4031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.562+0000 7fe4c37fe640 1 -- 172.21.15.78:0/2110882068 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4003c30 con 0x7fe4cc1025e0 2024-09-16T00:30:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.562+0000 7fe4c37fe640 1 -- 172.21.15.78:0/2110882068 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4c4003dd0 con 0x7fe4cc1025e0 2024-09-16T00:30:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.562+0000 7fe4c37fe640 1 -- 172.21.15.78:0/2110882068 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c402fc10 con 0x7fe4cc1025e0 2024-09-16T00:30:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.562+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4cc19fd20 con 0x7fe4cc1025e0 2024-09-16T00:30:20.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.562+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4cc1a0260 con 0x7fe4cc1025e0 2024-09-16T00:30:20.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.563+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4cc1a0490 con 0x7fe4cc1025e0 2024-09-16T00:30:20.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.564+0000 7fe4c37fe640 1 -- 172.21.15.78:0/2110882068 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fe4c40046c0 con 0x7fe4cc1025e0 2024-09-16T00:30:20.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.565+0000 7fe4c37fe640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4b80a32d0 0x7fe4b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:20.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.565+0000 7fe4ca59c640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4b80a32d0 0x7fe4b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:20.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.565+0000 7fe4c37fe640 1 -- 172.21.15.78:0/2110882068 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fe4c40f0fe0 con 0x7fe4cc1025e0 2024-09-16T00:30:20.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.565+0000 7fe4ca59c640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4b80a32d0 0x7fe4b80a5790 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc005fd0 tx=0x7fe4bc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:20.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.569+0000 7fe4c37fe640 1 -- 172.21.15.78:0/2110882068 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4c40bd9a0 con 0x7fe4cc1025e0 2024-09-16T00:30:20.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.784+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fe4cc1a07f0 con 0x7fe4cc1025e0 2024-09-16T00:30:20.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.791+0000 7fe4c37fe640 1 -- 172.21.15.78:0/2110882068 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+742336 (secure 0 0 0) 0x7fe4c40bd2a0 con 0x7fe4cc1025e0 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 20, 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: "active_gid": 14150, 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: "active_name": "x", 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addrs": { 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-16T00:30:20.802 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6800", 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 119552845 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:6801", 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 119552845 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addr": "172.21.15.78:6801/119552845", 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "active_change": "2024-09-16T00:04:02.295526+0000", 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "active_mgr_features": 4540701547738038271, 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "available": true, 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: "standbys": [ 2024-09-16T00:30:20.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24106, 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "z", 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.805 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.808 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.809 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.812 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.813 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-09-16T00:30:20.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-16T00:30:20.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-09-16T00:30:20.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:30:20.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:30:20.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:30:20.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.826 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.831 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-09-16T00:30:20.832 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.834 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:30:20.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.837 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.840 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.844 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-09-16T00:30:20.845 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-09-16T00:30:20.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-09-16T00:30:20.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-09-16T00:30:20.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-09-16T00:30:20.850 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-09-16T00:30:20.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.852 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.853 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.856 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.860 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.862 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.864 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.865 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.866 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-09-16T00:30:20.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-09-16T00:30:20.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-09-16T00:30:20.869 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-09-16T00:30:20.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.871 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.872 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.874 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.875 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.876 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.878 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:20.879 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.884 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:30:20.885 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-09-16T00:30:20.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-09-16T00:30:20.887 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:30:20.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.890 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.894 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-09-16T00:30:20.895 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-09-16T00:30:20.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-09-16T00:30:20.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-09-16T00:30:20.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:30:20.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-09-16T00:30:20.900 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:30:20.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.903 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:20.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.906 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.909 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.910 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.915 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.916 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.919 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.920 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.924 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.925 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.934 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.937 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.940 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.943 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.944 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.947 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.950 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:30:20.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.952 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.953 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.959 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.960 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:20.961 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-09-16T00:30:20.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:30:20.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:20.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-09-16T00:30:20.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-16T00:30:20.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-16T00:30:20.971 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.973 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.977 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.978 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-09-16T00:30:20.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:20.984 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.985 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-09-16T00:30:20.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:20.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:20.992 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-09-16T00:30:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-09-16T00:30:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:30:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:20.993 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.994 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-09-16T00:30:20.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:20.996 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:20.998 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:20.999 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.000 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.002 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.004 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:30:21.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:30:21.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.010 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.011 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.012 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.013 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.015 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.019 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.020 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-09-16T00:30:21.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.022 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.024 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.025 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.027 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.028 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.029 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.034 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.036 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:30:21.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.038 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.039 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.040 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.041 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.042 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.044 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.045 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.046 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.047 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.049 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.050 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.054 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.058 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.059 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.060 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-09-16T00:30:21.063 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.064 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.065 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.066 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.067 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.068 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.069 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.071 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.072 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.073 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.074 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-09-16T00:30:21.076 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.077 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.078 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.080 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.081 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.082 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-09-16T00:30:21.083 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.084 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.086 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.087 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.088 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.090 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-09-16T00:30:21.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.092 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.094 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.095 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.096 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.097 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.098 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.099 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.100 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.101 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:30:21.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-09-16T00:30:21.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:21.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.106 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.109 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.111 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.112 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.114 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:30:21.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:30:21.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-09-16T00:30:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.119 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.120 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.122 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.123 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.124 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.125 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.127 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.128 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.130 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.132 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.133 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.134 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.135 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.136 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.137 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.138 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.139 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.140 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.141 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.142 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.145 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.146 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-09-16T00:30:21.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-09-16T00:30:21.149 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-09-16T00:30:21.150 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.151 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.152 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.153 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.154 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-09-16T00:30:21.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-09-16T00:30:21.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-09-16T00:30:21.157 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-09-16T00:30:21.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.159 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:30:21.160 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.162 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.165 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.166 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.167 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.168 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.169 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.170 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.171 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.174 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.175 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.176 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-09-16T00:30:21.178 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.180 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.181 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.183 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.184 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.186 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.187 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.191 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.192 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.196 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.198 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.199 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.201 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.202 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.203 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.205 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-09-16T00:30:21.207 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.208 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.210 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.212 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.213 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.214 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.216 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.217 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-09-16T00:30:21.218 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.219 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.220 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.221 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.222 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.223 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.225 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.226 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.228 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.230 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.232 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.233 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 24110, 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "y", 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:30:21.234 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.235 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.236 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.237 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.238 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.239 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.240 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.241 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:30:21.242 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-09-16T00:30:21.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-09-16T00:30:21.246 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:30:21.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.248 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.249 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-09-16T00:30:21.250 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.251 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.252 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.253 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.259 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.260 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.262 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.264 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.265 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-09-16T00:30:21.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.267 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-09-16T00:30:21.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.271 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.273 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.275 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.276 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.278 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.279 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.280 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.281 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.282 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.283 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:30:21.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.285 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.286 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.288 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:30:21.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-09-16T00:30:21.290 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.292 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.293 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.294 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.295 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.297 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:30:21.300 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-09-16T00:30:21.301 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-09-16T00:30:21.302 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.303 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.307 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-09-16T00:30:21.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-09-16T00:30:21.309 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-09-16T00:30:21.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-09-16T00:30:21.311 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.312 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.313 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.314 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.315 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.317 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.319 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.320 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.321 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.323 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.326 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.328 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.331 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.334 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.335 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.336 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.337 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.339 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.340 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.342 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.344 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:30:21.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.347 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.348 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.349 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.350 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.352 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.354 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.356 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.357 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:30:21.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:30:21.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:30:21.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:30:21.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.362 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-09-16T00:30:21.363 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:30:21.364 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:30:21.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:30:21.366 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:30:21.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:30:21.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:30:21.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:30:21.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.372 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.374 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.375 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.377 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.378 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.379 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.380 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.382 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.383 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.384 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.387 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.388 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.389 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.390 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.392 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.393 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-09-16T00:30:21.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.395 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.398 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.399 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.401 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.402 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.404 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.406 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-09-16T00:30:21.407 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-16T00:30:21.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.409 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-16T00:30:21.410 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.411 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.412 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.414 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.415 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.417 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.419 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.420 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.425 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.426 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.427 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.428 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-09-16T00:30:21.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-09-16T00:30:21.430 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.431 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.433 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.435 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.436 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.437 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.438 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.439 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.440 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.444 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-09-16T00:30:21.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.447 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.449 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.451 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.452 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.454 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.457 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.458 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-09-16T00:30:21.459 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:30:21.460 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.461 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.462 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.463 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.464 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.465 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.467 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.468 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.469 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.470 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.472 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.473 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.474 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.475 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.478 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.480 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.481 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.483 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.484 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.485 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.486 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.489 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.491 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.492 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.493 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.494 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.496 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.497 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.498 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.500 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.501 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.502 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-16T00:30:21.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.504 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.506 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.507 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.509 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.510 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-09-16T00:30:21.511 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.512 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.515 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.516 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.519 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.520 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.521 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-09-16T00:30:21.525 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.528 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.529 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.532 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:30:21.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:30:21.534 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-09-16T00:30:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:30:21.543 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-09-16T00:30:21.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:21.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:21.546 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-09-16T00:30:21.547 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-16T00:30:21.548 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.551 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.554 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-09-16T00:30:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-09-16T00:30:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-09-16T00:30:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-09-16T00:30:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-09-16T00:30:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-09-16T00:30:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-09-16T00:30:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-09-16T00:30:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-09-16T00:30:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.652 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.654 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.655 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.658 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.659 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.660 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.663 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-09-16T00:30:21.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.666 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.668 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.672 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.674 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.676 INFO:teuthology.orchestra.run.smithi078.stdout: "modules": [ 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard", 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "iostat", 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "nfs", 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "restful" 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.679 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.682 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.683 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.684 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.687 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:30:21.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-09-16T00:30:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-09-16T00:30:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.711 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.713 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.714 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.716 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:30:21.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.719 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-09-16T00:30:21.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-09-16T00:30:21.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-09-16T00:30:21.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-09-16T00:30:21.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.725 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-09-16T00:30:21.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-09-16T00:30:21.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.728 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.737 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.738 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.739 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.742 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:30:21.745 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.746 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.747 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.748 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.752 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.753 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.754 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.755 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.756 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.757 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.759 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.761 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.763 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-09-16T00:30:21.764 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-09-16T00:30:21.765 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-09-16T00:30:21.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-09-16T00:30:21.767 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-09-16T00:30:21.768 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.769 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.771 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.772 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.773 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.775 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.776 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.777 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.778 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.779 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.780 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.782 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.783 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.784 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.787 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.788 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.793 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.794 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.795 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.797 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.798 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.800 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.801 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.802 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.803 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.804 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.805 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.806 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.807 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.808 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.809 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.812 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.815 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:30:21.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:30:21.818 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:30:21.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:30:21.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.821 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.822 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.823 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.827 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.830 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.832 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-09-16T00:30:21.834 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.837 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.839 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.840 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-09-16T00:30:21.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:30:21.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:21.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.844 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.845 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.846 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.847 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.849 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-09-16T00:30:21.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.852 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.853 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.854 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-09-16T00:30:21.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-09-16T00:30:21.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.859 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.860 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.861 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.864 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.866 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.870 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.871 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-09-16T00:30:21.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.876 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.877 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.878 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.879 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:30:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.885 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-09-16T00:30:21.887 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-09-16T00:30:21.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.890 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.891 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.895 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.896 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.897 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.901 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "k8sevents", 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_event_retention_days": { 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.902 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7", 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "config_check_secs": { 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_check_secs", 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.903 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "10", 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.904 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.907 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.909 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.910 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.913 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.914 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.916 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-09-16T00:30:21.919 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.920 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mds_autoscaler", 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.925 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.927 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.928 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.929 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.931 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.934 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.936 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.940 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.943 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.944 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.947 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.950 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.951 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.953 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.958 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.959 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.961 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-09-16T00:30:21.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.965 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.966 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.968 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.970 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.971 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-09-16T00:30:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-09-16T00:30:21.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.978 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.981 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.983 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.984 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.990 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:21.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.992 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.993 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.995 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.998 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:21.999 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.000 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.002 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.003 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.005 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.006 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.009 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.010 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.011 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rook", 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.019 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.020 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "storage_class": { 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "storage_class", 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "local", 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.024 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.025 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.026 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.027 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.032 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.033 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.034 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.035 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.036 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.038 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.040 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-09-16T00:30:22.041 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.044 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.045 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.048 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.049 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.051 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.052 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.053 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.055 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.056 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.058 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.060 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.062 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.063 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.064 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.065 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.066 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.067 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-09-16T00:30:22.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-09-16T00:30:22.069 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-09-16T00:30:22.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-09-16T00:30:22.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-09-16T00:30:22.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.073 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.076 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.077 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.078 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.079 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.080 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.081 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.082 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.083 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.084 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.085 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.086 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.087 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-09-16T00:30:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-16T00:30:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-16T00:30:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-16T00:30:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-16T00:30:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "services": { 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "dashboard": "https://172.21.15.78:8443/" 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "always_on_modules": { 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "octopus": [ 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-16T00:30:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "pacific": [ 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-16T00:30:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "quincy": [ 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-16T00:30:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "reef": [ 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-16T00:30:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "squid": [ 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-16T00:30:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "last_failure_osd_epoch": 3, 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "active_clients": [ 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1188369420 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "libcephsqlite", 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 956594699 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-16T00:30:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1075644778 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.78:0", 2024-09-16T00:30:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 118507306 2024-09-16T00:30:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-16T00:30:22.126 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:30:22.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:20 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2410933776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:30:22.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:20 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/24265055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:30:22.127 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:20 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2110882068' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:30:22.127 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:20 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2410933776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:30:22.128 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:20 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/24265055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:30:22.128 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:20 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2110882068' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4b80a32d0 msgr2=0x7fe4b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4b80a32d0 0x7fe4b80a5790 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc005fd0 tx=0x7fe4bc005850 comp rx=0 tx=0).stop 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 msgr2=0x7fe4cc19f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 0x7fe4cc19f420 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4031ee0 tx=0x7fe4c4031f10 comp rx=0 tx=0).stop 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 shutdown_connections 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fe4b80a32d0 0x7fe4b80a5790 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4cc1025e0 0x7fe4cc19f420 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4cc101c20 0x7fe4cc1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 --2- 172.21.15.78:0/2110882068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe4cc100a30 0x7fe4cc197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 >> 172.21.15.78:0/2110882068 conn(0x7fe4cc0fc190 msgr2=0x7fe4cc06a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 shutdown_connections 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:20.797+0000 7fe4d0867640 1 -- 172.21.15.78:0/2110882068 wait complete. 2024-09-16T00:30:22.187 DEBUG:tasks.check_counter:Getting stats from a 2024-09-16T00:30:22.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-16T00:30:22.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:20 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2410933776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:30:22.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:20 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/24265055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:30:22.188 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:20 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2110882068' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:30:22.203 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:21 smithi148 ceph-mon[35599]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T00:30:22.213 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:21 smithi078 ceph-mon[31157]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T00:30:22.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:21 smithi193 ceph-mon[35484]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-16T00:30:22.442 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:23.057 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:30:23.057 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:23.057 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1446, 2024-09-16T00:30:23.057 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1433, 2024-09-16T00:30:23.057 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1230224, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 547712, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.171702355, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.062335063, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.059445762, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.044447943, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1432, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.116214353, 2024-09-16T00:30:23.058 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000081155 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 856592, 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 194096 2024-09-16T00:30:23.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1037, 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1274, 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 255668, 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 821212, 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 6, 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 4, 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.114234262, 2024-09-16T00:30:23.060 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.040017135, 2024-09-16T00:30:23.061 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.024146669, 2024-09-16T00:30:23.061 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.039192060, 2024-09-16T00:30:23.061 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:23.061 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1272, 2024-09-16T00:30:23.061 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.066312866, 2024-09-16T00:30:23.061 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000052132 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 491696 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:23.062 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1299, 2024-09-16T00:30:23.063 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1299, 2024-09-16T00:30:23.063 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 2814144, 2024-09-16T00:30:23.063 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 412259, 2024-09-16T00:30:23.063 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-09-16T00:30:23.063 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-09-16T00:30:23.063 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.113210117, 2024-09-16T00:30:23.063 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.031681197, 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.035102163, 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.036439497, 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1298, 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.085430432, 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000065816 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.064 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:30:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:30:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-16T00:30:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-09-16T00:30:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-16T00:30:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.066 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-09-16T00:30:23.066 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-16T00:30:23.066 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-16T00:30:23.066 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 5590, 2024-09-16T00:30:23.066 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.202560263, 2024-09-16T00:30:23.066 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000036236 2024-09-16T00:30:23.067 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.067 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:23.067 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-16T00:30:23.067 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-16T00:30:23.067 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.067 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.067 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-09-16T00:30:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-09-16T00:30:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-09-16T00:30:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-09-16T00:30:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.068 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.069 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.069 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-09-16T00:30:23.069 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-09-16T00:30:23.069 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:23.069 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:23.069 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 325, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 325, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 325, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 10, 2024-09-16T00:30:23.070 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-09-16T00:30:23.071 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:23.072 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-09-16T00:30:23.073 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:23.074 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:30:23.075 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-09-16T00:30:23.076 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 2071, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 4, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 6669005, 2024-09-16T00:30:23.077 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 6668431, 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 2071 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 317, 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 322, 2024-09-16T00:30:23.078 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 5, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 18, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 18, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 0, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 325, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 325, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 0, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 50244, 2024-09-16T00:30:23.079 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215328 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.080 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.081 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.082 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-09-16T00:30:23.083 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.084 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.085 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.085 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-09-16T00:30:23.085 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.085 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.085 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.085 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-09-16T00:30:23.086 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.086 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.086 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.086 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-09-16T00:30:23.086 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.087 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.088 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.090 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.091 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.093 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-09-16T00:30:23.093 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.093 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.093 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.093 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.094 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.095 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.095 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:23.095 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:23.095 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:23.095 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.095 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.095 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 1, 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 2, 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 1, 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-09-16T00:30:23.096 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-09-16T00:30:23.097 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 2057, 2024-09-16T00:30:23.097 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:23.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.097 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-16T00:30:23.097 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:23.097 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:23.097 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:23.098 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:23.099 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:23.099 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:23.099 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:23.099 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:23.099 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:23.099 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:23.099 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:23.099 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:23.100 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:23.100 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:23.100 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:23.100 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:23.100 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:23.100 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:23.101 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:23.101 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:23.101 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:23.101 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:23.101 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-16T00:30:23.101 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:23.102 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:23.102 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:23.102 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:23.102 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 201640, 2024-09-16T00:30:23.102 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 362, 2024-09-16T00:30:23.102 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:23.102 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:23.103 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-16T00:30:23.103 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-16T00:30:23.103 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:23.103 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:23.103 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:23.103 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:23.103 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 163, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 990706, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 8929, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:23.104 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 3443, 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 3443, 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3443, 2024-09-16T00:30:23.105 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 8.018880374, 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.002329038 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3443, 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 3444 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 3443, 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 3443, 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-09-16T00:30:23.106 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 2145, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 1296, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-09-16T00:30:23.107 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-09-16T00:30:23.108 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-09-16T00:30:23.109 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 90, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-09-16T00:30:23.110 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 4, 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 5, 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 7, 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 2, 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-09-16T00:30:23.111 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:23.112 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:23.113 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 3778, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 3939846, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 3778, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 3778, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 3939846, 2024-09-16T00:30:23.114 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-16T00:30:23.115 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 3443, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 2474706, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 3443, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 2474706, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.116 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:23.117 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 3443, 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 3443, 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 3443, 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 3443, 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.118 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-16T00:30:23.119 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x55bb656aab40": { 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-16T00:30:23.120 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:23.121 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.122 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:23.122 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:30:24.009 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-16T00:30:24.010 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-16T00:30:24.010 DEBUG:tasks.check_counter:Getting stats from d 2024-09-16T00:30:24.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-16T00:30:24.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:23 smithi148 ceph-mon[35599]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T00:30:24.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:23 smithi078 ceph-mon[31157]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T00:30:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:24.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:23 smithi193 ceph-mon[35484]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-16T00:30:24.860 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:30:24.860 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:24.860 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 327, 2024-09-16T00:30:24.860 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 320, 2024-09-16T00:30:24.860 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 789072, 2024-09-16T00:30:24.860 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 196848, 2024-09-16T00:30:24.860 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.057214713, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.020642880, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.035476489, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 320, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.021464917, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000067077 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:24.861 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 789072, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 196848 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 243, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 320, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 200208, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-09-16T00:30:24.862 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.019311145, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.015130211, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001593999, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 241, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.019745629, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000081932 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.863 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 200208 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.000193513, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000115187, 2024-09-16T00:30:24.864 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.865 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:24.866 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:24.867 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148722, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 30, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:24.868 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-09-16T00:30:24.869 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 325, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 740390, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 325, 2024-09-16T00:30:24.870 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 325, 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 740390, 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:24.871 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:30:25.664 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-16T00:30:25.664 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-16T00:30:25.664 DEBUG:tasks.check_counter:Getting stats from g 2024-09-16T00:30:25.664 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-16T00:30:25.909 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:26.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:25 smithi148 ceph-mon[35599]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:30:26.212 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:25 smithi078 ceph-mon[31157]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:30:26.325 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:25 smithi193 ceph-mon[35484]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:30:26.422 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:30:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 1413, 2024-09-16T00:30:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 1400, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1219008, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 537862, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.182828908, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.070876216, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.064540500, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.041744063, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 1399, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.114039232, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000081514 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 857136, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 195376 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 2060, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2296, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 527132, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1145867, 2024-09-16T00:30:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 5, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.156312120, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.054404886, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.036078069, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.055976390, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 2295, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.120719436, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000052601 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 492320 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 4, 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 6, 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 1629, 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 1284, 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 8, 2024-09-16T00:30:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 3, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.002401584, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000056629, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.001925568, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000180215, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 4, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.001900105, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000475026 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 1, 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-MDSRank": { 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-16T00:30:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 5092, 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.131347890, 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000025794 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "queue_len": 0, 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "complete_latency": { 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": { 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "request": 0, 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "reply": 0, 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "reply_latency": { 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_reply": 0, 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "forward": 0, 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_commit": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_split": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_merge": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_top": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_bottom": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pin_tail": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_pinned": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_expired": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "inodes_with_caps": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "subtrees": 1, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse": 0, 2024-09-16T00:30:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_hit": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_forward": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_discover": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "traverse_lock": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "load_cent": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "q": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "exported": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "exported_inodes": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "imported": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "imported_inodes": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:30:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rfiles": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rbytes": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "root_rsnaps": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "scrub_file_inodes": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:30:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_cap_release": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "process_request_cap_release": 0 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_cache": { 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_try_discover": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_send_discover": 0, 2024-09-16T00:30:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "dir_handle_discover": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_created": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_enqueued": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_reintegrated": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "strays_migrated": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_started": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "recovery_completed": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:30:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_exportdir": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_flush": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "ireq_inodestats": 0 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_log": { 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "evlrg": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "evadd": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "evex": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "evtrm": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "ev": 4, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "evexg": 0, 2024-09-16T00:30:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "evexd": 0, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "segadd": 0, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "segex": 0, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "segtrm": 0, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "seg": 2, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "segmjr": 2, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "segexg": 0, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "segexd": 0, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "expos": 4194304, 2024-09-16T00:30:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "wrpos": 4195426, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "rdpos": 4195285, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "jlat": { 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "replayed": 4 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_mem": { 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "ino": 3, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "ino+": 3, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "ino-": 0, 2024-09-16T00:30:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "dir": 1, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "dir+": 1, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "dir-": 0, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "dn": 0, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "dn+": 0, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "dn-": 0, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "cap": 0, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "cap+": 0, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "cap-": 0, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "rss": 48884, 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "heap": 215328 2024-09-16T00:30:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_server": { 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_client_request": 0, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "dispatch_server_request": 0, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_request": 0, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_client_session": 0, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "handle_peer_request": 0, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "req_create_latency": { 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getattr_latency": { 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "req_link_latency": { 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookup_latency": { 2024-09-16T00:30:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupino_latency": { 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupname_latency": { 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "req_lssnap_latency": { 2024-09-16T00:30:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mkdir_latency": { 2024-09-16T00:30:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mknod_latency": { 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "req_mksnap_latency": { 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "req_open_latency": { 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "req_readdir_latency": { 2024-09-16T00:30:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rename_latency": { 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmdir_latency": { 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setattr_latency": { 2024-09-16T00:30:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setlayout_latency": { 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "req_setxattr_latency": { 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "req_symlink_latency": { 2024-09-16T00:30:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "req_unlink_latency": { 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_sessions": { 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "session_count": 0, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "session_add": 1, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "session_remove": 1, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_open": 1, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "sessions_stale": 0, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "total_load": 0, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "average_load": 0, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "avg_session_uptime": 1108, 2024-09-16T00:30:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 198183, 2024-09-16T00:30:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 54, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 163, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 8512, 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 13, 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "objecter": { 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "op_active": 0, 2024-09-16T00:30:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "op_laggy": 0, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send": 3136, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "op_send_bytes": 21, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "op_resend": 3, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "op_reply": 3136, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "op_latency": { 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3136, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 46.504306816, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.014829179 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "op_inflight": 0, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "oplen_avg": { 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 3136, 2024-09-16T00:30:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 3137 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "op": 3136, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "op_r": 3136, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "op_w": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "op_rmw": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "op_pg": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_stat": 2051, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_create": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_read": 1083, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_write": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writefull": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_writesame": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_append": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_zero": 0, 2024-09-16T00:30:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_truncate": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_delete": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_mapext": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_getxattr": 1, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_setxattr": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_call": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_watch": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_notify": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "osdop_other": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_active": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_send": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_resend": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "linger_ping": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_active": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_send": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "poolop_resend": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_active": 0, 2024-09-16T00:30:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_send": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "poolstat_resend": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_active": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_send": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "statfs_resend": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "command_active": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "command_send": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "command_resend": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "map_epoch": 90, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "map_full": 0, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "map_inc": 4, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_sessions": 6, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_open": 9, 2024-09-16T00:30:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_session_close": 3, 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_laggy": 0, 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_wr": 0, 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_rd": 2, 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_del": 0 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "oft": { 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_objs": 0, 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_updates": 0, 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "omap_total_removes": 0 2024-09-16T00:30:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "osd-slow-ops": { 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "slow_ops_count": 0 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "purge_queue": { 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing": 0, 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executing_high_water": 0, 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed_ops": 0, 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_executed": 0, 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 3473, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 1414925, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 3473, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 3473, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1414925, 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 3136, 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 1127, 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 3136, 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 1127, 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 1024, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 3136, 2024-09-16T00:30:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 3136, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 3136, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 3136, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-16T00:30:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-write_buf_throttle-0x55bad54ceb40": { 2024-09-16T00:30:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 3758096384, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:26.473 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:30:27.367 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-16T00:30:27.367 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-16T00:30:27.367 DEBUG:tasks.check_counter:Getting stats from j 2024-09-16T00:30:27.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-16T00:30:27.612 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:28.013 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:27 smithi078 ceph-mon[31157]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T00:30:28.130 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-16T00:30:28.130 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 325, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 320, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 786592, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 197680, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 0, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.050061345, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.022108594, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.033002695, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 319, 2024-09-16T00:30:28.131 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.023606930, 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000074002 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 786592, 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 197680 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 6, 2024-09-16T00:30:28.132 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 247, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 3344, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 201008, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 4, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 2, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.022942549, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.015231382, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.004746299, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 246, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.019238194, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000078204 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:28.133 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 201008 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_messages": 0, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages": 2, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_bytes": 0, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_created_connections": 2, 2024-09-16T00:30:28.134 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_active_connections": 1, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_total_time": 0.000344059, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_recv_time": 0.000268349, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:28.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "cct": { 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "total_workers": 0, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "unhealthy_workers": 0 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "mempool": { 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:28.136 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:28.137 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_items": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:28.138 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_bytes": 148704, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_anon_items": 29, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_bytes": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_items": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_bytes": 0, 2024-09-16T00:30:28.139 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_items": 0, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "pgmap_items": 0, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_bytes": 928, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "mds_co_items": 2, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_1_items": 0, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "unittest_2_items": 0 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:28.140 INFO:teuthology.orchestra.run.smithi078.stdout: "val": 0, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "max": 104857600, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "get_started": 0, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "get": 327, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "get_sum": 740650, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "get_or_fail_success": 327, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "take": 0, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "take_sum": 0, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "put": 327, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "put_sum": 740650, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "wait": { 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "avgcount": 0, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "sum": 0.000000000, 2024-09-16T00:30:28.141 INFO:teuthology.orchestra.run.smithi078.stdout: "avgtime": 0.000000000 2024-09-16T00:30:28.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:28.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-16T00:30:28.142 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-16T00:30:28.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:27 smithi148 ceph-mon[35599]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T00:30:28.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:27 smithi193 ceph-mon[35484]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-16T00:30:29.117 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-16T00:30:29.117 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-16T00:30:29.117 DEBUG:tasks.check_counter:Getting stats from b 2024-09-16T00:30:29.117 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-16T00:30:29.431 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:30:30.103 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:29 smithi148 ceph-mon[35599]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:30:30.122 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-16T00:30:30.122 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3042, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2578, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 845643, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 7291780, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 22, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.244027921, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.083448648, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.127625300, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.034417030, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2583, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.319061808, 2024-09-16T00:30:30.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000123523 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2858, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000601158, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000210 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 4266, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3991, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 2083219, 2024-09-16T00:30:30.124 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 2338195, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 10, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.326886747, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.144901141, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.203281488, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.016649573, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3993, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.321870371, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000080608 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5939, 2024-09-16T00:30:30.125 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001042916, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000175 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 902752, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 222800 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 6700, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 9526, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 2241968, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 7588443, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 215, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 11, 2024-09-16T00:30:30.126 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.596444446, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.189084369, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.406182287, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.081863967, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 9515, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.496358820, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000052165 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 14153, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002423655, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000171 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 84272, 2024-09-16T00:30:30.127 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 632272 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1359, 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.435205499, 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001056074 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:30.128 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 741, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.035743056, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000048236 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 2281, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 2160, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2160, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 13.560894792, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.006278192 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.129 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 121, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 39, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 3, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 38, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 4, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 11, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 23, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 32, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 964, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 12, 2024-09-16T00:30:30.130 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 12, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 3, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 3526, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 2724, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 120, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 74, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 31, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 977, 2024-09-16T00:30:30.131 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 11, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 4, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 263, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 1021, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 1589392717, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 263, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 143, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 18, 2024-09-16T00:30:30.132 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 125, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 120, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 2, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 182, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 1591, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 1341, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 1009, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 117, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 933 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-09-16T00:30:30.133 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 28, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 217, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 216, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 1, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-09-16T00:30:30.134 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 15, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 525, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 18, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 31, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 3, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 3095, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-09-16T00:30:30.135 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 3095, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 5, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 5, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 10533138, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-09-16T00:30:30.136 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 790, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 8.267558984, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.010465264 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 38, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 1221, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 1183, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 33, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 70, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 37, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 38, 2024-09-16T00:30:30.137 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 1466, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 1428, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 12, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 1235, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 1223, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 69032, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 231712 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 2878, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 1, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 2281, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 38, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 417, 2024-09-16T00:30:30.138 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1154, 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 9.312147087, 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.008069451 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 32, 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.082433016, 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002576031 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000902734, 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000902734 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 196, 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 3.026592773, 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.015441799 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:30.140 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000441441, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000220720 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:30.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.142 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 34, 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.282503232, 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.008308918 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.143 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 468, 2024-09-16T00:30:30.144 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.120090068, 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000256602 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 37, 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.075129793, 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002030534 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 215, 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.593545901, 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002760678 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.145 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.146 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001623439, 2024-09-16T00:30:30.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000811719 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.148 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 9, 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.022478218, 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002497579 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-09-16T00:30:30.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10, 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.043007090, 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004300709 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-09-16T00:30:30.150 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 4, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 3, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 73, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 73, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1023, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:30.151 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:30.152 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:30.153 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 287986, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 115, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:30.154 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 163, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 168377, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 1593, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-09-16T00:30:30.155 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 2514, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 6785592, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 33, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 2514, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2514, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 55.789685599, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.022191601 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-09-16T00:30:30.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2514, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2690 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 2514, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 383, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 2131, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 11, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 102, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 874, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 133, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-09-16T00:30:30.157 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 3, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 945, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 281, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 22, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-09-16T00:30:30.158 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-09-16T00:30:30.159 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 90, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 10, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-09-16T00:30:30.160 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 249, 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 70 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 504 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-09-16T00:30:30.161 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 10, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 5, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 432, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 216, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 14004, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 3877256, 2024-09-16T00:30:30.162 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 14004, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 14004, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 3877256, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:30.163 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 2514, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 6809136, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2514, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 6809136, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.164 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 2514, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 2514, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2514, 2024-09-16T00:30:30.165 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 2514, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 216, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 23544, 2024-09-16T00:30:30.166 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 216, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 84, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 23544, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x56411c196960": { 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-16T00:30:30.167 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 3095, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 6338834, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 3095, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 790, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 6338834, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.168 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:30.169 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-16T00:30:30.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:29 smithi078 ceph-mon[31157]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:30:30.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:29 smithi193 ceph-mon[35484]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-16T00:30:31.211 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.212 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.213 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:31.213 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:31.213 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:30 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:30:31.249 INFO:tasks.check_counter:Daemon mds.b mds.exported=31 2024-09-16T00:30:31.249 INFO:tasks.check_counter:Daemon mds.b mds.imported=11 2024-09-16T00:30:31.249 DEBUG:tasks.check_counter:Getting stats from e 2024-09-16T00:30:31.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-16T00:30:31.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:31.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:31.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:30 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:30:31.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:31.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:31.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:30 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' 2024-09-16T00:30:31.510 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1285, 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 1287, 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 4613985, 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 408640, 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 7, 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 2, 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.128259785, 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.048359404, 2024-09-16T00:30:32.132 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.032325941, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.035736069, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1285, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.088821607, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000069121 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:30:32.133 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 10, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 250, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 5000, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 584330, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 8, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.016581903, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.009744031, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.004518039, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000086944, 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:32.134 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 249, 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.019090312, 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000076667 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 582992 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:32.135 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 2652, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 2639, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 3587245, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 933317, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.199503034, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.055077629, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.076246462, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.065439751, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2638, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.119499513, 2024-09-16T00:30:32.136 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000045299 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 858656, 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 198736 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-09-16T00:30:32.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6022, 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.144657322, 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000024021 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.138 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:32.139 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 234, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 234, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 234, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 3, 2024-09-16T00:30:32.140 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:32.141 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 1021, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 1589392717, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:30:32.142 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-09-16T00:30:32.143 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-09-16T00:30:32.144 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-09-16T00:30:32.145 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 3095, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 5, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-09-16T00:30:32.146 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 10533138, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 10527925, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 3095 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 18, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 1209, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 1191, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 13, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 76, 2024-09-16T00:30:32.147 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 63, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 234, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 1223, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 989, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 53960, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215328 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-09-16T00:30:32.148 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.149 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.150 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.151 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.152 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.153 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-09-16T00:30:32.154 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-09-16T00:30:32.155 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.156 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.157 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.158 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.159 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 4, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 3, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 2, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 3543, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:32.160 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:32.161 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:32.162 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 199086, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 60, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:32.163 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 163, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 221987, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 1194, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-09-16T00:30:32.164 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 3599, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 26, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 4, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 3599, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3599, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 11.966721753, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003325012 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-09-16T00:30:32.165 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3599, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 3600 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 3599, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 3599, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 2238, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 1358, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2024-09-16T00:30:32.166 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 2, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-09-16T00:30:32.167 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-09-16T00:30:32.168 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 90, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 4, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 6, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 9, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 3, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-09-16T00:30:32.169 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-09-16T00:30:32.170 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 3943, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 7830737, 2024-09-16T00:30:32.171 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 3943, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 3943, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 7830737, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:32.172 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 3599, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 6338844, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 3599, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 6338844, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.173 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 3599, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 3599, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 3599, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 3599, 2024-09-16T00:30:32.174 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:32.175 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x555623ad2b40": { 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-16T00:30:32.176 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.177 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:32.178 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-16T00:30:32.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:31 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-16T00:30:32.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:31 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-16T00:30:32.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:31 smithi078 ceph-mon[31157]: Unable to set osd_memory_target on smithi078 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-16T00:30:32.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:31 smithi078 ceph-mon[31157]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T00:30:32.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:31 smithi078 ceph-mon[31157]: Adjusting osd_memory_target on smithi148 to 176.0M 2024-09-16T00:30:32.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:31 smithi078 ceph-mon[31157]: Unable to set osd_memory_target on smithi148 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-16T00:30:32.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:31 smithi078 ceph-mon[31157]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T00:30:32.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:31 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-16T00:30:32.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:31 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-16T00:30:32.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:31 smithi148 ceph-mon[35599]: Unable to set osd_memory_target on smithi078 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-16T00:30:32.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:31 smithi148 ceph-mon[35599]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T00:30:32.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:31 smithi148 ceph-mon[35599]: Adjusting osd_memory_target on smithi148 to 176.0M 2024-09-16T00:30:32.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:31 smithi148 ceph-mon[35599]: Unable to set osd_memory_target on smithi148 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-16T00:30:32.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:31 smithi148 ceph-mon[35599]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T00:30:32.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:31 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi078 to 176.0M 2024-09-16T00:30:32.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:31 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-16T00:30:32.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:31 smithi193 ceph-mon[35484]: Unable to set osd_memory_target on smithi078 to 184556646: error parsing value: Value '184556646' is below minimum 939524096 2024-09-16T00:30:32.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:31 smithi193 ceph-mon[35484]: Unable to set osd_memory_target on smithi193 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-16T00:30:32.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:31 smithi193 ceph-mon[35484]: Adjusting osd_memory_target on smithi148 to 176.0M 2024-09-16T00:30:32.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:31 smithi193 ceph-mon[35484]: Unable to set osd_memory_target on smithi148 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-16T00:30:32.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:31 smithi193 ceph-mon[35484]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-16T00:30:32.933 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-16T00:30:32.934 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-16T00:30:32.934 DEBUG:tasks.check_counter:Getting stats from h 2024-09-16T00:30:32.934 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-16T00:30:33.191 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:30:33.745 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 8607, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 8607, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 2173723, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 7465843, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 48, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.481930809, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.158240819, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.126337801, 2024-09-16T00:30:33.748 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.160371310, 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8606, 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.221327837, 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000025717 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 68, 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000015039, 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000221 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 7984, 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 6416 2024-09-16T00:30:33.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 10045, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 12758, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 2814417, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 9607596, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 15, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.562187284, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.205908577, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.251582394, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.110421307, 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:33.750 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 12756, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.383000322, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000030025 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 9696, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001662904, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000171 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 586816 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 4995, 2024-09-16T00:30:33.751 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 5566, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 3136038, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 2266155, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 7, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.409147614, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.187091690, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.215503831, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.038609730, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5565, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.399109550, 2024-09-16T00:30:33.752 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000071717 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7070, 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001279132, 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000180 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 888736, 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 218352 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.753 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 12911, 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 3.702224348, 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000286749 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 134, 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.004989276, 2024-09-16T00:30:33.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000037233 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 1976, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 1867, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1867, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 3.851068477, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002062704 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 109, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:33.755 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 11, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 1054, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 1025, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 29, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 1054, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 306, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 741, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 741, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 13, 2024-09-16T00:30:33.756 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 4190, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 3262, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 107, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 22, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 295, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 19, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 305, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 31, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 977, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:33.757 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 12445, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 1024, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 1606409783, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 12445, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 476, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 476, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 11969, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 26, 2024-09-16T00:30:33.758 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 1574, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 1428, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 2456, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 602, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 66, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 524 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-09-16T00:30:33.759 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 22, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 2, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 182, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 24, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 158, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-09-16T00:30:33.760 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 15, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 11054, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 19, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 1, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 2875, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-09-16T00:30:33.761 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 2875, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 5, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 5, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 11639727, 2024-09-16T00:30:33.762 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2260, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 12.594396196, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005572741 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 758, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 1082, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 324, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 66, 2024-09-16T00:30:33.763 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 69, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 3, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 1054, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 1392, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 338, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 741, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 2086, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 1345, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 64992, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 223520 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 2661, 2024-09-16T00:30:33.764 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 431, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 1976, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 355, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 863, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 116, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.108530408, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000935606 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 23, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.040324039, 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001753219 2024-09-16T00:30:33.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 158, 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.306726511, 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001941307 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 965, 2024-09-16T00:30:33.766 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.828334032, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001894646 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.767 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.768 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3, 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002138011, 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000712670 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.769 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 216, 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.138802449, 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000642603 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-09-16T00:30:33.770 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 39, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.037629009, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000964846 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 169, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.142680603, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.006761423 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.771 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.008893234, 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004446617 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.773 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-09-16T00:30:33.774 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 176, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.237010181, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001346648 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.775 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 3, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 2, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-09-16T00:30:33.776 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 548, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 548, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1025, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:33.777 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:33.778 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:33.779 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 323552, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 845, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 163, 2024-09-16T00:30:33.780 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 2973020, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 45934, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-09-16T00:30:33.781 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 15093, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 7579308, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 32, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 15093, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 15093, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 33.483218478, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002218460 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-09-16T00:30:33.782 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 15093, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 15325 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 15093, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 12459, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 2634, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 10, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 14, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 2274, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 79, 2024-09-16T00:30:33.783 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 76, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 12445, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 20, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-09-16T00:30:33.784 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-09-16T00:30:33.785 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 90, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 4, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2024-09-16T00:30:33.786 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 329, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 78 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-09-16T00:30:33.787 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 10323 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 11, 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 5, 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 63, 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 24, 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:33.788 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 23643, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 5971689, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 23643, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 23643, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 5971689, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.789 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 15093, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 7581908, 2024-09-16T00:30:33.790 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 15093, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 7581908, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:33.791 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 15093, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 15093, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 15093, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 15093, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:33.792 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 24, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 2600, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 24, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 14, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 2600, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.793 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x5564fa7d8960": { 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 78288, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 2875, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 7445423, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 2875, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 2260, 2024-09-16T00:30:33.794 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 7367135, 2024-09-16T00:30:33.795 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:33.795 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:33.795 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:33.795 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:33.795 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.795 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:33.795 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-16T00:30:34.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:33 smithi148 ceph-mon[35599]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T00:30:34.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:33 smithi078 ceph-mon[31157]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T00:30:34.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:33 smithi193 ceph-mon[35484]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-16T00:30:34.575 INFO:tasks.check_counter:Daemon mds.h mds.exported=19 2024-09-16T00:30:34.575 INFO:tasks.check_counter:Daemon mds.h mds.imported=31 2024-09-16T00:30:34.575 DEBUG:tasks.check_counter:Getting stats from k 2024-09-16T00:30:34.576 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-16T00:30:34.835 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:30:35.387 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3278, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 3278, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 929091, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 1041587, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 6, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 2, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.237746437, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.070164668, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.061761212, 2024-09-16T00:30:35.388 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.089316603, 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3277, 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.140289577, 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000042810 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-16T00:30:35.389 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 346, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 333, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 859868, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 201194, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 2, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.045785926, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.020102729, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.034406043, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000047424, 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:35.390 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 332, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.018500790, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000055725 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 859312, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 200496 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 3, 2024-09-16T00:30:35.391 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 248, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 588, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 586622, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 6, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.014405952, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.010195227, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.001837129, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000029149, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 246, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.015995795, 2024-09-16T00:30:35.392 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000065023 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 586304 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-09-16T00:30:35.393 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5435, 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.126512704, 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000023277 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.394 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:35.395 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 1, 2024-09-16T00:30:35.396 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:35.397 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:30:35.398 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-09-16T00:30:35.399 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:30:35.400 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 3, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-09-16T00:30:35.401 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4195200, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 4194892, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.402 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 3 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 3, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 3, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 0, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 1, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 1, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 0, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 0, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 0, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 0, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-09-16T00:30:35.403 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 50768, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215328 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.404 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.405 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.406 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.407 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-09-16T00:30:35.408 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.409 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.410 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.411 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.412 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.413 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.414 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:35.415 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 1, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 0, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-09-16T00:30:35.416 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 2288, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:35.417 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:35.418 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 182087, 2024-09-16T00:30:35.419 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 48, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 163, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:35.420 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 8512, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 13, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 3277, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 21, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 3, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 3277, 2024-09-16T00:30:35.421 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3277, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 13.326691207, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004066735 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3277, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 3278 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 3277, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 3277, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2024-09-16T00:30:35.422 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 2146, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 1129, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:35.423 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 1, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-09-16T00:30:35.424 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-09-16T00:30:35.425 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 90, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 4, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 4, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 6, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 2, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2024-09-16T00:30:35.426 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:35.427 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 3623, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 1442744, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 3623, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 3623, 2024-09-16T00:30:35.428 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1442744, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:35.429 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 3277, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 901, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 3277, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 901, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:35.430 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 3277, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 3277, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 3277, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 3277, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.431 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.432 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x557a21f68b40": { 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-16T00:30:35.433 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-09-16T00:30:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-09-16T00:30:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-16T00:30:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-16T00:30:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-16T00:30:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-16T00:30:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.434 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-16T00:30:35.434 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-16T00:30:36.145 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-16T00:30:36.145 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-16T00:30:36.145 DEBUG:tasks.check_counter:Getting stats from c 2024-09-16T00:30:36.145 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-16T00:30:36.216 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:35 smithi193 ceph-mon[35484]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-16T00:30:36.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:35 smithi078 ceph-mon[31157]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-16T00:30:36.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:35 smithi148 ceph-mon[35599]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-16T00:30:36.466 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:30:37.147 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3976, 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 3642, 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1458014, 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 4293238, 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 48, 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.285095665, 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.096445690, 2024-09-16T00:30:37.148 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.102156333, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.076135904, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3640, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.155467287, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000042710 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1221, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000184956, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000151 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-16T00:30:37.149 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 736 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 6428, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 8245, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1821861, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 4409774, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 11, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.344990197, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.137253292, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.199423767, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.037394687, 2024-09-16T00:30:37.150 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8243, 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.427427588, 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000051853 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8651, 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002161441, 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000249 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 587280 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.151 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3021, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2958, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1556957, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1131541, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.240551385, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.089507447, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.116411967, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.040074557, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2958, 2024-09-16T00:30:37.152 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.134648545, 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000045520 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2068, 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000439969, 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000212 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 886176, 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 218288 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-16T00:30:37.153 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5897, 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.426534144, 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000241908 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-16T00:30:37.154 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 20, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000620140, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000031007 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 2600, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 2589, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2589, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 78.515817290, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.030326696 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-16T00:30:37.155 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 11, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 11, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 329, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 329, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 329, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-16T00:30:37.156 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 306, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 306, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 11, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 3255, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 3131, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 11, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 23, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 8, 2024-09-16T00:30:37.157 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 301, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 5219, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1024, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1606409783, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 5219, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:30:37.158 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 5219, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 1092, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 1662, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 1, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 1998, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 529, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 25, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 30 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-16T00:30:37.159 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 5, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 5, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 5, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:37.160 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 15, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 4739, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-16T00:30:37.161 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 2101, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 2101, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 4, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 4, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-16T00:30:37.162 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 6702648, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1027, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 7.993250537, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.007783106 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 326, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 331, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 5, 2024-09-16T00:30:37.163 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 27, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 28, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 1, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 329, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 359, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 30, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 306, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 857, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 551, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 60100, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223520 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 2747, 2024-09-16T00:30:37.164 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 2600, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 20, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 16, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 11, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.019030648, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001730058 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1752, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 77.900052333, 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.044463500 2024-09-16T00:30:37.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 473, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.334854737, 2024-09-16T00:30:37.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000707938 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.167 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-16T00:30:37.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.169 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.169 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.169 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.169 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.170 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 339, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.218918602, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000645777 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.008924511, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001115563 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-16T00:30:37.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.173 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.027274615, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.027274615 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-16T00:30:37.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.006761844, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001352368 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 263, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 263, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2302, 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:37.177 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:37.178 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:37.179 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 299344, 2024-09-16T00:30:37.180 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 407, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 163, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:37.181 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 1180914, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 13900, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 6454, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 2610927, 2024-09-16T00:30:37.182 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 6454, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6454, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 23.655634966, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003665267 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6454, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6583 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 6454, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 5220, 2024-09-16T00:30:37.183 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 1234, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 1028, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 60, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 22, 2024-09-16T00:30:37.184 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 5219, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:37.185 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 1, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-16T00:30:37.186 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 90, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 4, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 186, 2024-09-16T00:30:37.187 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 36 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 4439 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-16T00:30:37.188 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 10, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 5, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 10, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 5, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 13421, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 3605125, 2024-09-16T00:30:37.189 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 13421, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 13421, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3605125, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:37.190 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 6454, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2611472, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6454, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2611472, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.191 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 6454, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 6454, 2024-09-16T00:30:37.192 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6454, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6454, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 5, 2024-09-16T00:30:37.193 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 545, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 5, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 545, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55a1c3df0960": { 2024-09-16T00:30:37.194 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 2101, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2508344, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 2101, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1027, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2508344, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:37.195 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:37.196 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.196 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:37.196 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-16T00:30:38.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:37 smithi078 ceph-mon[31157]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-16T00:30:38.323 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:37 smithi193 ceph-mon[35484]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-16T00:30:38.361 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-16T00:30:38.361 INFO:tasks.check_counter:Daemon mds.c mds.imported=8 2024-09-16T00:30:38.361 DEBUG:tasks.check_counter:Getting stats from f 2024-09-16T00:30:38.362 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-16T00:30:38.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:37 smithi148 ceph-mon[35599]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-16T00:30:38.620 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:30:39.181 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-16T00:30:39.181 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:39.181 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 495, 2024-09-16T00:30:39.181 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 793, 2024-09-16T00:30:39.181 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 131481, 2024-09-16T00:30:39.181 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 168728, 2024-09-16T00:30:39.181 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2024-09-16T00:30:39.181 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.074916881, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.025821891, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.036193993, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.006493975, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 791, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.073112088, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000092429 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1231, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000335413, 2024-09-16T00:30:39.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000272 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 426, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 411, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 880426, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 678861, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 48, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.065036810, 2024-09-16T00:30:39.183 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.022657990, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.049012504, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000175100, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 410, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.027430677, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000066904 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 45, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000005822, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000129 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.184 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 872432, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 205392 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 25, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 266, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 8000, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 642828, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 13, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.021364699, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.012108903, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.006511003, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000293684, 2024-09-16T00:30:39.185 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 265, 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.031953751, 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000120580 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4, 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000399, 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000099 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 586400 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-16T00:30:39.186 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 47, 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001099066, 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000023384 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-16T00:30:39.187 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000005571, 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000002785 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.188 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 11, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 10, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 10, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 10, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-09-16T00:30:39.189 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 30, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 30, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-16T00:30:39.190 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1024, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1606409783, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:30:39.191 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-16T00:30:39.192 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 1, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:39.193 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 15, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 15, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-16T00:30:39.194 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 4, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 4, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 2, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-16T00:30:39.195 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4195426, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.005177555, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001725851 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 13, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 13, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 0, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 11, 2024-09-16T00:30:39.196 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 12, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 1, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 10, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 10, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 0, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 52688, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223520 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-09-16T00:30:39.197 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 4, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.198 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.199 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.200 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.201 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.202 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.202 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.202 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-16T00:30:39.202 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.202 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.202 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.202 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.203 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.204 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:39.205 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.206 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.207 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.208 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 0, 2024-09-16T00:30:39.209 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 1, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 0, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 64, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:39.210 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-16T00:30:39.212 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 238967, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 79, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:39.213 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 163, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 50872, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 198, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-16T00:30:39.214 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 39, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 11424, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 39, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 39, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.195996912, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005025561 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 39, 2024-09-16T00:30:39.215 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 61 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 39, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 0, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 39, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 0, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 3, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 7, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-16T00:30:39.216 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 6, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:39.217 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-16T00:30:39.218 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 90, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 4, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 9, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 14, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 5, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 14, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 1 2024-09-16T00:30:39.219 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 28 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-16T00:30:39.220 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 942, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 912804, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 942, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:39.221 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 942, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 912804, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:39.222 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 39, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 11424, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 39, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 11424, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:39.223 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 39, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 39, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 39, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 39, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.224 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-16T00:30:39.225 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5561e8168960": { 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 4, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 1122, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:39.226 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 4, 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3, 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1122, 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.227 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:39.228 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-16T00:30:40.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:39 smithi193 ceph-mon[35484]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:30:40.214 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-16T00:30:40.214 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-16T00:30:40.214 DEBUG:tasks.check_counter:Getting stats from i 2024-09-16T00:30:40.214 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-16T00:30:40.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:39 smithi078 ceph-mon[31157]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:30:40.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:39 smithi148 ceph-mon[35599]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 10 op/s 2024-09-16T00:30:40.461 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:30:41.011 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-16T00:30:41.011 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:41.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 42, 2024-09-16T00:30:41.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 47, 2024-09-16T00:30:41.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 9064, 2024-09-16T00:30:41.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 63523, 2024-09-16T00:30:41.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 13, 2024-09-16T00:30:41.011 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.018252460, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.003443034, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.009663348, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000619938, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 45, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.023311333, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000518029 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 41, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000008155, 2024-09-16T00:30:41.012 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000198 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 40, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 282, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 7159, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 935535, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 38, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.025027223, 2024-09-16T00:30:41.013 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.010515301, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.011517498, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000195242, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 280, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.032239472, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000115140 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 28, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000003221, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000115 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:30:41.014 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 589008 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1146, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1291, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1093633, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 673032, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 16, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.139074631, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.053455044, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.086679810, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.007183001, 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:41.015 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1291, 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.078502642, 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000060807 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1645, 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000374706, 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000227 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 888976, 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 203184 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-16T00:30:41.016 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 37, 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002015551, 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000054474 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-16T00:30:41.017 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000014280, 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000007140 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 2, 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 1, 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001114345, 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001114345 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.018 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 1, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 11, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 14, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 14, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 14, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 1, 2024-09-16T00:30:41.019 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 1, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 2, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 33, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 31, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 1, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-09-16T00:30:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1024, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1606409783, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:30:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 3, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 3, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 3, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 1 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-16T00:30:41.022 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 2, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:41.023 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 15, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 75, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-16T00:30:41.024 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 3, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 3, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 2, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-16T00:30:41.025 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 4195200, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.003010441, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001505220 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 17, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 17, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 0, 2024-09-16T00:30:41.026 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 15, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 16, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 1, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 14, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 14, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 0, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 1, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 1, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 50528, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223520 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 3, 2024-09-16T00:30:41.027 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 2, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 14, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001114345, 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001114345 2024-09-16T00:30:41.028 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.030 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-16T00:30:41.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.032 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.032 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.032 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.032 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-16T00:30:41.032 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.032 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.032 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.033 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.034 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.035 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-16T00:30:41.036 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.037 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.038 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:41.039 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 1, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 1, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 1, 2024-09-16T00:30:41.040 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 1033, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:41.041 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:41.042 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 254727, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 80, 2024-09-16T00:30:41.043 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 163, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:41.044 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 68037, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 288, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 40, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 11574, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 29, 2024-09-16T00:30:41.045 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 40, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 40, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.212996661, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005324916 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 40, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 65 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 40, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 0, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 40, 2024-09-16T00:30:41.046 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 2, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 6, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 6, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-16T00:30:41.047 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-16T00:30:41.048 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-16T00:30:41.049 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 90, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 4, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 9, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 3, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 19, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 2 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-16T00:30:41.050 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 84 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-09-16T00:30:41.051 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1224, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 977559, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1224, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:41.052 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1224, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 977559, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:41.053 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 40, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 11574, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 40, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 11574, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:41.054 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 40, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 40, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 40, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 40, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.055 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:41.056 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5612b349c960": { 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 3, 2024-09-16T00:30:41.057 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 896, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 3, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 896, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:41.058 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-16T00:30:41.840 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-16T00:30:41.840 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-16T00:30:41.840 DEBUG:tasks.check_counter:Getting stats from l 2024-09-16T00:30:41.840 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-16T00:30:42.087 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:30:42.179 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:41 smithi193 ceph-mon[35484]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-16T00:30:42.256 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:41 smithi148 ceph-mon[35599]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-16T00:30:42.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:41 smithi078 ceph-mon[31157]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-16T00:30:42.625 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 597, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 599, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 4349453, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 189910, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.053076323, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.014359968, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.017806198, 2024-09-16T00:30:42.626 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.015502226, 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 597, 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.020937094, 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000035070 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:30:42.627 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1251, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1498, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3573542, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 985459, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.125678375, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.039619555, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.033755980, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.040334093, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1496, 2024-09-16T00:30:42.628 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.076210906, 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000050943 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 588272 2024-09-16T00:30:42.629 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 2128, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2113, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1401634, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 767950, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.228287394, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.070271846, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.083514479, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.067365795, 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:30:42.630 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2113, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.129913799, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000061483 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 863152, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 202864 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-16T00:30:42.631 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6102, 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.175132980, 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000028700 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.632 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-09-16T00:30:42.633 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 1072, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 1072, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 1072, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 11, 2024-09-16T00:30:42.634 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-16T00:30:42.635 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:30:42.636 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-16T00:30:42.637 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-16T00:30:42.638 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-16T00:30:42.639 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 2877, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 5, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 11644138, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 11642677, 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-16T00:30:42.640 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 2877 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 752, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 1077, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 325, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 60, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 62, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 2, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 1072, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 1379, 2024-09-16T00:30:42.641 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 307, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 53632, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 215328 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-16T00:30:42.642 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.643 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.644 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.645 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.646 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-16T00:30:42.647 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.648 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-16T00:30:42.649 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.649 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.649 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.649 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.649 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-16T00:30:42.649 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.649 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.649 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.650 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.651 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.652 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:30:42.653 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-09-16T00:30:42.654 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 3585, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:30:42.655 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:30:42.656 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 227216, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 800, 2024-09-16T00:30:42.657 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 8672, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 163, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-16T00:30:42.658 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 2592636, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 32725, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 3623, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 3623, 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-16T00:30:42.659 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3623, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 13.697788147, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003780786 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3623, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3624 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 3623, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 3623, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-16T00:30:42.660 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 2173, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1448, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-09-16T00:30:42.661 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-16T00:30:42.662 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 90, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-16T00:30:42.663 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 4, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 6, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 7, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 1, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.664 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:42.665 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 3972, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 8946260, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 3972, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3972, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 8946260, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.666 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3623, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 7449839, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3623, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 7449839, 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:42.667 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3623, 2024-09-16T00:30:42.668 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3623, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3623, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3623, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:42.669 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55cb54076b40": { 2024-09-16T00:30:42.670 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-16T00:30:42.671 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-16T00:30:42.672 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.672 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-16T00:30:42.672 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-16T00:30:43.485 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-16T00:30:43.485 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-16T00:30:43.485 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T00:30:43.496 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T00:30:43.496 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:30:43.496 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:30:43.529 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-16T00:30:43.529 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:30:43.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:30:43.891 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:44.168 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:43 smithi078 ceph-mon[31157]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:30:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.388+0000 7ff716770640 1 -- 172.21.15.78:0/3472775425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 msgr2=0x7ff710106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.388+0000 7ff716770640 1 --2- 172.21.15.78:0/3472775425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 0x7ff710106800 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff704009900 tx=0x7ff70402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 -- 172.21.15.78:0/3472775425 shutdown_connections 2024-09-16T00:30:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 --2- 172.21.15.78:0/3472775425 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff710106f30 0x7ff71010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 --2- 172.21.15.78:0/3472775425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 0x7ff710106800 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 --2- 172.21.15.78:0/3472775425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710100310 0x7ff7101006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 -- 172.21.15.78:0/3472775425 >> 172.21.15.78:0/3472775425 conn(0x7ff710077210 msgr2=0x7ff7100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 -- 172.21.15.78:0/3472775425 shutdown_connections 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 -- 172.21.15.78:0/3472775425 wait complete. 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 Processor -- start 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.389+0000 7ff716770640 1 -- start start 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff716770640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710100310 0x7ff710198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff716770640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 0x7ff7101989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff716770640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff710106f30 0x7ff71019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff716770640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff710111fb0 con 0x7ff710100cc0 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff716770640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff710111e30 con 0x7ff710106f30 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff716770640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff710112130 con 0x7ff710100310 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 0x7ff7101989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 0x7ff7101989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43166/0 (socket says 172.21.15.78:43166) 2024-09-16T00:30:44.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 -- 172.21.15.78:0/2973162162 learned_addr learned my addr 172.21.15.78:0/2973162162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70ffff640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710100310 0x7ff710198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff714d0d640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff710106f30 0x7ff71019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 -- 172.21.15.78:0/2973162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710100310 msgr2=0x7ff710198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710100310 0x7ff710198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 -- 172.21.15.78:0/2973162162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff710106f30 msgr2=0x7ff71019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff710106f30 0x7ff71019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 -- 172.21.15.78:0/2973162162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71019d420 con 0x7ff710100cc0 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70ffff640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710100310 0x7ff710198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70f7fe640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 0x7ff7101989b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff704005ab0 tx=0x7ff7040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.390+0000 7ff70d7fa640 1 -- 172.21.15.78:0/2973162162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70403f030 con 0x7ff710100cc0 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.391+0000 7ff70d7fa640 1 -- 172.21.15.78:0/2973162162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff704003ce0 con 0x7ff710100cc0 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.391+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71019d6b0 con 0x7ff710100cc0 2024-09-16T00:30:44.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.391+0000 7ff70d7fa640 1 -- 172.21.15.78:0/2973162162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704035070 con 0x7ff710100cc0 2024-09-16T00:30:44.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.391+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7101a4ff0 con 0x7ff710100cc0 2024-09-16T00:30:44.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.392+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7101006f0 con 0x7ff710100cc0 2024-09-16T00:30:44.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.393+0000 7ff70d7fa640 1 -- 172.21.15.78:0/2973162162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff704042030 con 0x7ff710100cc0 2024-09-16T00:30:44.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.393+0000 7ff70d7fa640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff6f80a32c0 0x7ff6f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:44.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.394+0000 7ff70ffff640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff6f80a32c0 0x7ff6f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:44.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.394+0000 7ff70d7fa640 1 -- 172.21.15.78:0/2973162162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff7040f05b0 con 0x7ff710100cc0 2024-09-16T00:30:44.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.394+0000 7ff70ffff640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff6f80a32c0 0x7ff6f80a5780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff700009930 tx=0x7ff700008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:44.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.398+0000 7ff70d7fa640 1 -- 172.21.15.78:0/2973162162 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7040bcf70 con 0x7ff710100cc0 2024-09-16T00:30:44.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:43 smithi193 ceph-mon[35484]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:30:44.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:43 smithi148 ceph-mon[35599]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-16T00:30:44.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.585+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff71002d7c0 con 0x7ff710100cc0 2024-09-16T00:30:44.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.585+0000 7ff70d7fa640 1 -- 172.21.15.78:0/2973162162 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7ff7040bc870 con 0x7ff710100cc0 2024-09-16T00:30:44.586 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1052494441 2024-09-17T00:04:02.295421+0000 2024-09-16T00:30:44.586 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3861325161 2024-09-17T00:04:02.295421+0000 2024-09-16T00:30:44.586 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1335742179 2024-09-17T00:03:37.955119+0000 2024-09-16T00:30:44.586 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/3957981561 2024-09-17T00:03:37.955119+0000 2024-09-16T00:30:44.586 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/2949703267 2024-09-17T00:03:37.955119+0000 2024-09-16T00:30:44.587 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1052494441 2024-09-17T00:04:02.295421+0000 2024-09-16T00:30:44.587 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6801/1515675480 2024-09-17T00:03:37.955119+0000 2024-09-16T00:30:44.587 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/486633606 2024-09-17T00:04:02.295421+0000 2024-09-16T00:30:44.587 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:0/1597437728 2024-09-17T00:04:02.295421+0000 2024-09-16T00:30:44.587 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6800/1515675480 2024-09-17T00:03:37.955119+0000 2024-09-16T00:30:44.587 INFO:teuthology.orchestra.run.smithi078.stderr:listed 10 entries 2024-09-16T00:30:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.587+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff6f80a32c0 msgr2=0x7ff6f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.587+0000 7ff716770640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff6f80a32c0 0x7ff6f80a5780 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff700009930 tx=0x7ff700008040 comp rx=0 tx=0).stop 2024-09-16T00:30:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.587+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 msgr2=0x7ff7101989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.587+0000 7ff716770640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 0x7ff7101989b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7ff704005ab0 tx=0x7ff7040042f0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.588+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 shutdown_connections 2024-09-16T00:30:44.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.588+0000 7ff716770640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff6f80a32c0 0x7ff6f80a5780 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.588+0000 7ff716770640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff710106f30 0x7ff71019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.588+0000 7ff716770640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff710100cc0 0x7ff7101989b0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.588+0000 7ff716770640 1 --2- 172.21.15.78:0/2973162162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710100310 0x7ff710198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.588+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 >> 172.21.15.78:0/2973162162 conn(0x7ff710077210 msgr2=0x7ff7101050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.588+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 shutdown_connections 2024-09-16T00:30:44.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:44.588+0000 7ff716770640 1 -- 172.21.15.78:0/2973162162 wait complete. 2024-09-16T00:30:45.019 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:44 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2973162162' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:45.376 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T00:30:45.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:45.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /tmp/cephfs 2024-09-16T00:30:45.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:44 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2973162162' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:45.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:44 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2973162162' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:45.523 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-16T00:30:45.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:45.524 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-16T00:30:45.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:45.559 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-16T00:30:45.629 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-16T00:30:45.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:45.629 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-16T00:30:45.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:45.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-09-16T00:30:45.805 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T00:30:45.816 INFO:tasks.cephadm:Teardown begin 2024-09-16T00:30:45.816 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T00:30:45.817 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:46.072 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:46.232 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:45 smithi078 ceph-mon[31157]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:46.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:45 smithi193 ceph-mon[35484]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:46.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:45 smithi148 ceph-mon[35599]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-16T00:30:46.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 -- 172.21.15.78:0/3995063445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330103660 msgr2=0x7fd33010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 --2- 172.21.15.78:0/3995063445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330103660 0x7fd33010bb50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd324009900 tx=0x7fd32402a480 comp rx=0 tx=0).stop 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 -- 172.21.15.78:0/3995063445 shutdown_connections 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 --2- 172.21.15.78:0/3995063445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd33010c170 0x7fd33010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 --2- 172.21.15.78:0/3995063445 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330103660 0x7fd33010bb50 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 --2- 172.21.15.78:0/3995063445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd330102d40 0x7fd330103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 -- 172.21.15.78:0/3995063445 >> 172.21.15.78:0/3995063445 conn(0x7fd3300fc7a0 msgr2=0x7fd3300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 -- 172.21.15.78:0/3995063445 shutdown_connections 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.613+0000 7fd335629640 1 -- 172.21.15.78:0/3995063445 wait complete. 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.614+0000 7fd335629640 1 Processor -- start 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.614+0000 7fd335629640 1 -- start start 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.614+0000 7fd335629640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd330102d40 0x7fd330198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd335629640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330103660 0x7fd330198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd335629640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd33010c170 0x7fd33019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd335629640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd330111fb0 con 0x7fd33010c170 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd335629640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd330111e30 con 0x7fd330103660 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd335629640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd330112130 con 0x7fd330102d40 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd33010c170 0x7fd33019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd33010c170 0x7fd33019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53492/0 (socket says 172.21.15.78:53492) 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 -- 172.21.15.78:0/1034143387 learned_addr learned my addr 172.21.15.78:0/1034143387 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32effd640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd330102d40 0x7fd330198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 -- 172.21.15.78:0/1034143387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd330102d40 msgr2=0x7fd330198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd330102d40 0x7fd330198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 -- 172.21.15.78:0/1034143387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330103660 msgr2=0x7fd330198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32e7fc640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330103660 0x7fd330198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330103660 0x7fd330198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 -- 172.21.15.78:0/1034143387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33019d520 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32effd640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd330102d40 0x7fd330198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32e7fc640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330103660 0x7fd330198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd32f7fe640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd33010c170 0x7fd33019ce40 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fd32800cad0 tx=0x7fd32800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd317fff640 1 -- 172.21.15.78:0/1034143387 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32800ae20 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd317fff640 1 -- 172.21.15.78:0/1034143387 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd328005020 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33019d810 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd317fff640 1 -- 172.21.15.78:0/1034143387 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32800f6f0 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.615+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3301a50f0 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.616+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd330103120 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.617+0000 7fd317fff640 1 -- 172.21.15.78:0/1034143387 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd3280051d0 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.618+0000 7fd317fff640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3180a3240 0x7fd3180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.618+0000 7fd317fff640 1 -- 172.21.15.78:0/1034143387 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7fd3280cfd60 con 0x7fd33010c170 2024-09-16T00:30:46.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.618+0000 7fd32effd640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3180a3240 0x7fd3180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.619+0000 7fd32effd640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3180a3240 0x7fd3180a5700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd320009930 tx=0x7fd320008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:46.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.622+0000 7fd317fff640 1 -- 172.21.15.78:0/1034143387 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd328014030 con 0x7fd33010c170 2024-09-16T00:30:46.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.823+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3301071e0 con 0x7fd33010c170 2024-09-16T00:30:46.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.824+0000 7fd317fff640 1 -- 172.21.15.78:0/1034143387 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11987 (secure 0 0 0) 0x7fd32809c720 con 0x7fd33010c170 2024-09-16T00:30:46.825 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:30:46.825 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":35,"btime":"2024-09-16T00:29:33:947759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:29:33.868463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:30:46.825 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 35 2024-09-16T00:30:46.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3180a3240 msgr2=0x7fd3180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3180a3240 0x7fd3180a5700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd320009930 tx=0x7fd320008040 comp rx=0 tx=0).stop 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd33010c170 msgr2=0x7fd33019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd33010c170 0x7fd33019ce40 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fd32800cad0 tx=0x7fd32800cf90 comp rx=0 tx=0).stop 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 shutdown_connections 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd3180a3240 0x7fd3180a5700 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd33010c170 0x7fd33019ce40 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd330103660 0x7fd330198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 --2- 172.21.15.78:0/1034143387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd330102d40 0x7fd330198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 >> 172.21.15.78:0/1034143387 conn(0x7fd3300fc7a0 msgr2=0x7fd3300fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 shutdown_connections 2024-09-16T00:30:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:46.827+0000 7fd335629640 1 -- 172.21.15.78:0/1034143387 wait complete. 2024-09-16T00:30:47.235 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:46 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1034143387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:47.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:46 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1034143387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:47.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:46 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1034143387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:47.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:47.927 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:48.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:47 smithi078 ceph-mon[31157]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-16T00:30:48.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:47 smithi193 ceph-mon[35484]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-16T00:30:48.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:47 smithi148 ceph-mon[35599]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-16T00:30:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- 172.21.15.78:0/2115329737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 msgr2=0x7f751c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 --2- 172.21.15.78:0/2115329737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 0x7f751c100ee0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f7508009930 tx=0x7f750802a420 comp rx=0 tx=0).stop 2024-09-16T00:30:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- 172.21.15.78:0/2115329737 shutdown_connections 2024-09-16T00:30:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 --2- 172.21.15.78:0/2115329737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f751c101420 0x7f751c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 --2- 172.21.15.78:0/2115329737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 0x7f751c100ee0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 --2- 172.21.15.78:0/2115329737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f751c106a80 0x7f751c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- 172.21.15.78:0/2115329737 >> 172.21.15.78:0/2115329737 conn(0x7f751c0fc7a0 msgr2=0x7f751c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- 172.21.15.78:0/2115329737 shutdown_connections 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- 172.21.15.78:0/2115329737 wait complete. 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 Processor -- start 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- start start 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 0x7f751c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f751c101420 0x7f751c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f751c106a80 0x7f751c1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f751c110ea0 con 0x7f751c100a80 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f751c110d20 con 0x7f751c106a80 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.441+0000 7f752300f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f751c111020 con 0x7f751c101420 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 0x7f751c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:48.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 0x7f751c199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53516/0 (socket says 172.21.15.78:53516) 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 -- 172.21.15.78:0/441934785 learned_addr learned my addr 172.21.15.78:0/441934785 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f751bfff640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f751c101420 0x7f751c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f75215ac640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f751c106a80 0x7f751c1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 -- 172.21.15.78:0/441934785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f751c101420 msgr2=0x7f751c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f751c101420 0x7f751c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 -- 172.21.15.78:0/441934785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f751c106a80 msgr2=0x7f751c1a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f751c106a80 0x7f751c1a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 -- 172.21.15.78:0/441934785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c1a48d0 con 0x7f751c100a80 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7520dab640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 0x7f751c199310 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f751000b4a0 tx=0x7f751000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:48.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7519ffb640 1 -- 172.21.15.78:0/441934785 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510014070 con 0x7f751c100a80 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7519ffb640 1 -- 172.21.15.78:0/441934785 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7510010af0 con 0x7f751c100a80 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f7519ffb640 1 -- 172.21.15.78:0/441934785 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510017d10 con 0x7f751c100a80 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c1a4aa0 con 0x7f751c100a80 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.442+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c1a4fb0 con 0x7f751c100a80 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.444+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f751c100ee0 con 0x7f751c100a80 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.444+0000 7f7519ffb640 1 -- 172.21.15.78:0/441934785 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7510017070 con 0x7f751c100a80 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.445+0000 7f7519ffb640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f750c0a3240 0x7f750c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.445+0000 7f751bfff640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f750c0a3240 0x7f750c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.446+0000 7f7519ffb640 1 -- 172.21.15.78:0/441934785 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f75100cefe0 con 0x7f751c100a80 2024-09-16T00:30:48.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.446+0000 7f751bfff640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f750c0a3240 0x7f750c0a5700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f75080058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.450+0000 7f7519ffb640 1 -- 172.21.15.78:0/441934785 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f751009b9a0 con 0x7f751c100a80 2024-09-16T00:30:48.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.654+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f751c109010 con 0x7f751c100a80 2024-09-16T00:30:48.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.655+0000 7f7519ffb640 1 -- 172.21.15.78:0/441934785 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11987 (secure 0 0 0) 0x7f751009b2a0 con 0x7f751c100a80 2024-09-16T00:30:48.656 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:30:48.656 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":35,"btime":"2024-09-16T00:29:33:947759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/4001113137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":4001113137},{"type":"v1","addr":"172.21.15.78:6835","nonce":4001113137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6837/3192121522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6836","nonce":3192121522},{"type":"v1","addr":"172.21.15.78:6837","nonce":3192121522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:11:05.234467+0000","modified":"2024-09-16T00:29:33.868463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24445,"mds_2":24440,"mds_3":24427,"mds_4":24421},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14571":{"gid":14571,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6839/1764484583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6838","nonce":1764484583},{"type":"v1","addr":"172.21.15.78:6839","nonce":1764484583}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.78:6841/1822173762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6840","nonce":1822173762},{"type":"v1","addr":"172.21.15.78:6841","nonce":1822173762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/1629662299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1629662299},{"type":"v1","addr":"172.21.15.193:6833","nonce":1629662299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/3203333506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":3203333506},{"type":"v1","addr":"172.21.15.148:6833","nonce":3203333506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6835/903408403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":903408403},{"type":"v1","addr":"172.21.15.148:6835","nonce":903408403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.193:6835/2292206006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2292206006},{"type":"v1","addr":"172.21.15.193:6835","nonce":2292206006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.193:6837/1139945107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1139945107},{"type":"v1","addr":"172.21.15.193:6837","nonce":1139945107}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.148:6837/3780846668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":3780846668},{"type":"v1","addr":"172.21.15.148:6837","nonce":3780846668}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":15,"addr":"172.21.15.193:6839/3313451522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3313451522},{"type":"v1","addr":"172.21.15.193:6839","nonce":3313451522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.148:6839/831570873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":831570873},{"type":"v1","addr":"172.21.15.148:6839","nonce":831570873}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24445,24440,24427,24421]},"id":1}]} 2024-09-16T00:30:48.656 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 35 2024-09-16T00:30:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f750c0a3240 msgr2=0x7f750c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f750c0a3240 0x7f750c0a5700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7508009900 tx=0x7f75080058d0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 msgr2=0x7f751c199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 0x7f751c199310 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f751000b4a0 tx=0x7f751000b960 comp rx=0 tx=0).stop 2024-09-16T00:30:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 shutdown_connections 2024-09-16T00:30:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f750c0a3240 0x7f750c0a5700 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f751c106a80 0x7f751c1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f751c101420 0x7f751c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 --2- 172.21.15.78:0/441934785 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f751c100a80 0x7f751c199310 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:48.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 >> 172.21.15.78:0/441934785 conn(0x7f751c0fc7a0 msgr2=0x7f751c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:48.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.657+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 shutdown_connections 2024-09-16T00:30:48.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:48.658+0000 7f752300f640 1 -- 172.21.15.78:0/441934785 wait complete. 2024-09-16T00:30:49.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:48 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/441934785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:49.337 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:30:49.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:48 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/441934785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:49.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:48 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/441934785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:49.583 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 -- 172.21.15.78:0/3753442196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 msgr2=0x7f4004103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3753442196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 0x7f4004103190 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f4000009cd0 tx=0x7f400002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 -- 172.21.15.78:0/3753442196 shutdown_connections 2024-09-16T00:30:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3753442196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40041082e0 0x7f400410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3753442196 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4004103760 0x7f4004107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3753442196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 0x7f4004103190 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 -- 172.21.15.78:0/3753442196 >> 172.21.15.78:0/3753442196 conn(0x7f40040fc7d0 msgr2=0x7f40040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 -- 172.21.15.78:0/3753442196 shutdown_connections 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.176+0000 7f400c7d9640 1 -- 172.21.15.78:0/3753442196 wait complete. 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400c7d9640 1 Processor -- start 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400c7d9640 1 -- start start 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400c7d9640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 0x7f4004198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400c7d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4004103760 0x7f4004198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400c7d9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40041082e0 0x7f400419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400c7d9640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f40041120b0 con 0x7f4004102db0 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400c7d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4004111f30 con 0x7f4004103760 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400c7d9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4004112230 con 0x7f40041082e0 2024-09-16T00:30:50.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400a575640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 0x7f4004198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400a575640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 0x7f4004198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53540/0 (socket says 172.21.15.78:53540) 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400a575640 1 -- 172.21.15.78:0/3626554819 learned_addr learned my addr 172.21.15.78:0/3626554819 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f4009d74640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4004103760 0x7f4004198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400ad76640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40041082e0 0x7f400419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400a575640 1 -- 172.21.15.78:0/3626554819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40041082e0 msgr2=0x7f400419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400a575640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40041082e0 0x7f400419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400a575640 1 -- 172.21.15.78:0/3626554819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4004103760 msgr2=0x7f4004198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400a575640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4004103760 0x7f4004198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400a575640 1 -- 172.21.15.78:0/3626554819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400419d540 con 0x7f4004102db0 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.177+0000 7f400ad76640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40041082e0 0x7f400419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.178+0000 7f400a575640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 0x7f4004198590 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f400002ae00 tx=0x7f4000003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.178+0000 7f3ffb7fe640 1 -- 172.21.15.78:0/3626554819 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000004240 con 0x7f4004102db0 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.178+0000 7f3ffb7fe640 1 -- 172.21.15.78:0/3626554819 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4000031d50 con 0x7f4004102db0 2024-09-16T00:30:50.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.178+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400419d7d0 con 0x7f4004102db0 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.178+0000 7f3ffb7fe640 1 -- 172.21.15.78:0/3626554819 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400003a400 con 0x7f4004102db0 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.178+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40041a5110 con 0x7f4004102db0 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.179+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4004103190 con 0x7f4004102db0 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.180+0000 7f3ffb7fe640 1 -- 172.21.15.78:0/3626554819 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f40000306c0 con 0x7f4004102db0 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.180+0000 7f3ffb7fe640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ff00a32c0 0x7f3ff00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.181+0000 7f4009d74640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ff00a32c0 0x7f3ff00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.181+0000 7f3ffb7fe640 1 -- 172.21.15.78:0/3626554819 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7f400002e030 con 0x7f4004102db0 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.181+0000 7f4009d74640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ff00a32c0 0x7f3ff00a5780 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f40040fee20 tx=0x7f3ff400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:50.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.185+0000 7f3ffb7fe640 1 -- 172.21.15.78:0/3626554819 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40000bd5c0 con 0x7f4004102db0 2024-09-16T00:30:50.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:49 smithi078 ceph-mon[31157]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-16T00:30:50.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.353+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f400410a2c0 con 0x7f4004102db0 2024-09-16T00:30:50.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.354+0000 7f3ffb7fe640 1 -- 172.21.15.78:0/3626554819 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24757 (secure 0 0 0) 0x7f40000bcec0 con 0x7f4004102db0 2024-09-16T00:30:50.355 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-16T00:30:50.355 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"eb73a744-73be-11ef-bceb-c7b262605968","created":"2024-09-16T00:03:16.287057+0000","modified":"2024-09-16T00:13:04.600749+0000","last_up_change":"2024-09-16T00:10:11.145820+0000","last_in_change":"2024-09-16T00:09:46.905900+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:06:20.608203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:11:01.500267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:11:03.483225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:11:09.892287+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"03b3fcd8-25ff-4e8d-b8ff-15e3cbc12576","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6803","nonce":1264645608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6805","nonce":1264645608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6809","nonce":1264645608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":1264645608},{"type":"v1","addr":"172.21.15.78:6807","nonce":1264645608}]},"public_addr":"172.21.15.78:6803/1264645608","cluster_addr":"172.21.15.78:6805/1264645608","heartbeat_back_addr":"172.21.15.78:6809/1264645608","heartbeat_front_addr":"172.21.15.78:6807/1264645608","state":["exists","up"]},{"osd":1,"uuid":"c2638a47-f838-4300-848b-e731ac3cce78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6801","nonce":2952692564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6803","nonce":2952692564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6807","nonce":2952692564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":2952692564},{"type":"v1","addr":"172.21.15.148:6805","nonce":2952692564}]},"public_addr":"172.21.15.148:6801/2952692564","cluster_addr":"172.21.15.148:6803/2952692564","heartbeat_back_addr":"172.21.15.148:6807/2952692564","heartbeat_front_addr":"172.21.15.148:6805/2952692564","state":["exists","up"]},{"osd":2,"uuid":"863c25a6-76c4-4097-9c6f-20e838ddb3e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6801","nonce":2579590991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6803","nonce":2579590991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6807","nonce":2579590991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2579590991},{"type":"v1","addr":"172.21.15.193:6805","nonce":2579590991}]},"public_addr":"172.21.15.193:6801/2579590991","cluster_addr":"172.21.15.193:6803/2579590991","heartbeat_back_addr":"172.21.15.193:6807/2579590991","heartbeat_front_addr":"172.21.15.193:6805/2579590991","state":["exists","up"]},{"osd":3,"uuid":"f34ec822-59f0-44e6-a33b-ae877c648c67","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6811","nonce":2314919115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6813","nonce":2314919115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6817","nonce":2314919115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2314919115},{"type":"v1","addr":"172.21.15.78:6815","nonce":2314919115}]},"public_addr":"172.21.15.78:6811/2314919115","cluster_addr":"172.21.15.78:6813/2314919115","heartbeat_back_addr":"172.21.15.78:6817/2314919115","heartbeat_front_addr":"172.21.15.78:6815/2314919115","state":["exists","up"]},{"osd":4,"uuid":"928fd68c-669c-4de2-aa53-11d764c071ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6809","nonce":336734377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6811","nonce":336734377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6815","nonce":336734377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":336734377},{"type":"v1","addr":"172.21.15.148:6813","nonce":336734377}]},"public_addr":"172.21.15.148:6809/336734377","cluster_addr":"172.21.15.148:6811/336734377","heartbeat_back_addr":"172.21.15.148:6815/336734377","heartbeat_front_addr":"172.21.15.148:6813/336734377","state":["exists","up"]},{"osd":5,"uuid":"538b1b72-96f3-41ed-bfc6-9796b5ae0578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6809","nonce":2332816249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6811","nonce":2332816249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6815","nonce":2332816249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2332816249},{"type":"v1","addr":"172.21.15.193:6813","nonce":2332816249}]},"public_addr":"172.21.15.193:6809/2332816249","cluster_addr":"172.21.15.193:6811/2332816249","heartbeat_back_addr":"172.21.15.193:6815/2332816249","heartbeat_front_addr":"172.21.15.193:6813/2332816249","state":["exists","up"]},{"osd":6,"uuid":"d19221b9-f4f0-42ca-ae31-7b1935b52a50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6819","nonce":445240021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6821","nonce":445240021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6825","nonce":445240021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":445240021},{"type":"v1","addr":"172.21.15.78:6823","nonce":445240021}]},"public_addr":"172.21.15.78:6819/445240021","cluster_addr":"172.21.15.78:6821/445240021","heartbeat_back_addr":"172.21.15.78:6825/445240021","heartbeat_front_addr":"172.21.15.78:6823/445240021","state":["exists","up"]},{"osd":7,"uuid":"c7efa41b-836b-419a-b2c0-8e2861cddeaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6817","nonce":155815362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6819","nonce":155815362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6823","nonce":155815362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":155815362},{"type":"v1","addr":"172.21.15.148:6821","nonce":155815362}]},"public_addr":"172.21.15.148:6817/155815362","cluster_addr":"172.21.15.148:6819/155815362","heartbeat_back_addr":"172.21.15.148:6823/155815362","heartbeat_front_addr":"172.21.15.148:6821/155815362","state":["exists","up"]},{"osd":8,"uuid":"f8e53741-a657-48d0-a667-fb8a28f6f922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6817","nonce":3026218583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6819","nonce":3026218583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6823","nonce":3026218583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":3026218583},{"type":"v1","addr":"172.21.15.193:6821","nonce":3026218583}]},"public_addr":"172.21.15.193:6817/3026218583","cluster_addr":"172.21.15.193:6819/3026218583","heartbeat_back_addr":"172.21.15.193:6823/3026218583","heartbeat_front_addr":"172.21.15.193:6821/3026218583","state":["exists","up"]},{"osd":9,"uuid":"7e15eca2-aea8-4693-807d-bfa2b2439c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6827","nonce":1947247517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6829","nonce":1947247517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6833","nonce":1947247517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":1947247517},{"type":"v1","addr":"172.21.15.78:6831","nonce":1947247517}]},"public_addr":"172.21.15.78:6827/1947247517","cluster_addr":"172.21.15.78:6829/1947247517","heartbeat_back_addr":"172.21.15.78:6833/1947247517","heartbeat_front_addr":"172.21.15.78:6831/1947247517","state":["exists","up"]},{"osd":10,"uuid":"92a86462-ac20-4b68-b8eb-d18ba5c604b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6825","nonce":1827220524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6827","nonce":1827220524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6831","nonce":1827220524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":1827220524},{"type":"v1","addr":"172.21.15.148:6829","nonce":1827220524}]},"public_addr":"172.21.15.148:6825/1827220524","cluster_addr":"172.21.15.148:6827/1827220524","heartbeat_back_addr":"172.21.15.148:6831/1827220524","heartbeat_front_addr":"172.21.15.148:6829/1827220524","state":["exists","up"]},{"osd":11,"uuid":"41c5e26a-ddcf-4933-95f1-5c868521acfd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6825","nonce":872081348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6827","nonce":872081348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6831","nonce":872081348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":872081348},{"type":"v1","addr":"172.21.15.193:6829","nonce":872081348}]},"public_addr":"172.21.15.193:6825/872081348","cluster_addr":"172.21.15.193:6827/872081348","heartbeat_back_addr":"172.21.15.193:6831/872081348","heartbeat_front_addr":"172.21.15.193:6829/872081348","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:28.208857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:05:53.075047+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:18.051649+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:06:43.619195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:10.056069+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:34.867259+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:00.181509+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:25.576278+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:08:51.939648+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:17.890429+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:09:43.208890+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:10:09.019099+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.e","mappings":[{"from":9,"to":1},{"from":5,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6801/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/3861325161":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1335742179":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/3957981561":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/2949703267":"2024-09-17T00:03:37.955119+0000","172.21.15.78:6800/1052494441":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6801/1515675480":"2024-09-17T00:03:37.955119+0000","172.21.15.78:0/486633606":"2024-09-17T00:04:02.295421+0000","172.21.15.78:0/1597437728":"2024-09-17T00:04:02.295421+0000","172.21.15.78:6800/1515675480":"2024-09-17T00:03:37.955119+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:30:50.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ff00a32c0 msgr2=0x7f3ff00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:50.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ff00a32c0 0x7f3ff00a5780 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f40040fee20 tx=0x7f3ff400a4c0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 msgr2=0x7f4004198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 0x7f4004198590 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f400002ae00 tx=0x7f4000003ba0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 shutdown_connections 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f3ff00a32c0 0x7f3ff00a5780 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40041082e0 0x7f400419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4004103760 0x7f4004198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 --2- 172.21.15.78:0/3626554819 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4004102db0 0x7f4004198590 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 >> 172.21.15.78:0/3626554819 conn(0x7f40040fc7d0 msgr2=0x7f4004105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 shutdown_connections 2024-09-16T00:30:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:50.357+0000 7f400c7d9640 1 -- 172.21.15.78:0/3626554819 wait complete. 2024-09-16T00:30:50.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:49 smithi193 ceph-mon[35484]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-16T00:30:50.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:49 smithi148 ceph-mon[35599]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-09-16T00:30:51.234 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:51 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3626554819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:51.262 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-16T00:30:51.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:51 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3626554819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:51.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:51 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3626554819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:51.507 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 -- 172.21.15.78:0/4148589942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 msgr2=0x7ff39010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 --2- 172.21.15.78:0/4148589942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 0x7ff39010bb50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff380009900 tx=0x7ff38002a480 comp rx=0 tx=0).stop 2024-09-16T00:30:52.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 -- 172.21.15.78:0/4148589942 shutdown_connections 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 --2- 172.21.15.78:0/4148589942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff39010c170 0x7ff39010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 --2- 172.21.15.78:0/4148589942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 0x7ff39010bb50 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 --2- 172.21.15.78:0/4148589942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff390102d40 0x7ff390103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 -- 172.21.15.78:0/4148589942 >> 172.21.15.78:0/4148589942 conn(0x7ff3900fc7a0 msgr2=0x7ff3900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 -- 172.21.15.78:0/4148589942 shutdown_connections 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.001+0000 7ff397f91640 1 -- 172.21.15.78:0/4148589942 wait complete. 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff397f91640 1 Processor -- start 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff397f91640 1 -- start start 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff397f91640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff390102d40 0x7ff3901985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff397f91640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 0x7ff390198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff397f91640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39010c170 0x7ff39019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:52.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff397f91640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff390112030 con 0x7ff390103660 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff397f91640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff390111eb0 con 0x7ff390102d40 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff397f91640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3901121b0 con 0x7ff39010c170 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff39552c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 0x7ff390198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff39552c640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 0x7ff390198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53566/0 (socket says 172.21.15.78:53566) 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.002+0000 7ff39552c640 1 -- 172.21.15.78:0/1530555290 learned_addr learned my addr 172.21.15.78:0/1530555290 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.003+0000 7ff395d2d640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff390102d40 0x7ff3901985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.003+0000 7ff39652e640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39010c170 0x7ff39019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.003+0000 7ff39552c640 1 -- 172.21.15.78:0/1530555290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39010c170 msgr2=0x7ff39019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.003+0000 7ff39552c640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39010c170 0x7ff39019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.003+0000 7ff39552c640 1 -- 172.21.15.78:0/1530555290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff390102d40 msgr2=0x7ff3901985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.003+0000 7ff39552c640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff390102d40 0x7ff3901985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.003+0000 7ff39552c640 1 -- 172.21.15.78:0/1530555290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39019d550 con 0x7ff390103660 2024-09-16T00:30:52.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.003+0000 7ff39552c640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 0x7ff390198ae0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff380005ab0 tx=0x7ff3800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.005+0000 7ff386ffd640 1 -- 172.21.15.78:0/1530555290 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38003f030 con 0x7ff390103660 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.005+0000 7ff386ffd640 1 -- 172.21.15.78:0/1530555290 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff380003ce0 con 0x7ff390103660 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.005+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39019d7e0 con 0x7ff390103660 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.005+0000 7ff386ffd640 1 -- 172.21.15.78:0/1530555290 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380035070 con 0x7ff390103660 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.005+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3901a5120 con 0x7ff390103660 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.006+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3901031a0 con 0x7ff390103660 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.008+0000 7ff386ffd640 1 -- 172.21.15.78:0/1530555290 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff380042030 con 0x7ff390103660 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.008+0000 7ff386ffd640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff37c0a32c0 0x7ff37c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.009+0000 7ff395d2d640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff37c0a32c0 0x7ff37c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.009+0000 7ff386ffd640 1 -- 172.21.15.78:0/1530555290 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9100+0+0 (secure 0 0 0) 0x7ff3800f0410 con 0x7ff390103660 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.009+0000 7ff395d2d640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff37c0a32c0 0x7ff37c0a5780 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009990 tx=0x7ff38c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:52.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.013+0000 7ff386ffd640 1 -- 172.21.15.78:0/1530555290 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3800bcdd0 con 0x7ff390103660 2024-09-16T00:30:52.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.213+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff390106c50 con 0x7ff390103660 2024-09-16T00:30:52.214 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-mon[31157]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-16T00:30:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.216+0000 7ff386ffd640 1 -- 172.21.15.78:0/1530555290 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7ff3800bc6d0 con 0x7ff390103660 2024-09-16T00:30:52.216 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T00:30:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff37c0a32c0 msgr2=0x7ff37c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff37c0a32c0 0x7ff37c0a5780 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009990 tx=0x7ff38c009110 comp rx=0 tx=0).stop 2024-09-16T00:30:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 msgr2=0x7ff390198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 0x7ff390198ae0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff380005ab0 tx=0x7ff3800042f0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 shutdown_connections 2024-09-16T00:30:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7ff37c0a32c0 0x7ff37c0a5780 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39010c170 0x7ff39019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff390103660 0x7ff390198ae0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 --2- 172.21.15.78:0/1530555290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff390102d40 0x7ff3901985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 >> 172.21.15.78:0/1530555290 conn(0x7ff3900fc7a0 msgr2=0x7ff3900fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.219+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 shutdown_connections 2024-09-16T00:30:52.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:52.220+0000 7ff397f91640 1 -- 172.21.15.78:0/1530555290 wait complete. 2024-09-16T00:30:52.267 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-mon[35484]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-16T00:30:52.291 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-mon[35599]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-16T00:30:52.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a[31153]: 2024-09-16T00:30:52.214+0000 7f684d020640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:30:52.521 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a[72583]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.521 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a[72583]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.521 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-mds[72591]: -- 172.21.15.78:0/1466092675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0aea25860 con 0x55e0aea02800 2024-09-16T00:30:52.521 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a[72583]: starting mds.a at 2024-09-16T00:30:52.521 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g[72169]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.521 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g[72169]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.521 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-mds[72176]: -- 172.21.15.78:0/2035341021 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640a21dd860 con 0x5640a21bb400 2024-09-16T00:30:52.522 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g[72169]: starting mds.g at 2024-09-16T00:30:52.522 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:30:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g[72169]: 2024-09-16T00:30:52.324+0000 7f752c1fc640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6838/0: (98) Address already in use 2024-09-16T00:30:52.679 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c[66196]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.679 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c[66196]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.679 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-mds[66200]: -- 172.21.15.193:0/838047256 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556eb27cb860 con 0x556eb27a9400 2024-09-16T00:30:52.679 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c[66196]: starting mds.c at 2024-09-16T00:30:52.680 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i[66508]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.680 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i[66508]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.680 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-mds[66512]: -- 172.21.15.193:0/3663209625 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f4ca56e680 con 0x55f4ca543400 2024-09-16T00:30:52.680 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-mds[66512]: -- 172.21.15.193:0/3663209625 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4ca565860 con 0x55f4ca543400 2024-09-16T00:30:52.680 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i[66508]: starting mds.i at 2024-09-16T00:30:52.680 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l[65596]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.681 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l[65596]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.681 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-mds[65600]: -- 172.21.15.193:0/4213200370 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55728e63b860 con 0x55728e618800 2024-09-16T00:30:52.681 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l[65596]: starting mds.l at 2024-09-16T00:30:52.681 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f[65895]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.681 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f[65895]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.681 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-mds[65899]: -- 172.21.15.193:0/827983270 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c1c397860 con 0x555c1c375000 2024-09-16T00:30:52.681 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:30:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f[65895]: starting mds.f at 2024-09-16T00:30:52.704 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-k[64131]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.704 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-k[64131]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.704 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-mds[64135]: -- 172.21.15.148:0/279161002 <== mon.2 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b459d5b860 con 0x55b459d38800 2024-09-16T00:30:52.704 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-k[64131]: starting mds.k at 2024-09-16T00:30:52.705 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.705 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.705 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-mds[65068]: -- 172.21.15.148:0/510142446 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563aff735860 con 0x563aff712800 2024-09-16T00:30:52.705 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: starting mds.b at 2024-09-16T00:30:52.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e[64448]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e[64448]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-mds[64452]: -- 172.21.15.148:0/69212093 <== mon.0 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c6af89860 con 0x555c6af67000 2024-09-16T00:30:52.705 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e[64448]: starting mds.e at 2024-09-16T00:30:52.706 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h[64748]: ignoring --setuser ceph since I am not root 2024-09-16T00:30:52.706 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h[64748]: ignoring --setgroup ceph since I am not root 2024-09-16T00:30:52.706 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-mds[64752]: -- 172.21.15.148:0/551259459 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd65643c20 con 0x55dd65620800 2024-09-16T00:30:52.706 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:30:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h[64748]: starting mds.h at 2024-09-16T00:30:52.947 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T00:30:53.200 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:53.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:53 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1530555290' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:30:53.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:53 smithi148 ceph-mon[35599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:30:53.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:53 smithi148 ceph-mon[35599]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:30:53.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:53 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1530555290' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:30:53.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:53 smithi148 ceph-mon[35599]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:30:53.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:53 smithi148 ceph-mon[35599]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T00:30:53.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:53 smithi148 ceph-mon[35599]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-16T00:30:53.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:53 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1530555290' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:30:53.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:53 smithi078 ceph-mon[31157]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:30:53.269 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:53 smithi078 ceph-mon[31157]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:30:53.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:53 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1530555290' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:30:53.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:53 smithi078 ceph-mon[31157]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:30:53.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:53 smithi078 ceph-mon[31157]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T00:30:53.270 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:53 smithi078 ceph-mon[31157]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-16T00:30:53.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:53 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1530555290' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:30:53.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:53 smithi193 ceph-mon[35484]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:30:53.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:53 smithi193 ceph-mon[35484]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:30:53.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:53 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1530555290' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:30:53.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:53 smithi193 ceph-mon[35484]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:30:53.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:53 smithi193 ceph-mon[35484]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-16T00:30:53.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:53 smithi193 ceph-mon[35484]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-16T00:30:53.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 -- 172.21.15.78:0/1963899210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4102790 msgr2=0x7f8ac41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:53.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 --2- 172.21.15.78:0/1963899210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4102790 0x7f8ac41067e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009900 tx=0x7f8aac02a410 comp rx=0 tx=0).stop 2024-09-16T00:30:53.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 -- 172.21.15.78:0/1963899210 shutdown_connections 2024-09-16T00:30:53.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 --2- 172.21.15.78:0/1963899210 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac4106f10 0x7f8ac410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 --2- 172.21.15.78:0/1963899210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4102790 0x7f8ac41067e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 --2- 172.21.15.78:0/1963899210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ac4101de0 0x7f8ac41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 -- 172.21.15.78:0/1963899210 >> 172.21.15.78:0/1963899210 conn(0x7f8ac40fd8c0 msgr2=0x7f8ac40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 -- 172.21.15.78:0/1963899210 shutdown_connections 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.705+0000 7f8aca791640 1 -- 172.21.15.78:0/1963899210 wait complete. 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8aca791640 1 Processor -- start 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8aca791640 1 -- start start 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8aca791640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ac4101de0 0x7f8ac41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8aca791640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac4102790 0x7f8ac4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8aca791640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4106f10 0x7f8ac419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8aca791640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4112010 con 0x7f8ac4106f10 2024-09-16T00:30:53.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8aca791640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4111e90 con 0x7f8ac4102790 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8aca791640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac4112190 con 0x7f8ac4101de0 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8ac8d2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4106f10 0x7f8ac419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8ac8d2e640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4106f10 0x7f8ac419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53612/0 (socket says 172.21.15.78:53612) 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.706+0000 7f8ac8d2e640 1 -- 172.21.15.78:0/1615222762 learned_addr learned my addr 172.21.15.78:0/1615222762 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac3fff640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ac4101de0 0x7f8ac41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac37fe640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac4102790 0x7f8ac4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac8d2e640 1 -- 172.21.15.78:0/1615222762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ac4101de0 msgr2=0x7f8ac41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac8d2e640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ac4101de0 0x7f8ac41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac8d2e640 1 -- 172.21.15.78:0/1615222762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac4102790 msgr2=0x7f8ac4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac8d2e640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac4102790 0x7f8ac4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac8d2e640 1 -- 172.21.15.78:0/1615222762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac419d560 con 0x7f8ac4106f10 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac37fe640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac4102790 0x7f8ac4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:53.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac8d2e640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4106f10 0x7f8ac419ce80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00b970 tx=0x7f8abc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac17fa640 1 -- 172.21.15.78:0/1615222762 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc019070 con 0x7f8ac4106f10 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac17fa640 1 -- 172.21.15.78:0/1615222762 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8abc00cb40 con 0x7f8ac4106f10 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac419d850 con 0x7f8ac4106f10 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8ac17fa640 1 -- 172.21.15.78:0/1615222762 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc007650 con 0x7f8ac4106f10 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.707+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac41a5130 con 0x7f8ac4106f10 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.708+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac4102240 con 0x7f8ac4106f10 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.709+0000 7f8ac17fa640 1 -- 172.21.15.78:0/1615222762 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8abc01c070 con 0x7f8ac4106f10 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.710+0000 7f8ac17fa640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ab00a3240 0x7f8ab00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.710+0000 7f8ac3fff640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ab00a3240 0x7f8ab00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.710+0000 7f8ac17fa640 1 -- 172.21.15.78:0/1615222762 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9530+0+0 (secure 0 0 0) 0x7f8abc0cf4c0 con 0x7f8ac4106f10 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.710+0000 7f8ac3fff640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ab00a3240 0x7f8ab00a5700 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009890 tx=0x7f8ab4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:53.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.714+0000 7f8ac17fa640 1 -- 172.21.15.78:0/1615222762 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8abc09bcd0 con 0x7f8ac4106f10 2024-09-16T00:30:53.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:53.914+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8ac410a150 con 0x7f8ac4106f10 2024-09-16T00:30:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.228+0000 7f8ac17fa640 1 -- 172.21.15.78:0/1615222762 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f8abc09b5d0 con 0x7f8ac4106f10 2024-09-16T00:30:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ab00a3240 msgr2=0x7f8ab00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ab00a3240 0x7f8ab00a5700 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009890 tx=0x7f8ab4008040 comp rx=0 tx=0).stop 2024-09-16T00:30:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4106f10 msgr2=0x7f8ac419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4106f10 0x7f8ac419ce80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00b970 tx=0x7f8abc00be30 comp rx=0 tx=0).stop 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 shutdown_connections 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f8ab00a3240 0x7f8ab00a5700 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8ac4106f10 0x7f8ac419ce80 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ac4102790 0x7f8ac4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 --2- 172.21.15.78:0/1615222762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ac4101de0 0x7f8ac41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 >> 172.21.15.78:0/1615222762 conn(0x7f8ac40fd8c0 msgr2=0x7f8ac40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 shutdown_connections 2024-09-16T00:30:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:54.231+0000 7f8aca791640 1 -- 172.21.15.78:0/1615222762 wait complete. 2024-09-16T00:30:54.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6832/1503237030,v1:172.21.15.193:6834/1503237030] up:boot 2024-09-16T00:30:54.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6833/2302739377,v1:172.21.15.193:6835/2302739377] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6836/1882026087,v1:172.21.15.193:6838/1882026087] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.193:6837/1625314313,v1:172.21.15.193:6839/1625314313] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6833/354156268,v1:172.21.15.148:6835/354156268] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6832/65733844,v1:172.21.15.148:6834/65733844] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6836/3333496352,v1:172.21.15.148:6837/3333496352] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.148:6838/1496190154,v1:172.21.15.148:6839/1496190154] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.78:6838/3280326815,v1:172.21.15.78:6840/3280326815] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: mds.? [v2:172.21.15.78:6839/844862306,v1:172.21.15.78:6841/844862306] up:boot 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:30:54.520 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:30:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:30:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:30:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:30:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:30:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:30:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:30:54.521 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:54 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1615222762' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6832/1503237030,v1:172.21.15.193:6834/1503237030] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6833/2302739377,v1:172.21.15.193:6835/2302739377] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6836/1882026087,v1:172.21.15.193:6838/1882026087] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.193:6837/1625314313,v1:172.21.15.193:6839/1625314313] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6833/354156268,v1:172.21.15.148:6835/354156268] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6832/65733844,v1:172.21.15.148:6834/65733844] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6836/3333496352,v1:172.21.15.148:6837/3333496352] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.148:6838/1496190154,v1:172.21.15.148:6839/1496190154] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.78:6838/3280326815,v1:172.21.15.78:6840/3280326815] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: mds.? [v2:172.21.15.78:6839/844862306,v1:172.21.15.78:6841/844862306] up:boot 2024-09-16T00:30:54.551 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:30:54.552 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:54 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1615222762' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:30:54.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6832/1503237030,v1:172.21.15.193:6834/1503237030] up:boot 2024-09-16T00:30:54.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6833/2302739377,v1:172.21.15.193:6835/2302739377] up:boot 2024-09-16T00:30:54.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6836/1882026087,v1:172.21.15.193:6838/1882026087] up:boot 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.193:6837/1625314313,v1:172.21.15.193:6839/1625314313] up:boot 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6833/354156268,v1:172.21.15.148:6835/354156268] up:boot 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6832/65733844,v1:172.21.15.148:6834/65733844] up:boot 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6836/3333496352,v1:172.21.15.148:6837/3333496352] up:boot 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.148:6838/1496190154,v1:172.21.15.148:6839/1496190154] up:boot 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.78:6838/3280326815,v1:172.21.15.78:6840/3280326815] up:boot 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: mds.? [v2:172.21.15.78:6839/844862306,v1:172.21.15.78:6841/844862306] up:boot 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:30:54.705 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:30:54.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:30:54.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:30:54.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:30:54.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:30:54.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='mgr.14150 172.21.15.78:0/3323699077' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:30:54.706 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:54 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1615222762' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:30:54.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T00:30:55.226 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:55.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:55 smithi078 ceph-mon[31157]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:30:55.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:55 smithi078 ceph-mon[31157]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:30:55.505 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:55 smithi078 ceph-mon[31157]: Cluster is now healthy 2024-09-16T00:30:55.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:55 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/1615222762' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:30:55.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:55 smithi078 ceph-mon[31157]: fsmap 12 up:standby 2024-09-16T00:30:55.506 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:55 smithi078 ceph-mon[31157]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-16T00:30:55.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:55 smithi193 ceph-mon[35484]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:30:55.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:55 smithi193 ceph-mon[35484]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:30:55.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:55 smithi193 ceph-mon[35484]: Cluster is now healthy 2024-09-16T00:30:55.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:55 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/1615222762' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:30:55.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:55 smithi193 ceph-mon[35484]: fsmap 12 up:standby 2024-09-16T00:30:55.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:55 smithi193 ceph-mon[35484]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-16T00:30:55.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:55 smithi148 ceph-mon[35599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:30:55.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:55 smithi148 ceph-mon[35599]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:30:55.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:55 smithi148 ceph-mon[35599]: Cluster is now healthy 2024-09-16T00:30:55.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:55 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/1615222762' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:30:55.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:55 smithi148 ceph-mon[35599]: fsmap 12 up:standby 2024-09-16T00:30:55.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:55 smithi148 ceph-mon[35599]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-09-16T00:30:55.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 -- 172.21.15.78:0/3803123880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc102d40 msgr2=0x7feadc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:55.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 --2- 172.21.15.78:0/3803123880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc102d40 0x7feadc103120 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7feacc009900 tx=0x7feacc02a480 comp rx=0 tx=0).stop 2024-09-16T00:30:55.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 -- 172.21.15.78:0/3803123880 shutdown_connections 2024-09-16T00:30:55.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 --2- 172.21.15.78:0/3803123880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc10c170 0x7feadc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 --2- 172.21.15.78:0/3803123880 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc103660 0x7feadc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 --2- 172.21.15.78:0/3803123880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc102d40 0x7feadc103120 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 -- 172.21.15.78:0/3803123880 >> 172.21.15.78:0/3803123880 conn(0x7feadc0fc7a0 msgr2=0x7feadc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:55.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 -- 172.21.15.78:0/3803123880 shutdown_connections 2024-09-16T00:30:55.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.732+0000 7feae2996640 1 -- 172.21.15.78:0/3803123880 wait complete. 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.733+0000 7feae2996640 1 Processor -- start 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.733+0000 7feae2996640 1 -- start start 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae2996640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc102d40 0x7feadc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae2996640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc103660 0x7feadc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae2996640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc10c170 0x7feadc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae2996640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feadc111fb0 con 0x7feadc10c170 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae2996640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feadc111e30 con 0x7feadc102d40 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae2996640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feadc112130 con 0x7feadc103660 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc10c170 0x7feadc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc10c170 0x7feadc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53640/0 (socket says 172.21.15.78:53640) 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 -- 172.21.15.78:0/2970756599 learned_addr learned my addr 172.21.15.78:0/2970756599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:55.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feadb7fe640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc103660 0x7feadc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feadbfff640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc102d40 0x7feadc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 -- 172.21.15.78:0/2970756599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc103660 msgr2=0x7feadc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc103660 0x7feadc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 -- 172.21.15.78:0/2970756599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc102d40 msgr2=0x7feadc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc102d40 0x7feadc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 -- 172.21.15.78:0/2970756599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc19d440 con 0x7feadc10c170 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feadbfff640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc102d40 0x7feadc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae0f33640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc10c170 0x7feadc19cd60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fead400b970 tx=0x7fead400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:55.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7fead97fa640 1 -- 172.21.15.78:0/2970756599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4019070 con 0x7feadc10c170 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7fead97fa640 1 -- 172.21.15.78:0/2970756599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead400cb40 con 0x7feadc10c170 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc19d730 con 0x7feadc10c170 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.734+0000 7fead97fa640 1 -- 172.21.15.78:0/2970756599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4007650 con 0x7feadc10c170 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.735+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a5010 con 0x7feadc10c170 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.735+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feadc103120 con 0x7feadc10c170 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.737+0000 7fead97fa640 1 -- 172.21.15.78:0/2970756599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fead401c070 con 0x7feadc10c170 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.737+0000 7fead97fa640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feac40a3240 0x7feac40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.737+0000 7feadbfff640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feac40a3240 0x7feac40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.737+0000 7fead97fa640 1 -- 172.21.15.78:0/2970756599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9530+0+0 (secure 0 0 0) 0x7fead40cf410 con 0x7feadc10c170 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.738+0000 7feadbfff640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feac40a3240 0x7feac40a5700 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7feacc005ab0 tx=0x7feacc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:55.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.741+0000 7fead97fa640 1 -- 172.21.15.78:0/2970756599 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fead409b5d0 con 0x7feadc10c170 2024-09-16T00:30:55.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:55.948+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7feadc02d7c0 con 0x7feadc10c170 2024-09-16T00:30:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.235+0000 7fead97fa640 1 -- 172.21.15.78:0/2970756599 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7fead409f8d0 con 0x7feadc10c170 2024-09-16T00:30:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-16T00:30:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.238+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feac40a3240 msgr2=0x7feac40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.238+0000 7feae2996640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feac40a3240 0x7feac40a5700 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7feacc005ab0 tx=0x7feacc005a20 comp rx=0 tx=0).stop 2024-09-16T00:30:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.238+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc10c170 msgr2=0x7feadc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.238+0000 7feae2996640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc10c170 0x7feadc19cd60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fead400b970 tx=0x7fead400be30 comp rx=0 tx=0).stop 2024-09-16T00:30:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.239+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 shutdown_connections 2024-09-16T00:30:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.239+0000 7feae2996640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7feac40a3240 0x7feac40a5700 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.239+0000 7feae2996640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feadc10c170 0x7feadc19cd60 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.239+0000 7feae2996640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feadc103660 0x7feadc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.239+0000 7feae2996640 1 --2- 172.21.15.78:0/2970756599 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feadc102d40 0x7feadc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.239+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 >> 172.21.15.78:0/2970756599 conn(0x7feadc0fc7a0 msgr2=0x7feadc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.240+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 shutdown_connections 2024-09-16T00:30:56.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:56.240+0000 7feae2996640 1 -- 172.21.15.78:0/2970756599 wait complete. 2024-09-16T00:30:56.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:56 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2970756599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:30:56.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:56 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2970756599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:30:56.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:56 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2970756599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:30:57.004 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T00:30:57.251 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:57.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:57 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/2970756599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:30:57.515 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:57 smithi078 ceph-mon[31157]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:30:57.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:57 smithi078 ceph-mon[31157]: pgmap v889: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:57.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:57 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/2970756599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:30:57.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:57 smithi193 ceph-mon[35484]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:30:57.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:57 smithi193 ceph-mon[35484]: pgmap v889: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:57.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:57 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/2970756599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:30:57.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:57 smithi148 ceph-mon[35599]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:30:57.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:57 smithi148 ceph-mon[35599]: pgmap v889: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 -- 172.21.15.78:0/47875700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868100e20 msgr2=0x7fd868101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/47875700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868100e20 0x7fd868101200 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009900 tx=0x7fd85c02a480 comp rx=0 tx=0).stop 2024-09-16T00:30:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 -- 172.21.15.78:0/47875700 shutdown_connections 2024-09-16T00:30:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/47875700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd868102ef0 0x7fd86810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/47875700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd868102570 0x7fd8681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/47875700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868100e20 0x7fd868101200 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 -- 172.21.15.78:0/47875700 >> 172.21.15.78:0/47875700 conn(0x7fd8680fc7d0 msgr2=0x7fd8680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 -- 172.21.15.78:0/47875700 shutdown_connections 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 -- 172.21.15.78:0/47875700 wait complete. 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 Processor -- start 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.770+0000 7fd86ebf6640 1 -- start start 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86ebf6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd868100e20 0x7fd86819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86ebf6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd868102570 0x7fd86819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86ebf6640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868102ef0 0x7fd86819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86ebf6640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd868110fd0 con 0x7fd868102ef0 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86ebf6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd868110e50 con 0x7fd868102570 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86ebf6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd868111150 con 0x7fd868100e20 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868102ef0 0x7fd86819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:57.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86c992640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd868100e20 0x7fd86819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868102ef0 0x7fd86819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50184/0 (socket says 172.21.15.78:50184) 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 -- 172.21.15.78:0/4130394571 learned_addr learned my addr 172.21.15.78:0/4130394571 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd867fff640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd868102570 0x7fd86819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 -- 172.21.15.78:0/4130394571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd868100e20 msgr2=0x7fd86819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd868100e20 0x7fd86819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 -- 172.21.15.78:0/4130394571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd868102570 msgr2=0x7fd86819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd868102570 0x7fd86819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 -- 172.21.15.78:0/4130394571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86819f6d0 con 0x7fd868102ef0 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd86d193640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868102ef0 0x7fd86819eff0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd86000cad0 tx=0x7fd86000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.771+0000 7fd865ffb640 1 -- 172.21.15.78:0/4130394571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86000ae20 con 0x7fd868102ef0 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.772+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86819f9c0 con 0x7fd868102ef0 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.772+0000 7fd865ffb640 1 -- 172.21.15.78:0/4130394571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd860005020 con 0x7fd868102ef0 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.772+0000 7fd865ffb640 1 -- 172.21.15.78:0/4130394571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86000f6f0 con 0x7fd868102ef0 2024-09-16T00:30:57.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.772+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8681a72a0 con 0x7fd868102ef0 2024-09-16T00:30:57.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.773+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd868101200 con 0x7fd868102ef0 2024-09-16T00:30:57.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.774+0000 7fd865ffb640 1 -- 172.21.15.78:0/4130394571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd8600051d0 con 0x7fd868102ef0 2024-09-16T00:30:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.774+0000 7fd865ffb640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8580a3240 0x7fd8580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.774+0000 7fd86c992640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8580a3240 0x7fd8580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.775+0000 7fd865ffb640 1 -- 172.21.15.78:0/4130394571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9034+0+0 (secure 0 0 0) 0x7fd8600cfc20 con 0x7fd868102ef0 2024-09-16T00:30:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.775+0000 7fd86c992640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8580a3240 0x7fd8580a5700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000c00 tx=0x7fd85c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.778+0000 7fd865ffb640 1 -- 172.21.15.78:0/4130394571 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd860014030 con 0x7fd868102ef0 2024-09-16T00:30:57.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:57.944+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd8681a7470 con 0x7fd868102ef0 2024-09-16T00:30:58.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.247+0000 7fd865ffb640 1 -- 172.21.15.78:0/4130394571 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7fd86009bfd0 con 0x7fd868102ef0 2024-09-16T00:30:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8580a3240 msgr2=0x7fd8580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8580a3240 0x7fd8580a5700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000c00 tx=0x7fd85c005e50 comp rx=0 tx=0).stop 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868102ef0 msgr2=0x7fd86819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868102ef0 0x7fd86819eff0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd86000cad0 tx=0x7fd86000cf90 comp rx=0 tx=0).stop 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 shutdown_connections 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7fd8580a3240 0x7fd8580a5700 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd868102ef0 0x7fd86819eff0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd868102570 0x7fd86819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 --2- 172.21.15.78:0/4130394571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd868100e20 0x7fd86819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 >> 172.21.15.78:0/4130394571 conn(0x7fd8680fc7d0 msgr2=0x7fd86806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.249+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 shutdown_connections 2024-09-16T00:30:58.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:58.250+0000 7fd86ebf6640 1 -- 172.21.15.78:0/4130394571 wait complete. 2024-09-16T00:30:58.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:58 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4130394571' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:30:58.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:58 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4130394571' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:30:58.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:58 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4130394571' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:30:59.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid eb73a744-73be-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-16T00:30:59.250 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:59.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:59 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/4130394571' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:30:59.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:59 smithi078 ceph-mon[31157]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:30:59.516 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:30:59 smithi078 ceph-mon[31157]: pgmap v891: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:59.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:59 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/4130394571' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:30:59.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:59 smithi193 ceph-mon[35484]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:30:59.680 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:30:59 smithi193 ceph-mon[35484]: pgmap v891: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:59.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:59 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/4130394571' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:30:59.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:59 smithi148 ceph-mon[35599]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:30:59.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:30:59 smithi148 ceph-mon[35599]: pgmap v891: 33 pgs: 33 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:59.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 -- 172.21.15.78:0/2605444743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41036e0 msgr2=0x7f9ee4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 --2- 172.21.15.78:0/2605444743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41036e0 0x7f9ee4107b90 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009930 tx=0x7f9ed802a420 comp rx=0 tx=0).stop 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 -- 172.21.15.78:0/2605444743 shutdown_connections 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 --2- 172.21.15.78:0/2605444743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee41082a0 0x7f9ee410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 --2- 172.21.15.78:0/2605444743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41036e0 0x7f9ee4107b90 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 --2- 172.21.15.78:0/2605444743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee4102d30 0x7f9ee4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 -- 172.21.15.78:0/2605444743 >> 172.21.15.78:0/2605444743 conn(0x7f9ee40fc7d0 msgr2=0x7f9ee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 -- 172.21.15.78:0/2605444743 shutdown_connections 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 -- 172.21.15.78:0/2605444743 wait complete. 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 Processor -- start 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.745+0000 7f9ee9623640 1 -- start start 2024-09-16T00:30:59.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee9623640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4102d30 0x7f9ee4198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee9623640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee41036e0 0x7f9ee4198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee9623640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41082a0 0x7f9ee419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee9623640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41120a0 con 0x7f9ee41082a0 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee9623640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4111f20 con 0x7f9ee4102d30 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee9623640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4112220 con 0x7f9ee41036e0 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41082a0 0x7f9ee419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41082a0 0x7f9ee419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:50210/0 (socket says 172.21.15.78:50210) 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 -- 172.21.15.78:0/3782933750 learned_addr learned my addr 172.21.15.78:0/3782933750 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee2ffd640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4102d30 0x7f9ee4198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee27fc640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee41036e0 0x7f9ee4198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 -- 172.21.15.78:0/3782933750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee41036e0 msgr2=0x7f9ee4198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:59.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee41036e0 0x7f9ee4198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 -- 172.21.15.78:0/3782933750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4102d30 msgr2=0x7f9ee4198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4102d30 0x7f9ee4198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 -- 172.21.15.78:0/3782933750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee419d610 con 0x7f9ee41082a0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee27fc640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee41036e0 0x7f9ee4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee37fe640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41082a0 0x7f9ee419cf30 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00eaa0 tx=0x7f9edc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ecbfff640 1 -- 172.21.15.78:0/3782933750 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00cdb0 con 0x7f9ee41082a0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee419d900 con 0x7f9ee41082a0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ecbfff640 1 -- 172.21.15.78:0/3782933750 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9edc005020 con 0x7f9ee41082a0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ecbfff640 1 -- 172.21.15.78:0/3782933750 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0116b0 con 0x7f9ee41082a0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.746+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a51e0 con 0x7f9ee41082a0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.747+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee406a850 con 0x7f9ee41082a0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.748+0000 7f9ecbfff640 1 -- 172.21.15.78:0/3782933750 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9edc005560 con 0x7f9ee41082a0 2024-09-16T00:30:59.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.749+0000 7f9ecbfff640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:59.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.749+0000 7f9ee2ffd640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:59.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.749+0000 7f9ecbfff640 1 -- 172.21.15.78:0/3782933750 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8475+0+0 (secure 0 0 0) 0x7f9edc0cfbe0 con 0x7f9ee41082a0 2024-09-16T00:30:59.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.750+0000 7f9ee2ffd640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009bd0 tx=0x7f9ed4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:59.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.753+0000 7f9ecbfff640 1 -- 172.21.15.78:0/3782933750 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9edc016030 con 0x7f9ee41082a0 2024-09-16T00:30:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:30:59.920+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9ee410a2b0 con 0x7f9ee41082a0 2024-09-16T00:31:00.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.261+0000 7f9ecbfff640 1 -- 172.21.15.78:0/3782933750 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v94) ==== 178+0+0 (secure 0 0 0) 0x7f9edc09c810 con 0x7f9ee41082a0 2024-09-16T00:31:00.263 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data_ec' removed 2024-09-16T00:31:00.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.264+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ecc0a3240 msgr2=0x7f9ecc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:00.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.264+0000 7f9ee9623640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009bd0 tx=0x7f9ed4005f90 comp rx=0 tx=0).stop 2024-09-16T00:31:00.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.265+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41082a0 msgr2=0x7f9ee419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:00.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.265+0000 7f9ee9623640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41082a0 0x7f9ee419cf30 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00eaa0 tx=0x7f9edc00ef60 comp rx=0 tx=0).stop 2024-09-16T00:31:00.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.265+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 shutdown_connections 2024-09-16T00:31:00.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.265+0000 7f9ee9623640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:6800/119552845,v1:172.21.15.78:6801/119552845] conn(0x7f9ecc0a3240 0x7f9ecc0a5700 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.265+0000 7f9ee9623640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ee41082a0 0x7f9ee419cf30 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.265+0000 7f9ee9623640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ee41036e0 0x7f9ee4198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.265+0000 7f9ee9623640 1 --2- 172.21.15.78:0/3782933750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9ee4102d30 0x7f9ee4198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.265+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 >> 172.21.15.78:0/3782933750 conn(0x7f9ee40fc7d0 msgr2=0x7f9ee4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:00.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.266+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 shutdown_connections 2024-09-16T00:31:00.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-16T00:31:00.266+0000 7f9ee9623640 1 -- 172.21.15.78:0/3782933750 wait complete. 2024-09-16T00:31:00.519 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:00 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3782933750' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:31:00.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:00 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3782933750' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:31:00.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:00 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3782933750' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:31:01.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:31:01.101 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:31:01.136 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:31:01.316 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T00:31:01.317 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:31:01.346 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:31:01.365 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:31:01.382 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T00:31:01.382 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T00:31:01.383 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:31:01.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:01 smithi078 ceph-mon[31157]: from='client.? 172.21.15.78:0/3782933750' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:31:01.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:01 smithi078 ceph-mon[31157]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:31:01.422 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:01 smithi078 ceph-mon[31157]: pgmap v893: 1 pgs: 1 active+clean; 577 KiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:31:01.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:01 smithi193 ceph-mon[35484]: from='client.? 172.21.15.78:0/3782933750' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:31:01.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:01 smithi193 ceph-mon[35484]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:31:01.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:01 smithi193 ceph-mon[35484]: pgmap v893: 1 pgs: 1 active+clean; 577 KiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:31:01.453 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:01 smithi148 ceph-mon[35599]: from='client.? 172.21.15.78:0/3782933750' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:31:01.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:01 smithi148 ceph-mon[35599]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:31:01.454 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:01 smithi148 ceph-mon[35599]: pgmap v893: 1 pgs: 1 active+clean; 577 KiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:31:01.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:01 smithi078 systemd[1]: Stopping Ceph mon.a for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:02.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a[31153]: 2024-09-16T00:31:01.791+0000 7f685282b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:02.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a[31153]: 2024-09-16T00:31:01.791+0000 7f685282b640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T00:31:02.133 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:01 smithi078 podman[122537]: 2024-09-16 00:31:01.824477568 +0000 UTC m=+0.171412292 container died 16361528f3de950291676d5378a55966c5370a41afc2a6b0f56bd00c6fa84411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:31:02.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:02 smithi078 podman[122537]: 2024-09-16 00:31:02.249530331 +0000 UTC m=+0.596465058 container cleanup 16361528f3de950291676d5378a55966c5370a41afc2a6b0f56bd00c6fa84411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T00:31:02.510 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:02 smithi078 bash[122537]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:31:02.769 INFO:journalctl@ceph.mon.a.smithi078.stdout:Sep 16 00:31:02 smithi078 podman[122551]: 2024-09-16 00:31:02.510012085 +0000 UTC m=+0.680145363 container remove 16361528f3de950291676d5378a55966c5370a41afc2a6b0f56bd00c6fa84411 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T00:31:02.816 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T00:31:02.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:02.909 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T00:31:02.909 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-16T00:31:02.909 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.b 2024-09-16T00:31:03.204 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:02 smithi148 systemd[1]: Stopping Ceph mon.b for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:03.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-b[35595]: 2024-09-16T00:31:03.320+0000 7f0054ee6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:03.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-b[35595]: 2024-09-16T00:31:03.320+0000 7f0054ee6640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:31:03.704 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:03 smithi148 podman[70363]: 2024-09-16 00:31:03.432745688 +0000 UTC m=+0.238496480 container died 99365af9c89318611e4dfdbb1261582ee1507eba4b3745e4a3730650f8302ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T00:31:04.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:03 smithi148 podman[70363]: 2024-09-16 00:31:03.845579794 +0000 UTC m=+0.651330574 container cleanup 99365af9c89318611e4dfdbb1261582ee1507eba4b3745e4a3730650f8302ffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T00:31:04.122 INFO:journalctl@ceph.mon.b.smithi148.stdout:Sep 16 00:31:03 smithi148 bash[70363]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-b 2024-09-16T00:31:04.245 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T00:31:04.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:04.337 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-16T00:31:04.338 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-16T00:31:04.338 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.c 2024-09-16T00:31:04.678 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:04 smithi193 systemd[1]: Stopping Ceph mon.c for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:04.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-c[35476]: 2024-09-16T00:31:04.676+0000 7f5704f95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:04.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-c[35476]: 2024-09-16T00:31:04.676+0000 7f5704f95640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:31:04.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:04 smithi193 podman[71949]: 2024-09-16 00:31:04.83157827 +0000 UTC m=+0.276778889 container died 3076c2d30b7bf21cdedcbe3e2d507ab2f4593d1bd667cfbd56e403130c08bdea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T00:31:05.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:05 smithi193 podman[71949]: 2024-09-16 00:31:05.223337833 +0000 UTC m=+0.668538451 container cleanup 3076c2d30b7bf21cdedcbe3e2d507ab2f4593d1bd667cfbd56e403130c08bdea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-16T00:31:05.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 16 00:31:05 smithi193 bash[71949]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mon-c 2024-09-16T00:31:05.655 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T00:31:05.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:05.707 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-16T00:31:05.707 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T00:31:05.707 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:31:06.019 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:05 smithi078 systemd[1]: Stopping Ceph mgr.x for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:06.519 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:06 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:31:06.128+0000 7fe6909f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:06.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:06 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x[31385]: 2024-09-16T00:31:06.128+0000 7fe6909f3640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-16T00:31:06.520 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:06 smithi078 podman[122666]: 2024-09-16 00:31:06.160727136 +0000 UTC m=+0.263856072 container died f1c3878bc64b3ca51ff83b23473e37c1c9762987ba7a0ce4fcbf4ea76007a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T00:31:07.019 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:06 smithi078 podman[122666]: 2024-09-16 00:31:06.789478452 +0000 UTC m=+0.892607375 container cleanup f1c3878bc64b3ca51ff83b23473e37c1c9762987ba7a0ce4fcbf4ea76007a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:07.019 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:06 smithi078 bash[122666]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x 2024-09-16T00:31:07.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T00:31:07.330 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:07 smithi078 podman[122679]: 2024-09-16 00:31:07.06680199 +0000 UTC m=+0.901387852 container remove f1c3878bc64b3ca51ff83b23473e37c1c9762987ba7a0ce4fcbf4ea76007a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:31:07.330 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:07 smithi078 systemd[1]: ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-16T00:31:07.330 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:07 smithi078 systemd[1]: Stopped Ceph mgr.x for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:31:07.330 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Sep 16 00:31:07 smithi078 systemd[1]: ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.x.service: Consumed 47.719s CPU time. 2024-09-16T00:31:07.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:07.346 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T00:31:07.346 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T00:31:07.346 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.y 2024-09-16T00:31:07.688 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:31:07 smithi148 systemd[1]: Stopping Ceph mgr.y for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:07.952 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:31:07 smithi148 podman[70483]: 2024-09-16 00:31:07.688454357 +0000 UTC m=+0.145771491 container died 151c57f5ff20e9f4c1a4f6b272a11a506bf7aa1e2d6571b2351979fcd2259897 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:08.204 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:31:08 smithi148 podman[70483]: 2024-09-16 00:31:08.068220034 +0000 UTC m=+0.525537168 container cleanup 151c57f5ff20e9f4c1a4f6b272a11a506bf7aa1e2d6571b2351979fcd2259897 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:31:08.204 INFO:journalctl@ceph.mgr.y.smithi148.stdout:Sep 16 00:31:08 smithi148 bash[70483]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-y 2024-09-16T00:31:08.506 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T00:31:08.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:08.540 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T00:31:08.540 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-16T00:31:08.540 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.z 2024-09-16T00:31:08.882 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:31:08 smithi193 systemd[1]: Stopping Ceph mgr.z for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:09.179 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:31:08 smithi193 podman[72075]: 2024-09-16 00:31:08.882254607 +0000 UTC m=+0.141489586 container died e3d4387421df7f64fec679205b305d35504a7c441915543cb058199afd3cbb53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T00:31:09.564 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:31:09 smithi193 podman[72075]: 2024-09-16 00:31:09.270989522 +0000 UTC m=+0.530224500 container cleanup e3d4387421df7f64fec679205b305d35504a7c441915543cb058199afd3cbb53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:09.564 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 16 00:31:09 smithi193 bash[72075]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mgr-z 2024-09-16T00:31:09.703 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-16T00:31:09.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:09.737 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-16T00:31:09.737 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T00:31:09.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.0 2024-09-16T00:31:10.019 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:09 smithi078 systemd[1]: Stopping Ceph osd.0 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:10.519 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:10 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0[45158]: 2024-09-16T00:31:10.052+0000 7f684f2c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:10.519 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:10 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0[45158]: 2024-09-16T00:31:10.052+0000 7f684f2c5640 -1 osd.0 94 *** Got signal Terminated *** 2024-09-16T00:31:10.520 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:10 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0[45158]: 2024-09-16T00:31:10.052+0000 7f684f2c5640 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:15.930 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:15 smithi078 podman[122787]: 2024-09-16 00:31:15.61711824 +0000 UTC m=+5.690483009 container died 6c698f6b413afc6e04342e4ab0421070ef0a7c1a1d9f7c16517740688b009354 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T00:31:16.224 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:16 smithi078 podman[122787]: 2024-09-16 00:31:16.03444307 +0000 UTC m=+6.107807838 container cleanup 6c698f6b413afc6e04342e4ab0421070ef0a7c1a1d9f7c16517740688b009354 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T00:31:16.224 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:16 smithi078 bash[122787]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0 2024-09-16T00:31:16.519 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:16 smithi078 podman[122803]: 2024-09-16 00:31:16.313816323 +0000 UTC m=+0.696323473 container remove 6c698f6b413afc6e04342e4ab0421070ef0a7c1a1d9f7c16517740688b009354 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e) 2024-09-16T00:31:17.109 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:16 smithi078 podman[122892]: 2024-09-16 00:31:16.704535487 +0000 UTC m=+0.016092452 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:17.109 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:16 smithi078 podman[122892]: 2024-09-16 00:31:16.831644922 +0000 UTC m=+0.143201922 container create 78faecdd586c1d52f7c9e44f3cd82c6c81de5f248a9221a0d6da6bb750802c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T00:31:17.109 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:17 smithi078 podman[122892]: 2024-09-16 00:31:17.109407672 +0000 UTC m=+0.420964631 container init 78faecdd586c1d52f7c9e44f3cd82c6c81de5f248a9221a0d6da6bb750802c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T00:31:17.373 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:17 smithi078 podman[122892]: 2024-09-16 00:31:17.112932539 +0000 UTC m=+0.424489500 container start 78faecdd586c1d52f7c9e44f3cd82c6c81de5f248a9221a0d6da6bb750802c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T00:31:17.373 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:17 smithi078 podman[122892]: 2024-09-16 00:31:17.176088417 +0000 UTC m=+0.487645391 container attach 78faecdd586c1d52f7c9e44f3cd82c6c81de5f248a9221a0d6da6bb750802c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:31:17.746 INFO:journalctl@ceph.osd.0.smithi078.stdout:Sep 16 00:31:17 smithi078 podman[122916]: 2024-09-16 00:31:17.398753741 +0000 UTC m=+0.017783484 container died 78faecdd586c1d52f7c9e44f3cd82c6c81de5f248a9221a0d6da6bb750802c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T00:31:18.063 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T00:31:18.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:18.102 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T00:31:18.102 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T00:31:18.103 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.1 2024-09-16T00:31:18.440 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:18 smithi148 systemd[1]: Stopping Ceph osd.1 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:18.704 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:18 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1[40051]: 2024-09-16T00:31:18.439+0000 7ffaa37c2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:18.704 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:18 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1[40051]: 2024-09-16T00:31:18.439+0000 7ffaa37c2640 -1 osd.1 94 *** Got signal Terminated *** 2024-09-16T00:31:18.704 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:18 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1[40051]: 2024-09-16T00:31:18.439+0000 7ffaa37c2640 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:24.204 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:23 smithi148 podman[70617]: 2024-09-16 00:31:23.940838497 +0000 UTC m=+5.643876279 container died de6bab0137e00754aab16324200236c150f9f2034c56616efc3af89b73deb12f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T00:31:24.591 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:24 smithi148 podman[70617]: 2024-09-16 00:31:24.323054882 +0000 UTC m=+6.026092659 container cleanup de6bab0137e00754aab16324200236c150f9f2034c56616efc3af89b73deb12f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e) 2024-09-16T00:31:24.591 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:24 smithi148 bash[70617]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1 2024-09-16T00:31:24.901 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:24 smithi148 podman[70632]: 2024-09-16 00:31:24.591101595 +0000 UTC m=+0.649693951 container remove de6bab0137e00754aab16324200236c150f9f2034c56616efc3af89b73deb12f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T00:31:25.186 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:24 smithi148 podman[70722]: 2024-09-16 00:31:24.804017972 +0000 UTC m=+0.015970540 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:25.186 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:24 smithi148 podman[70722]: 2024-09-16 00:31:24.933737663 +0000 UTC m=+0.145690228 container create 086fa2a363bb9b0c1f1dc30ec906fc8cd5f01a0e30cfbec477a7e5424770df38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:31:25.186 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:25 smithi148 podman[70722]: 2024-09-16 00:31:25.186459104 +0000 UTC m=+0.398411668 container init 086fa2a363bb9b0c1f1dc30ec906fc8cd5f01a0e30cfbec477a7e5424770df38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T00:31:25.454 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:25 smithi148 podman[70722]: 2024-09-16 00:31:25.190267617 +0000 UTC m=+0.402220184 container start 086fa2a363bb9b0c1f1dc30ec906fc8cd5f01a0e30cfbec477a7e5424770df38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:25.454 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:25 smithi148 podman[70722]: 2024-09-16 00:31:25.286563027 +0000 UTC m=+0.498515675 container attach 086fa2a363bb9b0c1f1dc30ec906fc8cd5f01a0e30cfbec477a7e5424770df38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T00:31:25.824 INFO:journalctl@ceph.osd.1.smithi148.stdout:Sep 16 00:31:25 smithi148 podman[70722]: 2024-09-16 00:31:25.455903382 +0000 UTC m=+0.667855945 container died 086fa2a363bb9b0c1f1dc30ec906fc8cd5f01a0e30cfbec477a7e5424770df38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T00:31:26.152 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T00:31:26.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:26.193 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T00:31:26.193 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T00:31:26.194 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.2 2024-09-16T00:31:26.592 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:26 smithi193 systemd[1]: Stopping Ceph osd.2 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:26.929 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2[40459]: 2024-09-16T00:31:26.592+0000 7f47de00b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:26.929 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2[40459]: 2024-09-16T00:31:26.592+0000 7f47de00b640 -1 osd.2 94 *** Got signal Terminated *** 2024-09-16T00:31:26.929 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2[40459]: 2024-09-16T00:31:26.592+0000 7f47de00b640 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:32.551 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:32 smithi193 podman[72214]: 2024-09-16 00:31:32.240277554 +0000 UTC m=+5.850848925 container died b68b92a398e691f4a49fc4dd9123f11c9c499c6b550ccaff9dfe928f74f2ed53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:31:32.918 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:32 smithi193 podman[72214]: 2024-09-16 00:31:32.642423226 +0000 UTC m=+6.252994599 container cleanup b68b92a398e691f4a49fc4dd9123f11c9c499c6b550ccaff9dfe928f74f2ed53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T00:31:32.919 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:32 smithi193 bash[72214]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2 2024-09-16T00:31:33.179 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:32 smithi193 podman[72232]: 2024-09-16 00:31:32.919194354 +0000 UTC m=+0.674358621 container remove b68b92a398e691f4a49fc4dd9123f11c9c499c6b550ccaff9dfe928f74f2ed53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T00:31:33.548 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:33 smithi193 podman[72320]: 2024-09-16 00:31:33.133033159 +0000 UTC m=+0.015879160 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:33.548 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:33 smithi193 podman[72320]: 2024-09-16 00:31:33.270156296 +0000 UTC m=+0.153002292 container create 8025ae5cd4d4c7e474de7968dd5f068f740088f933cec37d8bda04771020cc49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:33.930 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:33 smithi193 podman[72320]: 2024-09-16 00:31:33.548129819 +0000 UTC m=+0.430975817 container init 8025ae5cd4d4c7e474de7968dd5f068f740088f933cec37d8bda04771020cc49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T00:31:33.930 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:33 smithi193 podman[72320]: 2024-09-16 00:31:33.55175447 +0000 UTC m=+0.434600463 container start 8025ae5cd4d4c7e474de7968dd5f068f740088f933cec37d8bda04771020cc49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True) 2024-09-16T00:31:33.930 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:33 smithi193 podman[72320]: 2024-09-16 00:31:33.648208396 +0000 UTC m=+0.531054393 container attach 8025ae5cd4d4c7e474de7968dd5f068f740088f933cec37d8bda04771020cc49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T00:31:33.930 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 16 00:31:33 smithi193 podman[72320]: 2024-09-16 00:31:33.794455553 +0000 UTC m=+0.677301547 container died 8025ae5cd4d4c7e474de7968dd5f068f740088f933cec37d8bda04771020cc49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T00:31:34.474 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T00:31:34.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:34.517 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T00:31:34.517 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T00:31:34.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.3 2024-09-16T00:31:34.833 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:34 smithi078 systemd[1]: Stopping Ceph osd.3 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:35.093 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:34 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:31:34.832+0000 7fb8f3f4c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:35.093 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:34 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:31:34.832+0000 7fb8f3f4c640 -1 osd.3 94 *** Got signal Terminated *** 2024-09-16T00:31:35.093 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:34 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:31:34.832+0000 7fb8f3f4c640 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:38.269 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:37 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:31:37.808+0000 7fb8f0562640 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:11.554154+0000 front 2024-09-16T00:31:11.554129+0000 (oldest deadline 2024-09-16T00:31:36.854277+0000) 2024-09-16T00:31:38.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:38.360+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:38.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:38 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:38.268+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:39.269 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:38 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:31:38.812+0000 7fb8f0562640 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:11.554154+0000 front 2024-09-16T00:31:11.554129+0000 (oldest deadline 2024-09-16T00:31:36.854277+0000) 2024-09-16T00:31:39.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:39.370+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:39.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:39 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:39.220+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:40.250 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:39 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3[52006]: 2024-09-16T00:31:39.775+0000 7fb8f0562640 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:11.554154+0000 front 2024-09-16T00:31:11.554129+0000 (oldest deadline 2024-09-16T00:31:36.854277+0000) 2024-09-16T00:31:40.454 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:40 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:40.183+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:40.519 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:40 smithi078 podman[123006]: 2024-09-16 00:31:40.367718017 +0000 UTC m=+5.655486809 container died e97064930bcb608448647ec1802dacf0d78f639a460a5dbe9e02d0638cf96a50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-16T00:31:40.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:40 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:40.249+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:40.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:40 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:40.388+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:41.019 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:40 smithi078 podman[123006]: 2024-09-16 00:31:40.866589413 +0000 UTC m=+6.154358193 container cleanup e97064930bcb608448647ec1802dacf0d78f639a460a5dbe9e02d0638cf96a50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T00:31:41.020 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:40 smithi078 bash[123006]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3 2024-09-16T00:31:41.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:41 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:41.215+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:41.520 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:41 smithi078 podman[123033]: 2024-09-16 00:31:41.143661474 +0000 UTC m=+0.778165078 container remove e97064930bcb608448647ec1802dacf0d78f639a460a5dbe9e02d0638cf96a50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:41.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:41 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:41.386+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:41.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:41 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:41.221+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:41.930 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:41 smithi078 podman[123124]: 2024-09-16 00:31:41.523452785 +0000 UTC m=+0.015946571 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:41.930 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:41 smithi078 podman[123124]: 2024-09-16 00:31:41.655665509 +0000 UTC m=+0.148159287 container create 89e7990a90acef44909658cf695c2775541d3fff3b5564ff0dde9c9000576c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-16T00:31:41.930 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:41 smithi078 podman[123124]: 2024-09-16 00:31:41.930504945 +0000 UTC m=+0.422998726 container init 89e7990a90acef44909658cf695c2775541d3fff3b5564ff0dde9c9000576c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:42.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:42 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:42.168+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:42.270 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:41 smithi078 podman[123124]: 2024-09-16 00:31:41.934134083 +0000 UTC m=+0.426627861 container start 89e7990a90acef44909658cf695c2775541d3fff3b5564ff0dde9c9000576c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T00:31:42.270 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:41 smithi078 podman[123124]: 2024-09-16 00:31:41.988881668 +0000 UTC m=+0.481375457 container attach 89e7990a90acef44909658cf695c2775541d3fff3b5564ff0dde9c9000576c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:31:42.270 INFO:journalctl@ceph.osd.3.smithi078.stdout:Sep 16 00:31:42 smithi078 podman[123124]: 2024-09-16 00:31:42.125452617 +0000 UTC m=+0.617946395 container died 89e7990a90acef44909658cf695c2775541d3fff3b5564ff0dde9c9000576c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T00:31:42.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:42.369+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:42.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:42 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:42.270+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:42.745 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T00:31:42.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:42.784 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T00:31:42.784 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T00:31:42.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.4 2024-09-16T00:31:43.101 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:42 smithi148 systemd[1]: Stopping Ceph osd.4 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:43.454 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:43 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:43.299+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:43.454 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:43 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4[46507]: 2024-09-16T00:31:43.101+0000 7f5253af2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:43.454 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:43 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4[46507]: 2024-09-16T00:31:43.101+0000 7f5253af2640 -1 osd.4 94 *** Got signal Terminated *** 2024-09-16T00:31:43.454 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:43 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4[46507]: 2024-09-16T00:31:43.101+0000 7f5253af2640 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:43.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:43 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:43.121+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:43.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:43.415+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:44.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:44 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:44.118+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:44.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:44 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:44.328+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:44.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:44.444+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:45.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:44 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:44.754+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:45.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:45 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:45.161+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:45.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:45 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:45.303+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:45.749 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:45 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:45.482+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:46.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:45 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:45.752+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:46.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:45 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:45.748+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:46.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:46 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:46.120+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:46.605 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:46 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:46.281+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:46.929 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:46 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:46.733+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:46.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:46 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:46.493+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:47.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:46 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:46.732+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:47.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:47 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:47.093+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:47.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:47 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:47.240+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:47.760 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:47 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:47.462+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:48.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:47 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:47.756+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:48.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:47 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:47.760+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:48.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:48 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:48.100+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:48.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:48 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:48.100+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:48.623 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:48 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:48.283+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:48.623 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:48 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:48.283+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:48.623 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:48 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:48.492+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:48.680 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:48 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:48.224+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:48.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:48 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:48.441+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:48.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:48 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:48.441+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:48.920 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:48 smithi148 podman[70825]: 2024-09-16 00:31:48.623041363 +0000 UTC m=+5.637768245 container died 3e355e46559c71f00e8c70ce0d0c253dc7ebbf615bf47fe085fff994564f0cfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:49.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:48 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:48.720+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:49.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:48 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:48.775+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:49.204 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:49 smithi148 podman[70825]: 2024-09-16 00:31:49.034969178 +0000 UTC m=+6.049696046 container cleanup 3e355e46559c71f00e8c70ce0d0c253dc7ebbf615bf47fe085fff994564f0cfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:31:49.204 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:49 smithi148 bash[70825]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4 2024-09-16T00:31:49.469 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:49 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:49.199+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:49.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:49 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:49.127+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:49.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:49 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:49.127+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:49.704 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:49 smithi148 podman[70841]: 2024-09-16 00:31:49.269720109 +0000 UTC m=+0.649076897 container remove 3e355e46559c71f00e8c70ce0d0c253dc7ebbf615bf47fe085fff994564f0cfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T00:31:49.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:49 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:49.476+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:49.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:49 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:49.317+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:49.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:49 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:49.317+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:49.749 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:49 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:49.469+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:49.750 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:49 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:49.469+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:49.964 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:49 smithi148 podman[70929]: 2024-09-16 00:31:49.606902043 +0000 UTC m=+0.015825144 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:49.964 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:49 smithi148 podman[70929]: 2024-09-16 00:31:49.729429758 +0000 UTC m=+0.138352840 container create 8c7e34fc1170fe502aa8dc1eeed3f7d2ea670fa837b124faa96a6f129664b3e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T00:31:50.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:49 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:49.724+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:50.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:49 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:49.749+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:50.308 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:49 smithi148 podman[70929]: 2024-09-16 00:31:49.964509725 +0000 UTC m=+0.373432806 container init 8c7e34fc1170fe502aa8dc1eeed3f7d2ea670fa837b124faa96a6f129664b3e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:50.308 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:49 smithi148 podman[70929]: 2024-09-16 00:31:49.968110936 +0000 UTC m=+0.377034019 container start 8c7e34fc1170fe502aa8dc1eeed3f7d2ea670fa837b124faa96a6f129664b3e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:31:50.308 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:50 smithi148 podman[70929]: 2024-09-16 00:31:50.073032707 +0000 UTC m=+0.481955790 container attach 8c7e34fc1170fe502aa8dc1eeed3f7d2ea670fa837b124faa96a6f129664b3e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T00:31:50.309 INFO:journalctl@ceph.osd.4.smithi148.stdout:Sep 16 00:31:50 smithi148 podman[70929]: 2024-09-16 00:31:50.161431182 +0000 UTC m=+0.570354274 container died 8c7e34fc1170fe502aa8dc1eeed3f7d2ea670fa837b124faa96a6f129664b3e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:31:50.492 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:50 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:50.197+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:50.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:50 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:50.106+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:50.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:50 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:50.106+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:50.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:50 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:50.473+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:50.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:50 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:50.307+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:50.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:50 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:50.307+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:50.751 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:50 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:50.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:50.751 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:50 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:50.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:50.861 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T00:31:50.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:50.906 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T00:31:50.906 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T00:31:50.906 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.5 2024-09-16T00:31:51.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:50 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:50.690+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:51.164 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:50 smithi193 systemd[1]: Stopping Ceph osd.5 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:51.164 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:50 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:50.751+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:51.429 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:51 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:51.163+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:51.429 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:51 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:51.305+0000 7f1b803fd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:51.430 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:51 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:51.305+0000 7f1b803fd640 -1 osd.5 94 *** Got signal Terminated *** 2024-09-16T00:31:51.430 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:51 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:51.305+0000 7f1b803fd640 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:51.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:51 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:51.078+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:51.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:51 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:51.078+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:51.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:51 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:51.454+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:51.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:51 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:51.322+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:51.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:51 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:51.322+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:51.929 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:51 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:51.716+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:51.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:51 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:51.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:51.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:51 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:51.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:52.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:51 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:51.702+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:52.429 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:52.120+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:52.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:52.086+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:52.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:52.086+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:52.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:52.501+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:52.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:52.350+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:52.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:52 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:52.350+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:52.929 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:52.701+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:52.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:52.484+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:52.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:52 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:52.484+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:53.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:52 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:52.750+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:53.429 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:53 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:53.165+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:53.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:53 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:53.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:53.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:53 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:53.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:53.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:53 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:53.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:31:53.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:53 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:53.516+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:53.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:53 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:53.360+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:53.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:53 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:53.360+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:53.929 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:53 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:53.682+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:53.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:53 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:53.442+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:53.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:53 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:53.442+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:54.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:53 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:53.751+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:54.417 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:54 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:54.149+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:54.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:54 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:54.076+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:54.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:54 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:54.076+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:54.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:54 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:54.076+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:31:54.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:54 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:54.418+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:54.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:54 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:54.418+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:54.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:54 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:54.476+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:54.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:54 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:54.476+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:31:54.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:54 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:54.370+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:54.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:54 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:54.370+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:55.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:54 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:54.728+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:55.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:54 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:54.713+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:55.417 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:55 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:55.162+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:55.417 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:55 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:55.162+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.892944+0000 front 2024-09-16T00:31:29.892998+0000 (oldest deadline 2024-09-16T00:31:54.493455+0000) 2024-09-16T00:31:55.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:55 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:55.066+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:55.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:55 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:55.066+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:55.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:55 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:55.066+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:31:55.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:55 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:55.414+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:55.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:55 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:55.414+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:55.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:55 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:55.414+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:31:55.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:55 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:55.516+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:55.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:55 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:55.516+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:31:55.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:55 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:55.419+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:55.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:55 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:55.419+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:55.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:55 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:55.419+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:31:56.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:55 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:55.735+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:56.153 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:55 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:55.712+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:56.154 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:55 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:55.712+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:31:56.429 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:56 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:56.153+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:18.191344+0000 front 2024-09-16T00:31:18.191312+0000 (oldest deadline 2024-09-16T00:31:48.192053+0000) 2024-09-16T00:31:56.429 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:56 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5[47397]: 2024-09-16T00:31:56.153+0000 7f1b78a0b640 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.892944+0000 front 2024-09-16T00:31:29.892998+0000 (oldest deadline 2024-09-16T00:31:54.493455+0000) 2024-09-16T00:31:56.429 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:56 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:56.388+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:56.430 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:56 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:56.388+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:56.430 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:56 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:56.388+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:31:56.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:56 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:56.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:56.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:56 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:56.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:56.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:56 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:56.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:31:56.711 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:56 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:56.546+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:56.711 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:56 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:56.546+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:31:56.712 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:56 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:56.392+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:56.712 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:56 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:56.392+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:56.712 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:56 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:56.392+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:31:57.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:56 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:56.735+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:57.161 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:56 smithi193 podman[72436]: 2024-09-16 00:31:56.861415384 +0000 UTC m=+5.753806782 container died 0727ae72ae9f901943d054bd2a2dc828fd49520e4960df0a5742185561976efb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T00:31:57.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:56 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:56.685+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:57.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:56 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:56.685+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:31:57.413 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:57 smithi193 podman[72436]: 2024-09-16 00:31:57.264546943 +0000 UTC m=+6.156938350 container cleanup 0727ae72ae9f901943d054bd2a2dc828fd49520e4960df0a5742185561976efb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:31:57.413 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:57 smithi193 bash[72436]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5 2024-09-16T00:31:57.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:57.044+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:57.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:57.044+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:57.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:57.044+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:31:57.679 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:57 smithi193 podman[72453]: 2024-09-16 00:31:57.524477024 +0000 UTC m=+0.661991817 container remove 0727ae72ae9f901943d054bd2a2dc828fd49520e4960df0a5742185561976efb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T00:31:57.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:57 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:57.412+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:57.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:57 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:57.412+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:57.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:57 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:57.412+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:31:57.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:57 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:57.510+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:57.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:57 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:57.510+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:31:57.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:57 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:57.376+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:57.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:57 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:57.376+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:57.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:57 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:57.376+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:31:58.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:57.718+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:58.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:57 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:57.718+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.905487+0000 front 2024-09-16T00:31:29.905552+0000 (oldest deadline 2024-09-16T00:31:56.906142+0000) 2024-09-16T00:31:58.057 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:57 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:57.702+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:58.058 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:57 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:57.702+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:31:58.058 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:57 smithi193 podman[72543]: 2024-09-16 00:31:57.785669312 +0000 UTC m=+0.015804246 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:31:58.058 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:57 smithi193 podman[72543]: 2024-09-16 00:31:57.917134251 +0000 UTC m=+0.147269178 container create ffb6c881f1b4f8dd56b5fe257b433416d1a941317cf699e6b33a5f2027e22b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-16T00:31:58.388 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:58 smithi193 podman[72543]: 2024-09-16 00:31:58.194986404 +0000 UTC m=+0.425121332 container init ffb6c881f1b4f8dd56b5fe257b433416d1a941317cf699e6b33a5f2027e22b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T00:31:58.389 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:58 smithi193 podman[72543]: 2024-09-16 00:31:58.198388003 +0000 UTC m=+0.428522930 container start ffb6c881f1b4f8dd56b5fe257b433416d1a941317cf699e6b33a5f2027e22b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T00:31:58.389 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:58 smithi193 podman[72543]: 2024-09-16 00:31:58.294936449 +0000 UTC m=+0.525071389 container attach ffb6c881f1b4f8dd56b5fe257b433416d1a941317cf699e6b33a5f2027e22b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:31:58.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:58.051+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:58.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:58.051+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:58.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:58.051+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:31:58.679 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 00:31:58 smithi193 podman[72543]: 2024-09-16 00:31:58.388981593 +0000 UTC m=+0.619116523 container died ffb6c881f1b4f8dd56b5fe257b433416d1a941317cf699e6b33a5f2027e22b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:31:58.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:58 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:58.427+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:58.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:58 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:58.427+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:58.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:58 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:58.427+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:31:58.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:58 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:58.541+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:58.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:58 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:58.541+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:31:58.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:58 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:58.397+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:58.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:58 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:58.397+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:58.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:58 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:58.397+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:31:59.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:58.759+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:31:59.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:58 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:58.759+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.905487+0000 front 2024-09-16T00:31:29.905552+0000 (oldest deadline 2024-09-16T00:31:56.906142+0000) 2024-09-16T00:31:59.049 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:58 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:58.709+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:59.049 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:58 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:58.709+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:31:59.068 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T00:31:59.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:31:59.109 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T00:31:59.110 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T00:31:59.110 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.6 2024-09-16T00:31:59.269 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:59 smithi078 systemd[1]: Stopping Ceph osd.6 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:31:59.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:59.019+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:31:59.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:59.019+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:31:59.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:59.019+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:31:59.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:59 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:59.530+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:31:59.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:31:59 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:31:59.530+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:31:59.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:59 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:59.384+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:31:59.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:59 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:59.384+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:31:59.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:31:59 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:31:59.384+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:31:59.769 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:59.419+0000 7f58cd45e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:31:59.769 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:59.419+0000 7f58cd45e640 -1 osd.6 94 *** Got signal Terminated *** 2024-09-16T00:31:59.769 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:59.419+0000 7f58cd45e640 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:31:59.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:59 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:59.463+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:31:59.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:59 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:59.463+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:31:59.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:31:59 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:31:59.463+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:31:59.930 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:59 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:59.702+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:31:59.930 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:31:59 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:31:59.702+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:00.269 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:59.798+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:32:00.269 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:31:59.798+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.905487+0000 front 2024-09-16T00:31:29.905552+0000 (oldest deadline 2024-09-16T00:31:56.906142+0000) 2024-09-16T00:32:00.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:59.991+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:00.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:59.991+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:00.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:31:59 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:31:59.991+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:00.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:00.428+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:00.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:00.428+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:00.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:00.428+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:00.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:00 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:00.502+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:00.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:00 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:00.502+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:00.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:00 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:00.502+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:00.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:00 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:00.398+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:00.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:00 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:00.398+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:00.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:00 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:00.398+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:01.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:00.681+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:01.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:00 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:00.681+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:01.269 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:00.796+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:32:01.269 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:00.796+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.905487+0000 front 2024-09-16T00:31:29.905552+0000 (oldest deadline 2024-09-16T00:31:56.906142+0000) 2024-09-16T00:32:01.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:01.032+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:01.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:01.032+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:01.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:01.032+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:01.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:01 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:01.424+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:01.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:01 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:01.424+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:01.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:01 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:01.424+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:01.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:01 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:01.460+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:01.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:01 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:01.460+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:01.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:01 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:01.460+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:01.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:01 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:01.395+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:01.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:01 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:01.395+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:01.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:01 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:01.395+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:02.040 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:01 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:01.715+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:02.040 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:01 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:01.715+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:02.040 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:01 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:01.716+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:02.047 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:01.775+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:32:02.047 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:01 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:01.775+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.905487+0000 front 2024-09-16T00:31:29.905552+0000 (oldest deadline 2024-09-16T00:31:56.906142+0000) 2024-09-16T00:32:02.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:02 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:02.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:02.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:02 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:02.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:02.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:02 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:02.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:02.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:02 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:02.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:02.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:02.375+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:02.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:02.375+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:02.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:02.375+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:02.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:02.439+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:02.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:02.439+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:02.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:02.439+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:02.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:02.439+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:02.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:02.439+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:02.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:02.439+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:02.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:02 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:02.439+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:03.080 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:02 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:02.798+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:32:03.080 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:02 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:02.798+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.905487+0000 front 2024-09-16T00:31:29.905552+0000 (oldest deadline 2024-09-16T00:31:56.906142+0000) 2024-09-16T00:32:03.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:02.714+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:03.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:02.714+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:03.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:02 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:02.714+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:03.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:03 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:03.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:03.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:03 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:03.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:03.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:03 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:03.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:03.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:03 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:03.080+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:03.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:03.415+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:03.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:03.415+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:03.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:03.415+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:03.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:03.429+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:03.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:03.429+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:03.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:03.429+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:03.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:03.429+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:03.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:03.429+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:03.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:03.429+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:03.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:03 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:03.429+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:04.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:03.741+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:04.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:03.741+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:04.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:03 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:03.741+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:04.269 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:03 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:03.795+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.503506+0000 front 2024-09-16T00:31:22.503410+0000 (oldest deadline 2024-09-16T00:31:44.203301+0000) 2024-09-16T00:32:04.270 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:03 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:03.795+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.905487+0000 front 2024-09-16T00:31:29.905552+0000 (oldest deadline 2024-09-16T00:31:56.906142+0000) 2024-09-16T00:32:04.270 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:03 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6[58773]: 2024-09-16T00:32:03.795+0000 7f58c9a74640 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:36.906440+0000 front 2024-09-16T00:31:36.906488+0000 (oldest deadline 2024-09-16T00:32:02.806625+0000) 2024-09-16T00:32:04.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:04.032+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:04.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:04.032+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:04.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:04.032+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:04.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:04 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:04.032+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:04.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:04.369+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:04.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:04.369+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:04.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:04.369+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:04.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:04.382+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:04.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:04.382+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:04.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:04.382+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:04.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:04.442+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:04.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:04.442+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:04.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:04.442+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:04.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:04.442+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:05.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:04.738+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:05.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:04.738+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:05.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:04 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:04.738+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:05.237 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:04 smithi078 podman[123227]: 2024-09-16 00:32:04.948115786 +0000 UTC m=+5.642036842 container died a92f2939fed224f7d5d0b8c57983be35f56b0640a577df7216276e2189713bbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T00:32:05.238 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:05 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:05.026+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:05.238 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:05 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:05.026+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:05.238 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:05 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:05.026+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:05.238 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:05 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:05.026+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:05.520 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:05 smithi078 podman[123227]: 2024-09-16 00:32:05.436900055 +0000 UTC m=+6.130821110 container cleanup a92f2939fed224f7d5d0b8c57983be35f56b0640a577df7216276e2189713bbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:32:05.520 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:05 smithi078 bash[123227]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6 2024-09-16T00:32:05.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:05 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:05.358+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:05.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:05 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:05.358+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:05.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:05 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:05.358+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:05.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:05.350+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:05.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:05.350+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:05.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:05.350+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:05.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:05.438+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:05.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:05.438+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:05.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:05.438+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:05.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:05 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:05.438+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:06.019 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:05 smithi078 podman[123242]: 2024-09-16 00:32:05.717324722 +0000 UTC m=+0.765308521 container remove a92f2939fed224f7d5d0b8c57983be35f56b0640a577df7216276e2189713bbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-16T00:32:06.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:05 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:05.724+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:06.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:05 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:05.724+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:06.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:05 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:05.725+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:06.291 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:06 smithi078 podman[123342]: 2024-09-16 00:32:05.954076849 +0000 UTC m=+0.015986939 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:06.291 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:06 smithi078 podman[123342]: 2024-09-16 00:32:06.064311688 +0000 UTC m=+0.126221767 container create 0dcfba629ec0436c00db85943ad4557a0cae3c31bc7152d335bf04a5a4fcb6a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True) 2024-09-16T00:32:06.291 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:06 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:06.026+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:06.292 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:06 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:06.026+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:06.292 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:06 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:06.026+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:06.292 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:06 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:06.026+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:06.681 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:06 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:06.327+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:06.681 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:06 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:06.327+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:06.681 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:06 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:06.327+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:06.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:06.311+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:06.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:06.311+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:06.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:06.311+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:06.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:06.441+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:06.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:06.441+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:06.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:06.441+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:06.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:06 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:06.441+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:06.769 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:06 smithi078 podman[123342]: 2024-09-16 00:32:06.291626839 +0000 UTC m=+0.353536916 container init 0dcfba629ec0436c00db85943ad4557a0cae3c31bc7152d335bf04a5a4fcb6a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T00:32:06.770 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:06 smithi078 podman[123342]: 2024-09-16 00:32:06.295183526 +0000 UTC m=+0.357093606 container start 0dcfba629ec0436c00db85943ad4557a0cae3c31bc7152d335bf04a5a4fcb6a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e) 2024-09-16T00:32:06.770 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:06 smithi078 podman[123342]: 2024-09-16 00:32:06.383328181 +0000 UTC m=+0.445238281 container attach 0dcfba629ec0436c00db85943ad4557a0cae3c31bc7152d335bf04a5a4fcb6a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:32:06.770 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:06 smithi078 podman[123342]: 2024-09-16 00:32:06.488231724 +0000 UTC m=+0.550141814 container died 0dcfba629ec0436c00db85943ad4557a0cae3c31bc7152d335bf04a5a4fcb6a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T00:32:07.053 INFO:journalctl@ceph.osd.6.smithi078.stdout:Sep 16 00:32:07 smithi078 podman[123367]: 2024-09-16 00:32:07.037003978 +0000 UTC m=+0.542636615 container remove 0dcfba629ec0436c00db85943ad4557a0cae3c31bc7152d335bf04a5a4fcb6a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:32:07.058 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T00:32:07.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:32:07.100 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T00:32:07.100 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T00:32:07.100 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.7 2024-09-16T00:32:07.181 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:06 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:06.707+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:07.182 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:06 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:06.707+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:07.182 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:06 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:06.707+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:07.412 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:07 smithi148 systemd[1]: Stopping Ceph osd.7 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:32:07.413 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:07.280+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:07.413 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:07.280+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:07.413 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:07.280+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:07.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:07 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:07.055+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:07.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:07 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:07.055+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:07.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:07 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:07.055+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:07.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:07 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:07.055+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:07.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:07 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:07.286+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:07.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:07 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:07.286+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:07.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:07 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:07.286+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:07.703 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:07.411+0000 7f4e871e6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:07.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:07.411+0000 7f4e871e6640 -1 osd.7 94 *** Got signal Terminated *** 2024-09-16T00:32:07.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:07.411+0000 7f4e871e6640 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:32:07.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:07.416+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:07.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:07.416+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:07.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:07.416+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:07.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:07 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:07.416+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:08.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:07 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:07.702+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:08.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:07 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:07.702+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:08.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:07 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:07.702+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:08.521 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:08 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:08.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:08.521 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:08 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:08.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:08.521 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:08 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:08.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:08.521 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:08 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:08.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:08.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:08 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:08.279+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:08.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:08 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:08.279+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:08.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:08 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:08.279+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:08.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:08 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:08.249+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:08.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:08 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:08.249+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:08.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:08 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:08.249+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:08.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:08 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:08.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:08.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:08 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:08.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:08.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:08 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:08.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:08.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:08 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:08.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:09.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:08 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:08.741+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:09.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:08 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:08.741+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:09.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:08 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:08.741+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:09.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:09 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:09.062+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:09.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:09 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:09.062+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:09.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:09 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:09.062+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:09.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:09 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:09.062+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:09.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:09 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:09.062+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:09.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:09 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:09.293+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:09.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:09 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:09.293+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:09.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:09 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:09.293+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:09.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:09.211+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:09.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:09.211+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:09.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:09.211+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:09.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:09.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:09.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:09.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:09.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:09.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:09.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:09.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:10.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:09 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:09.733+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:10.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:09 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:09.733+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:10.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:09 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:09.733+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:10.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:09 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:09.733+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:10.454 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:10.175+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:10.454 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:10.175+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:10.454 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:10.175+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:10.454 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:10.408+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:10.454 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:10.408+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:10.455 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:10.408+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:10.455 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:10.408+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:10.455 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:10 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:10.408+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:10.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:10 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:10.089+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:10.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:10 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:10.089+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:10.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:10 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:10.089+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:10.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:10 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:10.089+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:10.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:10 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:10.089+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:10.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:10.291+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:10.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:10.291+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:10.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:10.291+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:11.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:10.781+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:11.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:10.781+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:11.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:10.781+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:11.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:10 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:10.781+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:11.454 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:11 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:11.131+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:11.454 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:11 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:11.131+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:11.454 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:11 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:11.131+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:11.454 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:11 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:11.364+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:11.454 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:11 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:11.364+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:11.454 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:11 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:11.364+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:11.454 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:11 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:11.364+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:11.455 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:11 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:11.364+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:11.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:11 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:11.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:11.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:11 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:11.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:11.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:11 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:11.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:11.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:11 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:11.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:11.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:11 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:11.046+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:11.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:11 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:11.254+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:11.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:11 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:11.254+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:11.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:11 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:11.254+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:12.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:11 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:11.794+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:12.193 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:11 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:11.794+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:12.193 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:11 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:11.794+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:12.193 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:11 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:11.794+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:12.355 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:12 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:12.085+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:21.943436+0000 front 2024-09-16T00:31:21.943565+0000 (oldest deadline 2024-09-16T00:31:47.843652+0000) 2024-09-16T00:32:12.356 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:12 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:12.085+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.544764+0000 front 2024-09-16T00:31:29.544821+0000 (oldest deadline 2024-09-16T00:31:53.644684+0000) 2024-09-16T00:32:12.356 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:12 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7[53023]: 2024-09-16T00:32:12.085+0000 7f4e837fc640 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.746984+0000 front 2024-09-16T00:31:38.747160+0000 (oldest deadline 2024-09-16T00:32:00.446910+0000) 2024-09-16T00:32:12.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:12 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:12.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:12.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:12 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:12.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:12.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:12 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:12.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:12.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:12 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:12.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:12.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:12 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:12.031+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:12.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:12 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:12.233+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:12.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:12 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:12.233+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:12.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:12 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:12.233+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:12.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:12 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:12.233+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:12.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:12 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:12.354+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:12.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:12 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:12.354+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:12.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:12 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:12.354+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:12.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:12 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:12.354+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:12.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:12 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:12.354+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:13.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:12 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:12.836+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:13.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:12 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:12.836+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:13.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:12 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:12.836+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:13.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:12 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:12.836+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:13.437 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:13 smithi148 podman[71032]: 2024-09-16 00:32:13.153667139 +0000 UTC m=+5.857130740 container died d06d5b99e1c546126dc4ee7c749a47c7bc3e21964510dcc032e4fc2f21e4cda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T00:32:13.438 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:13 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:13.320+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:13.438 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:13 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:13.320+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:13.438 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:13 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:13.320+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:13.438 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:13 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:13.320+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:13.438 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:13 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:13.320+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:13.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:13 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:13.040+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:13.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:13 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:13.040+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:13.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:13 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:13.040+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:13.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:13 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:13.040+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:13.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:13 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:13.040+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:13.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:13 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:13.283+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:13.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:13 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:13.283+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:13.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:13 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:13.283+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:13.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:13 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:13.283+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:13.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:13 smithi148 podman[71032]: 2024-09-16 00:32:13.537354903 +0000 UTC m=+6.240818503 container cleanup d06d5b99e1c546126dc4ee7c749a47c7bc3e21964510dcc032e4fc2f21e4cda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:13.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:13 smithi148 bash[71032]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7 2024-09-16T00:32:14.153 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:13 smithi148 podman[71047]: 2024-09-16 00:32:13.796942477 +0000 UTC m=+0.642204961 container remove d06d5b99e1c546126dc4ee7c749a47c7bc3e21964510dcc032e4fc2f21e4cda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T00:32:14.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:13 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:13.816+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:14.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:13 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:13.816+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:14.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:13 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:13.816+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:14.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:13 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:13.816+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:14.433 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:14 smithi148 podman[71137]: 2024-09-16 00:32:14.055913288 +0000 UTC m=+0.015954353 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:14.434 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:14 smithi148 podman[71137]: 2024-09-16 00:32:14.164512509 +0000 UTC m=+0.124553558 container create 41d51b66274f2d49300366061b03e8b8f34151fd397b7b88820d0d393a159818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:14.434 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:14 smithi148 podman[71137]: 2024-09-16 00:32:14.433551052 +0000 UTC m=+0.393592104 container init 41d51b66274f2d49300366061b03e8b8f34151fd397b7b88820d0d393a159818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:14.434 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:14 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:14.362+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:14.434 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:14 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:14.362+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:14.434 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:14 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:14.362+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:14.434 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:14 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:14.362+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:14.434 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:14 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:14.362+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:14.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:14 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:14.021+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:14.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:14 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:14.021+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:14.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:14 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:14.021+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:14.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:14 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:14.021+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:14.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:14 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:14.021+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:14.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:14 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:14.314+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:14.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:14 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:14.314+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:14.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:14 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:14.314+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:14.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:14 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:14.314+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:14.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:14 smithi148 podman[71137]: 2024-09-16 00:32:14.437611641 +0000 UTC m=+0.397652696 container start 41d51b66274f2d49300366061b03e8b8f34151fd397b7b88820d0d393a159818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T00:32:14.704 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:14 smithi148 podman[71137]: 2024-09-16 00:32:14.533578835 +0000 UTC m=+0.493619885 container attach 41d51b66274f2d49300366061b03e8b8f34151fd397b7b88820d0d393a159818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T00:32:14.705 INFO:journalctl@ceph.osd.7.smithi148.stdout:Sep 16 00:32:14 smithi148 podman[71137]: 2024-09-16 00:32:14.631282838 +0000 UTC m=+0.591323897 container died 41d51b66274f2d49300366061b03e8b8f34151fd397b7b88820d0d393a159818 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T00:32:15.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:14 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:14.808+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:15.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:14 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:14.808+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:15.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:14 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:14.808+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:15.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:14 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:14.808+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:15.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:15 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:15.012+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:15.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:15 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:15.012+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:15.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:15 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:15.012+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:15.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:15 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:15.012+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:15.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:15 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:15.012+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:15.326 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T00:32:15.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:32:15.366 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T00:32:15.366 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-16T00:32:15.367 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.8 2024-09-16T00:32:15.679 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:15 smithi193 systemd[1]: Stopping Ceph osd.8 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:32:15.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:15.308+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:15.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:15.308+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:15.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:15.308+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:15.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:15.308+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:15.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:15 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:15.395+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:15.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:15 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:15.395+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:15.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:15 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:15.395+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:15.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:15 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:15.395+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:15.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:15 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:15.395+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:16.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:15.684+0000 7fb6e306e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:16.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:15.684+0000 7fb6e306e640 -1 osd.8 94 *** Got signal Terminated *** 2024-09-16T00:32:16.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:15.684+0000 7fb6e306e640 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:32:16.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:15.770+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:16.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:15.770+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:16.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:15.770+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:16.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:15 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:15.770+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:16.445 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:16 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:16.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:16.446 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:16 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:16.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:16.446 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:16 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:16.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:16.446 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:16 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:16.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:16.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:16.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:16.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:16.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:16.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:16.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:16 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:16.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:16.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:16 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:16.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:16.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:16 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:16.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:16.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:16 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:16.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:16.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:16 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:16.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:17.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:16 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:16.810+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:17.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:16 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:16.810+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:17.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:16 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:16.810+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:17.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:16 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:16.810+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:17.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.982+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:17.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.982+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:17.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.982+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:17.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.982+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:17.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.982+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:17.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:16 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:16.982+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:17.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:17.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:17.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:17.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:17.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:17.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:17.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:17.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:17.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:17 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:17.357+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:17.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:17 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:17.357+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:17.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:17 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:17.357+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:17.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:17 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:17.357+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:17.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:17 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:17.357+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:18.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:17.833+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:18.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:17.833+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:18.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:17.833+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:18.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:17 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:17.833+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:18.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:18 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:18.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:18.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:18 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:18.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:18.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:18 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:18.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:18.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:18 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:18.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:18.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:18 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:18.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:18.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:18 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:18.022+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:18.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:18.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:18.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:18.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:18.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:18.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:18.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:18.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:18.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:18.325+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:18.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:18 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:18.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:18.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:18 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:18.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:18.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:18 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:18.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:18.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:18 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:18.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:18.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:18 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:18.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:19.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:18.878+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:19.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:18.878+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:19.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:18.878+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:19.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:18 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:18.878+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:19.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.025+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:19.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.025+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:19.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.025+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:19.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.025+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:19.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.025+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:19.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.025+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:19.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:19.343+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:19.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:19.343+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:19.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:19.343+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:19.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:19.343+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:19.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:19.343+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:19.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:19 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:19.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:19.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:19 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:19.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:19.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:19 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:19.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:19.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:19 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:19.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:19.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:19 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:19.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:20.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:19.904+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:19.175382+0000 front 2024-09-16T00:31:19.175134+0000 (oldest deadline 2024-09-16T00:31:45.075105+0000) 2024-09-16T00:32:20.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:19.904+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.676528+0000 front 2024-09-16T00:31:29.676497+0000 (oldest deadline 2024-09-16T00:31:55.576610+0000) 2024-09-16T00:32:20.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:19.904+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:35.577277+0000 front 2024-09-16T00:31:35.577286+0000 (oldest deadline 2024-09-16T00:32:01.477382+0000) 2024-09-16T00:32:20.180 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:19 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8[53928]: 2024-09-16T00:32:19.904+0000 7fb6df684640 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.878982+0000 front 2024-09-16T00:31:47.879003+0000 (oldest deadline 2024-09-16T00:32:09.578896+0000) 2024-09-16T00:32:20.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.984+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:20.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.984+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:20.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.984+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:20.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.984+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:20.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.984+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:20.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:19 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:19.984+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:20.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:20 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:20.390+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:20.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:20 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:20.390+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:20.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:20 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:20.390+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:20.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:20 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:20.390+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:20.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:20 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:20.390+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:20.703 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:20 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:20.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:20.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:20 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:20.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:20.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:20 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:20.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:20.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:20 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:20.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:20.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:20 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:20.387+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:21.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:21 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:21.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:21.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:21 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:21.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:21.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:21 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:21.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:21.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:21 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:21.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:21.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:21 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:21.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:21.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:21 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:21.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:21.461 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:21 smithi193 podman[72647]: 2024-09-16 00:32:21.200901621 +0000 UTC m=+5.636479101 container died 4db43b5b1c2898de5930fb8a8b537a40a584f3821969368cf18a159cc83d7ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:32:21.461 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:21 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:21.362+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:21.461 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:21 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:21.362+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:21.461 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:21 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:21.362+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:21.461 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:21 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:21.362+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:21.462 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:21 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:21.362+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:21.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:21 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:21.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:21.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:21 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:21.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:21.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:21 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:21.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:21.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:21 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:21.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:21.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:21 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:21.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:21.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:21 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:21.400+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:21.886 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:21 smithi193 podman[72647]: 2024-09-16 00:32:21.577095453 +0000 UTC m=+6.012672943 container cleanup 4db43b5b1c2898de5930fb8a8b537a40a584f3821969368cf18a159cc83d7ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-16T00:32:21.887 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:21 smithi193 bash[72647]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8 2024-09-16T00:32:22.179 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:21 smithi193 podman[72676]: 2024-09-16 00:32:21.886930321 +0000 UTC m=+0.684375599 container remove 4db43b5b1c2898de5930fb8a8b537a40a584f3821969368cf18a159cc83d7ff7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:32:22.454 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:22 smithi193 podman[72762]: 2024-09-16 00:32:22.098988313 +0000 UTC m=+0.016126465 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:22.454 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:22 smithi193 podman[72762]: 2024-09-16 00:32:22.212721872 +0000 UTC m=+0.129860052 container create eb80f8538d217db6c13487d28460e96b8efb6410d0f9b57e446eaec40a6685be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T00:32:22.454 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:22 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:22.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:22.454 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:22 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:22.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:22.454 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:22 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:22.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:22.454 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:22 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:22.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:22.455 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:22 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:22.357+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:22.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.041+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:22.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.041+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:22.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.041+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:22.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.041+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:22.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.041+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:22.520 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.041+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:22.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:22 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:22.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:22.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:22 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:22.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:22.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:22 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:22.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:22.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:22 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:22.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:22.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:22 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:22.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:22.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:22 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:22.381+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:22.929 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:22 smithi193 podman[72762]: 2024-09-16 00:32:22.45451839 +0000 UTC m=+0.371656546 container init eb80f8538d217db6c13487d28460e96b8efb6410d0f9b57e446eaec40a6685be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T00:32:22.929 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:22 smithi193 podman[72762]: 2024-09-16 00:32:22.457842259 +0000 UTC m=+0.374980403 container start eb80f8538d217db6c13487d28460e96b8efb6410d0f9b57e446eaec40a6685be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:32:22.929 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:22 smithi193 podman[72762]: 2024-09-16 00:32:22.598848797 +0000 UTC m=+0.515986948 container attach eb80f8538d217db6c13487d28460e96b8efb6410d0f9b57e446eaec40a6685be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:32:22.929 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 16 00:32:22 smithi193 podman[72762]: 2024-09-16 00:32:22.650989043 +0000 UTC m=+0.568127192 container died eb80f8538d217db6c13487d28460e96b8efb6410d0f9b57e446eaec40a6685be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T00:32:23.264 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.8.service' 2024-09-16T00:32:23.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.993+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:23.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.993+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:23.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.993+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:23.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.993+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:23.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.993+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:23.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:22 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:22.993+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:23.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:32:23.305 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-16T00:32:23.305 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-16T00:32:23.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.9 2024-09-16T00:32:23.631 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:23 smithi078 systemd[1]: Stopping Ceph osd.9 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:32:23.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:23 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:23.375+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:23.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:23 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:23.375+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:23.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:23 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:23.375+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:23.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:23 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:23.375+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:23.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:23 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:23.375+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:23.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:23 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:23.385+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:23.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:23 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:23.385+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:23.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:23 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:23.385+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:23.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:23 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:23.385+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:23.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:23 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:23.385+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:23.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:23 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:23.385+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:24.007 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:23.630+0000 7fa17ca2b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:24.007 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:23.630+0000 7fa17ca2b640 -1 osd.9 94 *** Got signal Terminated *** 2024-09-16T00:32:24.007 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:23 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:23.630+0000 7fa17ca2b640 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:32:24.007 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:24.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:24.007 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:24.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:24.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:24.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:24.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:24.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:24.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:24.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:24.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:24 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:24.006+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:24.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:24 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:24.403+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:24.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:24 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:24.403+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:24.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:24 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:24.403+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:24.679 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:24 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:24.403+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:24.680 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:24 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:24.403+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:24.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:24 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:24.433+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:24.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:24 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:24.433+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:24.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:24 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:24.433+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:24.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:24 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:24.433+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:24.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:24 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:24.433+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:24.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:24 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:24.433+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:25.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.011+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:25.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.011+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:25.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.011+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:25.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.011+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:25.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.011+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:25.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.011+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:25.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:25 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:25.413+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:25.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:25 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:25.413+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:25.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:25 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:25.413+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:25.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:25 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:25.413+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:25.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:25 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:25.413+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:25.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:25 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:25.413+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:25.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:25 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:25.450+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:25.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:25 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:25.450+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:25.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:25 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:25.450+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:25.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:25 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:25.450+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:25.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:25 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:25.450+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:26.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.980+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:26.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.980+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:26.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.981+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:26.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.981+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:26.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.981+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:26.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:25 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:25.981+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:26.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:26.369+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:26.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:26.369+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:26.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:26.369+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:26.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:26.369+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:26.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:26.369+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:26.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:26.369+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:26.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:26 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:26.369+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:26.916 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:26.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:26.917 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:26.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:26.917 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:26.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:26.917 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:26.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:26.917 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:26.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:26.917 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:26 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:26.492+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:27.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:26.976+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:27.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:26.976+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:27.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:26.976+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:27.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:26.976+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:27.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:26.976+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:27.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:26 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:26.976+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:27.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:27 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:27.390+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:27.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:27 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:27.390+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:27.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:27 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:27.390+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:27.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:27 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:27.390+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:27.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:27 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:27.390+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:27.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:27 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:27.390+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:27.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:27 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:27.390+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:27.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:27 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:27.470+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:27.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:27 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:27.470+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:27.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:27 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:27.470+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:27.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:27 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:27.470+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:27.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:27 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:27.470+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:27.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:27 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:27.470+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:28.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:27.965+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:10.165161+0000 front 2024-09-16T00:31:10.165181+0000 (oldest deadline 2024-09-16T00:31:40.166047+0000) 2024-09-16T00:32:28.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:27.965+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:23.067558+0000 front 2024-09-16T00:31:23.067403+0000 (oldest deadline 2024-09-16T00:31:47.167469+0000) 2024-09-16T00:32:28.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:27.965+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:27.168037+0000 front 2024-09-16T00:31:27.168138+0000 (oldest deadline 2024-09-16T00:31:53.068035+0000) 2024-09-16T00:32:28.269 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:27.965+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:38.968845+0000 front 2024-09-16T00:31:38.968885+0000 (oldest deadline 2024-09-16T00:32:01.268909+0000) 2024-09-16T00:32:28.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:27.965+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:47.070462+0000 front 2024-09-16T00:31:47.070370+0000 (oldest deadline 2024-09-16T00:32:08.770236+0000) 2024-09-16T00:32:28.270 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:27 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9[65743]: 2024-09-16T00:32:27.965+0000 7fa179041640 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.171771+0000 front 2024-09-16T00:31:55.171701+0000 (oldest deadline 2024-09-16T00:32:16.871582+0000) 2024-09-16T00:32:28.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:28 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:28.434+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:28.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:28 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:28.434+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:28.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:28 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:28.434+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:28.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:28 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:28.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:28.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:28 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:28.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:28.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:28 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:28.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:28.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:28 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:28.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:28.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:28 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:28.433+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:28.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:28 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:28.433+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:28.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:28 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:28.433+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:28.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:28 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:28.433+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:28.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:28 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:28.433+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:28.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:28 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:28.433+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:29.519 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:29 smithi078 podman[123449]: 2024-09-16 00:32:29.266695833 +0000 UTC m=+5.763187446 container died 5e239e9ceb471574fc3ad0ac0a4cd906c40f4dc4120a88e8187e8bb4c72622a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T00:32:29.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:29 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:29.447+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:29.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:29 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:29.447+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:29.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:29 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:29.447+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:29.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:29 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:29.447+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:29.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:29 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:29.447+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:29.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:29 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:29.447+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:29.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:29 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:29.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:29.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:29 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:29.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:29.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:29 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:29.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:29.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:29 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:29.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:29.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:29 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:29.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:29.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:29 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:29.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:29.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:29 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:29.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:30.019 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:29 smithi078 podman[123449]: 2024-09-16 00:32:29.714617769 +0000 UTC m=+6.211109375 container cleanup 5e239e9ceb471574fc3ad0ac0a4cd906c40f4dc4120a88e8187e8bb4c72622a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T00:32:30.019 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:29 smithi078 bash[123449]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9 2024-09-16T00:32:30.333 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:30 smithi078 podman[123465]: 2024-09-16 00:32:30.024363262 +0000 UTC m=+0.754871299 container remove 5e239e9ceb471574fc3ad0ac0a4cd906c40f4dc4120a88e8187e8bb4c72622a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-16T00:32:30.644 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:30 smithi078 podman[123557]: 2024-09-16 00:32:30.236070187 +0000 UTC m=+0.015993902 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:30.644 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:30 smithi078 podman[123557]: 2024-09-16 00:32:30.333755789 +0000 UTC m=+0.113679492 container create dbfb2de83498e65bcff62e04526a88fdcb01d56962cac9cfcb1f2d3f49f81dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-16T00:32:30.644 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:30 smithi078 podman[123557]: 2024-09-16 00:32:30.561169956 +0000 UTC m=+0.341093665 container init dbfb2de83498e65bcff62e04526a88fdcb01d56962cac9cfcb1f2d3f49f81dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:32:30.644 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:30 smithi078 podman[123557]: 2024-09-16 00:32:30.564476665 +0000 UTC m=+0.344400373 container start dbfb2de83498e65bcff62e04526a88fdcb01d56962cac9cfcb1f2d3f49f81dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:30.644 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:30 smithi078 podman[123557]: 2024-09-16 00:32:30.644389083 +0000 UTC m=+0.424312802 container attach dbfb2de83498e65bcff62e04526a88fdcb01d56962cac9cfcb1f2d3f49f81dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T00:32:30.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:30 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:30.468+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:30.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:30 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:30.468+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:30.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:30 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:30.468+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:30.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:30 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:30.468+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:30.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:30 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:30.468+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:30.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:30 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:30.468+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:30.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:30 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:30.507+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:30.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:30 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:30.507+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:30.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:30 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:30.507+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:30.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:30 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:30.507+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:30.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:30 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:30.507+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:30.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:30 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:30.507+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:30.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:30 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:30.507+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:31.019 INFO:journalctl@ceph.osd.9.smithi078.stdout:Sep 16 00:32:30 smithi078 podman[123557]: 2024-09-16 00:32:30.751328518 +0000 UTC m=+0.531252230 container died dbfb2de83498e65bcff62e04526a88fdcb01d56962cac9cfcb1f2d3f49f81dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T00:32:31.442 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.9.service' 2024-09-16T00:32:31.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:32:31.485 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-16T00:32:31.485 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-16T00:32:31.485 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.10 2024-09-16T00:32:31.806 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.477+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:31.806 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.477+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:31.806 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.477+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:31.807 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.477+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:31.807 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.477+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:31.807 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.477+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:31.807 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.477+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:31.807 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 systemd[1]: Stopping Ceph osd.10 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:32:31.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:31 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:31.507+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:31.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:31 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:31.507+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:31.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:31 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:31.507+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:31.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:31 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:31.507+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:31.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:31 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:31.507+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:31.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:31 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:31.507+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:32.204 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.805+0000 7f0de6d1f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:32.204 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.805+0000 7f0de6d1f640 -1 osd.10 94 *** Got signal Terminated *** 2024-09-16T00:32:32.204 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:31 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:31.805+0000 7f0de6d1f640 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:32:32.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:32 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:32.554+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:32.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:32 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:32.554+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:32.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:32 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:32.554+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:32.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:32 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:32.554+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:32.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:32 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:32.554+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:32.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:32 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:32.554+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:32.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:32 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:32.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:32.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:32 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:32.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:32.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:32 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:32.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:32.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:32 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:32.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:32.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:32 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:32.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:32.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:32 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:32.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:32.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:32 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:32.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:33.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:33 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:33.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:33.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:33 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:33.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:33.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:33 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:33.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:33.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:33 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:33.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:33.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:33 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:33.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:33.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:33 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:33.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:33.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:33 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:33.435+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:33.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:33.571+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:33.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:33.571+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:33.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:33.571+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:33.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:33.571+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:33.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:33.571+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:33.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:33.571+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:34.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:34 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:34.580+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:34.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:34 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:34.580+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:34.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:34 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:34.580+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:34.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:34 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:34.580+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:34.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:34 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:34.580+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:34.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:34 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:34.580+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:34.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:34 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:34.462+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:34.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:34 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:34.462+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:34.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:34 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:34.462+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:34.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:34 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:34.462+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:34.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:34 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:34.462+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:34.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:34 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:34.462+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:34.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:34 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:34.462+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:35.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:35 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:35.590+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:35.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:35 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:35.590+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:35.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:35 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:35.590+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:35.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:35 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:35.590+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:35.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:35 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:35.590+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:35.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:35 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:35.590+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:35.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:35 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:35.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:35.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:35 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:35.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:35.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:35 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:35.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:35.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:35 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:35.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:35.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:35 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:35.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:35.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:35 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:35.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:35.954 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:35 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:35.457+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:36.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:36 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:36.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:13.019476+0000 front 2024-09-16T00:31:13.019356+0000 (oldest deadline 2024-09-16T00:31:37.719411+0000) 2024-09-16T00:32:36.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:36 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:36.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.221300+0000 front 2024-09-16T00:31:22.221326+0000 (oldest deadline 2024-09-16T00:31:47.521457+0000) 2024-09-16T00:32:36.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:36 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:36.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:29.222680+0000 front 2024-09-16T00:31:29.222698+0000 (oldest deadline 2024-09-16T00:31:54.522785+0000) 2024-09-16T00:32:36.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:36 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:36.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6814 osd.3 since back 2024-09-16T00:31:39.724470+0000 front 2024-09-16T00:31:39.724414+0000 (oldest deadline 2024-09-16T00:32:02.024459+0000) 2024-09-16T00:32:36.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:36 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:36.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.725663+0000 front 2024-09-16T00:31:46.725623+0000 (oldest deadline 2024-09-16T00:32:09.625808+0000) 2024-09-16T00:32:36.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:36 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:36.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:55.427109+0000 front 2024-09-16T00:31:55.427171+0000 (oldest deadline 2024-09-16T00:32:20.727189+0000) 2024-09-16T00:32:36.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:36 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10[59541]: 2024-09-16T00:32:36.411+0000 7f0de3335640 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:03.628244+0000 front 2024-09-16T00:32:03.628285+0000 (oldest deadline 2024-09-16T00:32:25.928076+0000) 2024-09-16T00:32:36.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:36.582+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:36.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:36.582+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:36.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:36.582+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:36.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:36.582+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:36.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:36.582+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:36.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:36.582+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:36.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:36 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:36.582+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:37.730 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:37 smithi148 podman[71242]: 2024-09-16 00:32:37.476976136 +0000 UTC m=+5.797977181 container died 65a6c56fcc957195f1b090b828f41c9bb4f421d08a0f3b2184a129ef971268e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T00:32:37.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:37 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:37.555+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:37.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:37 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:37.555+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:37.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:37 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:37.555+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:37.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:37 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:37.555+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:37.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:37 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:37.555+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:37.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:37 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:37.555+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:37.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:37 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:37.555+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:38.014 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:37 smithi148 podman[71242]: 2024-09-16 00:32:37.805642666 +0000 UTC m=+6.126643702 container cleanup 65a6c56fcc957195f1b090b828f41c9bb4f421d08a0f3b2184a129ef971268e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:32:38.015 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:37 smithi148 bash[71242]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10 2024-09-16T00:32:38.418 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:38 smithi148 podman[71259]: 2024-09-16 00:32:38.015000971 +0000 UTC m=+0.533420953 container remove 65a6c56fcc957195f1b090b828f41c9bb4f421d08a0f3b2184a129ef971268e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:38.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:38 smithi148 podman[71346]: 2024-09-16 00:32:38.321667643 +0000 UTC m=+0.015717524 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:38.704 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:38 smithi148 podman[71346]: 2024-09-16 00:32:38.42393166 +0000 UTC m=+0.117981534 container create 27e94fa31c3a996447005438637910a5a9177df65e30f1bf8677f00da59f2c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:32:38.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:38 smithi148 podman[71346]: 2024-09-16 00:32:38.642831006 +0000 UTC m=+0.336880879 container init 27e94fa31c3a996447005438637910a5a9177df65e30f1bf8677f00da59f2c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:38.705 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:38 smithi148 podman[71346]: 2024-09-16 00:32:38.646215285 +0000 UTC m=+0.340265159 container start 27e94fa31c3a996447005438637910a5a9177df65e30f1bf8677f00da59f2c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-16T00:32:38.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:38.586+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:38.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:38.586+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:38.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:38.586+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:38.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:38.586+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:38.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:38.586+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:38.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:38.586+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:38.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:38 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:38.586+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:39.120 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:38 smithi148 podman[71346]: 2024-09-16 00:32:38.72959076 +0000 UTC m=+0.423640643 container attach 27e94fa31c3a996447005438637910a5a9177df65e30f1bf8677f00da59f2c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:32:39.120 INFO:journalctl@ceph.osd.10.smithi148.stdout:Sep 16 00:32:38 smithi148 podman[71346]: 2024-09-16 00:32:38.83423009 +0000 UTC m=+0.528279967 container died 27e94fa31c3a996447005438637910a5a9177df65e30f1bf8677f00da59f2c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T00:32:39.433 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.10.service' 2024-09-16T00:32:39.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:32:39.473 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-16T00:32:39.473 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-16T00:32:39.473 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.11 2024-09-16T00:32:39.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 systemd[1]: Stopping Ceph osd.11 for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:32:39.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.568+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:39.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.568+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:39.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.568+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:39.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.568+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:39.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.568+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:39.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.568+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:39.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.568+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:40.429 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.979+0000 7ff2f7063640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:40.430 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.979+0000 7ff2f7063640 -1 osd.11 94 *** Got signal Terminated *** 2024-09-16T00:32:40.430 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:39 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:39.979+0000 7ff2f7063640 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:32:40.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:40 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:40.593+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:40.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:40 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:40.593+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:40.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:40 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:40.593+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:40.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:40 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:40.593+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:40.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:40 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:40.593+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:40.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:40 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:40.593+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:40.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:40 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:40.593+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:41.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:41 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:41.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:41.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:41 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:41.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:41.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:41 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:41.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:41.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:41 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:41.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:41.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:41 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:41.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:41.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:41 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:41.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:41.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:41 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:41.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:42.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:42.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:42.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:42.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:42.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:42.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:42.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:42.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:42.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:42.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:42.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:42.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:42.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:42.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:42.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:42 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:42.596+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-16T00:32:20.451374+0000 front 2024-09-16T00:32:20.451402+0000 (oldest deadline 2024-09-16T00:32:42.151383+0000) 2024-09-16T00:32:43.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:43.562+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:43.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:43.562+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:43.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:43.562+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:43.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:43.562+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:43.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:43.562+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:43.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:43.562+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:43.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:43.562+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:43.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:43 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:43.562+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-16T00:32:20.451374+0000 front 2024-09-16T00:32:20.451402+0000 (oldest deadline 2024-09-16T00:32:42.151383+0000) 2024-09-16T00:32:44.929 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:44.530+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6806 osd.0 since back 2024-09-16T00:31:12.438448+0000 front 2024-09-16T00:31:12.438547+0000 (oldest deadline 2024-09-16T00:31:38.338576+0000) 2024-09-16T00:32:44.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:44.530+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6804 osd.1 since back 2024-09-16T00:31:22.841023+0000 front 2024-09-16T00:31:22.841048+0000 (oldest deadline 2024-09-16T00:31:48.141236+0000) 2024-09-16T00:32:44.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:44.530+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-16T00:31:28.141806+0000 front 2024-09-16T00:31:28.141844+0000 (oldest deadline 2024-09-16T00:31:54.443001+0000) 2024-09-16T00:32:44.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:44.530+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6812 osd.4 since back 2024-09-16T00:31:46.045794+0000 front 2024-09-16T00:31:46.045775+0000 (oldest deadline 2024-09-16T00:32:11.846269+0000) 2024-09-16T00:32:44.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:44.530+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-16T00:31:52.347623+0000 front 2024-09-16T00:31:52.347500+0000 (oldest deadline 2024-09-16T00:32:17.647686+0000) 2024-09-16T00:32:44.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:44.530+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.78:6822 osd.6 since back 2024-09-16T00:32:02.849534+0000 front 2024-09-16T00:32:02.849625+0000 (oldest deadline 2024-09-16T00:32:25.749425+0000) 2024-09-16T00:32:44.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:44.530+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.148:6820 osd.7 since back 2024-09-16T00:32:11.650407+0000 front 2024-09-16T00:32:11.650466+0000 (oldest deadline 2024-09-16T00:32:35.750301+0000) 2024-09-16T00:32:44.930 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:44 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11[60915]: 2024-09-16T00:32:44.530+0000 7ff2ef671640 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-16T00:32:20.451374+0000 front 2024-09-16T00:32:20.451402+0000 (oldest deadline 2024-09-16T00:32:42.151383+0000) 2024-09-16T00:32:45.888 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:45 smithi193 podman[72866]: 2024-09-16 00:32:45.620647069 +0000 UTC m=+5.960282541 container died b73a94c50e1309accab71354e876fdea44e011526ba778d70faf480d8ba33012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:32:46.164 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:45 smithi193 podman[72866]: 2024-09-16 00:32:45.962843793 +0000 UTC m=+6.302479253 container cleanup b73a94c50e1309accab71354e876fdea44e011526ba778d70faf480d8ba33012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:32:46.165 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:45 smithi193 bash[72866]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11 2024-09-16T00:32:46.429 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:46 smithi193 podman[72883]: 2024-09-16 00:32:46.164548363 +0000 UTC m=+0.542845654 container remove b73a94c50e1309accab71354e876fdea44e011526ba778d70faf480d8ba33012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T00:32:46.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:46 smithi193 podman[72973]: 2024-09-16 00:32:46.474730511 +0000 UTC m=+0.015785460 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:32:46.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:46 smithi193 podman[72973]: 2024-09-16 00:32:46.582117967 +0000 UTC m=+0.123172903 container create 23bb00995eede86b071f73cc49d518b0b69510a73a02e68cf238749f444d8487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:32:46.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:46 smithi193 podman[72973]: 2024-09-16 00:32:46.817907325 +0000 UTC m=+0.358962273 container init 23bb00995eede86b071f73cc49d518b0b69510a73a02e68cf238749f444d8487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-16T00:32:46.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:46 smithi193 podman[72973]: 2024-09-16 00:32:46.821082613 +0000 UTC m=+0.362137554 container start 23bb00995eede86b071f73cc49d518b0b69510a73a02e68cf238749f444d8487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11-deactivate, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:32:46.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:46 smithi193 podman[72973]: 2024-09-16 00:32:46.901170265 +0000 UTC m=+0.442225219 container attach 23bb00995eede86b071f73cc49d518b0b69510a73a02e68cf238749f444d8487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-16T00:32:47.179 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 16 00:32:47 smithi193 podman[72973]: 2024-09-16 00:32:47.012052677 +0000 UTC m=+0.553107621 container died 23bb00995eede86b071f73cc49d518b0b69510a73a02e68cf238749f444d8487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T00:32:47.609 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@osd.11.service' 2024-09-16T00:32:47.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:32:47.647 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-16T00:32:47.648 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-16T00:32:47.648 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.a 2024-09-16T00:32:47.958 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:32:47 smithi078 systemd[1]: Stopping Ceph mds.a for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:32:48.210 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:32:47 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a[72583]: 2024-09-16T00:32:47.957+0000 7fd3e045a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:48.211 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:32:47 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a[72583]: 2024-09-16T00:32:47.957+0000 7fd3e045a640 -1 mds.a *** got signal Terminated *** 2024-09-16T00:32:52.225 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:32:51 smithi078 podman[123660]: 2024-09-16 00:32:51.96480554 +0000 UTC m=+4.113877158 container died 38e369dd5f437043043fcad742558b16546b22d838b73fe9a860fb7d6e70a2fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T00:32:52.519 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:32:52 smithi078 podman[123660]: 2024-09-16 00:32:52.424709019 +0000 UTC m=+4.573780670 container cleanup 38e369dd5f437043043fcad742558b16546b22d838b73fe9a860fb7d6e70a2fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T00:32:52.519 INFO:journalctl@ceph.mds.a.smithi078.stdout:Sep 16 00:32:52 smithi078 bash[123660]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-a 2024-09-16T00:32:52.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.a.service' 2024-09-16T00:32:52.901 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:32:52.901 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-16T00:32:52.902 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-16T00:32:52.902 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.d 2024-09-16T00:32:53.269 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:32:52 smithi078 systemd[1]: Stopping Ceph mds.d for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:32:53.269 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:32:53 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d[71292]: 2024-09-16T00:32:53.228+0000 7f9c65df9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:53.269 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:32:53 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d[71292]: 2024-09-16T00:32:53.228+0000 7f9c65df9640 -1 mds.d *** got signal Terminated *** 2024-09-16T00:32:54.979 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:32:54 smithi078 podman[123794]: 2024-09-16 00:32:54.724198569 +0000 UTC m=+1.603334954 container died 1d0226a243e8a89277ea176c4ff39bbffc0049de9a3f7c479532a0cd92fd243a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:32:55.269 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:32:55 smithi078 podman[123794]: 2024-09-16 00:32:55.07854131 +0000 UTC m=+1.957677692 container cleanup 1d0226a243e8a89277ea176c4ff39bbffc0049de9a3f7c479532a0cd92fd243a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:32:55.269 INFO:journalctl@ceph.mds.d.smithi078.stdout:Sep 16 00:32:55 smithi078 bash[123794]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-d 2024-09-16T00:32:55.504 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.d.service' 2024-09-16T00:32:55.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:32:55.537 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-16T00:32:55.538 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-16T00:32:55.538 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.g 2024-09-16T00:32:56.019 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:32:55 smithi078 systemd[1]: Stopping Ceph mds.g for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:32:56.019 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:32:55 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g[72169]: 2024-09-16T00:32:55.857+0000 7f752a1f8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:32:56.019 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:32:55 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g[72169]: 2024-09-16T00:32:55.857+0000 7f752a1f8640 -1 mds.g *** got signal Terminated *** 2024-09-16T00:33:00.150 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:32:59 smithi078 podman[123914]: 2024-09-16 00:32:59.872130928 +0000 UTC m=+4.110714294 container died 0560a45c8536ca0b3e8f784f2d1a2c9f7e3a9c39a2ee16d1534f70fc75d82f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T00:33:00.492 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:33:00 smithi078 podman[123914]: 2024-09-16 00:33:00.249261093 +0000 UTC m=+4.487844500 container cleanup 0560a45c8536ca0b3e8f784f2d1a2c9f7e3a9c39a2ee16d1534f70fc75d82f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:33:00.492 INFO:journalctl@ceph.mds.g.smithi078.stdout:Sep 16 00:33:00 smithi078 bash[123914]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-g 2024-09-16T00:33:00.633 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.g.service' 2024-09-16T00:33:00.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:00.667 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-16T00:33:00.667 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-16T00:33:00.667 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.j 2024-09-16T00:33:01.019 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:33:00 smithi078 systemd[1]: Stopping Ceph mds.j for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:01.019 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:33:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j[70898]: 2024-09-16T00:33:00.986+0000 7fe58e486640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:01.019 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:33:00 smithi078 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j[70898]: 2024-09-16T00:33:00.986+0000 7fe58e486640 -1 mds.j *** got signal Terminated *** 2024-09-16T00:33:03.429 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:33:03 smithi078 podman[124034]: 2024-09-16 00:33:03.146413478 +0000 UTC m=+2.265578563 container died 980434cf6e6ddea37056e83169b46c00e6d2262c789ea3ac8e560b93b0d49c4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T00:33:03.762 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:33:03 smithi078 podman[124034]: 2024-09-16 00:33:03.527955993 +0000 UTC m=+2.647121087 container cleanup 980434cf6e6ddea37056e83169b46c00e6d2262c789ea3ac8e560b93b0d49c4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T00:33:03.762 INFO:journalctl@ceph.mds.j.smithi078.stdout:Sep 16 00:33:03 smithi078 bash[124034]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-j 2024-09-16T00:33:03.890 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.j.service' 2024-09-16T00:33:03.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:03.924 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-16T00:33:03.924 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-16T00:33:03.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.b 2024-09-16T00:33:04.204 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:33:03 smithi148 systemd[1]: Stopping Ceph mds.b for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:04.704 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:33:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: 2024-09-16T00:33:04.240+0000 7ff6d4be7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:04.704 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:33:04 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b[65064]: 2024-09-16T00:33:04.240+0000 7ff6d4be7640 -1 mds.b *** got signal Terminated *** 2024-09-16T00:33:08.580 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:33:08 smithi148 podman[71446]: 2024-09-16 00:33:08.255377726 +0000 UTC m=+4.131472315 container died 3899f630a044e7e011125c469409d4c6caf099039cb6841dbd9e8ebdbf156fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:33:08.888 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:33:08 smithi148 podman[71446]: 2024-09-16 00:33:08.645831017 +0000 UTC m=+4.521925594 container cleanup 3899f630a044e7e011125c469409d4c6caf099039cb6841dbd9e8ebdbf156fbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T00:33:08.888 INFO:journalctl@ceph.mds.b.smithi148.stdout:Sep 16 00:33:08 smithi148 bash[71446]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-b 2024-09-16T00:33:09.035 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.b.service' 2024-09-16T00:33:09.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:09.071 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-16T00:33:09.071 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-16T00:33:09.072 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.e 2024-09-16T00:33:09.431 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:33:09 smithi148 systemd[1]: Stopping Ceph mds.e for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:09.704 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:33:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e[64448]: 2024-09-16T00:33:09.431+0000 7f9b9fa2a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:09.704 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:33:09 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e[64448]: 2024-09-16T00:33:09.431+0000 7f9b9fa2a640 -1 mds.e *** got signal Terminated *** 2024-09-16T00:33:12.625 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:33:12 smithi148 podman[71568]: 2024-09-16 00:33:12.315270576 +0000 UTC m=+3.020584208 container died 4052c5131330d6e9d1163f0b3856c28dc49a4ab7ca6445436d16a1ceb4cdf458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T00:33:12.917 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:33:12 smithi148 podman[71568]: 2024-09-16 00:33:12.716288121 +0000 UTC m=+3.421601764 container cleanup 4052c5131330d6e9d1163f0b3856c28dc49a4ab7ca6445436d16a1ceb4cdf458 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:33:12.917 INFO:journalctl@ceph.mds.e.smithi148.stdout:Sep 16 00:33:12 smithi148 bash[71568]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-e 2024-09-16T00:33:13.043 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.e.service' 2024-09-16T00:33:13.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:13.076 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-16T00:33:13.076 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-16T00:33:13.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.h 2024-09-16T00:33:13.203 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:33:13 smithi148 systemd[1]: Stopping Ceph mds.h for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:13.703 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:33:13 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h[64748]: 2024-09-16T00:33:13.394+0000 7f1f44e81640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:13.704 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:33:13 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h[64748]: 2024-09-16T00:33:13.394+0000 7f1f44e81640 -1 mds.h *** got signal Terminated *** 2024-09-16T00:33:16.704 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:33:16 smithi148 podman[71688]: 2024-09-16 00:33:16.325269099 +0000 UTC m=+3.033790193 container died 2744b0beb61239049ef1426a181ccd7a139c5e9b3d8092eb81ece14ba98c04d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-16T00:33:16.954 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:33:16 smithi148 podman[71688]: 2024-09-16 00:33:16.786181249 +0000 UTC m=+3.494702343 container cleanup 2744b0beb61239049ef1426a181ccd7a139c5e9b3d8092eb81ece14ba98c04d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:33:16.954 INFO:journalctl@ceph.mds.h.smithi148.stdout:Sep 16 00:33:16 smithi148 bash[71688]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-h 2024-09-16T00:33:17.214 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.h.service' 2024-09-16T00:33:17.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:17.247 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-16T00:33:17.247 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-16T00:33:17.247 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.k 2024-09-16T00:33:17.628 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:33:17 smithi148 systemd[1]: Stopping Ceph mds.k for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:17.954 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:33:17 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-k[64131]: 2024-09-16T00:33:17.627+0000 7f05c1297640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:17.954 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:33:17 smithi148 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-k[64131]: 2024-09-16T00:33:17.627+0000 7f05c1297640 -1 mds.k *** got signal Terminated *** 2024-09-16T00:33:20.615 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:33:20 smithi148 podman[71809]: 2024-09-16 00:33:20.313342153 +0000 UTC m=+2.849041855 container died 3b76efa29f975698d1b045493f8ec3d27d75800bc6fb7a4d80bf6ee5a37386db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-k, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T00:33:20.940 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:33:20 smithi148 podman[71809]: 2024-09-16 00:33:20.706404863 +0000 UTC m=+3.242104565 container cleanup 3b76efa29f975698d1b045493f8ec3d27d75800bc6fb7a4d80bf6ee5a37386db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:33:20.940 INFO:journalctl@ceph.mds.k.smithi148.stdout:Sep 16 00:33:20 smithi148 bash[71809]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-k 2024-09-16T00:33:21.079 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.k.service' 2024-09-16T00:33:21.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:21.111 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-16T00:33:21.111 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-16T00:33:21.111 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.c 2024-09-16T00:33:21.412 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:33:21 smithi193 systemd[1]: Stopping Ceph mds.c for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:21.679 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:33:21 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c[66196]: 2024-09-16T00:33:21.412+0000 7fe96796f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:21.679 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:33:21 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c[66196]: 2024-09-16T00:33:21.412+0000 7fe96796f640 -1 mds.c *** got signal Terminated *** 2024-09-16T00:33:24.563 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:33:24 smithi193 podman[73086]: 2024-09-16 00:33:24.292375099 +0000 UTC m=+2.988867058 container died f4a64c18669736473c08f5c8604861f6608e66a0c5005793b0c3a175e29429d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T00:33:24.929 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:33:24 smithi193 podman[73086]: 2024-09-16 00:33:24.66198671 +0000 UTC m=+3.358478656 container cleanup f4a64c18669736473c08f5c8604861f6608e66a0c5005793b0c3a175e29429d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-16T00:33:24.929 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 16 00:33:24 smithi193 bash[73086]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-c 2024-09-16T00:33:25.094 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.c.service' 2024-09-16T00:33:25.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:25.131 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-16T00:33:25.131 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-16T00:33:25.131 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.f 2024-09-16T00:33:25.449 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:33:25 smithi193 systemd[1]: Stopping Ceph mds.f for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:25.929 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:33:25 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f[65895]: 2024-09-16T00:33:25.449+0000 7fc17c044640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:25.929 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:33:25 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f[65895]: 2024-09-16T00:33:25.449+0000 7fc17c044640 -1 mds.f *** got signal Terminated *** 2024-09-16T00:33:28.658 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:33:28 smithi193 podman[73219]: 2024-09-16 00:33:28.296608676 +0000 UTC m=+2.944573399 container died 3e9c74b0cac859ea087f88c6f613cd1e62d7ada02a07ee725d4c76a57cc488ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T00:33:28.929 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:33:28 smithi193 podman[73219]: 2024-09-16 00:33:28.741007736 +0000 UTC m=+3.388972456 container cleanup 3e9c74b0cac859ea087f88c6f613cd1e62d7ada02a07ee725d4c76a57cc488ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T00:33:28.929 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 16 00:33:28 smithi193 bash[73219]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-f 2024-09-16T00:33:29.178 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.f.service' 2024-09-16T00:33:29.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:29.214 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-16T00:33:29.214 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-16T00:33:29.214 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.i 2024-09-16T00:33:29.553 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:29 smithi193 systemd[1]: Stopping Ceph mds.i for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:29.929 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:29 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i[66508]: 2024-09-16T00:33:29.553+0000 7f28abd93640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:29.929 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:29 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i[66508]: 2024-09-16T00:33:29.553+0000 7f28abd93640 -1 mds.i *** got signal Terminated *** 2024-09-16T00:33:32.612 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:32 smithi193 podman[73341]: 2024-09-16 00:33:32.29862256 +0000 UTC m=+2.857061460 container died 39672d44980d49f7ed827a3b51f7232494faa74e54f3542e8e0b9c603669e1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:33:32.929 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:32 smithi193 podman[73341]: 2024-09-16 00:33:32.711953871 +0000 UTC m=+3.270392739 container cleanup 39672d44980d49f7ed827a3b51f7232494faa74e54f3542e8e0b9c603669e1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:33:32.929 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:32 smithi193 bash[73341]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i 2024-09-16T00:33:33.188 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.i.service' 2024-09-16T00:33:33.209 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:32 smithi193 podman[73353]: 2024-09-16 00:33:32.956308608 +0000 UTC m=+0.653839223 container remove 39672d44980d49f7ed827a3b51f7232494faa74e54f3542e8e0b9c603669e1da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:33:33.210 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:33 smithi193 systemd[1]: ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-16T00:33:33.210 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:33 smithi193 systemd[1]: Stopped Ceph mds.i for eb73a744-73be-11ef-bceb-c7b262605968. 2024-09-16T00:33:33.210 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 16 00:33:33 smithi193 systemd[1]: ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.i.service: Consumed 2.264s CPU time. 2024-09-16T00:33:33.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:33.220 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-16T00:33:33.220 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-16T00:33:33.221 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.l 2024-09-16T00:33:33.492 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:33:33 smithi193 systemd[1]: Stopping Ceph mds.l for eb73a744-73be-11ef-bceb-c7b262605968... 2024-09-16T00:33:33.929 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:33:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l[65596]: 2024-09-16T00:33:33.582+0000 7f9668327640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:33:33.929 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:33:33 smithi193 ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l[65596]: 2024-09-16T00:33:33.582+0000 7f9668327640 -1 mds.l *** got signal Terminated *** 2024-09-16T00:33:36.675 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:33:36 smithi193 podman[73459]: 2024-09-16 00:33:36.298457672 +0000 UTC m=+2.848936128 container died 6b6940ac814d1fa7074e65666c2b0f25c5dc3212e23ade81675e873bdf534dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-16T00:33:36.929 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:33:36 smithi193 podman[73459]: 2024-09-16 00:33:36.76599212 +0000 UTC m=+3.316470578 container cleanup 6b6940ac814d1fa7074e65666c2b0f25c5dc3212e23ade81675e873bdf534dd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T00:33:36.929 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 16 00:33:36 smithi193 bash[73459]: ceph-eb73a744-73be-11ef-bceb-c7b262605968-mds-l 2024-09-16T00:33:37.182 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-eb73a744-73be-11ef-bceb-c7b262605968@mds.l.service' 2024-09-16T00:33:37.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:33:37.215 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-16T00:33:37.215 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb73a744-73be-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:33:37.426 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:33:39.911 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb73a744-73be-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:33:40.116 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:33:42.682 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb73a744-73be-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:33:42.884 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:33:45.291 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:33:45.324 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:33:45.357 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:33:45.384 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T00:33:45.386 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi078/crash 2024-09-16T00:33:45.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:33:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:33:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:33:45.425 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi148/crash 2024-09-16T00:33:45.426 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:33:45.469 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:33:45.470 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:33:45.472 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi193/crash 2024-09-16T00:33:45.473 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:33:45.510 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/eb73a744-73be-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:33:45.510 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:33:45.511 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T00:33:45.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-16T00:33:45.604 INFO:tasks.cephadm:Compressing logs... 2024-09-16T00:33:45.605 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:33:45.607 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:33:45.610 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:33:45.640 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T00:33:45.641 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-16T00:33:45.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T00:33:45.647 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:33:45.647 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-16T00:33:45.647 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T00:33:45.647 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-16T00:33:45.647 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T00:33:45.647 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:33:45.652 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 87.0% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:33:45.652 INFO:teuthology.orchestra.run.smithi078.stderr: 94.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-16T00:33:45.652 INFO:teuthology.orchestra.run.smithi078.stderr: 94.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-16T00:33:45.653 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T00:33:45.653 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-16T00:33:45.653 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T00:33:45.653 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T00:33:45.653 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:33:45.661 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-16T00:33:45.661 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-16T00:33:45.661 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T00:33:45.661 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:33:45.661 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:33:45.661 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T00:33:45.661 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:33:45.662 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-16T00:33:45.662 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T00:33:45.662 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-16T00:33:45.662 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:33:45.662 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T00:33:45.662 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-16T00:33:45.663 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-16T00:33:45.664 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-16T00:33:45.664 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.z.log: /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:33:45.694 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-16T00:33:45.701 INFO:teuthology.orchestra.run.smithi193.stderr: 93.4% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-16T00:33:45.710 INFO:teuthology.orchestra.run.smithi193.stderr: 83.4% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:33:45.755 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 83.5% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:33:45.782 INFO:teuthology.orchestra.run.smithi078.stderr: 91.1% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:33:45.804 INFO:teuthology.orchestra.run.smithi078.stderr: 90.3% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:33:45.812 INFO:teuthology.orchestra.run.smithi078.stderr: 94.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:33:45.825 INFO:teuthology.orchestra.run.smithi148.stderr: 93.2% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T00:33:45.836 INFO:teuthology.orchestra.run.smithi193.stderr: 90.5% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:33:45.862 INFO:teuthology.orchestra.run.smithi193.stderr: 94.5% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-16T00:33:45.892 INFO:teuthology.orchestra.run.smithi148.stderr: 90.4% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:33:45.908 INFO:teuthology.orchestra.run.smithi078.stderr: 83.0% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:33:45.911 INFO:teuthology.orchestra.run.smithi193.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:33:45.943 INFO:teuthology.orchestra.run.smithi148.stderr: 87.0% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:33:46.033 INFO:teuthology.orchestra.run.smithi148.stderr: 90.9% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:33:46.067 INFO:teuthology.orchestra.run.smithi193.stderr: 91.1% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:33:46.146 INFO:teuthology.orchestra.run.smithi148.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:33:46.252 INFO:teuthology.orchestra.run.smithi193.stderr: 93.1% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-16T00:33:46.499 INFO:teuthology.orchestra.run.smithi193.stderr: 87.0% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:33:46.673 INFO:teuthology.orchestra.run.smithi078.stderr: 94.6% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-16T00:33:46.703 INFO:teuthology.orchestra.run.smithi148.stderr: 94.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-16T00:33:47.106 INFO:teuthology.orchestra.run.smithi078.stderr: 90.2% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T00:33:47.179 INFO:teuthology.orchestra.run.smithi078.stderr: 93.8% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-16T00:33:47.345 INFO:teuthology.orchestra.run.smithi148.stderr: 94.4% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-16T00:33:47.958 INFO:teuthology.orchestra.run.smithi193.stderr: 95.5% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-16T00:33:47.995 INFO:teuthology.orchestra.run.smithi148.stderr: 93.8% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-16T00:33:48.545 INFO:teuthology.orchestra.run.smithi148.stderr: 91.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T00:33:49.865 INFO:teuthology.orchestra.run.smithi193.stderr: 91.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T00:33:52.686 INFO:teuthology.orchestra.run.smithi078.stderr: 90.3% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T00:33:58.822 INFO:teuthology.orchestra.run.smithi148.stderr: 95.6% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-16T00:34:34.123 INFO:teuthology.orchestra.run.smithi148.stderr: 90.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-16T00:35:13.753 INFO:teuthology.orchestra.run.smithi148.stderr: 90.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T00:35:45.760 INFO:teuthology.orchestra.run.smithi193.stderr: 90.9% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T00:35:49.376 INFO:teuthology.orchestra.run.smithi148.stderr: 90.8% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T00:35:49.510 INFO:teuthology.orchestra.run.smithi193.stderr: 90.8% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T00:36:27.202 INFO:teuthology.orchestra.run.smithi193.stderr: 90.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-16T00:36:30.048 INFO:teuthology.orchestra.run.smithi148.stderr: 91.0% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T00:36:30.050 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-16T00:36:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:real 2m44.424s 2024-09-16T00:36:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:user 5m0.606s 2024-09-16T00:36:30.050 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m10.300s 2024-09-16T00:36:43.103 INFO:teuthology.orchestra.run.smithi193.stderr: 90.8% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-16T00:36:43.105 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-09-16T00:36:43.105 INFO:teuthology.orchestra.run.smithi193.stderr:real 2m57.480s 2024-09-16T00:36:43.105 INFO:teuthology.orchestra.run.smithi193.stderr:user 4m54.325s 2024-09-16T00:36:43.105 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m10.114s 2024-09-16T00:37:58.489 INFO:teuthology.orchestra.run.smithi078.stderr: 90.7% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T00:39:17.649 INFO:teuthology.orchestra.run.smithi078.stderr: 90.8% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T00:40:26.615 INFO:teuthology.orchestra.run.smithi078.stderr: 90.8% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T00:40:52.494 INFO:teuthology.orchestra.run.smithi078.stderr: 90.8% -- replaced with /var/log/ceph/eb73a744-73be-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-16T00:40:52.495 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-09-16T00:40:52.495 INFO:teuthology.orchestra.run.smithi078.stderr:real 7m6.874s 2024-09-16T00:40:52.496 INFO:teuthology.orchestra.run.smithi078.stderr:user 5m58.481s 2024-09-16T00:40:52.496 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m13.320s 2024-09-16T00:40:52.496 INFO:tasks.cephadm:Archiving logs... 2024-09-16T00:40:52.497 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi078/log 2024-09-16T00:40:52.499 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:41:56.697 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi148/log 2024-09-16T00:41:56.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:42:48.447 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi193/log 2024-09-16T00:42:48.448 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:43:41.910 INFO:tasks.cephadm:Removing cluster... 2024-09-16T00:43:41.910 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb73a744-73be-11ef-bceb-c7b262605968 --force 2024-09-16T00:43:42.126 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:43:42.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb73a744-73be-11ef-bceb-c7b262605968 --force 2024-09-16T00:43:43.067 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:43:43.724 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid eb73a744-73be-11ef-bceb-c7b262605968 --force 2024-09-16T00:43:44.057 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: eb73a744-73be-11ef-bceb-c7b262605968 2024-09-16T00:43:44.720 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T00:43:44.720 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:43:44.739 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:43:44.759 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:43:44.774 INFO:tasks.cephadm:Teardown complete 2024-09-16T00:43:44.775 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T00:43:44.786 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T00:43:44.786 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:43:44.789 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:43:44.801 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:43:44.942 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T00:43:44.952 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T00:43:44.952 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:43:44.959 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:43:44.962 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:43:44.973 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:43:44.976 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:43:44.976 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:43:44.998 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:43:44.998 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +11us[ +32us] +/- 51ms 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -1194us[-1173us] +/- 68ms 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +1372us[+1393us] +/- 68ms 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 -109us[ -137us] +/- 51ms 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -1183us[-1183us] +/- 68ms 2024-09-16T00:43:44.999 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +1379us[+1351us] +/- 68ms 2024-09-16T00:43:45.000 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:43:45.001 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:43:45.001 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-16T00:43:45.002 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 -144us[ -89us] +/- 51ms 2024-09-16T00:43:45.002 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -1222us[-1222us] +/- 68ms 2024-09-16T00:43:45.002 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +1350us[+1350us] +/- 68ms 2024-09-16T00:43:45.002 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:43:45.003 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T00:43:45.013 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T00:43:45.013 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T00:43:45.023 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:43:45.177 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:43:45.280 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:43:45.441 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:43:45.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:43:45.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:43:45.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:43:45.506 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:43:45.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:43:45.535 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T00:43:45.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T00:43:45.557 INFO:teuthology.task.internal:Duration was 2836.916964 seconds 2024-09-16T00:43:45.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T00:43:45.568 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T00:43:45.568 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:43:45.574 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:43:45.576 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:43:45.637 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:43:45.643 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:43:45.655 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:43:46.013 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T00:43:46.014 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-09-16T00:43:46.014 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:43:46.053 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-09-16T00:43:46.053 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:43:46.078 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-09-16T00:43:46.078 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:43:46.113 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T00:43:46.114 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:43:46.116 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:43:46.122 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:43:46.180 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T00:43:46.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:43:46.193 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:43:46.200 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:43:47.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T00:43:47.830 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T00:43:47.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:43:47.880 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:43:47.981 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:43:48.033 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T00:43:48.045 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:43:48.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:43:48.051 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:43:48.074 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-09-16T00:43:48.083 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-09-16T00:43:48.091 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-09-16T00:43:48.119 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:43:48.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:43:48.157 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:43:48.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:43:48.176 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:43:48.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:43:48.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T00:43:48.204 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T00:43:48.205 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi078 2024-09-16T00:43:48.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:43:48.279 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi148 2024-09-16T00:43:48.280 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:43:48.318 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906370/remote/smithi193 2024-09-16T00:43:48.318 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:43:48.358 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T00:43:48.359 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:43:48.361 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:43:48.363 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:43:48.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T00:43:48.422 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T00:43:48.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T00:43:48.436 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T00:43:48.436 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:43:48.438 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:43:48.441 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:43:48.451 INFO:teuthology.orchestra.run.smithi078.stdout: 265707 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:43 /home/ubuntu/cephtest 2024-09-16T00:43:48.454 INFO:teuthology.orchestra.run.smithi148.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:43 /home/ubuntu/cephtest 2024-09-16T00:43:48.465 INFO:teuthology.orchestra.run.smithi193.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:43 /home/ubuntu/cephtest 2024-09-16T00:43:48.466 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T00:43:48.478 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T00:43:48.509 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 2836.916963815689 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T00:43:48.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:43:48.598 INFO:teuthology.run:pass